docs(sample-app): add agent memory retrieval tracing example#4364
Open
FBISiri wants to merge 2 commits into
Open
docs(sample-app): add agent memory retrieval tracing example#4364FBISiri wants to merge 2 commits into
FBISiri wants to merge 2 commits into
Conversation
Adds packages/sample-app/sample_app/agents/agent_memory_tracing_example.py — a self-contained demo showing how to instrument memory read/write operations in an agent with OpenTelemetry spans. Key patterns demonstrated: - memory.write span with key and value_len attributes - memory.read span with hit/miss boolean and age_s on cache hits - Parent agent.turn span correlating memory latency + LLM latency - Two-session demo: warm memory hit vs cold-start miss Useful for teams that add a retrieval-augmented memory layer (Mem0, custom vector store, Redis) and want to measure retrieval quality and latency separately from model inference in Traceloop. No new dependencies — uses opentelemetry-api and traceloop-sdk already present in the sample-app requirements.
|
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughAdds a runnable OpenTelemetry/Traceloop example that traces session-scoped memory reads and writes around OpenAI agent turns, including hit/miss and age attributes, with a multi-session demo. ChangesAgent memory tracing workflow
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant User
participant run_agent_turn
participant SimpleMemoryStore
participant OpenAI
User->>run_agent_turn: submit message and session ID
run_agent_turn->>SimpleMemoryStore: read session preference
run_agent_turn->>OpenAI: send system prompt and user message
OpenAI-->>run_agent_turn: return completion
run_agent_turn->>SimpleMemoryStore: store detected preference
run_agent_turn-->>User: return agent response
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/sample-app/sample_app/agents/agent_memory_tracing_example.py`:
- Around line 34-37: Update the Traceloop initialization in
agent-memory-tracing-example to configure a ConsoleSpanExporter, importing it
from opentelemetry.sdk.trace.export. Preserve the existing app_name and
disable_batch settings while adding the exporter for debugging spans and
hierarchy issues.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 474e3cbd-4206-414a-9016-8f4e457be2e1
📒 Files selected for processing (1)
packages/sample-app/sample_app/agents/agent_memory_tracing_example.py
Comment on lines
+34
to
+37
| Traceloop.init( | ||
| app_name="agent-memory-tracing-demo", | ||
| disable_batch=True, # flush immediately so spans appear right away in dev | ||
| ) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win
Configure ConsoleSpanExporter for debugging.
The current initialization of Traceloop does not include a console exporter. As per coding guidelines, you must use ConsoleSpanExporter from opentelemetry.sdk.trace.export for debugging OpenTelemetry spans and hierarchy issues.
🛠️ Proposed fix to add the console exporter
Update the imports and initialization block to include the required exporter:
from opentelemetry import trace
+from opentelemetry.sdk.trace.export import ConsoleSpanExporter, SimpleSpanProcessor
from traceloop.sdk import Traceloop
load_dotenv()
+traceloop_processor = Traceloop.get_default_span_processor(disable_batch=True)
+console_processor = SimpleSpanProcessor(ConsoleSpanExporter())
+
Traceloop.init(
app_name="agent-memory-tracing-demo",
- disable_batch=True, # flush immediately so spans appear right away in dev
+ processor=[traceloop_processor, console_processor],
)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/sample-app/sample_app/agents/agent_memory_tracing_example.py` around
lines 34 - 37, Update the Traceloop initialization in
agent-memory-tracing-example to configure a ConsoleSpanExporter, importing it
from opentelemetry.sdk.trace.export. Preserve the existing app_name and
disable_batch settings while adding the exporter for debugging spans and
hierarchy issues.
Source: Coding guidelines
Author
|
I have read the CLA Document and I hereby sign the CLA |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
packages/sample-app/sample_app/agents/agent_memory_tracing_example.py— a self-contained, runnable example showing how to instrument memory read/write operations in an agent with OpenTelemetry spans.Motivation
The existing sample apps cover LLM inference tracing (OpenAI, LangChain, Agno, CrewAI) and agent tool-call tracing, but none demonstrate how to trace the memory retrieval layer that is increasingly common in production agents. Teams that add Mem0, a custom vector store, or Redis for cross-session memory have no reference pattern for measuring retrieval latency and hit/miss rates alongside model inference.
What the example shows
Key span attributes:
memory.operation"read"or"write"memory.keymemory.hitmemory.age_smemory.value_lenmemory.context_injectedThe demo runs a two-turn warm session followed by a cold-start session so you can compare traces in the Traceloop dashboard.
Checklist
opentelemetry-apiandtraceloop-sdkalready in sample-app requirements)python -m sample_app.agents.agent_memory_tracing_example(requiresOPENAI_API_KEY)Traceloop.init()+ tracer pattern as other sample-app examplesSummary by CodeRabbit