feat(agents): opt-in capture of raw LLM request/response bodies

[hvent90]

Contribution path

• Small, safe change that does not need a tracking issue

Problem

We need to save agent trajectories of our MCP client but it at present has no way of saving that information.

Solution

An opt-in process that instruments the httpx client for OpenAI requests. The default behavior does not affect any existing hot paths. The first consumer will be the eval framework: #3774

Auth headers are not saved.

Langchain does not have a provider-agnostic hook for capturing raw requests/responses, so it would require more tinkering for other providers. Because we just use OpenAI, I kept it simple. Maybe we'll be off Langchain by the time we need universal coverage :)

How to Test

uv run python - <<'EOF'
from pathlib import Path
from dimos.agents.llm_trace import tracing_http_client

tracing_http_client(Path("/tmp/llm-trace")).post(
    "https://api.openai.com/v1/chat/completions", json={"ping": 1})
print(sorted(p.name for p in Path("/tmp/llm-trace").iterdir()))
EOF # -> Prints `['000-request.json', '000-response.json']`

AI assistance

Fable 5 wrote 99.9% of this. I reviewed every line.

Checklist

• I have read and approved the CLA.

[codecov]

Codecov Report

❌ Patch coverage is 94.00000% with 6 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
dimos/agents/llm_trace.py	91.83%	4 Missing ⚠️
dimos/agents/mcp/mcp_client.py	84.61%	1 Missing and 1 partial ⚠️

@@            Coverage Diff             @@
##             main    #3843      +/-   ##
==========================================
+ Coverage   77.58%   77.60%   +0.01%     
==========================================
  Files        1315     1317       +2     
  Lines      124697   124794      +97     
  Branches    10867    10868       +1     
==========================================
+ Hits        96748    96841      +93     
- Misses      24816    24820       +4     
  Partials     3133     3133              

Flag	Coverage Δ
OS-ubuntu-24.04-arm	74.25% <94.00%> (+0.01%)	⬆️
OS-ubuntu-latest	74.79% <94.00%> (+0.01%)	⬆️
Py-3.10	74.79% <94.00%> (+0.01%)	⬆️
Py-3.11	74.79% <94.00%> (+0.01%)	⬆️
Py-3.12	74.79% <94.00%> (+0.01%)	⬆️
Py-3.13	74.79% <94.00%> (+0.01%)	⬆️
Py-3.14	74.80% <94.00%> (+0.01%)	⬆️
Py-3.14t	74.79% <94.00%> (+0.01%)	⬆️
SelfHosted-Large	30.22% <36.00%> (+<0.01%)	⬆️
SelfHosted-Linux	34.74% <39.00%> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
dimos/agents/test_llm_trace.py	100.00% <100.00%> (ø)
dimos/agents/mcp/mcp_client.py	83.55% <84.61%> (-0.17%)	⬇️
dimos/agents/llm_trace.py	91.83% <91.83%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Greptile Summary

This change adds opt-in raw OpenAI request/response tracing and exposes the trace directory through MCP client configuration.

A concurrent shared trace directory can lose or misassociate records: independently created tracing clients can select the same sequence number and overwrite a request/response pair. The non-OpenAI initialization path was also exercised; although it temporarily constructs an unused tracing client, that client is immediately collected before opening connections, so it does not produce the reported connection-pool leak.

The change is not safe to merge until trace sequence allocation is made atomic for all clients and processes writing to the same directory.

Confidence Score: 4/5

Not safe to merge while concurrent writers can overwrite trace pairs in a shared trace directory.

Focused executable checks reproduced the shared-directory overwrite using the real tracing hooks and disproved the reported retained-client leak on the non-OpenAI path.

Files Needing Attention: dimos/agents/llm_trace.py needs directory-scoped, cross-process-safe trace sequence reservation. dimos/agents/mcp/mcp_client.py was exercised and needs no change for the reported client-lifetime concern.

T-Rex Logs

What T-Rex did

• T-Rex ran both the sequential baseline harness and the concurrent collision harness and reviewed their outputs.
• T-Rex executed the focused non-OpenAI tracing-client lifecycle harness with and without a trace directory, and verified that no requests were made and no resources remained allocated after initialization.
• T-Rex verified the exact tracing commands and observed results, including per-client locking preventing cross-client synchronization and the difference in files produced by the baseline and the concurrent runs.
• T-Rex produced a formal finding proof for a posted P1 finding.
• T-Rex concluded that the non-OpenAI tracing path does not pose a security issue and has negligible overhead, based on the tracing harness runs.

_{Ran code and verified through T-Rex}

Comments Outside Diff (1)

1. General comment

   Shared trace directories allow independently-created clients to overwrite trace pairs

   • Bug
     • When two tracing_http_client instances submit requests concurrently to the same trace directory, both can count zero request files and choose sequence 0. The focused runtime harness made both production count operations complete before either write; both HTTP requests returned 200, but the directory retained only 000-request.json and 000-response.json, losing client-B's trace record.
   • Cause
     • _next_seq() is a non-atomic len(glob(...)) allocation at dimos/agents/llm_trace.py:43-44, while the threading.Lock() at line 73 is captured separately by every client instance. It cannot coordinate allocations across independently-created clients sharing trace_dir.
   • Fix
     • Use allocation synchronized at the trace-directory scope, including across processes if shared directories are supported. For example, hold a shared per-directory lock plus allocate with an atomic create (open(..., 'x')) and retry on FileExistsError; use an OS-backed file lock or another atomic allocator for multi-process callers. Preserve request/response pairing by reserving the sequence before writing the request.

   _{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "refactor(agents): drop the pair-reading ..." | Re-trigger Greptile}