fix(telemetry): report cached input tokens on LLM spans

[Panmax]

Summary

• export cached prompt token usage with the standard gen_ai.usage.cache_read.input_tokens attribute
• keep gen_ai.usage.input_tokens inclusive, as required by the OpenTelemetry GenAI semantic conventions, while allowing compatible backends such as Langfuse to normalize cache reads without double-counting
• apply the fix centrally so every provider that populates CompletionUsage.prompt_cached_tokens benefits
• preserve the existing Langfuse-specific cache attribute on realtime metrics and evaluation spans so current dashboards remain compatible
• add a provider-independent regression test that verifies the exported span attributes

Testing

• pytest tests/test_llm_telemetry.py --unit -q
• ruff check livekit-agents/livekit/agents/llm/llm.py livekit-agents/livekit/agents/telemetry/trace_types.py tests/test_llm_telemetry.py
• ruff format --check livekit-agents/livekit/agents/llm/llm.py livekit-agents/livekit/agents/telemetry/trace_types.py tests/test_llm_telemetry.py
• mypy --config-file pyproject.toml -p livekit.agents.llm.llm

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 1 additional finding in Devin Review.

[devin-ai-integration]

🟡 Cached-token reporting only updated for one of the three places it is sent to tracing

The standard cached-input-token field is added only for regular text model spans (ATTR_GEN_AI_USAGE_CACHE_READ_INPUT_TOKENS at livekit-agents/livekit/agents/llm/llm.py:405-407), while realtime-model spans and evaluation spans still send only the old non-standard field, so monitoring backends see cache usage inconsistently depending on which path produced the data.
Impact: Dashboards that rely on the standard cached-token field will show no cache usage for realtime sessions and evaluation runs, even though the PR claims those paths were updated.

Where the old non-standard attribute is still the only one emitted

The PR description states "use the same standard cache-read attribute for realtime metrics and evaluation spans", but two remaining call sites were not changed:

• livekit-agents/livekit/agents/telemetry/utils.py:41 (record_realtime_metrics) still sets only trace_types.ATTR_GEN_AI_USAGE_INPUT_CACHED_TOKENS (gen_ai.usage.input_cached_tokens) from ev.input_token_details.cached_tokens.
• livekit-agents/livekit/agents/voice/run_result.py:1068 (judge span attributes) still sets only ATTR_GEN_AI_USAGE_INPUT_CACHED_TOKENS from usage.prompt_cached_tokens.

The result is that only llm_request spans carry gen_ai.usage.cache_read.input_tokens, so consumers must handle both naming schemes.

Prompt for agents

The PR intends to emit the standard OpenTelemetry cache-read input token attribute everywhere cached prompt tokens are reported, but only livekit-agents/livekit/agents/llm/llm.py was updated. Two other span-attribute sites still emit only the legacy, unofficial attribute ATTR_GEN_AI_USAGE_INPUT_CACHED_TOKENS: record_realtime_metrics in livekit-agents/livekit/agents/telemetry/utils.py (using ev.input_token_details.cached_tokens) and the judge span attributes in livekit-agents/livekit/agents/voice/run_result.py (using usage.prompt_cached_tokens). Decide whether both attributes should be emitted (for backwards compatibility) or only the standard one, and apply the choice consistently across all three sites so telemetry consumers see one coherent schema.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[Panmax]

Thanks. This split is intentional and is now called out in the PR summary. The realtime and evaluation emitters are pre-existing call sites, so changing their key would break current dashboards, as the previous finding correctly noted. The generic LLM span did not report cached tokens before this PR, so adding only the standard gen_ai.usage.cache_read.input_tokens attribute there has no attribute-rename compatibility cost. I also avoided dual-emitting aliases on a single span because consumers may normalize both names into one cache bucket and double-count usage. Standardizing the two legacy paths should therefore be handled as a separate deprecation/migration with a compatibility plan; this PR remains scoped to adding cached-token reporting to generic LLM spans.