Skip to content

Docs: reconcile the tracing page with the generated telemetry contract #519

Description

@Rachael-Graham

kagent now defines its telemetry attribute names in one place and generates the reference from it. kagent#2906 added a Weaver registry at telemetry/registry, which generates the Go constants, the Python constants, and docs/architecture/telemetry-contract.md. A CI job fails when the generated files drift.

No attribute name changed in that pull request, so nothing on the docs site is wrong today. The problem is what the contract now says about the attributes the site does document.

The defect

The Correlation attributes table on observability/tracing.md presents four attributes as the way to find a request: gen_ai.task.id, gen_ai.conversation.id, kagent.app_name and kagent.user_id.

Three of those four are outside the contract. docs/architecture/telemetry.md lists kagent.user_id, gen_ai.task.id and kagent.app_name in a column named "Keys outside the contract", emitted by the Go ADK runtime alone, and its Blind spots section says they "are removed when the Go and Python runtimes move to the contract". The pull request describes itself as the first step of a larger cleanup with follow-up pull requests to come.

The contract's own request identity is a2a.task.id, gen_ai.conversation.id and enduser.id, with kagent.runtime on the resource and kagent.invocation.segment on request spans. Only gen_ai.conversation.id appears in both places. So a reader who builds a dashboard from the current page keys it on three names that upstream has already scheduled for removal, and does not learn the names that replace them.

The span table above it has the same problem in a milder form. The page names spans POST /lf.a2a.v1.A2AService/SendMessage, invocation, invoke_agent <agent>, generate_content <model> and execute_tool <tool>; the contract names kagent.a2a.http.server, kagent.a2a.rpc.server and kagent.invoke_agent.internal. Both descriptions can be true at once, since the contract names span definitions rather than emitted span names, but the page should not leave a reader unable to connect the two.

What to change

Rework the correlation section so that it leads with the contract attributes and marks the Go ADK keys as transitional. The reader's question is "what do I key a dashboard on", and the honest answer today is "these, and these others still appear but will go".

observability/audit-prompts.md carries the same attribute vocabulary and needs the same pass.

Also worth reconciling while the page is open: telemetry.md documents the capture controls as KAGENT_OTEL_CAPTURE_SENSITIVE_CONTENT, KAGENT_OTEL_CAPTURE_RAW_API_BODIES and KAGENT_OTEL_MAX_CAPTURE_BYTES, while the tracing page's warning tells a reader to set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT, and the contract records gen_ai.input.messages only under OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=SPAN_ONLY. Establish which variable a reader sets, and whether a plain true reaches spans at all.

What to check before starting

This one needs a live trace, not a reading of the contract. The registry is a declaration of what kagent intends to emit; telemetry.md says plainly that nothing yet compares emitted telemetry with the registry, and that a live check is still planned. Run an agent, export to a backend, and read the spans before changing a word.

  1. Read docs/architecture/telemetry.md and docs/architecture/telemetry-contract.md in the kagent repository.
  2. Run the existing tracing guide as written, on the version being documented, and capture a real trace.
  3. Compare the emitted attributes and span names with both tables on the page.
  4. Check the capture variables by setting each one and observing what reaches the backend.

Done when

  • The correlation table distinguishes contract attributes from the Go ADK keys that upstream plans to remove, and says so.
  • a2a.task.id, enduser.id, kagent.runtime and kagent.invocation.segment are covered, to the extent a live trace shows them.
  • The span table is reconcilable with the contract's span definitions.
  • The capture variable a reader sets is verified by observation, and the page and audit-prompts.md agree on it.
  • Every claim on the page was checked against a trace captured on the documented version.
  • audit-prompts.md got the same pass.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationkagent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions