Skip to content

[bot] AWS Bedrock Agent Runtime SDK (@aws-sdk/client-bedrock-agent-runtime) not instrumented — no tracing for managed agent invocation #2170

Description

@braintrust-bot

Summary

The AWS SDK v3 Bedrock Agent Runtime client (@aws-sdk/client-bedrock-agent-runtime) provides execution APIs for invoking pre-configured Amazon Bedrock Agents. This is a distinct package and API surface from @aws-sdk/client-bedrock-runtime (which instruments direct model calls via ConverseCommand/InvokeModelCommand and is already covered by js/src/auto-instrumentations/configs/bedrock-runtime.ts). The Agent Runtime client runs orchestrated, stateful agents that perform multi-turn reasoning with tool use — its own InvokeAgentCommand is not covered by any existing instrumentation. Users who invoke Bedrock Agents directly get no Braintrust spans.

What instrumentation is missing

The @aws-sdk/client-bedrock-agent-runtime package exposes these execution surfaces, none of which are instrumented:

SDK Method Description
client.send(new InvokeAgentCommand({ agentId, agentAliasId, sessionId, inputText })) Invokes a pre-configured Bedrock Agent; returns a streaming AsyncIterable of response chunks and trace events
client.send(new InvokeInlineAgentCommand({ ... })) Invokes an inline agent definition (no pre-configured agent required)
client.send(new RetrieveAndGenerateCommand({ ... })) RAG-based generation using Bedrock Knowledge Bases
client.send(new RetrieveAndGenerateStreamCommand({ ... })) Streaming variant of RAG generation

The streaming InvokeAgentCommand response includes both generated text chunks and structured trace events that carry model invocation details, rationale, and tool invocation metadata — exactly the kind of data Braintrust is designed to capture.

No coverage in any instrumentation layer:

  • No wrapper function (e.g. wrapBedrockAgentRuntime())
  • No diagnostics channels for Agent Runtime methods
  • No plugin handler in js/src/instrumentation/plugins/
  • No auto-instrumentation config in js/src/auto-instrumentations/configs/
  • No e2e test scenarios
  • Grep for bedrock-agent or InvokeAgent across js/src/ returns zero matches

Relationship to existing Bedrock instrumentation:

js/src/auto-instrumentations/configs/bedrock-runtime.ts instruments @aws-sdk/client-bedrock-runtime — the model inference layer (ConverseCommand, InvokeModelCommand). @aws-sdk/client-bedrock-agent-runtime is a separate npm package with a different API (agent orchestration layer). Both packages can be used together, but the agent invocation surface is not covered.

Braintrust docs status

unclear — Braintrust documents AWS Bedrock as a supported provider, but the JavaScript SDK documentation focuses on @aws-sdk/client-bedrock-runtime (direct model calls). No Braintrust docs page specifically addresses @aws-sdk/client-bedrock-agent-runtime or InvokeAgentCommand.

Upstream references

Local files inspected

  • js/src/auto-instrumentations/configs/bedrock-runtime.ts — instruments @aws-sdk/client-bedrock-runtime only; no entry for @aws-sdk/client-bedrock-agent-runtime
  • js/src/auto-instrumentations/configs/all.ts@aws-sdk/client-bedrock-agent-runtime not listed
  • js/src/instrumentation/plugins/bedrock-runtime-plugin.ts handles model runtime only; no agent runtime plugin
  • e2e/scenarios/ — Bedrock e2e tests cover model runtime, not agent runtime
  • Full repo grep for bedrock-agent, BedrockAgent, InvokeAgent in js/src/ — zero matches
  • e2e/scenarios/strands-agent-sdk-instrumentation/pnpm-lock.yaml — only mention of bedrock-agent-runtime in the entire repo (as a transitive dependency of Strands SDK)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions