Skip to content

feat: capture agent statistics (token usage, duration) and surface in safe output write actions #168

@jamesadevine

Description

@jamesadevine

Summary

When the Copilot CLI exits, capture agent execution details and statistics — particularly token usage (input/output/total tokens, model used) and wall-clock duration — and make them available for inclusion in safe output write actions.

Motivation

Today, when an agent creates a PR or work item, there is no visibility into how much work the agent did — how many tokens it consumed, how long it ran, which model was used, etc. Surfacing these stats in write actions (PR descriptions, work item comments) gives operators better observability and cost attribution without requiring them to dig through pipeline logs.

Proposed Behavior

  1. Capture on exit: After the Copilot CLI process exits, parse its stdout/stderr or exit artifacts for:

    • Token usage (input tokens, output tokens, total tokens)
    • Model name
    • Wall-clock duration
    • Number of tool calls / turns (if available)
  2. Expose as template variables: Make these stats available as template variables that can be interpolated into safe output payloads, e.g.:

    • {{ agent.tokens.input }}
    • {{ agent.tokens.output }}
    • {{ agent.tokens.total }}
    • {{ agent.model }}
    • {{ agent.duration }}
    • {{ agent.turns }}
  3. Surface in write actions: Automatically append (or allow opt-in appending of) a summary block to:

    • create-pull-request — append stats to PR description
    • create-work-item — append stats to work item description
    • comment-on-work-item — optionally include in comment body
    • add-pr-comment — optionally include in comment body

Example Output

A PR description might include:

---
🤖 **Agent Stats**
- Model: claude-opus-4.5
- Tokens: 45,230 input / 12,450 output (57,680 total)
- Duration: 4m 32s
- Tool calls: 23

Open Questions

  • What format does Copilot CLI emit stats in? (stdout JSON, exit artifact, etc.)
  • Should stats be opt-in per safe-output or always included?
  • Should there be a front matter toggle like include-agent-stats: true?
  • Should stats be appended as a collapsible section (<details>) in markdown bodies?

Acceptance Criteria

  • Agent stats are captured after Copilot CLI exits
  • Stats are available for interpolation in safe output write actions
  • At least create-pull-request and create-work-item support stats inclusion
  • Stats rendering is configurable (opt-in/opt-out)

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions