Skip to content

Add cache_control breakpoints to Anthropic requests to reuse expensive context #4256

Description

@Zelys-DFKH

Describe the feature or problem you'd like to solve

Requests to the Claude/Anthropic backend don't set cache_control breakpoints, so expensive, repeated context (system prompt, tool definitions, long-lived file/repo context) gets fully reprocessed on every turn instead of being served from Anthropic's prompt cache.

Proposed solution

Mark the static, reusable portions of the request (system prompt, tool schema, and other content unlikely to change turn-to-turn) with cache_control: {"type": "ephemeral"} breakpoints so Anthropic can serve them from cache on later turns. This is a narrower slice of #3808, focused only on wiring up the cache_control blocks themselves, not the TTL config option or a copilot cache status command.

Example prompts or workflows

  1. A long multi-turn CLI session: system prompt and tool definitions are marked cacheable, so turn 2+ reads them from cache instead of reprocessing.
  2. A large repo session: time-to-first-token on follow-up turns drops because the injected repo context is cached.
  3. Overall input token cost drops because cached tokens bill at Anthropic's reduced cache-read rate.

Additional context

Related to #3808, which additionally proposes configurable cache TTL and a diagnostics command. This issue is scoped to just the cache_control blocks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:modelsModel selection, availability, switching, rate limits, and model-specific behavior

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions