Skip to content

[Bug]: Context compaction trigger stays tripped after compaction — stale provider usage is trusted as the prefix sum #152

Description

@KevinBarrus

Description

With the default local token estimator (no remote tokenCounter configured), the context-compaction trigger stays permanently tripped after a compaction has already run, because kept assistant messages still carry provider usage from requests that included the entire pre-compaction context. This causes repeated unnecessary compactions (each costing a summary LLM call and dropping more real transcript) or leaves context growth unmanaged once no_safe_cut_point kicks in.

Source evidence

Audited against the 0.4.12 source preview (release/extraction.json sourceRevision 9b9885e42a3cf1a3df1cfa52a46e4fdb034cfcee); line numbers refer to that revision.

  • estimateContextTokens treats the last assistant message's provider-reported usage as the ground-truth prefix sum: packages/agent-modules/context-manager/src/token-estimator.ts:287-308 (tokens = usageTokens + trailing).
  • After compaction, the replacement transcript is [compactionSummary, ...keptMessages]: packages/agent-modules/context-manager/src/manager.ts:117-121. keptMessages are assistant/user messages from before the compaction, and kept assistant messages still carry their original usage blocks — whose input token counts covered the whole pre-compaction context.
  • The trigger evaluation uses that estimate: packages/agent-modules/context-manager/src/manager.ts:189-227 (shouldCompact = count.tokens > triggerAt, source local_estimate when no remote counter is wired).
  • Consequence: right after a compaction that brought the real context down to (summary + tail), the next checkpoint still estimates ≈ stale pre-compaction usage + trailing, so shouldCompact stays true. selectPlan (manager.ts:230-274) then either re-compacts the already-small kept window (another summary LLM call, more real messages dropped into a summary) or returns no_safe_cut_point once the kept window is below minMessagesToCompact, while real context grows unmanaged.

The "trust provider usage as prefix ground truth" assumption is only valid for an unmodified transcript; replaceMessages invalidates it. The estimate only self-heals once the stale-usage assistant message is itself compacted out of the kept window.

Expected behavior

After a compaction, the trigger estimate should reflect the post-compaction transcript (summary + kept tail), so the next compaction happens when the new context actually approaches the threshold again.

Suggested fix directions

Either of:

  1. At compaction time, strip/zero the usage blocks carried by kept assistant messages (the summary message does not carry usage), so the estimator falls back to local estimation for the new transcript.
  2. In estimateContextTokens, only trust provider usage whose assistant message is positioned after the latest compactionSummary message; otherwise fall back to full local estimation.

Option 2 avoids mutating persisted history.

Environment

Found by source audit; not runtime-reproduced yet. Applies to the default local-estimator path (manager.ts:191-209); sessions with a remote tokenCounter are unaffected.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions