fix: preserve DCP state across compaction, protect reasoning signatures, resolve ignored boundary refs#530
Open
tracycam wants to merge 16 commits into
Open
Conversation
Edit/write filediff.before/after were being stripped from every tool call, including recent edits not yet flagged for pruning. Gate the strip on state.prune.tools.has(part.callID) so only tool calls already entering the prune pipeline lose their filediff snapshots. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…estimate When a range consumed previously-compressed blocks, estimateSelectedTokens double-counted: it added every selected message's tokens AND added each consumed block's summaryTokens. Real current-context cost should skip messages already covered by the consumed blocks (their tokens are gone from context) and only add the consumed blocks' summary tokens. Build a consumedMessageIds Set from each block's effectiveMessageIds and skip those messages during selection accounting. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
…ilure
Batch compression previously called allocateBlockId() per plan inside the
apply loop. If the second plan's summary blew the size limit, the first
plan had already reserved a block id and mutated state.prune.messages,
leaving a half-written batch behind.
Split allocateBlockId into two phases:
- previewBlockIds(state, n): pure read, returns the next n ids without
mutating state
- reserveBlockIds(state, n): advances state.prune.messages.nextBlockId
compress message/range now (1) preview ids for the whole batch, (2) build
storedSummary + summaryTokens + estimateSelectedTokens for every plan and
run assertUsefulCompressedSummary, then (3) reserve ids and allocateRunId
only after every plan validates. allocateBlockId() is kept as a wrapper
around reserveBlockIds(state, 1) for backward compat.
Adds regression tests in compress-{message,range}.test.ts asserting that
when a later plan's summary exceeds limits, blocksById and activeBlockIds
remain empty and nextBlockId/nextRunId stay at 1.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
# Conflicts: # lib/compress/message.ts
Persist lastCompaction and messageIds so restarts keep the same mNNNN aliases, make native compaction reset inert because opencode preserves msg_* rows, and only deactivate compression blocks when sync is fed an authoritative full-session message list. Oracle findings: b1, b2, b3, b4, b5. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Add regression coverage for persisted lastCompaction and messageIds, backward-compatible loads, stale-save protection, native compaction restarts, authority-aware block sync, immediate ref persistence, and preservation of aliases after compaction. Oracle findings: b1, b2, b3, b4, b5. Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent) Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Anthropic's thinking blocks must stay byte-for-byte unchanged on subsequent API calls; signature lives in metadata.anthropic. DCP previously stripped that signature on differentModel and Anthropic rejected the next request as "thinking blocks ... cannot be modified". Reasoning parts now pass through unchanged while opencode native handles reasoning-to-text conversion on differentModel itself (message-v2.ts:958-972). Reproduction session: ses_1e893bcf3ffeZluDvQ4RHEsfbP.
assignMessageRefs allocates refs for user-summary slash-command messages when parts are not yet marked ignored (chat.params time). At compress time, fetchSessionMessages returns parts with ignored:true. buildBoundaryLookup then excluded these refs via isIgnoredUserMessage, causing "not available" errors for any boundary ID pointing to such a message. Remove the isIgnoredUserMessage filter from buildBoundaryLookup — the lookup only maps ref → msgId → rawIndex for address resolution. Downstream resolveSelection (line 129) already applies the ignored-message filter for content selection, so ignored messages remain non-compressible content but are now usable as positional anchors. Affects every session that uses slash commands (/cost, /share, etc.) and then attempts to compress ranges containing those message positions. Reproduced on ses_1e2de93a6ffeXwjU7kSyqHWeFL (ESCTools project) and ses_1e9066e4effeRkxz5jqH4YqvdZ (this session).
4 test cases: - ignored user msg ref resolves in resolveBoundaryIds - ignored user msg as both start and end (single-message compress) - unknown ref still rejected - compressed block anchored to ignored user msg resolves
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
lastCompactionandmessageIdsacross process restarts — previously in-memory only, causingresetOnCompactionto wipe all DCP state (compression blocks, prune.tools, messageIds, nudges) on every restart after native/compactreasoningparts fromstripStaleMetadata— was strippingmetadata.anthropic.signatureon model-switch, breaking Anthropic's signature verification requirementisIgnoredUserMessagefilter frombuildBoundaryLookup— refs allocated atchat.paramstime (whenparts.ignorednot yet set) became unresolvable at compress time whenfetchSessionMessagesreturnedignored:truepartsAlso includes from prior branch work:
previewBlockIds/reserveBlockIds) preventing partial state on validation failureassertUsefulCompressedSummary: >60K token cap, ratio check for >10K selections)prune.toolssyncCompressionBlocksauthoritative mode — only deactivate missing-origin blocks when called with full DB message listBug Details
Bug 1: State lost on compaction restart
lastCompactionwas never persisted → every process restart re-triggeredresetOnCompaction→ wiped ALL DCP state. Fix: persist bothlastCompactionandmessageIdsinPersistedSessionState; makeresetOnCompactioninert (opencode/compactpreserves allmsg_XIDs in DB).Bug 2: Reasoning signatures wiped on model switch
stripStaleMetadataincludedreasoninginRELEVANT_TYPES, droppingmetadata.anthropic.signature. Anthropic requires thinking block metadata byte-for-byte intact. Fix:RELEVANT_TYPES = Set(["text", "tool"])— opencode native already handles reasoning→text conversion for different-model requests.Bug 3: Compress boundary lookup rejects ignored user-message refs
assignMessageRefsallocated refs for slash-command user messages atchat.paramstime (beforeparts.ignoredwas set). At compress time,fetchSessionMessagesreturned these withignored:true→buildBoundaryLookupexcluded them → 'not available' error. Fix: removeisIgnoredUserMessagechecks from lookup (address resolution only; content filtering handled byresolveSelection:129).Test Results
Pre-existing failures (unrelated to this PR):
New test files
tests/compaction-resilience.test.ts(8 tests): persistence round-trips, compaction restart preservation, syncCompressionBlocks authoritative mode, chat transform ref persistencetests/reasoning-strip.test.ts(4 tests): reasoning signature preserved on model switch, text/tool metadata still stripped, empty text reasoning untouched, same-model reasoning untouchedtests/ignored-boundary-lookup.test.ts(4 tests): ignored ref resolves, both start+end ignored, unknown ref still rejected, compressed block anchored to ignored msg resolvestests/prune-tool-metadata.test.ts(1 test): filediff snapshot pruned only when callID in prune.toolstests/summary-limits.test.ts(1 test): consumed blocks counted as current summariestests/message-ids.test.ts: asserts preservation instead of reset after compactionKnown Limitations
saveSessionStatemonotonic-mergeslastCompactionbut overlapping async saves can still overwrite newermessageIds— would need an atomic write layer to fully solveresolveSelection:129still excludes ignored messages from selection content; active blocks anchored to ignored messages may not be auto-included unless referenced viabNboundarytruncateProtectedToolOutputretains 20K chars of original content plus marker text (~80 chars over cap)Files Changed
23 files changed, 1359 insertions(+), 67 deletions(-)