refactor: summarize only trimmed history in reactive_compact#364
Open
costajohnt wants to merge 1 commit into
Open
refactor: summarize only trimmed history in reactive_compact#364costajohnt wants to merge 1 commit into
costajohnt wants to merge 1 commit into
Conversation
reactive_compact summarized the full message history before slicing off the recent tail, so the kept tail was summarized and then re-attached verbatim, and the emergency prompt-too-long path re-summarized the whole oversized context. Move the summarize_history call below the tail computation (including the tool_use/tool_result pair adjustment) and summarize only messages[:tail_start], so the summary covers older history while the recent tail stays verbatim. Applied to the duplicated function in s08, s09, and s20, with the s08 README snippets (en/zh/ja) updated to match. Adds tests covering the normal path and the case where a tool pair straddles the tail boundary. Fixes shareAI-lab#350
|
@costajohnt is attempting to deploy a commit to the crazyboym's projects Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #350.
reactive_compactis the emergency compaction path that runs after the main call hitsprompt_too_long. It was callingsummarize_history(messages)on the full history first, then slicing off the last few messages as a verbatim tail. So the kept tail got summarized and then re-attached verbatim, and the emergency path re-summarized the whole oversized context it was trying to shrink.This moves the
summarize_historycall below the tail computation (including the tool_use/tool_result pair adjustment) and summarizes onlymessages[:tail_start], so the summary covers the older history while the recent tail stays verbatim. That matches the intent described in the issue.The same function is duplicated in s08, s09, and s20, so all three are updated together. s11 keeps its own tail only version since it never calls
summarize_history. The s08 README snippets (en/zh/ja) are updated to match.Added two tests, one for the normal path and one where a tool_use/tool_result pair straddles the tail boundary, confirming the summary excludes the tool_use that gets pulled into the verbatim tail.
Two notes. When a conversation is 5 messages or fewer,
tail_startis 0 and the whole list is kept verbatim, sosummarize_historyis called with an empty list. That is cheaper than the previous full re-summary and nothing is lost, but happy to add a guard to skip the call entirely if you prefer. I also leftweb/src/data/generated/docs.jsonuntouched since the build regenerates it from source vianpm run extract, let me know if you would rather I commit the regenerated file.