Skip to content

fix(session): drop the clarification option from the run-mode nudge and apply the completion contract to every agent in run mode - #1641

Draft
RaghwendraSingh wants to merge 1 commit into
mainfrom
fix/run-mode-compaction-nudge
Draft

RaghwendraSingh wants to merge 1 commit into
mainfrom
fix/run-mode-compaction-nudge

Conversation

@RaghwendraSingh

@RaghwendraSingh RaghwendraSingh commented Nov 3, 2026 •

Copy link
Copy Markdown

Issue for this PR

Closes #1640

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Headless altimate-code run runs could end on a question to the user, who isn't there to answer. In an eight-pass headless evaluation, all six turn-ending questions came right after the post-compaction nudge, whose option (2) was "stop and ask for clarification". This PR fixes that and a second gap that let the same thing happen.

1. The run-mode nudge no longer offers a clarification stop (session/termination.ts).
COMPLETION_NUDGE is sent only when ALTIMATE_RUN_MODE is on (compaction.ts already branches on the flag; interactive compaction uses its own continuation string). It now reads:

Context was compacted; the summary above is the record of the work so far. No one can answer questions during this run, so do not end your turn with a question. Choose exactly one: (1) if concrete next steps remain toward the original task, continue with them; if you are unsure how to proceed, pick the most reasonable interpretation, state your assumption, and continue; (2) if the deliverable is complete and verified, reply with DONE alone on the final line and stop.

The interactive continuation ("Continue if you have next steps, or stop and ask for clarification if you are unsure how to proceed.") is not touched; it is now pinned byte-for-byte by tests.

2. Every agent gets the completion contract in run mode (session/termination.ts, session/prompt.ts).
completionInstruction() was gated by COMPLETION_CONTRACT_AGENTS = {"builder"}, so custom agents and subagents started by the task tool ran without the "end with DONE" contract. The allowlist and the agent input are removed; the function now depends only on run mode. Run mode is the existing Flag.ALTIMATE_RUN_MODE signal (set by run via cli/cmd/run/run-mode.ts); no new global. Task subagents are prompted through the same SessionPrompt loop in the same process, so they pick it up with no change to tool/task.ts. Interactive sessions still get no contract for any agent.

Behaviour change to review: in run mode, the subagent's final text now ends with a DONE line, and that text is what the task tool returns inside <task_result>. The parent's completion detection only reads the parent's own text parts, so this does not end the parent run, but the parent model will see the token in the tool result.

Not in this PR: a "blocked: report and stop" option for genuinely blocked runs (left out on purpose; the eval showed the escape option was the problem), and stripping DONE from task results.

How did you verify your code works?

  • termination.test.ts: the run-mode nudge has two options, no "clarification", says not to end on a question, and still instructs DONE; completionInstruction({ runMode: true }) returns the contract and { runMode: false } returns nothing; a source guard checks that prompt.ts passes only the run-mode flag and that the agent allowlist is gone; the interactive continuation string in compaction.ts is pinned byte-for-byte.
  • compaction-summarizer-integrity.test.ts: a run-mode compaction injects the new nudge with no "clarification"; an interactive compaction (ALTIMATE_RUN_MODE=0) injects the unchanged interactive string, checked byte-for-byte.
  • New loop-level test in prompt.test.ts: a custom reviewer subagent (configured as a subagent, started from a subtask part and run through the task tool) is driven against the mock LLM server. In run mode every request, the child's and the parent's, carries the contract in its system messages; with ALTIMATE_RUN_MODE=0 none does. Neither mode's requests contain "stop and ask for clarification". The test does not force a compaction inside the subagent; the compaction nudge itself is covered at the session level by the integrity test above.
  • The run-mode tests fail against main (checked by reverting the source files) and pass on this branch; the interactive pins pass on both.
  • bun test (packages/opencode): 20,098 pass, 14 fail, 1 error between tests. All are unrelated: the 7 mcp.headers / mcp HttpApi status failures fail identically on main; the 3 --time-budget cases in json-output.test.ts are the known timing-sensitive ones; budget-finalization, session-diff-missing-patch (2), v2 pty HttpApi and the between-tests error (database is locked in session-diff) are load failures that pass when run alone.
  • Typecheck clean; script/check-tracker-leaks.ts and analyze.ts --markers --strict pass.

No real-gateway run; mock server only.

Cost side effects

Small increase per run-mode request for agents other than builder: the completion contract adds one fixed sentence (about 50 tokens) to the system prompt of custom agents and subagents. It is constant for the session, so it is cached with the rest of the prefix. Interactive sessions are unchanged. Runs that would have ended on an unanswerable question should now finish, which is the expected direction for cost per completed task.

Screenshots / recordings

N/A

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR
  • I checked for cost side effects (model calls, queries or cloud resources that grow with user data) and described them, or wrote "none"

🤖 Generated with Claude Code


Summary by cubic

Fixes headless altimate-code run sessions ending on a question nobody can answer, and makes the run-mode completion contract (DONE) apply to every agent instead of just builder.

The post-compaction nudge no longer offers a "stop and ask for clarification" option; it now instructs agents to pick the most reasonable interpretation and continue, or reply DONE. The completion contract is injected for every agent in run mode — custom agents and task subagents included — while interactive sessions still receive nothing. Behavior change to review: in run mode a subagent's final text now ends with a DONE line, which appears inside the <task_result> the parent sees; parent completion detection only reads the parent's own text parts, so a parent run still ends normally.

Written for commit da56132. Summary will update on new commits.

View guided diff Turn on auto-fix

…nd apply the completion contract to every agent in run mode

In headless `altimate-code run` nobody can answer a question, but the
post-compaction nudge offered "stop and ask for clarification" as one of
its three options, so runs ended on a question instead of finishing. And
the run-mode completion contract (end with a trailing `DONE`) was gated on
an allowlist holding only `builder`, so custom agents and task subagents
ran without it.

- `COMPLETION_NUDGE` (sent only when `ALTIMATE_RUN_MODE` is on) now offers
  two options: continue (picking the most reasonable interpretation and
  stating the assumption when unsure) or reply `DONE`. It says no one can
  answer questions during the run.
- `completionInstruction()` returns the contract for every agent in run
  mode; the `COMPLETION_CONTRACT_AGENTS` allowlist and the `agent` input are
  removed. Task subagents are prompted through the same loop, so they get
  it too. Interactive sessions still get nothing.
- The interactive compaction continuation in `compaction.ts` is unchanged
  and is now pinned byte-for-byte by tests.
- Loop-level test: a custom subagent started from a subtask receives the
  contract in run mode and not in interactive mode.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

👋 This PR was automatically closed by our quality checks.

Common reasons:

  • New GitHub account with limited contribution history
  • PR description doesn't meet our guidelines
  • Contribution appears to be AI-generated without meaningful review

If you believe this was a mistake, please open an issue explaining your intended contribution and a maintainer will help you.

@RaghwendraSingh

Copy link
Copy Markdown
Author

Gateway run: no hand-backs after compaction

One headless run on the real gateway (model altimate-backend/altimate-default), on a local branch that stacks this PR with #1631, #1632, #1639 and #1643, against a 240-model dbt project with the same whole-project task used in our earlier runs. 1M-token cap, one leg, hard-stopped at the cap after 10 minutes: 50 requests across the parent session (custom primary agent) and one worker subagent. Measured from the session database.

check earlier runs on main (passes 1–13) this run
post-compaction nudge offers "stop and ask for clarification" yes no — the injected text reads "No one can answer questions during this run, so do not end your turn with a question."
turns ending on a question or a status recap 6 in passes 1–8, 1 in pass 11 0 (parent and worker)
worker ended its task with a question returned to the parent once (pass 6) 0

The run ended only because of the external token cap (the last parent message was mid-work, not a hand-back).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Headless run ends on a question after compaction; custom agents and subagents get no completion contract

1 participant