Skip to content

feat: cut the cost of scode-build-goal overseer runs - #527

Merged
scode merged 1 commit into
mainfrom
build-goal-overseer-cost
Nov 15, 2026
Merged

scode merged 1 commit into
mainfrom
build-goal-overseer-cost

Conversation

@scode

@scode scode commented Nov 15, 2026

Copy link
Copy Markdown
Owner

In scode-build-goal's overseer mode, a strong overseer session hands a goal to a cheap executor session, typically
Haiku, which it starts through farhelm (the tool that launches and supervises agent sessions) and steers by reading the
executor's transcript. The first real run, a ten-PR goal, cost about $42 at API list prices, and most of that paid for
re-reading context rather than for work. The executor never compacted: its context grew to 919K of its 1M-token window,
and because Haiku bills any call whose prompt exceeds 100K tokens at five times its base rate, re-reading cached context
made up about 90% of its cost. The opus overseer had been started in the session that planned the goal, so each of its
looks also carried about 300K tokens of unrelated history.

A haiku executor is now launched with Claude Code's auto-compact window set to 125K tokens. On Claude Code 2.1.325,
compaction happens about 31K short of the window (measured at 100K and 130K windows), so it lands at roughly 94K,
leaving some margin under Haiku's price step. The alternative was a fresh executor session per PR, but that ties a
session's lifetime to the size of a PR, drops all context between PRs, and needs a farhelm launch for each one, which
cannot yet happen fully unattended. Other models have no such price step and keep the default window. The setting
travels as a farhelm launch wrapper, and farhelm holds a wrapper for the user's approval unless the host allows YOLO
launches without asking, so an unattended haiku run now needs that host permission even when the overseer is not running with
permissions bypassed. The skill's questions and closing instructions also tell the user to start the overseer in a
fresh session, or to compact the session first.

Each PR's review saw only that PR, so in that run nothing ever checked a PR together with the later PR that consumes
its API. The overseer's final report now asks whether to run one review over the whole stack. It does not run one by
default, since whether a stack warrants it is the user's call.

In scode-build-goal's overseer mode, a strong overseer session hands a goal to a cheap executor session, typically
Haiku, which it starts through farhelm (the tool that launches and supervises agent sessions) and steers by reading the
executor's transcript. The first real run, a ten-PR goal, cost about $42 at API list prices, and most of that paid for
re-reading context rather than for work. The executor never compacted: its context grew to 919K of its 1M-token window,
and because Haiku bills any call whose prompt exceeds 100K tokens at five times its base rate, re-reading cached context
made up about 90% of its cost. The opus overseer had been started in the session that planned the goal, so each of its
looks also carried about 300K tokens of unrelated history.

A haiku executor is now launched with Claude Code's auto-compact window set to 125K tokens. On Claude Code 2.1.325,
compaction happens about 31K short of the window (measured at 100K and 130K windows), so it lands at roughly 94K,
leaving some margin under Haiku's price step. The alternative was a fresh executor session per PR, but that ties a
session's lifetime to the size of a PR, drops all context between PRs, and needs a farhelm launch for each one, which
cannot yet happen fully unattended. Other models have no such price step and keep the default window. The setting
travels as a farhelm launch wrapper, and farhelm holds a wrapper for the user's approval unless the host allows YOLO
launches without asking, so an unattended haiku run now needs that host permission even when the overseer is not running with
permissions bypassed. The skill's questions and closing instructions also tell the user to start the overseer in a
fresh session, or to compact the session first.

Each PR's review saw only that PR, so in that run nothing ever checked a PR together with the later PR that consumes
its API. The overseer's final report now asks whether to run one review over the whole stack. It does not run one by
default, since whether a stack warrants it is the user's call.
@scode
scode marked this pull request as ready for review November 15, 2026 01:36
@scode
scode merged commit 59c14b7 into main Nov 15, 2026
5 checks passed
@scode
scode deleted the build-goal-overseer-cost branch November 15, 2026 01:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant