fix(security-audit): end each orchestrator wait call before the Bash cap - #626
fix(security-audit): end each orchestrator wait call before the Bash cap#626dormouse-bot wants to merge 4 commits into
Conversation
A Bash call that reaches the harness's ten-minute cap is moved to the background rather than returned, so the wait loop handed the orchestrator no fragment list and no DEADLINE — nothing to decide the next step on. Whether it re-issued a third time was then a judgement call: run 34457954349 did, its deadline landed inside that call, and it merged and published both completed domains; run 34581574869 spent one extra call checking the fragments, ended its turn four minutes short of the deadline, and published no report at all. Each call now breaks on its own 540-second sub-cap and prints STILL WAITING or DEADLINE, so every re-issue is driven by the loop's own output, and §4 states that the turn must not end before audit-report.md exists.
application-security has now failed to report inside the 25-minute deadline two nights running — it expired on 2026-09-10, and on 2026-09-11 the domain was still sweeping when the run ended at 21 minutes. Both nights left roughly 13 of the job's 40 minutes unused, so the deadline moves into that headroom and timeout-minutes: 40 stays above it.
Deploying mouseterm with
|
| Latest commit: |
e0a9399
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://17dab339.mouseterm.pages.dev |
| Branch Preview URL: | https://fix-audit-wait-loop-returns.mouseterm.pages.dev |
dormouse-bot
left a comment
There was a problem hiding this comment.
Feedback on work in progress — not a merge verdict.
On the separable second commit: the headroom is fine. On run 34457954349 the steps before the agent step took 21 seconds and the steps after it took 4, so a 32-minute wait puts the job at roughly 33 of its 40 minutes — timeout-minutes: 40 does not need to move with the deadline.
Three inline findings. One point beyond the diff: timeout: 600000 is now load-bearing in a way it was not. Before this change a short timeout only cost extra re-issues; now it costs the printed answer entirely, and the only thing holding it is the orchestrator reading one sentence. BASH_DEFAULT_TIMEOUT_MS: 600000 in the audit step's env: block — beside AUDIT_PAT, which already shows the action forwards step env into the agent's Bash — would make a forgotten timeout harmless rather than fatal, if the harness honors it.
Three corrections from review, plus the env backstop behind them. The `ls` that follows the loop wrote to stderr and exited 2 whenever no fragment existed yet, which is the most common `STILL WAITING` call on a slow night. That was invisible while capped calls were backgrounded and their tail never read; now §2 tells the orchestrator to act on what the call printed, so the call has to end cleanly. `2>/dev/null || echo "no fragments yet"` gives it a tail and exit 0. The `timeout: 600000` sentence still described the pre-sub-cap world, where a short timeout only bought more re-issues. At the harness default of two minutes the call is now backgrounded before the loop's own 540-second break can print, so no answer comes back at all — the run 34581574869 failure the block exists to end. `BASH_DEFAULT_TIMEOUT_MS: 600000` in the audit step's `env:` makes that harmless rather than fatal: the prompt still asks for the per-call `timeout`, but forgetting it no longer costs the printed answer. The AUDIT_PAT entry beside it already shows the action forwards step env to the agent. The spec's `FAIL IF` asserted that every call prints `STILL WAITING` or `DEADLINE`; a call that finds all three fragments never enters the loop body and prints neither, answering with the `ls` listing alone, as §2 already says. Narrowed to the capped call, word-neutral so the 1,750-word budget needs no ratchet.
dormouse-bot
left a comment
There was a problem hiding this comment.
Feedback on work in progress — not a merge verdict. Mark it ready when you want the full review.
Two inline findings, both on material this commit added. The loop itself behaves as the four documented cases claim — I re-ran the committed block against a fake $RUNNER_TEMP with the sub-cap shrunk to 2s and each case printed what §2 says it does.
The orchestrator was told to act on "the loop's own last line", but the `ls` after the loop always runs, so a re-issue call's last line is `no fragments yet` or a partial listing — neither of the two answers. Act on what the call printed instead, and call the two things decisions: the `ls` listing all three fragments is one of the inputs, not a third answer the loop was supposed to print. Also state the true scope of `BASH_DEFAULT_TIMEOUT_MS`: it is a process-wide default, so it raises the cap on the domain subagents' Bash calls too, not just the orchestrator's wait.
The nightly audit has now gone two nights without a verdict, and last night it published no report at all — issue #598. The cause is in the orchestrator's wait loop, not in the deadline it enforces.
A wait call that reaches the harness's ten-minute Bash cap is moved to the background rather than returned. It hands back no fragment listing, no
DEADLINE, and nothing to decide the next step on, so whether the orchestrator re-issued a third time was a judgement call rather than a step in the recipe. Run 34457954349 (2026-09-10) happened to re-issue, its 25-minute deadline fell inside that third call, and it merged and published both completed domains asINCONCLUSIVE. Run 34581574869 (2026-09-11) spent one extra call listing the fragments between re-issues, which shifted the phase enough that a fourth call would have been needed, ended its turn at 21 minutes instead — four minutes short of its own deadline — and the session terminated with §3 never reached.supply-chainandci-and-secretshad both writtenVERDICT: PASSfragments by 09:02; neither reached the issue, which says only_No audit-report.md was produced._.So each call now ends itself at 540 seconds and prints
STILL WAITINGorDEADLINE, and §2 tells the orchestrator to act on that line: re-issue on the first, merge on the second. §4 gains the backstop the lost run needed — the turn must not end whileaudit-report.mddoes not exist. Nothing else about the technique changes: the deadline is still persisted in$RUNNER_TEMP, still accumulates across re-issues, and the loop is still the sameuntilover the three fragment files.Because the orchestrator now acts on what each call printed, the call has to end cleanly: the
lsafter the loop is2>/dev/null || echo "no fragments yet", since on a slow night's first call it otherwise wrote to stderr and exited 2. And the per-calltimeout: 600000is load-bearing in a way it was not — at the harness default of two minutes the call is backgrounded before the 540-second break can print, so no answer comes back at all.BASH_DEFAULT_TIMEOUT_MS: "600000"in the audit step'senv:, besideAUDIT_PAT, makes forgetting it harmless rather than fatal.The second commit is separable and you may not want it:
application-securityhas missed the 25-minute deadline two nights running, while roughly 13 of the job'stimeout-minutes: 40went unused on both, so it moves the deadline into that headroom (32 minutes). It is the knob, not a diagnosis — the domain spent last night's run fanning out to 15 nested subagents, re-runningvitest,deploy-lint-selftestand the other repo lints, and then busy-waiting ~10 minutes on its own children, so 32 minutes may still not be enough for it. What the first commit guarantees regardless is that the two domains that do finish get published.How the loop was verified
The
shblock was extracted verbatim from the committed.github/audit/orchestrator.md, syntax-checked withbash -n, and run against a fake$RUNNER_TEMPwith the two constants shrunk for the test:STILL WAITING, thenno fragments yet, exit 0DEADLINElslists thempnpm lint:specspasses;docs/specs/security-audit.mdcame in 3 words over its 1,750-word budget and was tightened to fit rather than ratcheted. TheFAIL IFcovering the wait loop is scoped to the capped call, since the all-fragments call prints neither answer and §2 already names itslslisting as its own.