Repository navigation
Description
Problem
amend --accept-protected-failures refuses a valid, by-design protected failure with protected regression observation 1 does not match the approved check whenever the plan lists a costly regression check (for example go test ./...) before the cheaper one that failed.
Seen on #403, Molecule mol-403-5e0af62179c7cb56a75b9b2b, v0.4.5:
- The regression checks in plan order were
go-test-all(go test -timeout 20m ./...),go-vet,gofmt,protocol-and-cache-regression(a targetedgo test -run …), the hygiene scripts, then the baselines. - The protected view ran
go-vet,gofmtandprotocol-and-cache-regression, which failed by design onTestAdapterInvocationChangesRequireProtocolBumpafter a protocol bump. It never rango-test-all; that check is recorded undernotRunas "an earlier required check failed". failingProtectedCheck(lab/internal/molecule/accept_protected_failures.go) comparesranwith the plan's regression checks position by position, from the start.regression[0]isgo-test-allandran[0]isgo-vet, so it refuses.
The comments in executionOrder (lab/internal/verification/order.go) and in runViewDeltas say the protected view keeps plan order and that --accept-protected-failures reads its run as a plan-order prefix. In practice the protected run skipped the costlier earlier check, so its run isn't a prefix.
The workaround was to reorder the plan's regression checks by cost with amend --parent … --plan … (revision 3), then run checks-only, then accept. That works, but it needs a hand-edited plan.
Expected
A by-design protected failure can be accepted whatever order the plan lists its regression checks in, without weakening the rule that the run must have stopped on that check for its own exit.
Find out which side is wrong:
- if the protected view is meant to keep plan order, it should run
go-test-allfirst (and then stop there); - if cost order is intended in the protected view too,
failingProtectedCheckshould compareranwith the regression checks in the same execution order (executionOrder), and treat checks recorded asnotRunafter the stopping check as not part of the prefix.
Fix it on one side, and correct the comments that describe the other.
Acceptance
- A regression test reproduces the Probe the Codex sandbox before coding and declare environment skips (#43, slice 3) #403 order (a full-suite check listed first, a cheaper targeted check that fails by design) and the protected delta is then accepted.
- The existing refusals still hold: a run that didn't fail on a regression check, a setup failure, a check that failed before the last one recorded without accepted deltas, or a command that doesn't match the plan.
- No change to which checks a candidate view runs or how it orders them.
Activity
Keep covered checks out of the protected prefix after an earlier stop (…
DanielNetzer commentedon Nov 8, 2026
Fixed by #410: a covered check left unrun by an earlier stop now keeps CoveredBy, so --accept-protected-failures leaves it out of the plan-order prefix. The protected view keeps plan order; the plan-reorder workaround is no longer needed.