test: the accounting breakdown says which LINE it means, because twelve suites were called unseedable and ten of them record - #1003
Conversation
…t#752) The runner printed: N via lib.sh's accounting; by their own mechanism: audit bench_guards ... and "by their own mechanism" was read as "emits no RESULT records" -- by two different readers, hours apart -- and a bound on how far suites_not_covered can fall was derived from that reading: twelve suites called impossible to seed. IT IS FALSE. Measured by running all twelve and counting RESULT records: audit 31 smoke 9 phase2 42 phase3 32 phase4 38 phase5 36 phase6 43 concurrency 7 unique_conc 31 update_conc 25 bench_guards 0 docs_style 0 TEN OF THE TWELVE EMIT RECORDS. The set is the suites whose log lacks lib.sh's `accounting:` line and carries their own `checks run:` instead -- about the accounting LINE and nothing else. Only bench_guards and docs_style emit none, for the reason pgc_log_shows_any_accounting's own comment already gives: those two never source lib.sh. Defining a private `check` is orthogonal; audit does it and records 31. So the bound is two suites, not twelve, with ~294 records in the other ten. The label now names the line and disclaims the records in the same breath, with the measurement beside it so the next reader does not re-derive it. WHAT IS NOT DONE, stated rather than omitted: the disclaimer is not separately pinned. Pinning it needs either a new check -- a ledger row -- or renaming the existing arm, which would ORPHAN its row, and main has no tool to remove an orphan until commandprompt#983 lands. So the arm keeps its name and greps the reworded label, the sequencing is in its comment, and widening it belongs to the change that regenerates the ledger anyway. NO LEDGER CHANGE: no check is added, removed or renamed, so the check set does not move. Verified by running harness_selftest on this tree and on main and comparing the counts rather than assuming it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a
jdatcmd
left a comment
There was a problem hiding this comment.
Approving. Fixing the label rather than either reader's habits is the right call, and "two readers independently reached the same wrong conclusion from one line" is the correct standard for deciding that.
I reviewed this from the file list first, then verified each claim.
Verified
| nothing else in the tree greps the old label | only run_all_versions.sh:1477 and 390:699, and this PR changes both |
| the reworded arm matches | grep -c "printed lib.sh's accounting line" → 1; the old string → 0 |
| no ledger movement | no check name is added, removed or renamed — the arm keeps its name and only its value expression changes, so the key set is untouched |
That first row was the one I most wanted, because a label reworded in one place and grepped in another is precisely the silent-red shape that put #993 and #998 on a collision course.
Your "forty lines" is about 438, and the correction strengthens your point
pgc_log_shows_any_accounting's comment is at 1036-1040; the label that misled us is at 1477.
bench_guardsanddocs_stylekeep private counters and print their ownchecks run: N; they never source lib.sh
So the real discriminator was not sitting forty lines away where a reader might have stumbled on it. It was four hundred and thirty-eight lines away, in a different function, attached to a different concept. That makes the case for fixing the label stronger, not weaker: the correct information existed and was not reachable from the place the wrong inference was made. Worth correcting in the body, since it is a number in prose and this tree polices those.
One finding, non-blocking: the disclaimer is over-broad for two of its own members
these printed their own `checks run:` instead (a different accounting LINE, not a missing RESULT record)
Strictly this is right — it says what membership in the set indicates. But bench_guards and docs_style are printed inside that very list, and they do lack RESULT records. A reader scanning the line now takes away "none of these are missing RESULT records", which is false for two of the names in front of them.
That is the same shape as the original defect, one turn on: a parenthetical that is accurate about the set and misleading about two of its elements. And the reason to care is the reason you wrote this PR — this is a line that has already been misread twice, so the bar is what a scanning reader concludes, not what the sentence can be defended as meaning.
Cheapest fix that keeps the disclaimer true of everything it sits beside:
(a different accounting LINE; ten of these emit RESULT records normally, bench_guards
and docs_style emit none because they never source lib.sh)
Longer, but it ends the question instead of narrowing it, and it puts the real discriminator at the point of misreading rather than 438 lines away. Not blocking — the current wording is a large improvement on what it replaces, and I would rather this land than argue about a parenthetical.
The orphan constraint is the interesting part
renaming this one ORPHANS its existing row, and
maincurrently has no tool to remove an orphan
Stating that in the arm's comment rather than leaving the omission unexplained is right, and noting it is the first time #983's gap has constrained a design decision rather than been the thing under repair is the observation I would keep. It is also a concrete argument for the merge order I have already stated to jd and will not restate here.
Verdict
Approved. No ledger movement, no cross-file coupling left behind, the arm follows the string it guards, and the reasoning for what is deliberately not pinned is written where the next reader meets it.
…tion (commandprompt#752) commandprompt#1002's changelog entry said "the census is `grep -c` over the ledger". It is not. The gate compares the budget against the count of rows whose LAST-RED is `never`: never = sum(1 for v in rows.values() if v[0] == NEVER) ... elif stated != never: awk -F'\t' '$4=="never"' test/check_ledger.tsv | wc -l A plain row count agrees with that only while nothing has ever been seen red. On this tree today: 1155 rows, 1155 never, 0 ever red -- so the NUMBER I committed was right and the METHOD was wrong, in an entry whose whole subject is derive rather than carry. Reported by @OffgridwithJD. DRIVEN, because the two derivations are indistinguishable on today's tree and the claim is about a tree that does not exist yet. Three rows, one of them observed red: grep -c 3 awk $4=="never" 2 budget 3, from grep -c gate rc=1 "the budget states checks_never_observed_red 3, the ledger holds 2: these describe the same file and disagree" budget 2, from awk gate rc=0 AND THAT REFINES THE REPORTED CONSEQUENCE. The report was that the wrong method would agree with a budget written the same wrong way and pass. It does not: the gate compares the budget against the ledger, not against whatever command produced it, so a wrongly-derived budget FAILS CLOSED. The harm is not an undetected lie, it is an unexplainable red on a correct tree the first time a check is attacked -- which is still worth removing, because a guard that reddens for a reason nobody can find is a guard somebody turns off. The derivation now sits in check_ledger_budget.txt beside the number it governs, which is where someone looking for it will be, rather than in a changelog entry they would have to know to search. Same principle as commandprompt#1003: a summary worth reading is worth defining where it prints. The wrong sentence is left visible in the entry rather than silently replaced, because it was an instruction and somebody may already have followed it. Verified: budget still parses ({'suites_not_covered': 249, 'checks_never_observed_red': 1155}), documented command returns 1155, gate rc=0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
A label in the runner's output was read as saying something it does not say, by two different readers hours apart, and a planning number was derived from the misreading. This says what it means instead.
The runner printed:
by their own mechanismwas taken to mean emits no RESULT records, which would make those twelve suites impossible to seed into the mutation ledger — and a bound on how farsuites_not_coveredcan fall came out of that reading.It is false, and here is the measurement
Every one of the twelve, run, counting
RESULTrecords:auditphase4bench_guardsphase5concurrencyphase6docs_stylesmokephase2unique_concphase3update_concTen of the twelve emit records. The set is the suites whose log lacks lib.sh's
accounting:line and carries their ownchecks run:instead — a statement about the accounting line and nothing else.Only
bench_guardsanddocs_styleemit none, and the tree already knows why.pgc_log_shows_any_accounting's own comment says it:That is the discriminator — sourcing
lib.sh, which is what reachespgc_record. Defining a privatecheckis orthogonal:auditdoes it and records 31 anyway.So the bound on how far the ceiling can fall before anything needs converting is two suites, not twelve, with roughly 294 records sitting in the other ten.
Why a wording change is worth a PR
The label is computed from a real set difference and is correct about that difference. What it lacked was a statement of which difference, and the consequence was not cosmetic: two readers independently reached the same wrong conclusion and one of them put it in a PR body as a constraint on the project's next step.
Both of my own attempts to check it statically were also wrong, in opposite directions — a grep for "contains a
checkcall" returned one suite (a false positive:harness_selftestsources parts that do the calling), and a predictor built on "defines its own recorder" agreed with the claim on 0 of 12. What settled it was two runtime spot-checks that could not both fit the story:docs_style0 againstsmoke9.What is not done, stated rather than omitted
The disclaimer in the new label is not separately pinned. Pinning it needs either a second check — a ledger row — or renaming the existing arm to match a wider assertion, and a rename orphans that arm's existing row.
mainhas no tool to remove an orphan until #983 lands. So the arm keeps its name, greps the reworded label, and its comment records the sequence: #993 lands the prune, then this arm can be renamed and widened in the change that regenerates the ledger anyway.Choosing the ledger-free option here is deliberate — there are already two PRs in flight touching
check_ledger.tsv, and a third would buy a conflict for a wording fix.No ledger change, measured not assumed
The arm that follows the reword passes, and its grep discriminates: it matches the new label once and
main's old label zero times, so it is a pin rather than a tautology.shellcheck -S error -s bashover both changed files: 0.bash -nclean.Context: #752, and the framing this corrects appears in #1002.
🤖 Generated with Claude Code
https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a