Skip to content

test: the accounting breakdown says which LINE it means, because twelve suites were called unseedable and ten of them record - #1003

Merged
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:fix/the-own-mechanism-label-says-what-it-measures
Sep 12, 2026
Merged

test: the accounting breakdown says which LINE it means, because twelve suites were called unseedable and ten of them record#1003
jdatcmd merged 1 commit into
commandprompt:mainfrom
OffgridwithJD:fix/the-own-mechanism-label-says-what-it-measures

Conversation

@OffgridwithJD

Copy link
Copy Markdown
Collaborator

A label in the runner's output was read as saying something it does not say, by two different readers hours apart, and a planning number was derived from the misreading. This says what it means instead.

The runner printed:

N via lib.sh's accounting; by their own mechanism: audit bench_guards concurrency ...

by their own mechanism was taken to mean emits no RESULT records, which would make those twelve suites impossible to seed into the mutation ledger — and a bound on how far suites_not_covered can fall came out of that reading.

It is false, and here is the measurement

Every one of the twelve, run, counting RESULT records:

suite records suite records
audit 31 phase4 38
bench_guards 0 phase5 36
concurrency 7 phase6 43
docs_style 0 smoke 9
phase2 42 unique_conc 31
phase3 32 update_conc 25

Ten of the twelve emit records. The set is the suites whose log lacks lib.sh's accounting: line and carries their own checks run: instead — a statement about the accounting line and nothing else.

Only bench_guards and docs_style emit none, and the tree already knows why. pgc_log_shows_any_accounting's own comment says it:

bench_guards and docs_style keep private counters and print their own checks run: N; they never source lib.sh

That is the discriminator — sourcing lib.sh, which is what reaches pgc_record. Defining a private check is orthogonal: audit does it and records 31 anyway.

So the bound on how far the ceiling can fall before anything needs converting is two suites, not twelve, with roughly 294 records sitting in the other ten.

Why a wording change is worth a PR

The label is computed from a real set difference and is correct about that difference. What it lacked was a statement of which difference, and the consequence was not cosmetic: two readers independently reached the same wrong conclusion and one of them put it in a PR body as a constraint on the project's next step.

Both of my own attempts to check it statically were also wrong, in opposite directions — a grep for "contains a check call" returned one suite (a false positive: harness_selftest sources parts that do the calling), and a predictor built on "defines its own recorder" agreed with the claim on 0 of 12. What settled it was two runtime spot-checks that could not both fit the story: docs_style 0 against smoke 9.

What is not done, stated rather than omitted

The disclaimer in the new label is not separately pinned. Pinning it needs either a second check — a ledger row — or renaming the existing arm to match a wider assertion, and a rename orphans that arm's existing row. main has no tool to remove an orphan until #983 lands. So the arm keeps its name, greps the reworded label, and its comment records the sequence: #993 lands the prune, then this arm can be renamed and widened in the change that regenerates the ledger anyway.

Choosing the ledger-free option here is deliberate — there are already two PRs in flight touching check_ledger.tsv, and a third would buy a conflict for a wording fix.

No ledger change, measured not assumed

this branch   rc=0  checks run 867  records 867  FAIL 0
main          rc=0  checks run 867                FAIL 0
-> the check set does not move, so no regeneration

The arm that follows the reword passes, and its grep discriminates: it matches the new label once and main's old label zero times, so it is a pin rather than a tautology.

shellcheck -S error -s bash over both changed files: 0. bash -n clean.

Context: #752, and the framing this corrects appears in #1002.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

…t#752)

The runner printed:

    N via lib.sh's accounting; by their own mechanism: audit bench_guards ...

and "by their own mechanism" was read as "emits no RESULT records" -- by two
different readers, hours apart -- and a bound on how far suites_not_covered can fall
was derived from that reading: twelve suites called impossible to seed.

IT IS FALSE. Measured by running all twelve and counting RESULT records:

    audit 31   smoke 9   phase2 42   phase3 32   phase4 38   phase5 36
    phase6 43  concurrency 7   unique_conc 31   update_conc 25
    bench_guards 0   docs_style 0

TEN OF THE TWELVE EMIT RECORDS. The set is the suites whose log lacks lib.sh's
`accounting:` line and carries their own `checks run:` instead -- about the accounting
LINE and nothing else. Only bench_guards and docs_style emit none, for the reason
pgc_log_shows_any_accounting's own comment already gives: those two never source
lib.sh. Defining a private `check` is orthogonal; audit does it and records 31.

So the bound is two suites, not twelve, with ~294 records in the other ten.

The label now names the line and disclaims the records in the same breath, with the
measurement beside it so the next reader does not re-derive it.

WHAT IS NOT DONE, stated rather than omitted: the disclaimer is not separately
pinned. Pinning it needs either a new check -- a ledger row -- or renaming the
existing arm, which would ORPHAN its row, and main has no tool to remove an orphan
until commandprompt#983 lands. So the arm keeps its name and greps the reworded label, the
sequencing is in its comment, and widening it belongs to the change that regenerates
the ledger anyway.

NO LEDGER CHANGE: no check is added, removed or renamed, so the check set does not
move. Verified by running harness_selftest on this tree and on main and comparing the
counts rather than assuming it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Fixing the label rather than either reader's habits is the right call, and "two readers independently reached the same wrong conclusion from one line" is the correct standard for deciding that.

I reviewed this from the file list first, then verified each claim.

Verified

nothing else in the tree greps the old label only run_all_versions.sh:1477 and 390:699, and this PR changes both
the reworded arm matches grep -c "printed lib.sh's accounting line"1; the old string → 0
no ledger movement no check name is added, removed or renamed — the arm keeps its name and only its value expression changes, so the key set is untouched

That first row was the one I most wanted, because a label reworded in one place and grepped in another is precisely the silent-red shape that put #993 and #998 on a collision course.

Your "forty lines" is about 438, and the correction strengthens your point

pgc_log_shows_any_accounting's comment is at 1036-1040; the label that misled us is at 1477.

bench_guards and docs_style keep private counters and print their own checks run: N; they never source lib.sh

So the real discriminator was not sitting forty lines away where a reader might have stumbled on it. It was four hundred and thirty-eight lines away, in a different function, attached to a different concept. That makes the case for fixing the label stronger, not weaker: the correct information existed and was not reachable from the place the wrong inference was made. Worth correcting in the body, since it is a number in prose and this tree polices those.

One finding, non-blocking: the disclaimer is over-broad for two of its own members

these printed their own `checks run:` instead (a different accounting LINE, not a missing RESULT record)

Strictly this is right — it says what membership in the set indicates. But bench_guards and docs_style are printed inside that very list, and they do lack RESULT records. A reader scanning the line now takes away "none of these are missing RESULT records", which is false for two of the names in front of them.

That is the same shape as the original defect, one turn on: a parenthetical that is accurate about the set and misleading about two of its elements. And the reason to care is the reason you wrote this PR — this is a line that has already been misread twice, so the bar is what a scanning reader concludes, not what the sentence can be defended as meaning.

Cheapest fix that keeps the disclaimer true of everything it sits beside:

(a different accounting LINE; ten of these emit RESULT records normally, bench_guards
 and docs_style emit none because they never source lib.sh)

Longer, but it ends the question instead of narrowing it, and it puts the real discriminator at the point of misreading rather than 438 lines away. Not blocking — the current wording is a large improvement on what it replaces, and I would rather this land than argue about a parenthetical.

The orphan constraint is the interesting part

renaming this one ORPHANS its existing row, and main currently has no tool to remove an orphan

Stating that in the arm's comment rather than leaving the omission unexplained is right, and noting it is the first time #983's gap has constrained a design decision rather than been the thing under repair is the observation I would keep. It is also a concrete argument for the merge order I have already stated to jd and will not restate here.

Verdict

Approved. No ledger movement, no cross-file coupling left behind, the arm follows the string it guards, and the reasoning for what is deliberately not pinned is written where the next reader meets it.

@jdatcmd
jdatcmd merged commit 181db5e into commandprompt:main Sep 12, 2026
13 checks passed
jdatcmd added a commit to linuxhikerpm/pgcolumnar that referenced this pull request Sep 12, 2026
…tion (commandprompt#752)

commandprompt#1002's changelog entry said "the census is `grep -c` over the ledger". It is not.
The gate compares the budget against the count of rows whose LAST-RED is `never`:

    never = sum(1 for v in rows.values() if v[0] == NEVER)
    ...
    elif stated != never:

    awk -F'\t' '$4=="never"' test/check_ledger.tsv | wc -l

A plain row count agrees with that only while nothing has ever been seen red. On
this tree today: 1155 rows, 1155 never, 0 ever red -- so the NUMBER I committed was
right and the METHOD was wrong, in an entry whose whole subject is derive rather
than carry. Reported by @OffgridwithJD.

DRIVEN, because the two derivations are indistinguishable on today's tree and the
claim is about a tree that does not exist yet. Three rows, one of them observed red:

    grep -c                     3
    awk $4=="never"             2

    budget 3, from grep -c      gate rc=1
      "the budget states checks_never_observed_red 3, the ledger holds 2:
       these describe the same file and disagree"
    budget 2, from awk          gate rc=0

AND THAT REFINES THE REPORTED CONSEQUENCE. The report was that the wrong method
would agree with a budget written the same wrong way and pass. It does not: the
gate compares the budget against the ledger, not against whatever command produced
it, so a wrongly-derived budget FAILS CLOSED. The harm is not an undetected lie, it
is an unexplainable red on a correct tree the first time a check is attacked --
which is still worth removing, because a guard that reddens for a reason nobody can
find is a guard somebody turns off.

The derivation now sits in check_ledger_budget.txt beside the number it governs,
which is where someone looking for it will be, rather than in a changelog entry
they would have to know to search. Same principle as commandprompt#1003: a summary worth reading
is worth defining where it prints.

The wrong sentence is left visible in the entry rather than silently replaced,
because it was an instruction and somebody may already have followed it.

Verified: budget still parses ({'suites_not_covered': 249,
'checks_never_observed_red': 1155}), documented command returns 1155, gate rc=0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbyGSaU93XYQr8aH4NrUiw
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants