From dc21a26744f27c649e89756cf502ea3f31cfbafc Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Fri, 11 Sep 2026 21:39:15 +0000 Subject: [PATCH 1/2] test: assert the matrix controller records the installed library (#961) run_all_versions.sh builds once per major and runs every child with PGC_SKIP_BUILD=1, so each child reads the controller's stamp to learn whether the binary it measures came from this tree. If the controller's stamp write loses its third argument, every child reaches source-only and PASSES -- because source-only is also the state of every stamp written before #959, so it cannot be a failure. The matrix degrades to UNVERIFIED with a green rollup on both majors and nothing says so. jdatcmd asked for this while approving #960: an arm catching a green failure is worth more than most arms. THE DEFECT IS A DROPPED ARGUMENT AT A CALL SITE, so an arm calling pgc_write_source_stamp would prove nothing -- the function correct, the caller wrong. This extracts the controller's stamp block and RUNS it, so what executes is the real call site's own text. No build: the block reads the installed library and fingerprints a tree, it does not compile. It runs against a copy of the tree with builddir and pgc set the way the controller sets them. 24MB at 31ms a copy, against minutes for a build. Proven by mutating run_all_versions.sh itself, dropping the argument in a way that still parses: the controller's stamp carries BOTH fields got [1] want [2] a child reaches verified, not source-only got [source-only] want [verified] every caller records the installed library's digest got [2] want [3] The part carries its own control: it removes the argument from the extracted block and asserts the same driver reaches source-only, so the arms cannot pass for a reason about the driver rather than the controller. The static sweep covers pgc_setup and devloop.sh, where driving either costs a build. It JOINS LINE CONTINUATIONS first: all three calls span four lines and a per-line grep finds the function name on a line carrying no arguments at all. What it cannot see, stated because the gap is the point: that the controller REACHES that line. It asserts what the line does, not that flow arrives there. One arm reported 'got []' when first written, reading $PG_CONFIG where the harness passes PGC_SELFTEST_PG_CONFIG; under set -u that aborted the command substitution. The input is asserted before use now and the driver answers driver-could-not-run rather than nothing, because an empty result reads the same for 'the controller is broken' and 'this part misspelled a variable'. harness_selftest 815 checks 0 FAIL, docs_style 9/9, shellcheck clean. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a --- CHANGELOG.md | 52 +++++++ ...0-the-controller-must-record-the-binary.sh | 140 ++++++++++++++++++ test/selftest/parts.manifest | 1 + 3 files changed, 193 insertions(+) create mode 100644 test/selftest/460-the-controller-must-record-the-binary.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 9adad267..7beedfa9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1664,6 +1664,58 @@ true until the next version shipped. rate of the fixture that has no branches. It is anchored, and a control with a genuinely zero-covered file proves the anchor did not defeat the assertion. +- The matrix controller is asserted to record the installed library, because the + failure if it stops is a green matrix-wide downgrade (#961). + + `run_all_versions.sh` builds once per major and runs every child suite with + `PGC_SKIP_BUILD=1`, so each child checks the controller's stamp to learn whether + the binary it measures came from this tree. If the controller's stamp write loses + its third argument, every child reaches `source-only` and **passes** -- because + `source-only` is also the state of every stamp written before #959, so it cannot be + a failure. The whole matrix degrades to UNVERIFIED with a green rollup on both + majors, and nothing says so. + + `@jdatcmd` asked for this arm while approving #960, on the ground that an arm + catching a green failure is worth more than most arms. + + **The defect is a dropped argument at a call site**, so an arm that calls + `pgc_write_source_stamp` itself would prove nothing: the function would be correct + and the caller wrong. The new part extracts the controller's stamp block and runs + it, so what executes is the real call site's own text. + + It costs no build. The block reads the installed library and fingerprints a tree; + it does not compile. So it runs against a copy of the tree with `builddir` and + `pgc` set the way the controller sets them -- 24 MB at 31 ms a copy, against + minutes for a per-major build. + + Measured by mutating `run_all_versions.sh` itself, dropping the third argument in a + way that still parses: + + the controller's stamp carries BOTH fields got [1] want [2] + a child reaches verified, not source-only got [source-only] want [verified] + every caller records the installed library's digest got [2] want [3] + + Three arms, on an edit a careless hand would make. The part also carries its own + control: it removes the argument from the extracted block and asserts the same + driver reaches `source-only`, so the two arms above cannot pass for a reason that + is about the driver rather than the controller. + + The static sweep covers the other two call sites, `pgc_setup` and `devloop.sh`, + where driving either would cost a build. It joins line continuations first, because + all three calls are written across four lines and a per-line grep finds the + function name on a line carrying no arguments at all. + + What it cannot see, which is worth stating: that the controller REACHES that line. + The `make install` guard above it could start failing closed and this part would + not notice. It asserts what the line does, not that control flow arrives there. + + One of the arms reported `got []` when first written, because it read `$PG_CONFIG` + and no selftest part sets that -- the harness passes `PGC_SELFTEST_PG_CONFIG`. + Under `set -u` that aborted the command substitution the driver runs in. The input + is now asserted before use and the driver answers `driver-could-not-run` rather + than nothing, because an empty result reports the same emptiness for "the + controller is broken" and "this part misspelled a variable". + ## [1.0-alpha3] - 2026-09-02 ### Added diff --git a/test/selftest/460-the-controller-must-record-the-binary.sh b/test/selftest/460-the-controller-must-record-the-binary.sh new file mode 100644 index 00000000..832d5ea2 --- /dev/null +++ b/test/selftest/460-the-controller-must-record-the-binary.sh @@ -0,0 +1,140 @@ +# ---- the controller must record the BINARY, not only the source ------------- +# +# #961. #960 made a suite able to refuse a binary it has not examined, by recording +# the installed library's digest in the source stamp. Nothing asserted that the +# MATRIX CONTROLLER records it, and the failure if it stops is green. +# +# `run_all_versions.sh` builds once per major and runs every child with +# `PGC_SKIP_BUILD=1`. If the controller's stamp write loses its third argument, +# every child reaches `source-only` and PASSES -- because `source-only` is also the +# state of every stamp written before #959, so it cannot be a failure. The whole +# matrix degrades to UNVERIFIED with a green rollup on both majors. +# +# THE DEFECT IS A DROPPED ARGUMENT AT A CALL SITE, so an arm that calls +# `pgc_write_source_stamp` itself proves nothing: the function would be correct and +# the caller wrong. The block must RUN. @jdatcmd asked for this arm while approving +# #960, on the ground that the failure it catches is green. +# +# WHY IT COSTS NO BUILD. The controller's stamp block reads the installed library +# and fingerprints a tree; it does not compile. So the block is extracted from the +# runner and evaluated against a COPY of this tree, with `builddir` and `pgc` set +# the way the controller sets them. 24M at 31ms a copy, against minutes for a real +# per-major build, and what runs is the real call site's own text. +# +# WHAT THIS CANNOT SEE, stated because the gap is the point of #961: that the +# controller REACHES that line. The `make install` guard above it could start +# failing closed and this part would not notice. It asserts what the line does, not +# that control flow arrives there. + +_c961_rav="$TESTDIR/run_all_versions.sh" +_c961_tab="$(printf '\t')" + +check "premise: the matrix controller is present and parses" \ + "$([ -r "$_c961_rav" ] && bash -n "$_c961_rav" 2>/dev/null && echo yes || echo no)" "yes" + +# THE PG_CONFIG THIS SUITE WAS GIVEN, by the name the harness uses. The first +# version of this part read `$PG_CONFIG`, which no part sets: under `set -u` that +# aborted the command substitution the driver runs in, and both arms below reported +# `got []`. An empty result is not a value, and an arm whose inputs are unchecked +# reports the same emptiness for "the controller is broken" and "I misspelled a +# variable". So the input is asserted before it is used. +_c961_pgc="${PGC_SELFTEST_PG_CONFIG:-}" +check "premise: this part was given an executable pg_config to read the prefix from" \ + "$([ -n "$_c961_pgc" ] && [ -x "$_c961_pgc" ] && echo yes || echo no)" "yes" + +# The subshell block, from `if (` to `); then`. A LITERAL TAB, not `\t`: GNU grep's +# BRE does not read `\t` as a tab, and the first version of this premise counted 0 +# and would have let the arms below run against an empty block. +_c961_block="$(awk -v t="$_c961_tab" ' + $0 == t "if (" {f=1} f {print} f && $0 == t "); then" {exit}' "$_c961_rav")" + +check "premise: the controller's stamp block was extracted exactly once" \ + "$(printf '%s\n' "$_c961_block" | grep -c "^${_c961_tab}if ($")" "1" +check "premise: and it holds exactly one stamp write" \ + "$(printf '%s\n' "$_c961_block" | grep -c 'pgc_write_source_stamp')" "1" + +# _c961_drive BLOCK -> " " +# +# The block names `builddir` and `pgc`, which is why setting those is enough to run +# it. Evaluated with the leading tab stripped so the heredoc-free `eval` parses. +_c961_drive() { + local block="$1" tmp builddir pgc stamp src_v bin_v claim lines + tmp="$(mktemp -d "${TMPDIR:-/tmp}/pgc-c961.XXXXXX")" + mkdir -p "$tmp/copy" + cp -a "$TESTDIR/../." "$tmp/copy/" 2>/dev/null + builddir="$tmp/copy" + pgc="$_c961_pgc" + if [ -z "$pgc" ] || [ ! -d "$builddir/test" ]; then + # A SENTINEL, NOT EMPTY. `driver-could-not-run` cannot be mistaken for a + # verdict, so no arm below can pass or fail for a reason that is about this + # part rather than about the controller. + printf 'driver-could-not-run 0\n' + rm -rf "$tmp" + return 0 + fi + eval "$(printf '%s\n' "$block" | sed "s/^${_c961_tab}//") + : +else + echo 'the controller block reported failure' >&2 +fi" >/dev/null 2>&1 + stamp="$(pgc_source_stamp_path "$builddir" "$pgc")" + if [ ! -f "$stamp" ]; then + printf 'no-stamp 0\n' + rm -rf "$tmp" + return 0 + fi + lines="$(grep -c . "$stamp")" + src_v="$(pgc_freshness_verdict "$(pgc_read_source_stamp "$stamp")" \ + "$(pgc_source_fingerprint "$builddir")")" + bin_v="$(pgc_binary_identity_verdict "$(pgc_read_installed_stamp "$stamp")" \ + "$(pgc_installed_library_digest "$pgc")")" + claim="$(pgc_freshness_claim "$src_v" "$bin_v")" + printf '%s %s\n' "$claim" "$lines" + rm -rf "$tmp" +} + +_c961_real="$(_c961_drive "$_c961_block")" + +check "the controller's stamp carries BOTH fields (#961)" \ + "$(printf '%s' "$_c961_real" | awk '{print $2}')" "2" +check "so a child suite under PGC_SKIP_BUILD reaches verified, not source-only (#961)" \ + "$(printf '%s' "$_c961_real" | awk '{print $1}')" "verified" + +# THE CONTROL, because an arm asserting `verified` passes for any reason that makes +# the claim verified, including a driver that never ran the block. Drop the third +# argument -- the careless edit #961 describes -- and the SAME driver must reach +# `source-only`. Without this the two arms above could both be vacuous. +_c961_mut="$(printf '%s\n' "$_c961_block" \ + | sed '/pgc_installed_library_digest/d' \ + | sed "s|\"\$(pgc_source_fingerprint \"\$builddir\")\" \\\\|\"\$(pgc_source_fingerprint \"\$builddir\")\"|")" +check "premise: the mutation removed the installed-digest argument" \ + "$(printf '%s\n' "$_c961_mut" | grep -c 'pgc_installed_library_digest')" "0" + +_c961_degraded="$(_c961_drive "$_c961_mut")" +check "control: without that argument the same driver degrades to source-only (#961)" \ + "$(printf '%s' "$_c961_degraded" | awk '{print $1}')" "source-only" +check "control: and the stamp carries one field rather than two" \ + "$(printf '%s' "$_c961_degraded" | awk '{print $2}')" "1" + +# ---- and the same argument at every other call site ------------------------- +# +# A static sweep, which is weaker than driving and is the right shape here: the +# other two callers are `pgc_setup` (every suite's own build) and `devloop.sh` (a +# developer tool), and driving either costs a build. CONTINUATIONS ARE JOINED +# first, because every one of these calls is written across four lines and a +# per-line grep would find the function name on a line carrying no arguments at +# all -- a guard that passes because it read half a statement. +_c961_joined="$( + for f in "$TESTDIR/lib.sh" "$TESTDIR/devloop.sh" "$_c961_rav"; do + sed -e :a -e '/\\$/N; s/\\\n//; ta' "$f" + done +)" +_c961_calls="$(printf '%s\n' "$_c961_joined" | grep -c 'pgc_write_source_stamp .*pgc_source_stamp_path')" +check "premise: all three stamp call sites were found with their arguments joined" \ + "$_c961_calls" "3" +check "every caller records the installed library's digest (#961)" \ + "$(printf '%s\n' "$_c961_joined" | grep 'pgc_write_source_stamp .*pgc_source_stamp_path' \ + | grep -c 'pgc_installed_library_digest')" "3" + +unset _c961_rav _c961_tab _c961_pgc _c961_block _c961_real _c961_mut _c961_degraded _c961_joined _c961_calls +unset -f _c961_drive diff --git a/test/selftest/parts.manifest b/test/selftest/parts.manifest index 3d87c0a5..26090671 100644 --- a/test/selftest/parts.manifest +++ b/test/selftest/parts.manifest @@ -42,3 +42,4 @@ 430-the-self-test-must-not-report.sh 440-a-count-grep-never-produced.sh 450-a-red-nightly-must-be-findable.sh +460-the-controller-must-record-the-binary.sh From 36f58dd35ebf45299cb62d0f67e9507af4e739a8 Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Fri, 11 Sep 2026 22:17:23 +0000 Subject: [PATCH 2/2] test: regenerate the ledger for the eleven controller-stamp checks (#961) THIRD pass for this PR. #975 moved the census while it was open, then #978 moved it again, so each earlier derivation was stale before anyone could review it. That is not a defect in the PRs -- it is what a committed census costs when more than one change is in flight, and it is the measurement #432 needed. Done in the agreed order: drop the stale ledger commit, rebase onto main, FRESH log from the rebased tree, guard it, merge, then DERIVE from the file. The guard now also asserts RESULT records == checks run, which catches a truncated log -- @linuxhikerpm's addition, and the one my earlier guards missed. No expected census value was written down before the count ran. Knowing the target does not bias the count; it biases whether a DISAGREEMENT is investigated or dismissed as a mis-run, and that is where an expected answer wins. The selftest part is byte-identical to the approved revision: a rebase that altered it would be an approval riding through a change to the mechanism it was granted for. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a --- test/check_ledger.tsv | 11 +++++++++++ test/check_ledger_budget.txt | 2 +- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index b885d8b6..19433997 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -843,6 +843,17 @@ harness_selftest 450-a-red-nightly-must-be-findable the reporter runs on every o harness_selftest 450-a-red-nightly-must-be-findable the reporter waits on EVERY other nightly job (#973) never - harness_selftest 450-a-red-nightly-must-be-findable the title carries no run number, so every red lands on one issue never - harness_selftest 450-a-red-nightly-must-be-findable the verdict comes from the whole needs context, not a second list never - +harness_selftest 460-the-controller-must-record-the-binary control: and the stamp carries one field rather than two never - +harness_selftest 460-the-controller-must-record-the-binary control: without that argument the same driver degrades to source-only (#961) never - +harness_selftest 460-the-controller-must-record-the-binary every caller records the installed library's digest (#961) never - +harness_selftest 460-the-controller-must-record-the-binary premise: all three stamp call sites were found with their arguments joined never - +harness_selftest 460-the-controller-must-record-the-binary premise: and it holds exactly one stamp write never - +harness_selftest 460-the-controller-must-record-the-binary premise: the controller's stamp block was extracted exactly once never - +harness_selftest 460-the-controller-must-record-the-binary premise: the matrix controller is present and parses never - +harness_selftest 460-the-controller-must-record-the-binary premise: the mutation removed the installed-digest argument never - +harness_selftest 460-the-controller-must-record-the-binary premise: this part was given an executable pg_config to read the prefix from never - +harness_selftest 460-the-controller-must-record-the-binary so a child suite under PGC_SKIP_BUILD reaches verified, not source-only (#961) never - +harness_selftest 460-the-controller-must-record-the-binary the controller's stamp carries BOTH fields (#961) never - native_join_runtime_filter native_join_runtime_filter 3-table answer equals filter-off never - native_join_runtime_filter native_join_runtime_filter 3-table join order matches filter-off never - native_join_runtime_filter native_join_runtime_filter 3-table plan has coordinator never - diff --git a/test/check_ledger_budget.txt b/test/check_ledger_budget.txt index 63961bdf..e251bd20 100644 --- a/test/check_ledger_budget.txt +++ b/test/check_ledger_budget.txt @@ -34,4 +34,4 @@ suites_not_covered 250 # Without that it is a hand-maintained count that drifts, which is the failure # this repository has spent a day proving. It is not a ceiling; it is a # measurement that must be true. -checks_never_observed_red 889 +checks_never_observed_red 900