From bf50eb44343ee6144c9c4423f2690ad2fe779d4d Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Fri, 11 Sep 2026 20:27:44 +0000 Subject: [PATCH 1/2] test: the unprivileged fingerprint reader needs a reachable harness (#907) The nightly coverage job has been red for two nights and nothing said so. Every suite job was green, the PR gate was green, and the only failure was harness_selftest inside the one job nobody reads: 249 passed, 1 failed. Three checks in 340-the-binary-must-be-built-from failed, with six 'test/lib.sh: Permission denied' lines beside them. The arms there read a fixture tree as a second user, because chmod 000 is invisible to root, and they sourced lib.sh from the checkout. In GitHub Actions the checkout is under /home/runner/work, which postgres and nobody cannot traverse. TWO OF THE ARMS PASSED ANYWAY, which is the defect worth naming. The reader did '. "$PGC_TESTDIR/lib.sh" || exit 1', so a denied source produced empty output -- exactly what an arm asserting 'an unreadable file yields no fingerprint' wants to see. Measured under a tree the reader cannot traverse: everything readable, lib.sh UNREACHABLE -> [] arms PASS on this b.c unreadable, lib.sh reachable -> [] what they test everything readable, lib.sh reachable -> cde49bff94a7 Three changes. A readable copy of lib.sh, portlib.sh and pgc_fingerprint.py is staged beside the fixture, which the reader can always reach; the same three files read from another directory, which is an environment property rather than anything this part asserts. A failed source now answers harness-unreadable, which cannot be mistaken for a hash or for empty, so no arm can pass that way again. And the arms run only when their premises were met, skipping loudly otherwise. Measured, PG18: reader can traverse the tree unfixed 803 0 FAIL fixed 804 0 FAIL tree under a mode-750 parent unfixed 3 FAIL, 6 denied, 2 arms PASS fixed 0 FAIL, 0 denied, arms run reader cannot fingerprint fixed 2 FAIL, 5 honest SKIP The last row is a probe, not a condition anyone meets: python3 was removed from the reader's PATH to prove the skip branch is reachable, since an else that cannot run proves nothing either. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a --- CHANGELOG.md | 46 ++++++++++++ .../340-the-binary-must-be-built-from.sh | 74 +++++++++++++++++-- 2 files changed, 115 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da905eb8..da69a331 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1545,6 +1545,52 @@ true until the next version shipped. four. The per-major table is in #965, which is where the remainder should be read from rather than from either run alone. +- The unprivileged reader in the fingerprint arms needs a harness it can reach, + and a failed source must not be reported as a refused fingerprint (#907). + + The nightly coverage job went red for two nights and nothing said so: every suite + job was green, the PR gate was green, and the only failure was one suite inside the + job nobody reads. `harness_selftest` reported 249 passed, 1 failed. + + Three checks in `340-the-binary-must-be-built-from` failed, with six + `test/lib.sh: Permission denied` lines beside them. The arms there read a fixture + tree as a second user, because `chmod 000` is invisible to root, and they sourced + `lib.sh` from the checkout. In GitHub Actions the checkout lives under + `/home/runner/work`, which `postgres` and `nobody` cannot traverse, so the + unprivileged shell could not load the harness at all. + + **Two of the arms passed anyway, and that is the defect worth naming.** The reader + did `. "$PGC_TESTDIR/lib.sh" || exit 1`, so a denied source produced empty output -- + which is exactly what an arm asserting "an unreadable file yields no fingerprint" + wants to see. Measured under a tree the reader cannot traverse: + + everything readable, lib.sh UNREACHABLE -> [] the arms PASS on this + b.c unreadable, lib.sh reachable -> [] what they mean to test + everything readable, lib.sh reachable -> cde49bff94a7 + + So the premises were the only thing standing between the suite and a clean report + on two arms that proved nothing. + + Three changes, each with its own reason. A readable copy of `lib.sh`, `portlib.sh` + and `pgc_fingerprint.py` is staged beside the fixture, which the reader can always + reach -- the same three files, read from a different directory, which is an + environment property rather than anything the part asserts. A failed source now + answers `harness-unreadable`, a value that cannot be mistaken for a hash or for + empty, so no arm can pass that way again. And the arms run only when the premises + they rest on were met, skipping loudly otherwise, because a guard that cannot run + is not a guard that held. + + Measured, three conditions, PG18: + + tree readable by the reader unfixed 803 checks 0 FAIL fixed 804 checks 0 FAIL + tree under a mode-750 parent unfixed 3 FAIL, 6 denied, fixed 0 FAIL, 0 denied, + 2 arms PASS vacuously arms run + reader cannot fingerprint at all -- 2 FAIL, 5 honest SKIP + + The last row is a probe rather than a condition anyone meets today: `python3` was + removed from the reader's `PATH` to prove the skip branch can be reached, since an + `else` that cannot run proves nothing either. + ## [1.0-alpha3] - 2026-09-02 ### Added diff --git a/test/selftest/340-the-binary-must-be-built-from.sh b/test/selftest/340-the-binary-must-be-built-from.sh index 352d9c5b..49df9815 100644 --- a/test/selftest/340-the-binary-must-be-built-from.sh +++ b/test/selftest/340-the-binary-must-be-built-from.sh @@ -617,6 +617,24 @@ printf 'all:\n\ttrue\n' > "$_fp/tree/Makefile" printf 'x\n' > "$_fp/tree/pgcolumnar.control" chmod -R a+rX "$_fp" +# A READABLE COPY OF THE HARNESS, because the reader may not be able to reach the +# checkout. The arms below source `lib.sh` as a second user, and in GitHub Actions +# the tree lives under `/home/runner/work`, which `postgres` and `nobody` cannot +# traverse. The nightly coverage job went red for two nights on exactly that, with +# six `lib.sh: Permission denied` lines in its log, and the premise checks below +# are what caught it. +# +# Copying is not a weaker test: it is the same three files, and which directory +# they are read from is an environment property rather than anything this part +# asserts. `_pgc_fp_module` resolves `pgc_fingerprint.py` beside `lib.sh` through +# BASH_SOURCE, and `lib.sh` sources `portlib.sh` the same way, so all three must +# sit together. +_fp_harness="$_fp/harness" +mkdir -p "$_fp_harness" +cp "$PGC_TESTDIR/lib.sh" "$PGC_TESTDIR/portlib.sh" "$PGC_TESTDIR/pgc_fingerprint.py" \ + "$_fp_harness/" 2>/dev/null +chmod -R a+rX "$_fp" + _fp_user="" if [ "$(id -u)" -ne 0 ]; then _fp_user="-" # already unprivileged; read in this shell @@ -627,26 +645,72 @@ else fi # _fp_as reads as the unprivileged user, or in this shell when already one. -_fp_as() { # _fp_as EXPR -> stdout +# +# IT ANSWERS `harness-unreadable` RATHER THAN NOTHING when the source fails, and +# that is the whole defect this replaces. The first version did `|| exit 1`, so a +# reader that could not source `lib.sh` produced empty output -- which is exactly +# what the arms below want. Measured under a tree the reader cannot traverse: +# +# everything readable, lib.sh UNREACHABLE -> [] <- the ARMS PASS on this +# b.c unreadable, lib.sh reachable -> [] <- what they mean to test +# everything readable, lib.sh reachable -> [cde49bff94a7] +# +# Two arms reported PASS having proved nothing. A sentinel cannot be mistaken for +# a hash or for empty, so no arm can ever pass that way again. +_fp_as() { # _fp_as EXPR -> stdout, or `harness-unreadable` + local _fp_cmd + # `echo`, not `printf`: a `\n` inside this concatenation is correct, and the + # static checker cannot see through the concatenation to know that (SC1012). A + # warning that has to be explained is worse than a form needing no explanation. + # And this comment does not open with that checker's name, because a comment + # that does is parsed as a DIRECTIVE -- the first wording here turned into + # SC1072/SC1073 parse errors on the comment itself. + _fp_cmd=". \"$_fp_harness/lib.sh\" 2>/dev/null || { echo harness-unreadable; exit 0; }; $1" if [ "$_fp_user" = "-" ]; then - bash -c ". \"$PGC_TESTDIR/lib.sh\" || exit 1; $1" + bash -c "$_fp_cmd" else - runuser -u "$_fp_user" -- bash -c ". \"$PGC_TESTDIR/lib.sh\" || exit 1; $1" + runuser -u "$_fp_user" -- bash -c "$_fp_cmd" fi } if [ -z "$_fp_user" ]; then check_skip "the unreadable-source refusal" "SKIP no non-root user to read as; root ignores chmod 000" "no non-root user to read as" else + # Premise nought: the reader can source the copy at all. Stated separately + # from the fingerprint premise so a reachability failure and a fingerprint + # failure cannot be read as each other. + check "premise: the unprivileged reader can source the staged harness" \ + "$(_fp_as "printf sourced")" "sourced" + _fp_base="$(_fp_as "pgc_source_fingerprint \"$_fp/tree\"")" check "premise: the tree fingerprints to something when it is readable" \ - "$([ -n "$_fp_base" ] && echo yes || echo empty)" "yes" + "$([ -n "$_fp_base" ] && [ "$_fp_base" != harness-unreadable ] && echo yes || echo "${_fp_base:-empty}")" "yes" # Premise for the mechanism: the unprivileged reader must agree with this # shell while nothing is denied, or the arms below measure the user switch. check "premise: the unprivileged read agrees while everything is readable" \ "$_fp_base" "$(pgc_source_fingerprint "$_fp/tree")" +fi +# THE ARMS run only when the premises above were met. A guard that cannot run is +# not a guard that held, so the alternative to running them is SKIPPING them +# loudly -- never running them against a reader that is broken for a reason they +# do not describe. The premises stay FAILED in that case, so the suite is still +# red and still says why. +if [ -z "$_fp_user" ]; then + : # already skipped above +elif [ -z "$_fp_base" ] || [ "$_fp_base" = harness-unreadable ] \ + || [ "$_fp_base" != "$(pgc_source_fingerprint "$_fp/tree")" ]; then + for _fp_n in "an unreadable b.c yields no fingerprint, not a wrong one" \ + "an unreadable c.c yields no fingerprint, not a wrong one" \ + "control: and the tree fingerprints again once it is readable" \ + "so the verdict is unknown -- UNVERIFIED -- and never stale" \ + "control: a readable run still reads fresh"; do + check_skip "$_fp_n" \ + "SKIP $_fp_n (the unprivileged reader could not fingerprint a readable tree)" \ + "the unprivileged reader could not fingerprint a readable tree" + done +else # THE ARM. One unreadable file, and the answer must be EMPTY, not a hash. for _fp_n in b.c c.c; do chmod 000 "$_fp/tree/src/$_fp_n" @@ -670,7 +734,7 @@ else "$(pgc_freshness_verdict "$_fp_base" \ "$(_fp_as "pgc_source_fingerprint \"$_fp/tree\"")")" "fresh" fi -unset _fp_user _fp_u _fp_got _fp_n +unset _fp_user _fp_u _fp_got _fp_n _fp_base _fp_harness _fp_cmd unset -f _fp_as # --------------------------------------------------------------------------- From a5201e1f4e378b7af7c649369806fe043489e7cd Mon Sep 17 00:00:00 2001 From: OffgridwithJD Date: Fri, 11 Sep 2026 20:46:33 +0000 Subject: [PATCH 2/2] test: regenerate the ledger for the new premise check (#907) The gate refused the previous head, correctly and by design: not in the ledger: harness_selftest 340-the-binary-must-be-built-from premise: the unprivileged reader can source the staged harness 1 check(s) the ledger has never seen PG18 has a check the ledger has never seen, which is not a pass harness_selftest is one of the two suites the ledger covers, so a check added to it is refused until the ledger has seen it. Every suite in that run reported PASS or SKIP; the job verdict was the gate, not the change. One row, and the census DERIVED from the updated ledger rather than written as 846+1: arithmetic across merges has been right by accident before and is not evidence. rows 846 -> 847 checks_never_observed_red 846 -> 847 (derived: awk '$4=="never"' | wc -l) suites_not_covered 250 unchanged; this adds no suite Order matters and @linuxhikerpm corrected mine: rebase FIRST, then regenerate. Regenerating before the rebase derives the census against the wrong tree. I had merged on the pre-rebase tree, discarded it, rebased onto 972820e5, re-ran the suite on the rebased tree for a fresh log, and merged that. The log was guarded before it went near the merge -- rc=0, a numeric checks-run total above a floor, zero FAILs, and the suite's own PASSED verdict -- because a red run merged into the ledger records a red observation as the baseline. Gate on the rebased tree, with the registered list from run_all_versions.sh --list-suites rather than a grep of the SUITES array: ledger census: rows=847 | never observed red=847, ever red=0, new this run=0 census stated 847, ledger holds 847: they agree ledger coverage: registered=252 | covered=2, not covered=250, ceiling=250 rc=0 Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01Uf6UoeBRZYLQZa4KxNiw8a --- test/check_ledger.tsv | 1 + test/check_ledger_budget.txt | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/test/check_ledger.tsv b/test/check_ledger.tsv index e45b869b..30752640 100644 --- a/test/check_ledger.tsv +++ b/test/check_ledger.tsv @@ -404,6 +404,7 @@ harness_selftest 340-the-binary-must-be-built-from premise: the spelling fixture harness_selftest 340-the-binary-must-be-built-from premise: the sweep finds the call sites it is meant to police never - harness_selftest 340-the-binary-must-be-built-from premise: the tree fingerprints to something when it is readable never - harness_selftest 340-the-binary-must-be-built-from premise: the unprivileged read agrees while everything is readable never - +harness_selftest 340-the-binary-must-be-built-from premise: the unprivileged reader can source the staged harness never - harness_selftest 340-the-binary-must-be-built-from premise: the writer wrote a stamp at all never - harness_selftest 340-the-binary-must-be-built-from renaming a source file moves the fingerprint too never - harness_selftest 340-the-binary-must-be-built-from so its verdict is unknown rather than a spurious stale never - diff --git a/test/check_ledger_budget.txt b/test/check_ledger_budget.txt index bf938dac..e923ee03 100644 --- a/test/check_ledger_budget.txt +++ b/test/check_ledger_budget.txt @@ -34,4 +34,4 @@ suites_not_covered 250 # Without that it is a hand-maintained count that drifts, which is the failure # this repository has spent a day proving. It is not a ceiling; it is a # measurement that must be true. -checks_never_observed_red 846 +checks_never_observed_red 847