Skip to content

test: the ten suites that recorded nothing now record their checks (#965, 10 of 10) #969

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdatcmd merged 4 commits into from
Sep 11, 2026
Merged

test: the ten suites that recorded nothing now record their checks (#965, 10 of 10) #969

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
626373b
test: smoke.sh records its checks, so the ledger can see them (#965)
Sep 11, 2026
3bd66aa
test: say that checks run: is not read yet (#969 review)
Sep 11, 2026
66507a9
test: the other nine suites record their checks too (#965, 10 of 10)
Sep 11, 2026
19ddb4a
test: the remainder is fifty-eight on PG18, not fifty-seven (#965)
Sep 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1445,6 +1445,53 @@ true until the next version shipped.
The refusal on a changed source had never been exercised by anyone before this
change, only read. It is now driven end to end, along with the three other states.

- The ten suites that recorded nothing now record their checks (#965).

Ten registered suites print their own `PASS <name>: <value>` lines and their own
verdict and emit no machine-readable records at all. Counted across the ten, 293
checks were invisible to every mechanism built on that vocabulary: the ledger, the
census, the count of checks never observed red, the red-observation record, and the
duplicate-name detection. They pass, and nothing that reads records can see them.

This converts the first of the ten. Its nine checks now emit a record each and a
total, measured on PG18: nine records where there were none, and `checks run: 9`
where there was no total.

The human output is unchanged, byte for byte. `pgc_record` takes the display whole,
so each line still reads `PASS count(*): 100000` with the measured value rather
than a label. That value is the comparison, not a message, which is why the local
helper records through `pgc_record` rather than delegating to `lib.sh`'s own
`check` -- the latter composes its own display and would drop the value.

The suite's own verdict line stays, and so does its exit logic. That line is
load-bearing until a reconciliation replaces it: `smoke.sh` runs under
`set -euo pipefail`, so a failing command aborts it, and the verdict is what
distinguishes a suite that finished from one that stopped. Removing it in the same
change would make the suite report less than it did before.

This converts nine more of the ten, the same way, after the shape was reviewed on
the first. Measured on PG18: 227 records where there were none, and every suite's
human output byte-for-byte unchanged.

It also fixes a call that was failing silently. `unique_conc.sh` calls `check_skip`
for the case where the citext extension is absent, and that function lives in
`lib.sh`, which the suite did not source -- so the baseline log carries
`line 392: check_skip: command not found`, the suite continued under
`set -uo pipefail`, and the case was reported nowhere at all. It now emits a SKIP
record with its reason.

Fifty-eight checks across four of the suites are still not recorded -- fifty-eight
on PG18 and fifty-seven on PG16, because one of them sits behind a version gate --
since those suites have further check-like helpers of their own, `eq_on_off` in
`phase6` alone accounting for thirty-nine, each printing its own display. Those need a
second pass rather than the same substitution, and the count in #965 should be read
as the number of checks rather than the number of helpers.

It does not add the suite to the ledger. Making a suite coverable and covering it
are separate decisions, and the second one is blocked on a measurement: check NAMES
differ between majors in at least one suite, the ledger has no major dimension, and
a gate seeded from one major would refuse runs on another.

## [1.0-alpha3] - 2026-09-02

### Added
Expand Down
31 changes: 28 additions & 3 deletions test/audit.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,10 @@ trap 'rc=$?; if [ "$AUDIT_REACHED_END" = 0 ]; then
echo "The checks after the failing command did not run.";
fi' EXIT

. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh"
# lib.sh for the check vocabulary (#965). It sources portlib.sh itself
# (lib.sh:110), so this is a superset of what was here, and its top level is
# assignments and function definitions only, so sourcing it starts nothing.
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

PG_CONFIG="${1:-/usr/local/pg17/bin/pg_config}"
BINDIR="$("$PG_CONFIG" --bindir)"
Expand Down Expand Up @@ -119,12 +122,32 @@ PSQL="psql -p $PORT -d audit -qAtX -v ON_ERROR_STOP=1"
q() { run_pg "$PSQL -c \"$1\""; }

fail=0
# RECORDS RATHER THAN ONLY PRINTING (#965). This suite emitted human PASS lines
# and no RESULT records, so every mechanism built on the record vocabulary -- the
# ledger, the census, checks_never_observed_red, the red-observation record,
# duplicate-name detection -- was blind to all of them. The ledger did not merely
# return nothing on this log, it REFUSED it: "no RESULT records, so there is
# nothing to reconcile".
#
# `pgc_record` takes the DISPLAY whole, so the human output below is byte-for-byte
# what it was. NOT lib.sh's own `check`: that composes its own display and would
# drop the `: $got` suffix, which is the measured value rather than a label.
#
# `fail` is still set, so this suite's exit logic is untouched. Its verdict line
# stays for the same reason: under `set -euo pipefail` a failing command aborts the
# suite, and the verdict is what distinguishes finished from stopped.
#
# The `checks run:` line is READ BY NOTHING YET. The matrix gates reconciliation on
# the ACCOUNTING line via `pgc_log_shows_accounting`, and this suite emits none
# because it emits no accounting line. The line is still correct and wanted --
# it is the total the records reconcile against -- so the remaining step is a gate
# flip rather than new work (@jdatcmd, #969 review).
check() {
local name="$1" got="$2" want="$3"
if [ "$got" = "$want" ]; then
echo "PASS $name: $got"
pgc_record PASS "$name" "PASS $name: $got"
else
echo "FAIL $name: got [$got] want [$want]"
pgc_record FAIL "$name" "FAIL $name: got [$got] want [$want]"
fail=1
fi
}
Expand Down Expand Up @@ -389,8 +412,10 @@ q "DROP TABLE ss_dup;" >/dev/null
echo
AUDIT_REACHED_END=1
if [ "$fail" = "0" ]; then
echo "checks run: $PGC_CHECKS"
echo "AUDIT TEST PASSED"
else
echo "checks run: $PGC_CHECKS"
echo "AUDIT TEST FAILED"
echo "---- server log tail ----"
run_pg "tail -30 '$LOGFILE'" || true
Expand Down
31 changes: 28 additions & 3 deletions test/concurrency.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,10 @@

# portlib.sh alone, not lib.sh: this suite carries its own harness, and the port
# band is needed before any of it runs. Sourcing portlib twice is harmless.
. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh"
# lib.sh for the check vocabulary (#965). It sources portlib.sh itself
# (lib.sh:110), so this is a superset of what was here, and its top level is
# assignments and function definitions only, so sourcing it starts nothing.
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

set -uo pipefail

Expand Down Expand Up @@ -168,12 +171,32 @@ SPSQL="psql -h '$WORKDIR' -p $PORT -d conc -qAtX"
ctl_q() { run_pg "$PSQL -c \"$1\""; }

fail=0
# RECORDS RATHER THAN ONLY PRINTING (#965). This suite emitted human PASS lines
# and no RESULT records, so every mechanism built on the record vocabulary -- the
# ledger, the census, checks_never_observed_red, the red-observation record,
# duplicate-name detection -- was blind to all of them. The ledger did not merely
# return nothing on this log, it REFUSED it: "no RESULT records, so there is
# nothing to reconcile".
#
# `pgc_record` takes the DISPLAY whole, so the human output below is byte-for-byte
# what it was. NOT lib.sh's own `check`: that composes its own display and would
# drop the `: $got` suffix, which is the measured value rather than a label.
#
# `fail` is still set, so this suite's exit logic is untouched. Its verdict line
# stays for the same reason: under `set -euo pipefail` a failing command aborts the
# suite, and the verdict is what distinguishes finished from stopped.
#
# The `checks run:` line is READ BY NOTHING YET. The matrix gates reconciliation on
# the ACCOUNTING line via `pgc_log_shows_accounting`, and this suite emits none
# because it emits no accounting line. The line is still correct and wanted --
# it is the total the records reconcile against -- so the remaining step is a gate
# flip rather than new work (@jdatcmd, #969 review).
check() {
local name="$1" got="$2" want="$3"
if [ "$got" = "$want" ]; then
echo "PASS $name: $got"
pgc_record PASS "$name" "PASS $name: $got"
else
echo "FAIL $name: got [$got] want [$want]"
pgc_record FAIL "$name" "FAIL $name: got [$got] want [$want]"
fail=1
fi
}
Expand Down Expand Up @@ -369,8 +392,10 @@ send s2 "\\q"

echo
if [ "$fail" = 0 ]; then
echo "checks run: $PGC_CHECKS"
echo "CONCURRENCY TEST PASSED"
else
echo "checks run: $PGC_CHECKS"
echo "CONCURRENCY TEST FAILED"
fi
exit "$fail"
31 changes: 28 additions & 3 deletions test/phase2.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,10 @@

set -euo pipefail

. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh"
# lib.sh for the check vocabulary (#965). It sources portlib.sh itself
# (lib.sh:110), so this is a superset of what was here, and its top level is
# assignments and function definitions only, so sourcing it starts nothing.
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

PG_CONFIG="${1:-/usr/local/pg17/bin/pg_config}"
BINDIR="$("$PG_CONFIG" --bindir)"
Expand Down Expand Up @@ -68,12 +71,32 @@ q() { run_pg "$PSQL -c \"$1\""; }
qq() { run_pg "$PSQL -c \"$1\"" | tr '\n' ',' ; }

fail=0
# RECORDS RATHER THAN ONLY PRINTING (#965). This suite emitted human PASS lines
# and no RESULT records, so every mechanism built on the record vocabulary -- the
# ledger, the census, checks_never_observed_red, the red-observation record,
# duplicate-name detection -- was blind to all of them. The ledger did not merely
# return nothing on this log, it REFUSED it: "no RESULT records, so there is
# nothing to reconcile".
#
# `pgc_record` takes the DISPLAY whole, so the human output below is byte-for-byte
# what it was. NOT lib.sh's own `check`: that composes its own display and would
# drop the `: $got` suffix, which is the measured value rather than a label.
#
# `fail` is still set, so this suite's exit logic is untouched. Its verdict line
# stays for the same reason: under `set -euo pipefail` a failing command aborts the
# suite, and the verdict is what distinguishes finished from stopped.
#
# The `checks run:` line is READ BY NOTHING YET. The matrix gates reconciliation on
# the ACCOUNTING line via `pgc_log_shows_accounting`, and this suite emits none
# because it emits no accounting line. The line is still correct and wanted --
# it is the total the records reconcile against -- so the remaining step is a gate
# flip rather than new work (@jdatcmd, #969 review).
check() {
local name="$1" got="$2" want="$3"
if [ "$got" = "$want" ]; then
echo "PASS $name: $got"
pgc_record PASS "$name" "PASS $name: $got"
else
echo "FAIL $name: got [$got] want [$want]"
pgc_record FAIL "$name" "FAIL $name: got [$got] want [$want]"
fail=1
fi
}
Expand Down Expand Up @@ -193,8 +216,10 @@ check "orphan chunks" "$(q 'SELECT count(*) FROM pgcolumnar.column_chunk;')"

echo
if [ "$fail" = "0" ]; then
echo "checks run: $PGC_CHECKS"
echo "PHASE 2 TEST PASSED"
else
echo "checks run: $PGC_CHECKS"
echo "PHASE 2 TEST FAILED"
echo "---- server log tail ----"
run_pg "tail -40 '$LOGFILE'" || true
Expand Down
31 changes: 28 additions & 3 deletions test/phase3.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@

set -euo pipefail

. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh"
# lib.sh for the check vocabulary (#965). It sources portlib.sh itself
# (lib.sh:110), so this is a superset of what was here, and its top level is
# assignments and function definitions only, so sourcing it starts nothing.
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

PG_CONFIG="${1:-/usr/local/pg17/bin/pg_config}"
BINDIR="$("$PG_CONFIG" --bindir)"
Expand Down Expand Up @@ -77,12 +80,32 @@ q() { run_pg "$PSQL" <<<"$1"; }
qq() { run_pg "$PSQL" <<<"$1" | tr '\n' ',' ; }

fail=0
# RECORDS RATHER THAN ONLY PRINTING (#965). This suite emitted human PASS lines
# and no RESULT records, so every mechanism built on the record vocabulary -- the
# ledger, the census, checks_never_observed_red, the red-observation record,
# duplicate-name detection -- was blind to all of them. The ledger did not merely
# return nothing on this log, it REFUSED it: "no RESULT records, so there is
# nothing to reconcile".
#
# `pgc_record` takes the DISPLAY whole, so the human output below is byte-for-byte
# what it was. NOT lib.sh's own `check`: that composes its own display and would
# drop the `: $got` suffix, which is the measured value rather than a label.
#
# `fail` is still set, so this suite's exit logic is untouched. Its verdict line
# stays for the same reason: under `set -euo pipefail` a failing command aborts the
# suite, and the verdict is what distinguishes finished from stopped.
#
# The `checks run:` line is READ BY NOTHING YET. The matrix gates reconciliation on
# the ACCOUNTING line via `pgc_log_shows_accounting`, and this suite emits none
# because it emits no accounting line. The line is still correct and wanted --
# it is the total the records reconcile against -- so the remaining step is a gate
# flip rather than new work (@jdatcmd, #969 review).
check() {
local name="$1" got="$2" want="$3"
if [ "$got" = "$want" ]; then
echo "PASS $name: $got"
pgc_record PASS "$name" "PASS $name: $got"
else
echo "FAIL $name: got [$got] want [$want]"
pgc_record FAIL "$name" "FAIL $name: got [$got] want [$want]"
fail=1
fi
}
Expand Down Expand Up @@ -217,8 +240,10 @@ check "row group cleaned" "$(q 'SELECT count(*) FROM pgcolumnar.row_group;')" "0

echo
if [ "$fail" = "0" ]; then
echo "checks run: $PGC_CHECKS"
echo "PHASE 3 TEST PASSED"
else
echo "checks run: $PGC_CHECKS"
echo "PHASE 3 TEST FAILED"
echo "---- server log tail ----"
run_pg "tail -40 '$LOGFILE'" || true
Expand Down
31 changes: 28 additions & 3 deletions test/phase4.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,10 @@

set -euo pipefail

. "$(dirname "${BASH_SOURCE[0]}")/portlib.sh"
# lib.sh for the check vocabulary (#965). It sources portlib.sh itself
# (lib.sh:110), so this is a superset of what was here, and its top level is
# assignments and function definitions only, so sourcing it starts nothing.
. "$(dirname "${BASH_SOURCE[0]}")/lib.sh"

PG_CONFIG="${1:-/usr/local/pg17/bin/pg_config}"
BINDIR="$("$PG_CONFIG" --bindir)"
Expand Down Expand Up @@ -74,12 +77,32 @@ q() { run_pg "$PSQL -c \"$1\""; }
qq() { run_pg "$PSQL -c \"$1\"" | tr '\n' ',' ; }

fail=0
# RECORDS RATHER THAN ONLY PRINTING (#965). This suite emitted human PASS lines
# and no RESULT records, so every mechanism built on the record vocabulary -- the
# ledger, the census, checks_never_observed_red, the red-observation record,
# duplicate-name detection -- was blind to all of them. The ledger did not merely
# return nothing on this log, it REFUSED it: "no RESULT records, so there is
# nothing to reconcile".
#
# `pgc_record` takes the DISPLAY whole, so the human output below is byte-for-byte
# what it was. NOT lib.sh's own `check`: that composes its own display and would
# drop the `: $got` suffix, which is the measured value rather than a label.
#
# `fail` is still set, so this suite's exit logic is untouched. Its verdict line
# stays for the same reason: under `set -euo pipefail` a failing command aborts the
# suite, and the verdict is what distinguishes finished from stopped.
#
# The `checks run:` line is READ BY NOTHING YET. The matrix gates reconciliation on
# the ACCOUNTING line via `pgc_log_shows_accounting`, and this suite emits none
# because it emits no accounting line. The line is still correct and wanted --
# it is the total the records reconcile against -- so the remaining step is a gate
# flip rather than new work (@jdatcmd, #969 review).
check() {
local name="$1" got="$2" want="$3"
if [ "$got" = "$want" ]; then
echo "PASS $name: $got"
pgc_record PASS "$name" "PASS $name: $got"
else
echo "FAIL $name: got [$got] want [$want]"
pgc_record FAIL "$name" "FAIL $name: got [$got] want [$want]"
fail=1
fi
}
Expand Down Expand Up @@ -244,8 +267,10 @@ assert_plan_seq

echo
if [ "$fail" = "0" ]; then
echo "checks run: $PGC_CHECKS"
echo "PHASE 4 TEST PASSED"
else
echo "checks run: $PGC_CHECKS"
echo "PHASE 4 TEST FAILED"
echo "---- server log tail ----"
run_pg "tail -60 '$LOGFILE'" || true
Expand Down
Loading
Loading