Nothing in the gate runs a sanitizer build, so the alignment class is ungated

[jdatcmd]

Found while closing the first #214 finding. The fix landed; the thing that
catches the next one did not exist and still does not.

What happened

The #214 fuzzer was run against an ASAN and UBSAN build of PostgreSQL 18.4 and
reported, at src/columnar_encoding.c:1135:

runtime error: load of misaligned address 0x75064ac5813e for type 'uint32',
which requires 4 byte alignment

Values in a chunk's raw buffer are packed end to end with no alignment padding,
so a four-byte varlena header begins wherever the previous value ended.
VARSIZE_ANY reads it by casting to varattrib_4b and loading a uint32.
Undefined on an unaligned address; works on x86_64; SIGBUS on a
strict-alignment target, which docs/limitations.md says is supported.

The part that matters for this issue: it was not a crafted-input defect. A
plain INSERT of low-cardinality text reports it six times. It had been on the
ordinary write path the whole time. Three call sites were affected
(encode_dict, and two buffer walks in columnar_reader.c).

Every one of the 92 suites passed on all five majors, before and after, on both
sides of the fix. The full matrix is green either way. Nothing in the gate can
see this class of defect, because nothing in the gate is instrumented.

Why the fuzzer is not the answer on its own

test/fuzz_parquet.sh classifies a sanitizer report as a finding and is in the
matrix, so this looks covered. It is not. The check is only live when the suite
runs against a sanitizer build, and the matrix's five majors are all ordinary
assert builds. On those, the sanitizer check passes vacuously, every time. That
is the same shape as a suite that greps a file for a guard that is not there:
green because nothing was looked at.

What to build

A sixth configuration in the matrix, or a second gate that runs beside it: one
PostgreSQL major built with -fsanitize=address,undefined, running a subset of
the suites. It does not need all 92 and it does not need five majors. It needs
the write path, the read path, the encoders, and the import and decode paths,
which is where hand-rolled buffer walks live.

Everything needed to build it is already worked out

The environment fights back in four specific ways, all now solved. They are
recorded here so nobody rediscovers them:

• Build in-tree, not VPATH. /root/postgresql-18.4 has a config.status
  and a built postgres in it, so it was configured in place. A VPATH build
  against such a tree fails with subdirectories that "enter" and "leave" without
  building, then a link that cannot find the objects. Extract a fresh tarball.
• libclang-rt-<N>-dev must match the clang major, or -fsanitize=address
  cannot link at all. gcc works too, via libasan.
• ASAN_OPTIONS=detect_leaks=0. pg_config is itself instrumented and leaks
  by design at exit, so LeakSanitizer fires on every invocation and poisons the
  values PGXS reads. The extension build fails before it starts.
• ASAN_OPTIONS=detect_stack_use_after_return=0. Without it, initdb fails
  with FATAL: stack depth limit exceeded during bootstrap, and raising
  max_stack_depth to 32MB does not help, because it is not really running out
  of stack. Clang now defaults that option on, which relocates locals to a heap
  fake stack, and PostgreSQL's check_stack_depth() is pointer arithmetic
  against stack_base_ptr. The arithmetic becomes meaningless and the check
  misfires.
• UBSAN needs a suppression for function. Every backend start reports
  dynahash.c:1022: call to function string_compare through pointer to incorrect function type. That is a PostgreSQL-wide idiom, not a defect, and unsuppressed
  it buries real reports. A suppressions file works; -fno-sanitize=function
  would too.

The working build is /usr/local/pg18_san in the dev container, and
/root/build-san.sh is the script that produced it.

Todo

• Decide where it runs: a sixth entry in run_all_versions.sh, or a
  separate test/run_sanitizer.sh invoked on its own cadence. A sanitizer
  build is several times slower, and the matrix is currently six minutes,
  which is worth protecting.
• Choose the suite subset, favouring paths that walk buffers by hand:
  native_encoding, native_roundtrip, native_writer, differential,
  parquet_import, native_read_parquet, fuzz_parquet, hardening.
• Script the build so it is reproducible rather than a container artifact,
  with the five gotchas above encoded rather than remembered.
• Bake the required ASAN_OPTIONS and UBSAN_OPTIONS into the runner, so a
  correct run is the default and not something the operator has to know.
• Prove the gate by removal: revert the ColumnarVarSizeAnyUnaligned change
  on a branch and confirm the sanitizer gate goes red and the ordinary matrix
  stays green. Until that is shown, the gate is not known to work.
• Decide whether the sanitizer build blocks a merge or reports. Given it
  found a real defect on its first run, blocking is defensible.

Related

Fix and full account: #214 and the PR that closes it. test/mutate_guard.py
already records that ASAN cannot see overruns inside palloc'd chunks, because
palloc sub-allocates from larger blocks, and that only a USE_VALGRIND build
shows those. So a sanitizer gate is a floor, not a ceiling, and a Valgrind
configuration is the natural follow-on.

[jdatcmd]

@ChronicallyJD this one is unclaimed and it is now the gate for two things rather
than one: the alignment class from #225, and the ASAN pass on the Arrow corpus
that #228 deliberately deferred here. Your argument on #228 is the best case for
it, so I am handing it to you rather than writing it myself: an OOB read does not
always fault, the assert build understates the class, and the defect you found
only faulted every time because it landed 3.6 MB out.

Everything needed is already in the issue body, including the four environment
traps that cost me an evening (in-tree not VPATH, matching libclang-rt,
detect_leaks=0, detect_stack_use_after_return=0) and the UBSAN suppression
for the dynahash function-pointer noise. The working build is /usr/local/pg18_san
in my container; /root/build-san.sh is the script.

The one item I would not skip is the last todo: prove the gate by removal. Revert
ColumnarVarSizeAnyUnaligned on a branch and confirm the sanitizer gate goes red
while the ordinary matrix stays green. Until that is shown, the gate is not known
to work, and this whole issue exists because a green matrix could not see the
defect.

I am on #212 (the quadratic walk in ColumnarBufferedRowByNumber).

[jdatcmd]

Two follow-ups from verifying #230, recorded here so they are not lost now that
the PR is merged. Neither was a blocker; both are about the gate being trustworthy
rather than merely green.

1. The removal instruction is ambiguous, and one reading proves nothing

run_san.sh's header says to revert ColumnarVarSizeAnyUnaligned to the direct
4-byte cast. There are two ways to read that and they do not give the same
result:

• Replace the helper's body with return (uint32) VARSIZE_ANY(p);, keeping
  the static inline wrapper. Gate goes green. A direct low-cardinality-text
  INSERT reports zero UBSAN violations.
• Delete the helper and restore VARSIZE_ANY at the three call sites, which
  is the true pre-Fuzz the Parquet decoder, and fix the first defect it found (#214) #225 form. Gate goes red, 20 of 23, 7 hits on
  columnar_encoding.c:1135.

Same undefined behaviour, same expression, one form detected and the other not:
clang stops emitting the alignment check behind that inline boundary at -O1.
The header should say "delete the helper and restore the three call sites",
because the other reading looks like a successful proof that the gate does not
work.

2. The gate has a limit worth stating

Following from the above: a future regression that reintroduces the same
unaligned read from inside an inline wrapper would not be reported by this
gate. That does not diminish it -- it catches the defect in the form the defect
took -- but "the sanitizer gate is green" should not be read as "there is no
unaligned access", and the file is the right place to say so.

3. A prerequisite check, one line

run_san.sh verifies a build exists at the prefix, not that it has the flags it
needs. Pointed at a build lacking -fno-sanitize=function it reports:

FAIL  smoke  (rc=124, sanitizer_lines=0)

which reads like a hang, not like "wrong build". A grep of that prefix's
Makefile.global for -fno-sanitize=function, next to the existing existence
check, turns a confusing failure into a clear one. This issue exists because a
gate must not pass or fail for reasons unrelated to what it tests, and that
applies to its own inputs.

Leaving this issue open for these three. The gate itself is merged and proven.

[jdatcmd]

Closed by #230 (the gate) and #232 (making it trustworthy).

The gate exists, is proven by removal, states its own limit, and validates its
inputs.

Four things from building it are worth keeping, because each is a way a gate can
be green and mean nothing:

1. It has to be clang. This issue said "gcc works too, via libasan". It does
   not, for this defect class. Measured: a low-cardinality-text INSERT does about
   45,000 unaligned 4-byte reads; a gcc -fsanitize=address,undefined build
   reports zero, because gcc drops the alignment check on the guarded load at
   -O1; clang reports them. A gcc gate would have built, run clean, and proven
   nothing.

2. The removal proof has exactly one correct form. Deleting
   ColumnarVarSizeAnyUnaligned and restoring VARSIZE_ANY at the three call
   sites turns about 20 of 23 suites red. Rewriting the helper's body to the
   same cast, keeping the static inline wrapper, leaves the gate green with
   zero reports -- clang stops emitting the check across that inline boundary.
   Same UB, same expression, one form seen and the other not. run_san.sh now
   names both readings.

3. Therefore the gate has a limit, now written down: a regression that
   reintroduces an unaligned read from inside an inlined wrapper would not be
   reported. Green means the defects in the forms it exercises are absent.

4. A prerequisite can fail the wrong way. Pointed at a build lacking
   -fno-sanitize=function, the runner used to fail as a smoke timeout that
   read like a hang. It now names the missing flags and exits 1. Writing that
   check had the same trap one level down: the sanitizer build's own pg_config
   leaks by design, so without detect_leaks=0 exported first it rejected a
   correct build.

The environment recipe in the issue body stands: build in-tree not VPATH,
matching libclang-rt, detect_leaks=0, detect_stack_use_after_return=0, and
-fno-sanitize=function for the dynahash noise. test/build_san.sh encodes all
of it.

It found nothing new on its first green run, which is the expected outcome for a
gate added after its motivating defect was already fixed. Its value is the next
one.