Alpha gate: fuzz the hand-rolled Parquet and Arrow parsers

[jdatcmd]

Alpha gate 1 of 4, from the pre-alpha review note. Ranked first there as the
highest-leverage item. @ChronicallyJD for review of the plan below before any
code is written.

What is true today

There is no byte-level fuzzing of any parser in this tree. grep -rniE 'LLVMFuzzer|afl-|libfuzzer|-fsanitize=fuzzer' over the whole repository returns
nothing.

test/fuzz.sh is not a parser fuzzer despite the name. It is a seeded
randomized differential suite at the SQL level: it varies row count, null
density, chunk and stripe limits, codec and level, loads a heap mirror with the
same data, and compares result-set hashes. It fuzzes data shapes, not byte
streams, and it never presents malformed input to a decoder.

Coverage of malformed input today is entirely example-based, one hand-written
case per known defect:

suite	what it feeds
test/native_parquet_hardening.sh	out-of-range physical values, wrong column-chunk count
test/native_parquet_stack.sh	the two #210 recursion vectors
test/hardening.sh	native on-disk bytes
test/corruption.sh	native catalog fields

The surface those cases cover a fraction of:

file	lines	contents
src/columnar_parquet_reader.c	4,098	Parquet footer, schema walk, page decode, FLBA, FDW
src/columnar_arrow.c	1,899	Arrow IPC, FlatBuffers
src/columnar_thrift.c	-	hand-written Thrift parser

#210 was found by reading code and grepping for an absent guard, not by a
machine. Before #211 landed, grep check_stack_depth src/ returned zero across
the entire tree. That is the evidence that this surface has not been searched
systematically: a coverage-guided fuzzer finds unbounded recursion in minutes,
and this one survived until someone read the file.

Scope correction from the review note

The note assumed these entry points are reachable by any user. They are not.
Every file-touching entry point is gated on superuser() in C. See #216 for the enumeration. That lowers the severity from remote attack
surface to "a superuser reads a file they did not produce", which is still the
normal data-lake case and still worth closing, but it is not an unauthenticated
surface and should not be described as one.

Plan

Two harnesses, in this order.

Stage A, file mutation through a live backend. Build a seed corpus of valid
Parquet and Arrow files from the existing export suites, mutate them (bit flips,
truncation, length-field and count-field substitution, structural nesting), and
feed each through pgcolumnar.parquet_schema, pgcolumnar.read_parquet,
pgcolumnar.import_parquet and the Arrow import path. The pass condition is
that the backend either succeeds or raises an ERROR, and never dies. Any signal
death, assertion failure, or sanitizer report is a finding, and the input is
saved to a corpus directory named by seed.

This stage is first because it needs no decoupling of the parsers from the
backend. They use palloc, ereport and memory contexts throughout, so a
standalone harness is real work; driving a real backend is not. #210 would have
been caught by this stage.

Stage B, coverage-guided in-process fuzzing. A libFuzzer or AFL++ harness
linking the parser translation units against a minimal memory-context and error
environment, for depth and execs per second that stage A cannot reach. Scope
this only after stage A stops finding things, and treat the decoupling cost as
the reason it is second rather than first.

Both stages need a build the existing matrix does not have. /usr/local carries
pgNN (assert), pgNN_nc (non-assert), pg18_uring and pg20. There is no
ASAN or UBSAN build. Memory-safety bugs in a decoder are exactly what a plain
assert build misses, so an instrumented build is part of this work, not an
optional extra.

Todo

• Build an ASAN and UBSAN enabled PostgreSQL (18.4 to start) in the dev
  container as pg18_san, and confirm the extension builds and the existing
  Parquet suites pass under it. Any pre-existing sanitizer report found here
  is a finding in its own right and gets its own issue.
• Add test/parquet_corpus.sh or equivalent to emit a seed corpus of valid
  files covering every physical type, encoding, codec, nesting depth and
  multi-file or partitioned layout the reader claims to support.
• Write the stage A mutator and driver as test/fuzz_parquet.sh, seeded and
  reproducible with PGC_SEED in the same style as test/fuzz.sh, so a
  failure reproduces exactly.
• Extend the same driver to the Arrow and FlatBuffers path.
• Run a soak, not a single pass. Record wall-clock hours, executions, and
  distinct crashes. A single green pass is not evidence.
• File one issue per distinct crash found, with the minimized input attached.
• Add the suite to the SUITES array in test/run_all_versions.sh at a
  bounded iteration count. A suite not in that array is never gated, which is
  how the multi-file Parquet path went ungated between Phase G: read a directory or glob of Parquet files as one relation #111 and Phase G multifile: skip non-regular-file matches; add the suite to the matrix #113.
• Decide, and write down, whether stage B is worth its decoupling cost given
  what stage A found.

What closes this

Stage A running in the matrix, a soak of stated duration with its findings
filed, and every crash it found either fixed or documented as accepted. Not a
single green run.

The set

Alpha gates from the pre-alpha review note, ranked cheapest-highest-value first:
#214 fuzz the parsers, #215 limitations and release status, #216 untrusted-input
boundary, #217 backend-crash blast radius.

[jdatcmd]

Stage A is merged (#225, 8e7e97c) and it found a real defect on its first
instrumented run.

Done

• test/parquet_corpus.py, test/parquet_mutate.py, test/fuzz_parquet.sh, in
  the matrix at 100 mutants (about ninety seconds).
• ASAN and UBSAN build of PostgreSQL 18.4 exists at /usr/local/pg18_san,
  built by /root/build-san.sh in the dev container.
• Campaigns: 8,000 mutants / 24,000 statements on the assert build with zero
  crashes and zero hangs; 500 mutants under ASAN and UBSAN.
• Defect found and fixed: an unaligned four-byte varlena header read on the
  packed value buffer, at three call sites. Not a crafted-input defect -- an
  ordinary INSERT of low-cardinality text reported it six times. Benign on
  x86_64, SIGBUS on a strict-alignment target, which docs/limitations.md
  promises to support. Same seeds now report zero.
• The fuzzer is proven against a known bug class: with both Crafted Parquet footer crashes the backend (SIGSEGV): unbounded Thrift recursion in ColumnarThriftSkip, no check_stack_depth #210 guards removed
  it finds the crash on all three entry points. The first version did not --
  250 mutants against the guard-removed build found nothing, because byte
  mutation does not produce a deep Thrift chain by chance. Two structural
  mutations fixed that. That is the difference between a fuzzer and a fuzzer with
  a known detection rate.

Still open on this issue

1. The Arrow and FlatBuffers path. The driver only covers the three Parquet
   entry points. columnar_arrow.c is 1,899 lines of hand-rolled decoding with
   the same shape and none of this coverage. import_arrow is the obvious next
   target and the corpus generator already has the layouts.
2. A soak of stated duration, not the single passes done so far. 8,000
   mutants is one afternoon's worth, not evidence of exhaustion.
3. Stage B, coverage-guided in-process fuzzing, still to be decided on the
   evidence. Stage A found something on day one, which argues for more stage A
   breadth before paying the decoupling cost of stage B.

Filed alongside

#224: nothing in the gate runs a sanitizer build, so the sanitizer check in
this very suite passes vacuously on all five majors. Every one of the 92 suites
passed before and after the alignment fix. The class this issue exists to catch
is caught only when someone runs the instrumented build by hand. #224 carries the
whole build recipe, including the four environment traps that cost time here.

Leaving this open for the Arrow half and the soak.