Parser performance experiments (record)

[JanJakes]

Record-keeping branch consolidating the MySQL parser/lexer performance experiments explored while optimizing the pure-PHP parser. The shipped optimizations are in #378 (built on #373/#375/#376); the optional native Rust extension is #381/#423. This branch is the catalog of everything else that was prototyped, measured, or evaluated — most of it previously living only in throwaway local branches or ephemeral sessions.

Each experiment is its own directory (code where it exists + a NOTES.md: idea, how to run, result, verdict) and its own commit. Branch is based directly on trunk and only adds files under experiments/. Catalog + harness: be95d4b.

Numbers are parse-only on the 69,577-query MySQL server corpus, warm tracing JIT, MacBook Pro M4 / PHP 8.5.5 (≈10–15% drift; treat as ratios). Baselines: trunk parser ≈28K QPS, optimized parser (#378) ≈57K, pure-regex recognition ≈98K, validate-only ≈246K. "—" = not measured (proposal/reasoning only).

Draft, closed immediately — kept purely as a record.

Experiment	Result	Verdict	Commit
Whole-grammar PHP compilation	+18–20% no JIT, −6–8% with JIT	Wrong shape for the JIT	c9b364f
Method-size capping + runtime fallback	~0.92×	Always slower than the interpreter	dfcf256
Alternative AST data structures	array −5%, tape +144%	AST build is ~77% of parse time; no easy node-shape win	0b998c1
Pratt parser for the expression cascade	—	Estimated 5–25%; worth a try	4fb6027
LL(2) selectors	—	Estimated 5–15%; high effort	eba6f84
Table-driven LALR(1)	—	No clear win in PHP	e050bd8
pack / unpack for hot-path lookups	~4× slower lookups	Good for bulk decode only	8a159ee
Full PCRE grammar recognizer	~98K QPS, 99.85% recognized	Recognizer, not a parser	c2cb3d9
Regex pre-validate + parser hybrid	~50K vs ~65K	The gate is pure overhead	0213f11
Multi-shape regex → direct AST	~1.2× overall	Works; modest and orthogonal	942d18e
PCRE2 captures for AST extraction	~4.6× JIT collapse	Infeasible in stock PHP	4ad1df5
PCRE2 callouts via FFI	29K–314K by callout density	Works, but FFI rarely available	03257ba
Iterative preg_replace_callback shift-reduce	~2.5× slower	Per-call cost too high	810a499
Bottom-up reduction with binary encoding	~20–30K ceiling	Same wall as shift-reduce	7b1311f
Oniguruma (?@...) capture trees	31-group cap	Too small; unreachable in PHP	38868e0
strtr blind reduction	~2650× slower	Dead end — scans the whole table per call	2af208b
Native tree builders (json_decode/unserialize/DOM)	—	Circular: the transform is the parse	90aa5dc
parle PECL extension	—	Estimated 3–10×; needs PECL	85476a7
Other PHP parser libs (PHP-PEG/Hoa/Phlexy)	—	None beat the current parser	ead2eb2
SQLite as the parser	—	No parse tree; classifier at best	071a8ca
AST cache keyed by parameterized template	~2.4× on repeats	Win on repeats, loss on unique queries	60d6035
Native Rust parser extension	1.33× with JIT, 2.19× without	Only ~1.3× over optimized PHP under JIT	fec4ae6