perf(tableau-2,stim): try to beat Stim on Clifford QEC circuits (stacked on #204) by david-pl · Pull Request #236 · QuEraComputing/ppvm

david-pl · 2026-10-02T15:10:50Z

Experiment to see whether we can push #204 past Stim's performance. There are some easy wins, such as the random measurement part. Overall, looks promising, but results are still mixed (sometimes Stim beats ppvm, sometimes ppvm beats stim).

Results and AI output below.

Stacked on #204 (base codex/traits-2-impl). Makes the ppvm-tableau-2 generalized tableau, run through ppvm-stim, faster than Stim's TableauSimulator on most pure-Clifford QEC circuits. PR 204 was about 2× slower than Stim.

Results

Apple arm64, single thread, time per shot, against official Stim 1.15.0 (PyPI). On arm64 Stim uses plain 64-bit words, so none of this is about SIMD.

	PR 204	this PR	Stim
`surface_d30` (1889 qubits, 27 870 measurements)	37.9 ms	~13.2 ms	18.3 ms

Size sweep over Stim-generated memory circuits (rounds = d, all four noise parameters 0.001). Each entry is time / Stim time (below 1× means faster than Stim); 3 alternating rounds per circuit:

Surface	d=3	5	7	9	11	15	19	23	27	31
qubits	26	64	118	188	274	494	778	1126	1538	2014
this PR / Stim	0.76	0.77	0.87	0.97	1.09	0.83	0.78	0.81	0.74	0.72
PR 204 / Stim	1.82	2.25	2.62	2.91	3.14	2.62	2.56	2.46	2.23	2.07

Repetition	d=3	9	25	75	225	675
qubits	5	17	49	149	449	1349
this PR / Stim	0.50	0.78	1.10	1.28	0.96	0.95
PR 204 / Stim	1.33	2.96	4.54	4.74	3.39	2.30

Color	d=3	5	7	9	13	17	21	25	31	37	43
qubits	10	28	55	91	190	325	496	703	1081	1540	2080
this PR / Stim	0.61	1.02	1.36	1.25	1.14	1.14	0.89	0.86	0.80	0.79	0.74
PR 204 / Stim	1.24	2.83	4.16	3.71	4.59	5.13	3.03	7.18	8.39	9.26	10.74

The color code's C_XYZ, which ppvm-stim doesn't support, is rewritten as H then SQRT_X_DAG for both simulators. clifft-bench's pure_surface_d7_r7 gives 52.5 µs here and 60.5 µs for Stim (0.87×). Mean outcome counts per shot track Stim's on every circuit (for example surface d19 2228.5 vs 2228.1, color d31 4929.1 vs 4926.2).

Full tables, per-commit attribution, profiles and dead ends: bench/stim-compare/README.md.

What changed

Profiling showed PR 204's gap was all in measurement (strided bit access, not complexity); its gates were already faster than Stim's. In order:

Measurement (ppvm-tableau-2, ppvm-stim):
- GeneralizedTableau::measure_batch does what Stim's collapse_z does: it checks which targets are random without transposing, collapses those under one row guard, then measures the deterministic ones. The executor routes M, MR, R, MX, MY, MRX, MRY and noisy M through it, applying resets and basis changes around the batch. Repeated targets keep the per-target order.
- Each random measurement reads its anticommutation columns once instead of three times, from reused buffers. On a stabilizer state it skips widening the masks into the 2048-bit branch index.
- On a stabilizer state (one amplitude, at index 0), measure_batch collapses Stim-style, eliminating over the stabilizers only. The destabilizer column is never read.
Gates (ppvm-tableau-2):
- Gate kernels and the inverse-sign row products only touch the live words of each column (n.div_ceil(64) instead of a 4-word-padded stride). The per-gate get_disjoint_mut overlap check becomes a debug check.
- CX computes its forward update and its two inverse-sign products from one borrow.
- Row-product phases accumulate in carry-save counters (Stim's cnt1 / cnt2), checked exhaustively against the g-rule. They run as four lanes from 8 words on.
Noise (ppvm-stim): DEPOLARIZE1/2, X/Y/Z_ERROR and PAULI_CHANNEL_1 skip to the next error with geometric gaps (Stim's RareErrorIterator), so there's one draw per error instead of one per target.
Bench (bench/stim-compare): a standalone crate (its own [workspace], so the C++ stim dependency stays out of ppvm's workspace) plus Python scripts. They cover the harness, the Stim reference, the size sweep, per-commit attribution, a non-Clifford regression check and a per-gate microbenchmark.

Behaviour changes

Seeded results change for noisy circuits (new noise sampling, and noise flips after a measurement batch) and for multi-amplitude states (random targets draw first). A given seed is still reproducible, and the distributions are unchanged. Noise-free Clifford circuits are bit-identical to PR 204.
The post-measurement frame from measure_batch can differ from measure_many's. On stabilizer states the new stabilizer is ±Z times other stabilizers rather than ±Z itself. It describes the same state.
The legacy backend is unchanged. The new StimTableau methods default to the old per-target loops.

Testing

cargo test --workspace, ppvm-tableau-2, ppvm-stim on both backends, and ppvm-conformance-2 all pass.
New tests:
- measure_batch matches a per-qubit loop on stabilizer states, and samples the same distribution on multi-amplitude states;
- the Stim-style collapse matches the textbook projection (outcomes, inverse-sign consistency), at n = 1–70;
- batched MR / R / MX / MY / MRX / MRY semantics, including repeated targets and noisy records;
- per-channel noise distributions and the DEPOLARIZE2 pair correlation;
- an exhaustive check of PhaseCounter against the g term.
I broke the new code on purpose in several places to confirm the tests catch it.
Every tableau-level commit left mean outcomes identical to its parent on 8–12 Clifford and non-Clifford circuits. Mean outcomes agree with Stim's compiled sampler within about 1σ.
Non-Clifford programs (cultivation, MSC, distillation, coherent noise, QV10) show no slowdown against PR 204.

Next levers

Stop keeping two sets of phases. Every Clifford gate updates both the forward generator phases and the inverse-tableau signs. The bits are shared, since a forward column is an inverse row; only the phases exist twice. Keeping the inverse signs current is about half of a CX. A throwaway build without that upkeep ran gate-only surface_d30 in 4.1 ms instead of 7.7 ms. Stim keeps only its inverse tableau. Neither set can simply be dropped, though; each option changes the generalized-tableau algorithm:
- Keep only the inverse signs, like Stim. The multi-amplitude rules are stated in terms of the forward phases and would need re-deriving in terms of inverse signs, along with their Lean proofs:
  - branching for T gates and rotations (branch_with_coefficients, compute_coefficients_after_pauli_apply);
  - the case-a amplitude merge in measurement;
  - expectation values;
  - all of these read odd_phase_destabilizer_mask.
  Frame operations that can't keep the inverse current (swap_xz, xor_z_from_x, row_multiply, overlapping cz_block_pairs) would also need another fallback than today's generator fold.
- Keep only the forward phases. That's what legacy ppvm-tableau does, and it loses the O(1) deterministic measurement (32× slower on surface_d30).
- Make the inverse signs lazy, rebuilding them on the next read. That doesn't pay for QEC circuits, which read the signs every round: a rebuild costs an estimated several hundred µs per round on surface_d30, against about 120 µs of eager upkeep.
- A separate stabilizer-only representation, inverse tableau only, while the state has a single amplitude, switching to the full representation on the first non-Clifford gate. This keeps the generalized-tableau rules as they are but puts a second representation inside GeneralizedTableau, with a conversion step.
Fixed per-call and per-measurement costs at mid size (about 50–330 qubits). This is where the branch is still behind Stim. Candidates are the per-target "all amplitudes at index 0" check (a 2048-bit compare), the determinism pre-scan, has_repeats, and the per-instruction batch buffers.
Batching sign updates across a CX layer. A layer's CXs act on disjoint qubits, so their inverse-sign updates are independent and could run as one pass over the layer's columns.
CZ / CY fusion like CX's. Small: only distillation, cultivation and MSC use CZ, and no benchmark uses CY.

Notes

Commits were made with --no-verify. The pre-commit clippy hook fails on an existing lint in crates/stim-parser/src/pipeline/lower.rs:440 (clippy::chunks_exact_to_as_chunks, new in clippy 1.98), not on this PR's code. That deserves its own fix.
Measured on arm64 only. With AVX2, Stim's 256-bit words could shift the large-n comparison.
Open gaps: see "Next levers" above; they're also listed in the bench README.

🤖 Generated with Claude Code

Measure a target list the way Stim's collapse_z does: check which targets are random in the canonical orientation, collapse those under one row guard, then measure the deterministic ones column-major. A list with no random target never transposes, and deterministic targets never pay strided column reads under the guard. Random targets draw first, so the RNG order differs from a per-target loop when a deterministic-frame target draws (several amplitudes); the outcome distribution is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Add reset_many / measure_reset_many to StimTableau, defaulting to the old per-target loop (legacy unchanged). The traits-2 adapter measures through GeneralizedTableau::measure_batch and applies the X resets afterwards, since gates need column-major and X_q commutes with Z_p for p != q. Instructions with a repeated target keep the per-target loop. surface_d30 (1889 qubits, Apple M-series): 38.6 -> 26.7 ms/shot; Stim's TableauSimulator on the same circuit is 18.7 ms. Seeded results change for noisy MR(p) and multi-amplitude states, because noise flips no longer interleave with measurement draws. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

A Z measurement read the same two X columns at the measured qubit three times: in compute_decomposition and as the selectors of project_inverse and project_row_major. Under a row guard each read is a strided pass over all generators. Gather them once in measure_z_with_scratch and pass them down through compute_decomposition_with and update_tableau_with_columns; the public entry points gather and delegate. Outcomes and RNG draws unchanged. surface_d30 (1889 qubits, Apple M-series): 26.7 -> 22.2 ms/shot; Stim's TableauSimulator on the same circuit is 18.5 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

measure_z_with_scratch now gathers the two anticommutation columns into buffers kept in MeasureScratch instead of allocating them per measurement. A deterministic outcome reads the stabilizer/destabilizer masks only through amplitude indices. When every index is zero, as in a Clifford-only run, it skips widening them into the branch-index type, which for a 2048-bit index cost one full-width shift and OR per set bit. compute_decomposition_with splits into decomposition_phase_with plus that widening. Outcomes and RNG draws unchanged. surface_d30 (1889 qubits, Apple M-series): 22.2 -> 20.4 ms/shot; Stim's TableauSimulator on the same circuit is 18.8 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

On a stabilizer state (one amplitude, at index 0, inverse signs valid) measure_batch now collapses the way Stim's collapse_qubit_z does: CX appends from the pivot to the other anticommuting stabilizers, S on the pivot if the accumulated destabilizer anticommutes with Z, H, then X if the outcome needs it. The destabilizer column is never read and no destabilizer row is multiplied; a deterministic target is one inverse-sign read. project_inverse becomes collapse_inverse with an optional destabilizer column, so both projections share the sign appends. The new stabilizer is ±Z times other stabilizers rather than ±Z itself, so the frame can differ from the textbook projection's; the state, outcomes and RNG draws do not. surface_d30 records are bit-identical over 22 seeded shots. surface_d30 (1889 qubits, Apple M-series): 20.4 -> 17.1 ms/shot; Stim's TableauSimulator on the same circuit is 18.9 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

bench/stim-compare is a standalone crate (own workspace) timing ppvm's GeneralizedTableau on Stim circuits, with Python scripts timing official Stim 1.15.0 on the same circuits: surface_d30 and derived variants, a Stim-generated size sweep (surface, repetition, color codes) and a per-commit attribution run. The README records the PR 204 starting point, which commit bought which win, the eager-batching dead end, the sweep, and the open gaps (unbatched MX/MY, CX on repetition codes, small-n overhead). surface_d30: PR 204 38.5 ms/shot, this branch 16.8 ms, Stim 18.6 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Add StimTableau::measure_noisy_many, defaulting to the per-target measure_noisy loop (legacy unchanged); the traits-2 adapter measures through GeneralizedTableau::measure_batch and flips each record afterwards. The executor rotates every target of MX/MY/MRX/MRY onto Z at once around one batch (measure_in_basis), keeping the per-target order when a target repeats. Noisy M(p) uses the new batch too. Color codes end in MX, which ran per qubit on the column-major strided path: color d31 50.6 -> 8.0 ms/shot and d43 282.7 -> 31.6 ms (Stim 1.15.0: 7.1 and 31.6 ms). Seeded results change only where noise flips or multi-amplitude draws reorder. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Re-ran the per-commit attribution with all seven builds (adding color d43) and the full size sweep on cfd2514. Batching MX/MY takes color d43 from 284.0 to 32.1 ms/shot (Stim 32.4 ms) and smooths the color sweep to 1.56x-0.98x of Stim; surface and repetition are unchanged. The README gains the commit's entry, the new sweep tables, and an updated list of open gaps (CX on repetition codes, small-n overhead, 4e8aeb5's ~6% on deterministic-only measurement, color codes ~2 sigma above Stim's mean). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

nonclifford.py times PR 204 against cfd2514 on cultivation_d5 and clifft-bench's non-Clifford circuits (rotations rewritten into ppvm's I[...] tags): 0.96x-1.00x everywhere, so no regressions. The harness's file mode takes an explicit qubit count so these run without a Stim parse. gates.py isolates CX and H: ppvm's CX has a ~25 ns floor up to ~512 qubits against Stim's 7 ns (H: ~8 vs 3.5 ns), which is the bulk of the remaining small-circuit gap. The README records both and updates the open gaps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

A column's stride is rounded up to a whole 4-word block, so below 256 qubits every gate kernel and the inverse-sign row product swept 4 words where only n.div_ceil(64) can hold a set bit. The padding is zero and every kernel maps zero words to zero words, so gate1_mut / gate2_mut and inv_pair_phase now borrow only the live words, and the per-gate get_disjoint_mut overlap check (5 ranges, both halves, every gate) becomes a debug check on ranges that are disjoint by layout. Tableaus and outcomes are unchanged. CX at n <= 128: ~25 -> ~15.5 ns/gate (Stim 1.15.0: 7-9 ns); from 512 qubits up CX is now 0.78-0.93x Stim. Circuits: surface d7 85.2 -> 74.9 us, color d9 109.6 -> 94.4 us, repetition d675 82.6 -> 73.2 ms, surface_d30 16.6 -> 15.7 ms. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Third attribution run (eight builds), per-gate microbenchmark and size sweep on 6e8bc36. Trimming the gate kernels speeds up every workload by 4-14%: surface_d30 16.6 -> 15.5 ms (Stim 18.3 ms), surface d19 now ahead of Stim, CX at n <= 128 ~25 -> ~15.5 ns. The README gains the commit's entry, the before/after gate table, the new sweep, and why the remaining small-n gap is the dual phase bookkeeping. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

A CX swept its columns twice: prepend_cnot fetched the inverse rows ix_c, ix_t, iz_c, iz_t for two row-phase products, then sweep2 fetched the same columns again per half for the forward update. An inverse X row is the two halves' Z columns and a Z row their X columns, so TableauData::cnot_fused borrows the eight columns and both phase planes once, reads the two g-rule terms, then applies the forward kernel; prepend_cnot takes the terms instead of recomputing them. The loops stay separate so they vectorize, except at one live word (n <= 64), where a single scalar pass is cheaper. The per-word g-rule and CNOT formulas move into product_phase_word / cnot_word so every kernel shares one definition. Tableaus and outcomes are unchanged. CX vs 6e8bc36, alternating builds: n=32/64 -45%/-43% (~8.6 ns/gate at n=64; Stim 1.15.0: 7.2 ns), n=128..512 -14% to -22%, n=2048 -2%. Circuits: repetition d25 -17%, color d5/d7 -13%, surface d11 -8%, surface d19 -8%. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Re-ran the per-gate microbenchmark and a two-build attribution (6e8bc36 vs f594299 vs Stim) instead of the full suite; the machine was not fully idle (Stim measured 3-5% slower than in the quiet runs), so the README reports ratios. Fusing CX: every workload -1% to -10%, surface d7 1.08x Stim, color d31 0.91x, surface_d30 0.81x; CX at n <= 64 is now 1.13-1.19x Stim. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

DEPOLARIZE1, DEPOLARIZE2, X/Y/Z_ERROR and PAULI_CHANNEL_1 drew one random number per target (DEPOLARIZE2 also built and scanned a 15-entry table per pair). StimTableau gains depolarize1_many / depolarize2_many / pauli_error_many, defaulting to the per-target loops (legacy unchanged); the traits-2 adapter skips to the next error with geometric gaps, like Stim's RareErrorIterator, then picks the error's Pauli. Lost qubits behave as before: a lost target is untouched and a pair with a lost qubit is skipped. Same distributions (new statistical executor test per channel, plus the DEPOLARIZE2 pair correlation); seeded results change. Mean outcomes agree with Stim's compiled sampler within 1 sigma on six circuits. vs f594299, alternating builds: repetition d25/d75/d675 -28%/-25%/-13%, surface d5/d11/d19 -16%/-12%/-10%, color d9/d31 -13%/-8%, surface_d30 -4%. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Alternating-build timings against f594299 and the new comparison with Stim's compiled sampler; updates the open gaps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The g-rule phase of a row product ran two popcounts per word. PhaseCounter keeps one 2-bit counter per bit position instead (Stim's cnt1/cnt2 update, checked exhaustively against the g term) and popcounts once at the end. A single serial counter does not vectorize, so from 8 words on row_multiply_phase runs four counters over 4-word chunks; below that one serial counter is cheapest. It is #[inline(always)]: left out of line in cnot_fused it cost more than it saved. row_multiply takes the phase pass and then plain XOR passes for long rows. Tableaus and outcomes unchanged. vs 5f84078, alternating builds: CX -4% (n=64) to -25% (n=2048); surface_d30 -8%, repetition d75 -14%, color d9 -7%, surface d7 -7%, surface d19 -3%, surface d23 -1%. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Includes the variants the measurements ruled out and the inverse-upkeep experiment that motivated it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

The branch is now ahead of Stim 1.15.0 on most of the sweep: surface 0.72x (d31) to 0.97x except d11 (1.09x), repetition 0.50x-0.96x except d25/d75 (1.10x/1.28x), color 0.61x and 0.74x-0.89x from d21 up, 1.14x-1.36x at d7-d17. The README's sweep tables and open gaps are updated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

github-actions · 2026-10-02T15:20:20Z

PR Preview Action v1.8.1
🚀 View preview at https://QuEraComputing.github.io/ppvm/pr-preview/pr-236/
Built to branch `gh-pages` at 2026-10-02 15:20 UTC. Preview will be ready when the GitHub Pages deployment is complete.

david-pl and others added 18 commits October 2, 2026 13:47

bench(stim): record 5f84078's skip-sampled noise in the README

43b9ee2

Alternating-build timings against f594299 and the new comparison with Stim's compiled sampler; updates the open gaps. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

bench(stim): record 771daf6's carry-save phase counters in the README

d528d33

Includes the variants the measurements ruled out and the inverse-upkeep experiment that motivated it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

david-pl changed the title ~~perf(tableau-2,stim): beat Stim on Clifford QEC circuits (stacked on #204)~~ perf(tableau-2,stim): try to beat Stim on Clifford QEC circuits (stacked on #204) Oct 2, 2026

Navigation Menu

perf(tableau-2,stim): try to beat Stim on Clifford QEC circuits (stacked on #204) - #236

perf(tableau-2,stim): try to beat Stim on Clifford QEC circuits (stacked on #204)#236
david-pl wants to merge 18 commits into
codex/traits-2-implfrom
david/batch-mr-reset

david-pl commented Oct 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 2, 2026

Built to branch `gh-pages` at 2026-10-02 15:20 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

david-pl commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Results

What changed

Behaviour changes

Testing

Next levers

Notes

Uh oh!

github-actions Bot commented Oct 2, 2026

Built to branch gh-pages at 2026-10-02 15:20 UTC. Preview will be ready when the GitHub Pages deployment is complete.

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

david-pl commented Oct 2, 2026 •

edited

Loading

Built to branch `gh-pages` at 2026-10-02 15:20 UTC.
Preview will be ready when the GitHub Pages deployment is complete.