Skip to content

bench: cross-library Pauli-propagation benchmark, with truncation-accuracy measurement #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Roger-luo wants to merge 5 commits into
base: main
Choose a base branch
from
Open

bench: cross-library Pauli-propagation benchmark, with truncation-accuracy measurement #211

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
38f160c
bench(cross-library): compare ppvm against four other Pauli-propagati…
Roger-luo Aug 12, 2026
6e0506c
docs(bench): quantify the accuracy cost of prune-at-creation truncation
Roger-luo Aug 12, 2026
6e84129
bench(cross-library): record the atol/accuracy sweep as a CSV
Roger-luo Aug 12, 2026
6cbfbf4
bench(cross-library): add a scrambling workload to locate where trunc…
Roger-luo Aug 12, 2026
58d5100
bench(cross-library): plot the truncation-error scaling and record it…
Roger-luo Aug 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions benchmarks/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,19 @@ uv run --with matplotlib python benchmarks/plot_branch_coalesce.py \
- `plot_branch_coalesce.py` — left panel: time vs `m` (log-log); right panel:
sort-merge speedup `t_hash / t_sortmerge` vs `m`, with the crossover line and
the "hash wins" band.

# Cross-library Pauli propagation

`ppvm` against [PauliPropagation.jl][xpp], [PauliStrings.jl][xps], Qiskit's
[pauli-prop][xqk] and Algorithmiq's [monoprop][xmp], on TFIM Trotter and
Heisenberg autocorrelator workloads, all single-threaded and all validated to
propagate the identical operator term-for-term before anything is timed.

Lives in [`cross-library/`](cross-library/README.md) — see that README for the
circuit spec, the shared environment contract, the per-engine caveats (notably
that monoprop is parallel unless capped), and a measured run.

[xpp]: https://github.com/MSRudolph/PauliPropagation.jl
[xps]: https://github.com/nicolasloizeau/PauliStrings.jl
[xqk]: https://github.com/Qiskit/pauli-prop
[xmp]: https://github.com/Algorithmiq/monoprop
382 changes: 382 additions & 0 deletions benchmarks/cross-library/README.md
View file
Open in desktop

Large diffs are not rendered by default.

51 changes: 51 additions & 0 deletions benchmarks/cross-library/accuracy.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
model,library,qubits,steps,dt,seed,atol,ref_atol,ref_terms,ref_norm,terms,l2_err,l1_err,max_coeff_err,lost_mass,kept_subthreshold,dropped_above_atol,observable,ref_observable,obs_abs_err
tfim,ppvm,8,10,0.1,12345,1e-3,1e-16,27051,2.828427e+00,834,5.730178e-02,1.956886e+00,6.556158e-03,1.180394e+00,0,388,1.889668769617,1.888902654574,7.661150e-04
tfim,pauli-propagation-jl,8,10,0.1,12345,1e-3,1e-16,27051,2.828427e+00,834,5.730178e-02,1.956886e+00,6.556158e-03,1.180394e+00,0,388,1.88966876961681,1.888902654574,7.661150e-04
tfim,pauli-strings-jl,8,10,0.1,12345,1e-3,1e-16,27051,2.828427e+00,834,5.730178e-02,1.956886e+00,6.556158e-03,1.180394e+00,0,388,1.88966876961681,1.888902654574,7.661150e-04
tfim,pauli-prop,8,10,0.1,12345,1e-3,1e-16,27051,2.828427e+00,743,9.747655e-02,3.201066e+00,8.889589e-03,1.519868e+00,0,479,1.9013036238629,1.888902654574,1.240097e-02
tfim,monoprop,8,10,0.1,12345,1e-3,1e-16,27051,2.828427e+00,804,6.429517e-02,2.126366e+00,6.563651e-03,1.286381e+00,0,418,1.895370105228,1.888902654574,6.467451e-03
tfim,ppvm,8,10,0.1,12345,1e-4,1e-16,27051,2.828427e+00,1730,7.713204e-03,3.399904e-01,5.538029e-04,1.657662e-01,0,558,1.887148304814,1.888902654574,1.754350e-03
tfim,pauli-propagation-jl,8,10,0.1,12345,1e-4,1e-16,27051,2.828427e+00,1730,7.713204e-03,3.399904e-01,5.538029e-04,1.657662e-01,0,558,1.88714830481379,1.888902654574,1.754350e-03
tfim,pauli-strings-jl,8,10,0.1,12345,1e-4,1e-16,27051,2.828427e+00,1730,7.713204e-03,3.399904e-01,5.538029e-04,1.657662e-01,0,558,1.88714830481379,1.888902654574,1.754350e-03
tfim,pauli-prop,8,10,0.1,12345,1e-4,1e-16,27051,2.828427e+00,1527,1.655369e-02,6.909180e-01,1.396127e-03,2.516797e-01,0,761,1.89026934442961,1.888902654574,1.366690e-03
tfim,monoprop,8,10,0.1,12345,1e-4,1e-16,27051,2.828427e+00,1705,8.349062e-03,3.553170e-01,6.351504e-04,1.787774e-01,1,584,1.888182135464,1.888902654574,7.205191e-04
tfim,ppvm,8,10,0.1,12345,1e-5,1e-16,27051,2.828427e+00,3001,9.762681e-04,5.198656e-02,5.615238e-05,2.218986e-02,0,715,1.888799248154,1.888902654574,1.034064e-04
tfim,pauli-propagation-jl,8,10,0.1,12345,1e-5,1e-16,27051,2.828427e+00,3001,9.762681e-04,5.198656e-02,5.615238e-05,2.218986e-02,0,715,1.88879924815445,1.888902654574,1.034064e-04
tfim,pauli-strings-jl,8,10,0.1,12345,1e-5,1e-16,27051,2.828427e+00,3001,9.762681e-04,5.198656e-02,5.615238e-05,2.218986e-02,0,715,1.88879924815445,1.888902654574,1.034064e-04
tfim,pauli-prop,8,10,0.1,12345,1e-5,1e-16,27051,2.828427e+00,2628,2.357978e-03,1.202946e-01,1.384898e-04,3.877258e-02,0,1088,1.88885571834255,1.888902654574,4.693623e-05
tfim,monoprop,8,10,0.1,12345,1e-5,1e-16,27051,2.828427e+00,2998,1.023503e-03,5.342297e-02,5.628422e-05,2.245608e-02,0,718,1.888873511516,1.888902654574,2.914306e-05
tfim,ppvm,8,10,0.1,12345,1e-6,1e-16,27051,2.828427e+00,4701,1.118273e-04,6.914840e-03,7.337651e-06,2.638321e-03,0,875,1.888902953057,1.888902654574,2.984830e-07
tfim,pauli-propagation-jl,8,10,0.1,12345,1e-6,1e-16,27051,2.828427e+00,4701,1.118273e-04,6.914840e-03,7.337651e-06,2.638321e-03,0,875,1.88890295305684,1.888902654574,2.984828e-07
tfim,pauli-strings-jl,8,10,0.1,12345,1e-6,1e-16,27051,2.828427e+00,4701,1.118273e-04,6.914840e-03,7.337651e-06,2.638321e-03,0,875,1.88890295305684,1.888902654574,2.984828e-07
tfim,pauli-prop,8,10,0.1,12345,1e-6,1e-16,27051,2.828427e+00,4161,2.838847e-04,1.707832e-02,1.911237e-05,5.130690e-03,0,1415,1.88893188165528,1.888902654574,2.922708e-05
tfim,monoprop,8,10,0.1,12345,1e-6,1e-16,27051,2.828427e+00,4662,1.182467e-04,7.215810e-03,7.389538e-06,2.782631e-03,1,915,1.888910840223,1.888902654574,8.185649e-06
tfim,ppvm,8,10,0.1,12345,1e-7,1e-16,27051,2.828427e+00,6780,1.239548e-05,8.753233e-04,7.429950e-07,2.963253e-04,0,996,1.888901406583,1.888902654574,1.247991e-06
tfim,pauli-propagation-jl,8,10,0.1,12345,1e-7,1e-16,27051,2.828427e+00,6780,1.239548e-05,8.753233e-04,7.429950e-07,2.963253e-04,0,996,1.88890140658276,1.888902654574,1.247991e-06
tfim,pauli-strings-jl,8,10,0.1,12345,1e-7,1e-16,27051,2.828427e+00,6780,1.239548e-05,8.753233e-04,7.429950e-07,2.963253e-04,0,996,1.88890140658276,1.888902654574,1.247991e-06
tfim,pauli-prop,8,10,0.1,12345,1e-7,1e-16,27051,2.828427e+00,6056,3.227852e-05,2.215272e-03,1.725103e-06,6.248028e-04,0,1720,1.88890165053338,1.888902654574,1.004041e-06
tfim,monoprop,8,10,0.1,12345,1e-7,1e-16,27051,2.828427e+00,6753,1.309534e-05,9.190766e-04,7.516440e-07,3.063929e-04,1,1024,1.888902140509,1.888902654574,5.140650e-07
heisenberg,ppvm,8,10,0.1,12345,1e-3,1e-16,16384,1.000000e+00,4566,1.935867e-01,1.683728e+01,6.612049e-03,8.467940e+00,370,2994,0.1843817263679,0.1835878942086,7.938322e-04
heisenberg,pauli-propagation-jl,8,10,0.1,12345,1e-3,1e-16,16384,1.000000e+00,4566,1.935867e-01,1.683728e+01,6.612049e-03,8.467940e+00,370,2994,0.184381726367865,0.1835878942086,7.938322e-04
heisenberg,pauli-strings-jl,8,10,0.1,12345,1e-3,1e-16,16384,1.000000e+00,4566,1.935867e-01,1.683728e+01,6.612049e-03,8.467940e+00,370,2994,0.184381726367865,0.1835878942086,7.938322e-04
heisenberg,pauli-prop,8,10,0.1,12345,1e-3,1e-16,16384,1.000000e+00,5072,2.384650e-01,2.082594e+01,1.022082e-02,9.244654e+00,1051,3169,0.186064764275212,0.1835878942086,2.476870e-03
heisenberg,monoprop,8,10,0.1,12345,1e-3,1e-16,16384,1.000000e+00,5106,1.937250e-01,1.714593e+01,6.293714e-03,8.315984e+00,836,2920,0.18421262641,0.1835878942086,6.247322e-04
heisenberg,ppvm,8,10,0.1,12345,1e-4,1e-16,16384,1.000000e+00,11923,2.564185e-02,2.511013e+00,1.063198e-03,5.963471e-01,1107,2274,0.1834786826111,0.1835878942086,1.092116e-04
heisenberg,pauli-propagation-jl,8,10,0.1,12345,1e-4,1e-16,16384,1.000000e+00,11923,2.564185e-02,2.511013e+00,1.063198e-03,5.963471e-01,1107,2274,0.183478682611051,0.1835878942086,1.092116e-04
heisenberg,pauli-strings-jl,8,10,0.1,12345,1e-4,1e-16,16384,1.000000e+00,11923,2.564185e-02,2.511013e+00,1.063198e-03,5.963471e-01,1107,2274,0.183478682611051,0.1835878942086,1.092116e-04
heisenberg,pauli-prop,8,10,0.1,12345,1e-4,1e-16,16384,1.000000e+00,13628,4.451207e-02,4.398424e+00,2.046005e-03,6.386030e-01,2836,2298,0.183655881056651,0.1835878942086,6.798685e-05
heisenberg,monoprop,8,10,0.1,12345,1e-4,1e-16,16384,1.000000e+00,13143,2.725847e-02,2.714465e+00,1.193874e-03,5.551494e-01,2074,2021,0.183546769437,0.1835878942086,4.112477e-05
heisenberg,ppvm,8,10,0.1,12345,1e-5,1e-16,16384,1.000000e+00,16101,2.426734e-03,2.449232e-01,1.074420e-04,1.125860e-02,963,420,0.1835902065899,0.1835878942086,2.312381e-06
heisenberg,pauli-propagation-jl,8,10,0.1,12345,1e-5,1e-16,16384,1.000000e+00,16101,2.426734e-03,2.449232e-01,1.074420e-04,1.125860e-02,963,420,0.18359020658986,0.1835878942086,2.312381e-06
heisenberg,pauli-strings-jl,8,10,0.1,12345,1e-5,1e-16,16384,1.000000e+00,16101,2.426734e-03,2.449232e-01,1.074420e-04,1.125860e-02,963,420,0.183590206589861,0.1835878942086,2.312381e-06
heisenberg,pauli-prop,8,10,0.1,12345,1e-5,1e-16,16384,1.000000e+00,17803,4.507725e-03,4.730143e-01,1.675695e-04,1.250100e-02,2677,432,0.183584912095183,0.1835878942086,2.982113e-06
heisenberg,monoprop,8,10,0.1,12345,1e-5,1e-16,16384,1.000000e+00,17223,2.557524e-03,2.663380e-01,1.096506e-04,9.925007e-03,2027,362,0.183585575353,0.1835878942086,2.318856e-06
heisenberg,ppvm,8,10,0.1,12345,1e-6,1e-16,16384,1.000000e+00,16324,1.873617e-04,1.844112e-02,8.671387e-06,1.957258e-04,191,82,0.1835880798476,0.1835878942086,1.856390e-07
heisenberg,pauli-propagation-jl,8,10,0.1,12345,1e-6,1e-16,16384,1.000000e+00,16324,1.873617e-04,1.844112e-02,8.671387e-06,1.957258e-04,191,82,0.183588079847589,0.1835878942086,1.856390e-07
heisenberg,pauli-strings-jl,8,10,0.1,12345,1e-6,1e-16,16384,1.000000e+00,16324,1.873617e-04,1.844112e-02,8.671387e-06,1.957258e-04,191,82,0.183588079847589,0.1835878942086,1.856390e-07
heisenberg,pauli-prop,8,10,0.1,12345,1e-6,1e-16,16384,1.000000e+00,17004,3.764296e-04,3.802784e-02,1.821500e-05,2.767212e-04,894,105,0.183587851042604,0.1835878942086,4.316600e-08
heisenberg,monoprop,8,10,0.1,12345,1e-6,1e-16,16384,1.000000e+00,16910,1.993482e-04,2.007446e-02,1.047708e-05,1.937806e-04,774,79,0.183587892322,0.1835878942086,1.886600e-09
heisenberg,ppvm,8,10,0.1,12345,1e-7,1e-16,16384,1.000000e+00,16377,1.541357e-05,1.514858e-03,7.285205e-07,3.268561e-06,39,14,0.1835878872944,0.1835878942086,6.914200e-09
heisenberg,pauli-propagation-jl,8,10,0.1,12345,1e-7,1e-16,16384,1.000000e+00,16377,1.541357e-05,1.514858e-03,7.285205e-07,3.268561e-06,39,14,0.183587887294405,0.1835878942086,6.914195e-09
heisenberg,pauli-strings-jl,8,10,0.1,12345,1e-7,1e-16,16384,1.000000e+00,16377,1.541357e-05,1.514858e-03,7.285205e-07,3.268561e-06,39,14,0.183587887294405,0.1835878942086,6.914195e-09
heisenberg,pauli-prop,8,10,0.1,12345,1e-7,1e-16,16384,1.000000e+00,16528,3.026389e-05,3.008580e-03,1.430835e-06,5.571167e-06,195,19,0.18358789229432,0.1835878942086,1.914280e-09
heisenberg,monoprop,8,10,0.1,12345,1e-7,1e-16,16384,1.000000e+00,16558,1.539943e-05,1.536256e-03,8.884821e-07,3.199623e-06,217,11,0.183587893774,0.1835878942086,4.346000e-10
Loading
Loading