benchmarks.md

Benchmarks — PGSync v0.1.0

All numbers below were measured, not estimated. Raw samples and JSON summaries are in bench-results/ (generated by scripts/bench.sh, scripts/catchup-bench.sh, scripts/scale.sh). PGSync commit cfadb53, release build (lto = thin).

Environment

Item	Value
Machine	Apple M1 Pro, 10 cores, 16 GiB RAM, macOS 14.4 (Darwin 23.4.0), Apple SSD
PostgreSQL	18.6 (Homebrew), two host-local clusters started by scripts/local-pg.sh
Source config	wal_level=logical, max_slot_wal_keep_size=4GB, wal_sender_timeout=30s, shared_buffers=256MB, fsync=on
Target config	defaults, shared_buffers=256MB, fsync=on
PGSync config	defaults (max_inflight_bytes 16 MiB, max_batch_rows 500, max_batch_bytes 4 MiB, buffer_size_kb 256, max_parallel_copies 2)
Writer	psql processes issuing one transaction each (20 updates + 30 inserts + 5 deletes); process spawn per transaction limits the offered load
Sampling	ps -o rss,%cpu of the pgsync process every second

Source, target and PGSync share the machine, so throughput numbers are conservative and CPU percentages include contention with two PostgreSQL servers and the writers. The Docker VM on this machine had no free disk, which is why host clusters were used (see docs/test-report.md).

Initial copy and continuous replication (scripts/bench.sh)

Dataset	Source size on disk	COPY bytes	Snapshot time	Copy throughput	RSS avg / peak during copy	CPU avg during copy
100 MB (2 × 400k rows)	156 MB	114 MB	5.0 s	22.8 MB/s	20.0 / 28.7 MB	21 %
1 GB (2 × 4M rows)	1.57 GB	1.17 GB	32.0 s	36.5 MB/s	23.1 / 33.2 MB	8 %

Dataset	CDC window	Transactions applied	Rows applied	Txn/s	Rows/s	Drain lag after writers stop	RSS avg / peak during CDC	CPU avg
100 MB	60 s, 4 writers	7,138	375,625	119	6,260	0 s	13.2 / 27.9 MB	14 %
1 GB	120 s, 4 writers	10,116	538,489	84	4,450	1 s	15.0 / 25.6 MB	11 %

Both runs ended with identical content hashes on source and target for both tables.

Observations:

• RSS is independent of dataset size (28.7 MB peak at 100 MB vs 33.2 MB at 1 GB). The working set is the configured buffers plus connection state, as designed.
• The CDC phase is bounded by the writer (one psql process per transaction); drain lag of 0–1 s after the writers stop shows PGSync was keeping up, not the ceiling. See the catch-up benchmark for apply throughput without that limit.
• Copy throughput rises with table size (per-table overhead amortised); two parallel copies share one target disk.

Catch-up throughput (scripts/catchup-bench.sh)

With the bench job bootstrapped and stopped, a backlog of 5,000 committed transactions (100 inserts + 10 updates + 5 deletes each, 575,000 row operations, 118 MB of WAL) was generated on the source; then pgsync run was timed until it acknowledged the WAL head.

Metric	Value
Catch-up time	8.0 s
Transactions per second	625
Row operations per second	≈ 72,000
WAL consumed	14.7 MB/s
RSS average / peak	34.9 / 44.3 MB
CPU	98 % of one core (the applier is single-threaded by design: one target connection, pipelined statements)
Contents after catch-up	identical hashes on both tables

An earlier catch-up run before the feedback fix (1.75 GB backlog including bulk updates) took 26 s from stream open to acknowledgement of the head, ≈ 192 transactions/s, 20 MB RSS; its hash check was lost to a script error and is not counted.

The applier is CPU-bound at ~70 k row operations/s on this machine. Beta 2's multi-row coalescing (one INSERT ... VALUES (...), (...) per batch, DELETE ... WHERE pk = ANY($1)) is the obvious next step; see docs/research/pgstream-audit.md.

Multiple workers (scripts/scale.sh)

One process per job, each with its own schema, publication, slot and target tables (20,000 rows per job, 60 s of writes to every job).

Workers	Jobs consistent at end	Aggregate RSS avg / peak	Per-process RSS avg / peak
5	5 / 5	48.8 / 73.3 MB	9.8 / 15.0 MB
10	10 / 10	95.2 / 143.5 MB	9.5 / 14.9 MB
20	20 / 20	190.0 / 278.4 MB	9.5 / 15.1 MB

Per-process memory is flat at about 9.5 MB average (15 MB peak) regardless of how many jobs run; aggregate memory grows linearly with the number of jobs. Process-level overhead is therefore ~10 MB per stream on this platform (release build, no allocator tuning). Running 20 streams in one process instead of 20 processes would save at most that baseline; it is not a Beta 1 concern. The 20-worker run created 20 slots and 20 walsenders on one source, which stayed healthy (max_wal_senders = 32).

Soak (scripts/bench.sh with CDC_SECONDS=2400)

100 MB dataset, two psql writers, 40 minutes of continuous writes (35 minutes of CDC samples after the snapshot phase), release build, defaults.

Metric	Value
Transactions applied	174,790
Rows applied	9,186,943
Bytes received from the walsender	1.86 GB
Reconnects	0
RSS average / peak over the whole CDC window	15.7 / 32.7 MB
RSS average per quarter (≈9 min each)	18.4, 14.9, 14.6, 14.8 MB
RSS peak per quarter	32.7, 15.7, 15.5, 15.6 MB
RSS first minute vs last minute	31.2 MB vs 14.7 MB
CPU average	9.2 %

No memory growth: after the first minute (tail of the snapshot phase and prepared-statement cache warm-up) the process stays at 14.6–14.9 MB for the remaining half hour.

This run also found a defect: after the writers stopped, the acknowledged position stayed 10 MB behind the WAL head (autovacuum records) because the reader reported progress on a fixed timer, which kept the walsender from sending the keepalive that carries skipped-WAL positions. Fixed (idle_reader_reports_only_changed_positions, idle_slot_advances_over_unpublished_wal); the convergence benchmark below was repeated with the fix: 100 MB dataset, snapshot 3.0 s (38.1 MB/s), drain lag 2 s after the writers stopped, identical hashes. That rerun overlapped with a rebuild and the Docker e2e suite on the same machine, so its CDC throughput figure is not reported. A multi-hour run is still outstanding.

Not measured

• Datasets of 10 GB or more (host disk had 13 GB free; Docker VM none).
• Multi-hour soak; the 40-minute run above is the longest performed.
• Comparison with pgstream: not run; no claim is made.
• Disk I/O and network throughput: not instrumented separately.