[jkaczman]

The idea here is to compile PgDog with profile-guided optimization (https://doc.rust-lang.org/rustc/profile-guided-optimization.html), which is..

"The basic concept of PGO is to collect data about the typical execution of a program (e.g. which branches it is likely to take) and then use this data to inform optimizations such as inlining, machine-code layout, register allocation, etc."

The profile we use here comes from running our integration suites (sql, python, pgbench, complex, schema_sync, tls, two_pc, copy_data/data_sync) against an instrumented build of PgDog. Coverage is prioritized right now as anything the profile doesn't cover isn't optimized for speed. After CI runs the suites to record the PGO profile, it proceeds to build the release binary with the profile. This probably adds another 30 mins to our release CI.

End-to-end test on AWS with 16 workers, 2 shards, 256 pgbench clients:

• x86 (c7i.4xlarge): 4-6% higher TPS, 9-10% less CPU per transaction
• arm64 (Graviton) (c7g.4xlarge): 9-17% higher TPS, 22-25% less CPU per transaction, 14-17% fewer instructions per transaction

BOLT (re: #190 mentioning it) was intentionally not included due to very minor gain not worth the extra complexity (plus it's not supported for arm64).

Fixes #190.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[levkk]

I think 10% is the difference between pgbouncer and pgdog currently, so this is cool

[github-actions]

Metal bench (arm64) passed. All 148 measures within bounds.

suite	throughput	cpu/txn	user cycles/txn	user instr/txn	kernel cycles/txn	ctx/txn
pooler		21.76 (-0.1% ± 0.5%)	22,467 (+0.3% ± 1.3%)	35,916 (+0.0% ± 0.06%)	32,188 (-0.3% ± 0.7%)
lb		22.45 (-0.1% ± 0.6%)	23,274 (-0.4% ± 1.0%)	36,888 (+0.0% ± 0.03%)	32,966 (+0.3% ± 1.1%)
sharding-w2		40.43 (-0.2% ± 1.2%)	55,711 (+0.1% ± 0.9%)	102,575 (+0.0% ± 0.01%)	47,636 (-0.5% ± 2.6%)	1.543 (-0.0% ± 0.2%)
sharding-w8	218,716 (+0.1% ± 0.1%)	33.35 (-0.1% ± 0.1%)	51,986 (-0.0% ± 0.3%)	57,901 (-0.0% ± 0.02%)	40,268 (-0.3% ± 0.4%)	0.1466 (+0.2% ± 0.6%)

Counters per txn

suite	frontend stall	l1i refill	backend stall
pooler	11,162 (+1.0% ± 3.5%)	3,146 (+0.3% ± 1.6%)	2,548 (-0.9% ± 2.8%)
lb	11,766 (-0.9% ± 2.2%)	3,235 (-0.6% ± 1.2%)	2,565 (+0.2% ± 1.8%)
sharding-w2	27,238 (+0.8% ± 2.9%)	5,245 (+0.3% ± 1.5%)	7,322 (-2.0% ± 5.6%)
sharding-w8	26,646 (-0.1% ± 0.5%)	4,808 (-0.1% ± 0.3%)	9,782 (+0.4% ± 0.7%)

Microbench: all 135 measures within bounds.

lb flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

microbench flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

pooler flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

sharding-w2 flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

sharding-w8 flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

Logs

lb, microbench, pooler, sharding-w2, sharding-w8, verdict.json

Compared 58cf508 with dbcf92a on m8g.metal-24xl (spot).

[github-actions]

Metal bench (x86) passed. All 148 measures within bounds.

suite	throughput	cpu/txn	user cycles/txn	user instr/txn	kernel cycles/txn	ctx/txn
pooler		26.14 (+0.0% ± 0.2%)	20,539 (+0.0% ± 0.09%)	36,742 (-0.0% ± 0.04%)	33,249 (+0.0% ± 0.3%)
lb		26.54 (-0.0% ± 0.2%)	21,138 (+0.0% ± 0.1%)	37,675 (+0.0% ± 0.05%)	33,350 (-0.1% ± 0.3%)
sharding-w2		42.54 (-0.0% ± 0.6%)	50,397 (+0.2% ± 1.1%)	106,036 (-0.0% ± 0.01%)	38,052 (-0.3% ± 2.1%)	1.522 (+0.0% ± 0.2%)
sharding-w8	202,456 (-0.0% ± 0.4%)	34.25 (-0.2% ± 0.4%)	49,458 (-0.3% ± 0.6%)	60,307 (-0.0% ± 0.06%)	31,249 (-0.1% ± 0.3%)	0.1636 (+0.7% ± 0.6%)

Counters per txn

suite	frontend stall	l1i refill	backend stall
pooler	3,628 (+0.1% ± 0.5%)	1,192 (-0.1% ± 0.6%)	41,517 (+0.0% ± 0.5%)
lb	3,759 (+0.1% ± 0.3%)	1,223 (+0.1% ± 0.3%)	42,796 (-0.0% ± 0.3%)
sharding-w2	6,408 (-0.1% ± 0.6%)	1,704 (-0.4% ± 0.8%)	121,960 (+0.5% ± 2.6%)
sharding-w8	3,369 (+0.0% ± 0.6%)	1,024 (-0.2% ± 0.4%)	187,278 (-0.4% ± 0.9%)

Microbench: all 135 measures within bounds.

lb flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

microbench flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

pooler flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

sharding-w2 flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

sharding-w8 flamegraph vs dbcf92a

Open interactive for hover, zoom and search.

Open interactive.

Logs

lb, microbench, pooler, sharding-w2, sharding-w8, verdict.json

Compared 58cf508 with dbcf92a on c7i.metal-48xl (spot).