Skip to content

Use talc as the guest heap allocator - #1928

Open
ludfjig wants to merge 2 commits into
ll/2-lazy-sandbox-memoryfrom
ll/3-talc-allocator
Open

ludfjig wants to merge 2 commits into
ll/2-lazy-sandbox-memoryfrom
ll/3-talc-allocator

Conversation

@ludfjig

@ludfjig ludfjig commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Switch the guest heap allocator from buddy_system_allocator to talc. Guest init touches fewer heap pages, so sandbox creation gets 25 to 37% faster on MSHV and 9 to 11% faster on KVM.

buddy_system_allocator init writes a free-list pointer at the start of every power-of-two heap block. talc writes only the start and end of the heap. The host populates heap pages on first guest touch. MSHV populates a whole 2 MiB chunk per touch, and buddy's init touches 2 to 7 more chunks than talc.

talc runs behind a spin mutex, as talc recommends. spin is already a dependency.

Benchmarks

Compared to the PR below.

  • MSHV: Linux 6.6 mshv.
  • KVM: Linux 6.17.
Benchmark MSHV KVM
sandboxes/create_initialized/small 3.09 ms → 2.11 ms (−32%) 2.92 ms → 2.58 ms (−11%)
sandboxes/create_initialized/medium 5.93 ms → 3.76 ms (−37%) 3.11 ms → 2.81 ms (~)
sandboxes/create_initialized/large 11 ms → 8.24 ms (−25%) 3.59 ms → 3.27 ms (−9%)
snapshot_files/cold_start_via_evolve/small 3.22 ms → 1.95 ms (−40%) 3.14 ms → 2.91 ms (−7%)

No benchmark regresses.

Full results

All benchmarks on both hosts (click to expand)

Before → after (change). Negative is faster. Bold is a regression. ~ is no consistent change. A change needs agreement between a 0.5 s and a 3 s run, or a 3 s run alone.

snapshots

Benchmark Size MSHV KVM
call_and_create default 366 µs → 363 µs (~) 379 µs → 390 µs (~)
call_and_create small 411 µs → 408 µs (~) 462 µs → 440 µs (~)
call_and_create medium 777 µs → 723 µs (~) 769 µs → 791 µs (~)
call_and_create large 1.82 ms → 1.96 ms (~) 1.87 ms → 1.89 ms (~)
create default 234 µs → 237 µs (~) 254 µs → 256 µs (~)
create small 282 µs → 274 µs (~) 313 µs → 305 µs (~)
create medium 612 µs → 662 µs (~) 629 µs → 631 µs (~)
create large 1.68 ms → 1.69 ms (~) 1.68 ms → 1.72 ms (~)
restore default 71.4 µs → 70.4 µs (~) 24.7 µs → 24.3 µs (~)
restore small 73.9 µs → 73.8 µs (~) 23.5 µs → 21.9 µs (~)
restore medium 77.2 µs → 74.8 µs (~) 28 µs → 25.7 µs (~)
restore large 110 µs → 108 µs (~) 61.7 µs → 53.3 µs (~)
restore_alternate default 82.5 µs → 81.6 µs (~) 92.1 µs → 94.1 µs (~)
restore_alternate small 82.2 µs → 83.2 µs (~) 94.4 µs → 73.7 µs (~)
restore_alternate medium 85.6 µs → 86.9 µs (~) 98.8 µs → 101 µs (~)
restore_alternate large 125 µs → 113 µs (−10%) 129 µs → 128 µs (~)

guest_calls

Benchmark Size MSHV KVM
call default 47.9 µs → 45.8 µs (~) 18.4 µs → 18.3 µs (~)
call small 48.7 µs → 45.5 µs (~) 17.2 µs → 16.7 µs (~)
call medium 46 µs → 44.3 µs (~) 18.4 µs → 18 µs (~)
call large 44.4 µs → 45.7 µs (~) 18.1 µs → 17.7 µs (~)
call_with_host_function default 75.5 µs → 77.1 µs (~) 34.9 µs → 34.9 µs (~)
call_with_host_function small 73.6 µs → 76.6 µs (~) 34.1 µs → 35.5 µs (~)
call_with_host_function medium 74.8 µs → 75.9 µs (~) 34.5 µs → 34.7 µs (~)
call_with_host_function large 75.5 µs → 74.7 µs (~) 34.5 µs → 32.9 µs (~)
call_with_restore default 148 µs → 141 µs (~) 62.5 µs → 65.4 µs (~)
call_with_restore small 139 µs → 142 µs (~) 71.9 µs → 64.4 µs (~)
call_with_restore medium 141 µs → 146 µs (~) 67.4 µs → 68 µs (~)
call_with_restore large 166 µs → 165 µs (~) 89.8 µs → 95.2 µs (~)
different_thread - 72.3 µs → 71.7 µs (~) 42.3 µs → 43 µs (~)
interrupt_latency - 64.5 µs → 68.9 µs (~) 39.2 µs → 30.1 µs (−23%)

guest_functions_with_large_parameters

Benchmark Size MSHV KVM
guest_call_with_large_parameters - 172 ms → 172 ms (~) 890 ms → 877 ms (~)

sample_workloads

Benchmark Size MSHV KVM
24K_in_8K_out_c - 57.5 µs → 59.7 µs (~) 27.6 µs → 27.6 µs (~)
24K_in_8K_out_rust - 55.3 µs → 57.8 µs (~) 27.8 µs → 28 µs (~)

sandboxes

Benchmark Size MSHV KVM
create_initialized default 1.49 ms → 1.5 ms (~) 2.83 ms → 2.59 ms (~)
create_initialized small 3.09 ms → 2.11 ms (−32%) 2.92 ms → 2.58 ms (−11%)
create_initialized medium 5.93 ms → 3.76 ms (−37%) 3.11 ms → 2.81 ms (~)
create_initialized large 11 ms → 8.24 ms (−25%) 3.59 ms → 3.27 ms (−9%)
create_initialized_and_drop default 24.3 ms → 23.3 ms (~) 11 ms → 12.8 ms (~)
create_initialized_and_drop small 25.9 ms → 21.9 ms (~) 15.7 ms → 15.2 ms (~)
create_initialized_and_drop medium 28.3 ms → 28.8 ms (~) 14.8 ms → 17.5 ms (~)
create_initialized_and_drop large 30.7 ms → 28.4 ms (~) 12.4 ms → 15.1 ms (~)
sandbox_from_snapshot default 631 µs → 620 µs (~) 947 µs → 837 µs (−12%)
sandbox_from_snapshot small 805 µs → 803 µs (~) 971 µs → 930 µs (~)
sandbox_from_snapshot medium 2.04 ms → 2.13 ms (~) 999 µs → 967 µs (~)
sandbox_from_snapshot large 6.16 ms → 6.07 ms (~) 1.21 ms → 1.12 ms (~)

snapshot_files

Benchmark Size MSHV KVM
cold_start_via_evolve default 1.58 ms → 1.67 ms (~) 3.05 ms → 2.78 ms (~)
cold_start_via_evolve small 3.22 ms → 1.95 ms (~) 3.14 ms → 2.91 ms (~)
cold_start_via_evolve medium 5.31 ms → 3.46 ms (~) 3.52 ms → 2.92 ms (~)
cold_start_via_evolve large 10.7 ms → 7.82 ms (~) 4.24 ms → 3.37 ms (~)
cold_start_via_snapshot default 2.01 ms → 1.99 ms (~) 2.73 ms → 2.69 ms (~)
cold_start_via_snapshot small 9.13 ms → 9.08 ms (~) 8.9 ms → 8.61 ms (~)
cold_start_via_snapshot medium 54 ms → 53.9 ms (~) 50.5 ms → 50.4 ms (~)
cold_start_via_snapshot large 205 ms → 205 ms (~) 196 ms → 193 ms (~)
cold_start_via_snapshot_unverified default 1.39 ms → 1.36 ms (~) 2.05 ms → 2.09 ms (~)
cold_start_via_snapshot_unverified small 2.21 ms → 2.14 ms (~) 2.3 ms → 2.06 ms (~)
cold_start_via_snapshot_unverified medium 3.52 ms → 3.65 ms (~) 2.1 ms → 2.14 ms (~)
cold_start_via_snapshot_unverified large 7.67 ms → 8.92 ms (~) 2.34 ms → 2.38 ms (~)
load_snapshot default 703 µs → 702 µs (~) 692 µs → 692 µs (~)
load_snapshot small 6.63 ms → 6.65 ms (~) 6.56 ms → 6.46 ms (~)
load_snapshot medium 50.3 ms → 50.3 ms (~) 48.3 ms → 48.2 ms (~)
load_snapshot large 197 ms → 198 ms (~) 192 ms → 191 ms (~)
load_snapshot_unverified default 116 µs → 113 µs (~) 130 µs → 118 µs (~)
load_snapshot_unverified small 117 µs → 114 µs (~) 118 µs → 120 µs (~)
load_snapshot_unverified medium 123 µs → 115 µs (~) 123 µs → 120 µs (~)
load_snapshot_unverified large 122 µs → 118 µs (~) 127 µs → 121 µs (−5%)
save_snapshot default 943 µs → 946 µs (~) 39.5 ms → 41.1 ms (~)
save_snapshot small 9.11 ms → 9.14 ms (~) 52.2 ms → 52.9 ms (~)
save_snapshot medium 67.7 ms → 67.4 ms (~) 396 ms → 398 ms (~)
save_snapshot large 266 ms → 266 ms (~) 1.74 s → 1.75 s (~)

@ludfjig ludfjig added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Oct 12, 2026
@ludfjig
ludfjig added this pull request to stack #1929 October 12, 2026 20:01
@hyperlight-gh-bot

This comment has been minimized.

Copilot AI balanced review requested due to automatic review settings October 13, 2026 18:56
@ludfjig
ludfjig force-pushed the ll/3-talc-allocator branch from 6919f7f to b364493 Compare October 13, 2026 18:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The unsafe heap claim lacks the required safety justification for its exclusive memory ownership invariant.

1 open finding
What changed in this PR

Switches the guest heap allocator to talc to reduce page touches during sandbox initialization.

Changes:

  • Replaces buddy_system_allocator with mutex-protected talc.
  • Updates dependency locks and changelog documentation.
File Description
src/​hyperlight_guest_bin/​src/​lib.rs Implements and initializes the talc guest allocator.
src/​hyperlight_guest_bin/​Cargo.toml Replaces the allocator dependency.
Cargo.lock Updates workspace dependencies.
src/​tests/​rust_guests/​Cargo.lock Updates guest test dependencies.
CHANGELOG.md Documents the performance improvement.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

.try_lock()
.expect("Failed to access HEAP_ALLOCATOR")
.init(heap_start, heap_size);
.claim(heap_start as *mut u8, heap_size)
@ludfjig
ludfjig force-pushed the ll/3-talc-allocator branch from b364493 to c3a949a Compare October 13, 2026 21:01
@hyperlight-gh-bot

This comment has been minimized.

@ludfjig
ludfjig force-pushed the ll/3-talc-allocator branch from c3a949a to 844a1bf Compare October 13, 2026 23:00
@hyperlight-gh-bot

This comment has been minimized.

andreiltd
andreiltd previously approved these changes Oct 14, 2026

@andreiltd andreiltd left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I wonder about using locking api rather than cell which talc seems to support as well? This would require unsafe impl Sync wrapper but would avoid atomic locking overhead?

Off-topic: there is an interesting article: https://matklad.github.io/2020/01/02/spinlocks-considered-harmful.html. The interrupts part I think is relevant to hyperligiht because interrupt handler can allocate while the interrupted code holds the heap lock.

@ludfjig

ludfjig commented Oct 15, 2026

Copy link
Copy Markdown
Contributor Author

Nice! I wonder about using locking api rather than cell which talc seems to support as well? This would require unsafe impl Sync wrapper but would avoid atomic locking overhead?

Off-topic: there is an interesting article: https://matklad.github.io/2020/01/02/spinlocks-considered-harmful.html. The interrupts part I think is relevant to hyperligiht because interrupt handler can allocate while the interrupted code holds the heap lock.

I tried the Cell option first, and it works, but decided to go with the lock instead to mimic current behavior (and I think deadlocking is slightly preferable to silent heap corruption?). The article was interesting indeed, we need to be careful we don't allocate heap in an exception handler

buddy_system_allocator writes a free-list pointer at the start of every
power-of-two heap block during guest init. talc writes only the start
and end of the heap. The host populates snapshot memory on first guest
touch, so fewer written pages make sandbox creation faster.

Criterion sandboxes/create_initialized:

* MSHV: 25% to 37% faster for small to large heaps
* KVM: 9% to 11% faster for small and large heaps

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
Add a SAFETY comment to the guest heap claim.

Signed-off-by: Ludvig Liljenberg <4257730+ludfjig@users.noreply.github.com>
@hyperlight-gh-bot

Copy link
Copy Markdown

Benchmark Results

Measured commit: b55b4ad9d7fa
Baseline commit: 421a4538a965

kvm / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 763.61 ns (➖ 1.02x faster)
vec_bytes 589.45 ns (➖ 1.01x faster)
384.29 µs (➖ 1.02x faster)

payload_allocation

slot_pool_segmented
262144 526.02 ns (➖ 1.01x slower)
65536 157.08 ns (➖ 1.11x slower)

sandboxes

create_initialized_and_drop
medium 12.55 ms (➖ 1.04x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
7.75 ns (➖ 1.00x slower) 7.76 ns (➖ 1.00x slower) 7.76 ns (➖ 1.00x slower)

snapshot_files

load_snapshot_unverified
small 103.72 µs (➖ 1.00x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.07 µs (➖ 1.02x faster) 7.12 µs (➖ 1.02x slower)
65536 2.06 µs (➖ 1.07x slower) 1.94 µs (➖ 1.08x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.88 µs (➖ 1.16x slower) 6.24 µs (➖ 1.06x faster)
8192 1.08 µs (➖ 1.02x slower) 1.09 µs (➖ 1.03x slower)
262144 27.29 µs (➖ 1.03x faster)
kvm / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 681.54 ns (➖ 1.01x slower)
vec_bytes 513.34 ns (➖ 1.01x slower)
677.78 µs (➖ 1.00x faster)

payload_allocation

slot_pool_segmented
262144 506.78 ns (➖ 1.02x slower)
65536 134.81 ns (➖ 1.00x faster)

sandboxes

create_initialized_and_drop
medium 14.95 ms (➖ 1.03x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
6.87 ns (➖ 1.00x faster) 6.86 ns (➖ 1.00x slower) 6.86 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 55.06 µs (➖ 1.00x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.57 µs (➖ 1.01x slower) 7.60 µs (➖ 1.01x slower)
65536 2.12 µs (➖ 1.01x slower) 2.12 µs (➖ 1.01x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.15 µs (➖ 1.00x faster) 7.24 µs (➖ 1.00x slower)
8192 745.08 ns (➖ 1.04x faster) 726.75 ns (➖ 1.00x faster)
262144 29.95 µs (➖ 1.00x slower)
mshv3 / amd (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 923.80 ns (➖ 1.01x slower)
vec_bytes 709.26 ns (➖ 1.01x faster)
322.46 µs (➖ 1.03x slower)

payload_allocation

slot_pool_segmented
262144 680.15 ns (➖ 1.01x slower)
65536 182.52 ns (➖ 1.03x faster)

sandboxes

create_initialized_and_drop
medium 64.36 ms (➖ 1.00x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.93 ns (➖ 1.00x slower) 9.96 ns (➖ 1.01x slower) 9.92 ns (➖ 1.03x faster)

snapshot_files

load_snapshot_unverified
small 108.08 µs (➖ 1.01x faster)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 9.42 µs (➖ 1.00x slower) 9.63 µs (➖ 1.04x slower)
65536 2.65 µs (➖ 1.02x faster) 2.47 µs (➖ 1.00x faster)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.04 µs (➖ 1.02x faster) 8.07 µs (➖ 1.04x faster)
8192 1.35 µs (➖ 1.01x slower) 1.33 µs (➖ 1.01x slower)
262144 34.83 µs (➖ 1.03x faster)
mshv3 / intel (Linux) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 920.00 ns (➖ 1.02x slower)
vec_bytes 651.81 ns (➖ 1.00x faster)
627.57 µs (➖ 1.03x faster)

payload_allocation

slot_pool_segmented
262144 679.50 ns (➖ 1.04x slower)
65536 172.54 ns (➖ 1.00x slower)

sandboxes

create_initialized_and_drop
medium 72.52 ms (➖ 1.07x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
9.52 ns (➖ 1.00x faster) 9.65 ns (➖ 1.01x slower) 9.29 ns (➖ 1.00x faster)

snapshot_files

load_snapshot_unverified
small 56.25 µs (➖ 1.04x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.98 µs (➖ 1.00x faster) 7.94 µs (➖ 1.01x faster)
65536 2.26 µs (➖ 1.00x faster) 2.27 µs (➖ 1.01x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 7.58 µs (➖ 1.01x slower) 7.58 µs (➖ 1.01x faster)
8192 932.32 ns (➖ 1.05x faster) 943.37 ns (➖ 1.04x slower)
262144 37.26 µs (➖ 1.17x faster)
hyperv-ws2025 / amd (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.17 µs (➖ 1.06x faster)
vec_bytes 786.68 ns (➖ 1.02x faster)
1.93 ms (➖ 1.00x slower)

payload_allocation

slot_pool_segmented
262144 786.93 ns (➖ 1.00x slower)
65536 237.78 ns (➖ 1.03x slower)

sandboxes

create_initialized_and_drop
medium 9.72 ms (➖ 1.23x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
11.10 ns (➖ 1.01x faster) 10.21 ns (➖ 1.01x faster) 11.36 ns (➖ 1.06x slower)

snapshot_files

load_snapshot_unverified
small 776.47 µs (➖ 1.10x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 9.17 µs (➖ 1.05x faster) 9.24 µs (➖ 1.01x slower)
65536 2.44 µs (➖ 1.03x slower) 2.38 µs (➖ 1.01x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 9.55 µs (➖ 1.07x slower) 8.82 µs (➖ 1.02x faster)
8192 1.32 µs (➖ 1.06x faster) 1.34 µs (➖ 1.02x slower)
262144 39.88 µs (➖ 1.04x slower)
hyperv-ws2025 / intel (Windows) (➖ stable)

No benchmark improved or regressed.

Benchmark Results

function_call_codec

encode_control decode_vec_bytes_copy
byte_chunks 1.20 µs (➖ 1.02x slower)
vec_bytes 766.16 ns (➖ 1.03x faster)
2.89 ms (➖ 1.02x faster)

payload_allocation

slot_pool_segmented
262144 730.58 ns (➖ 1.01x faster)
65536 212.00 ns (➖ 1.01x slower)

sandboxes

create_initialized_and_drop
medium 13.07 ms (➖ 2.42x faster)

slot_pool

alloc_dealloc_1500 alloc_dealloc_4096 alloc_dealloc_128
10.22 ns (➖ 1.02x faster) 10.08 ns (➖ 1.01x slower) 10.04 ns (➖ 1.01x slower)

snapshot_files

load_snapshot_unverified
small 669.76 µs (➖ 1.08x slower)

virtq_readonly

slot_pool_segmented_fragmented slot_pool_segmented
262144 7.94 µs (➖ 1.04x slower) 7.86 µs (➖ 1.02x slower)
65536 2.50 µs (➖ 1.08x slower) 2.39 µs (➖ 1.02x slower)

virtq_readwrite

slot_pool_segmented_fragmented slot_pool_segmented
65536 8.50 µs (➖ 1.04x faster) 9.54 µs (➖ 1.12x slower)
8192 1.14 µs (➖ 1.09x faster) 1.11 µs (➖ 1.00x slower)
262144 41.30 µs (➖ 1.00x faster)

Reported by cargo ci bench-report --candidate run:38814039092 --config-file bench_report.toml.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants