fix(divan): support threaded walltime benchmarks - #189
Conversation
Exercise Divan native threads with four workers in the walltime benchmark suite. Keep the simulation compatibility path single-threaded because its benchmark options do not expose the native threads setting.
Serialize the process-wide instrument-hooks transport across all Rust FFI handles. Aggregate worker timestamps into one benchmark marker envelope per Divan sampling round so benchmark bodies remain parallel without corrupting FIFO requests.
Merging this PR will degrade performance by 0.2%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | iter_batched_large_input |
5 ns | 8 ns | -37.5% |
| ❌ | WallTime | graph_coloring[5] |
1 µs | 1.5 µs | -29.12% |
| ❌ | Simulation | count_set_bits[0] |
170.3 ns | 225.9 ns | -24.59% |
| ❌ | Simulation | count_set_bits[1024] |
171.6 ns | 227.2 ns | -24.47% |
| ❌ | Simulation | count_set_bits[42] |
173.5 ns | 229 ns | -24.27% |
| ❌ | Simulation | count_set_bits[255] |
178 ns | 233.6 ns | -23.79% |
| ❌ | Simulation | count_set_bits[65535] |
185.1 ns | 240.7 ns | -23.09% |
| ❌ | Simulation | add_two_integers[(42, 13)] |
288 ns | 343.5 ns | -16.18% |
| ❌ | Simulation | add_two_integers[(255, 255)] |
288 ns | 343.5 ns | -16.18% |
| ❌ | Simulation | add_two_integers[(1024, -1024)] |
349.4 ns | 405 ns | -13.72% |
| ❌ | Simulation | add_two_integers[(65535, -65535)] |
361.1 ns | 416.7 ns | -13.34% |
| ❌ | Simulation | recursive_memoized[BTreeMap<u64, u64>, 0] |
771.4 ns | 880.8 ns | -12.42% |
| ❌ | WallTime | n_queens_solver[4] |
2.1 µs | 2.3 µs | -11.29% |
| ❌ | WallTime | recursive_memoized[HashMap<u64, u64>, 0] |
16 ns | 18 ns | -11.11% |
| ❌ | Simulation | iter_manual_simple |
839.4 ns | 894.9 ns | -6.21% |
| ❌ | Simulation | recursive_memoized[BTreeMap<u64, u64>, 5] |
4.5 µs | 4.8 µs | -5.8% |
| ❌ | WallTime | iterative[20] |
39 ns | 41 ns | -4.88% |
| ❌ | Simulation | recursive_memoized[BTreeMap<u64, u64>, 10] |
5.5 µs | 5.8 µs | -4.79% |
| ❌ | Simulation | b |
4.8 µs | 5 µs | -4.42% |
| ❌ | WallTime | graph_coloring[4] |
980 ns | 1,022 ns | -4.11% |
| ... | ... | ... | ... | ... | ... |
ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing cod-3324-codspeed-runner-hangs-when-using-divans-threads-feature (224cf47) with main (69f74b6)
Unable to generate the flame graphsThe performance report has correctly been generated, but there was an internal error while generating the flame graphs for this run. We're working on fixing the issue. Feel free to contact us on Discord or at support@codspeed.io if the issue persists. |
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
f6b60a3 to
224cf47
Compare
Greptile SummaryThe PR enables threaded Divan walltime benchmarks without concurrent writes to the process-wide instrument-hooks transport.
Confidence Score: 5/5The PR appears safe to merge, with no actionable correctness or security issues identified. The transport is serialized without enclosing benchmark execution, worker samples are fully collected before aggregation, and the runner accepts timestamped markers independently of their delivery time.
|
| Filename | Overview |
|---|---|
| crates/codspeed/src/instrument_hooks/mod.rs | Adds process-wide serialization around Linux instrument-hooks FFI operations and keeps each timestamp pair adjacent. |
| crates/divan_compat/divan_fork/src/bench/mod.rs | Moves marker transport out of worker threads and emits aggregate bounds after each parallel sample round. |
| crates/divan_compat/divan_fork/src/stats/sample.rs | Extends internal raw samples with the monotonic benchmark bounds required for coordinator aggregation. |
| crates/divan_compat/benches/thread_example.rs | Adds a four-worker benchmark on the native Divan path while retaining single-threaded simulation compatibility. |
Sequence Diagram
sequenceDiagram
participant C as Divan coordinator
participant W as Worker threads
participant H as InstrumentHooks
participant R as Runner FIFO
C->>W: Start parallel sampling round
par Worker samples
W->>W: Capture monotonic start
W->>W: Run benchmark iterations
W->>W: Capture monotonic end
end
W-->>C: Return samples and timestamp bounds
C->>C: Select minimum start and maximum end
C->>H: Add aggregate benchmark timestamps
H->>H: Acquire process-wide transport lock
H->>R: BenchmarkStart(minimum start)
H->>R: BenchmarkEnd(maximum end)
H->>H: Release transport lock
Reviews (1): Last reviewed commit: "fix(divan): support threaded walltime be..." | Re-trigger Greptile
Prevent threaded Divan walltime benchmarks from concurrently writing to the process-wide instrument-hooks FIFOs.
The Rust FFI boundary now serializes every transport operation across all handles. Divan workers record monotonic bounds locally, then the coordinator emits one marker envelope for the complete parallel sampling round. Benchmark bodies remain parallel.
A native four-worker benchmark keeps this path covered by the existing walltime integration matrix. The simulation compatibility path remains single-threaded because it does not expose Divan's native thread option.