Skip to content

fix(divan): support threaded walltime benchmarks - #189

Open
not-matthias wants to merge 2 commits into
mainfrom
cod-3324-codspeed-runner-hangs-when-using-divans-threads-feature
Open

fix(divan): support threaded walltime benchmarks#189
not-matthias wants to merge 2 commits into
mainfrom
cod-3324-codspeed-runner-hangs-when-using-divans-threads-feature

Conversation

@not-matthias

Copy link
Copy Markdown
Member

Prevent threaded Divan walltime benchmarks from concurrently writing to the process-wide instrument-hooks FIFOs.

The Rust FFI boundary now serializes every transport operation across all handles. Divan workers record monotonic bounds locally, then the coordinator emits one marker envelope for the complete parallel sampling round. Benchmark bodies remain parallel.

A native four-worker benchmark keeps this path covered by the existing walltime integration matrix. The simulation compatibility path remains single-threaded because it does not expose Divan's native thread option.

Exercise Divan native threads with four workers in the walltime benchmark suite. Keep the simulation compatibility path single-threaded because its benchmark options do not expose the native threads setting.
Serialize the process-wide instrument-hooks transport across all Rust FFI handles. Aggregate worker timestamps into one benchmark marker envelope per Divan sampling round so benchmark bodies remain parallel without corrupting FIFO requests.
@codspeed-hq

codspeed-hq Bot commented Sep 1, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 0.2%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 25 improved benchmarks
❌ 22 regressed benchmarks
✅ 544 untouched benchmarks
🆕 3 new benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
WallTime iter_batched_large_input 5 ns 8 ns -37.5%
WallTime graph_coloring[5] 1 µs 1.5 µs -29.12%
Simulation count_set_bits[0] 170.3 ns 225.9 ns -24.59%
Simulation count_set_bits[1024] 171.6 ns 227.2 ns -24.47%
Simulation count_set_bits[42] 173.5 ns 229 ns -24.27%
Simulation count_set_bits[255] 178 ns 233.6 ns -23.79%
Simulation count_set_bits[65535] 185.1 ns 240.7 ns -23.09%
Simulation add_two_integers[(42, 13)] 288 ns 343.5 ns -16.18%
Simulation add_two_integers[(255, 255)] 288 ns 343.5 ns -16.18%
Simulation add_two_integers[(1024, -1024)] 349.4 ns 405 ns -13.72%
Simulation add_two_integers[(65535, -65535)] 361.1 ns 416.7 ns -13.34%
Simulation recursive_memoized[BTreeMap<u64, u64>, 0] 771.4 ns 880.8 ns -12.42%
WallTime n_queens_solver[4] 2.1 µs 2.3 µs -11.29%
WallTime recursive_memoized[HashMap<u64, u64>, 0] 16 ns 18 ns -11.11%
Simulation iter_manual_simple 839.4 ns 894.9 ns -6.21%
Simulation recursive_memoized[BTreeMap<u64, u64>, 5] 4.5 µs 4.8 µs -5.8%
WallTime iterative[20] 39 ns 41 ns -4.88%
Simulation recursive_memoized[BTreeMap<u64, u64>, 10] 5.5 µs 5.8 µs -4.79%
Simulation b 4.8 µs 5 µs -4.42%
WallTime graph_coloring[4] 980 ns 1,022 ns -4.11%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cod-3324-codspeed-runner-hangs-when-using-divans-threads-feature (224cf47) with main (69f74b6)

Open in CodSpeed

@codspeed-hq

codspeed-hq Bot commented Sep 1, 2026

Copy link
Copy Markdown

Unable to generate the flame graphs

The performance report has correctly been generated, but there was an internal error while generating the flame graphs for this run. We're working on fixing the issue. Feel free to contact us on Discord or at support@codspeed.io if the issue persists.

@avalanche-staging

avalanche-staging Bot commented Sep 2, 2026

Copy link
Copy Markdown

Congrats! CodSpeed is installed 🎉

🆕 622 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Open in CodSpeed

@not-matthias
not-matthias force-pushed the cod-3324-codspeed-runner-hangs-when-using-divans-threads-feature branch from f6b60a3 to 224cf47 Compare September 2, 2026 13:42
@not-matthias
not-matthias marked this pull request as ready for review September 2, 2026 13:42
@greptile-apps

greptile-apps Bot commented Sep 2, 2026

Copy link
Copy Markdown

Greptile Summary

The PR enables threaded Divan walltime benchmarks without concurrent writes to the process-wide instrument-hooks transport.

  • Serializes all Linux instrument-hooks FFI transport operations across handles.
  • Collects per-worker monotonic bounds and emits one aggregate marker envelope per parallel sampling round.
  • Adds a four-worker native Divan benchmark to the walltime integration fixture.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable correctness or security issues identified.

The transport is serialized without enclosing benchmark execution, worker samples are fully collected before aggregation, and the runner accepts timestamped markers independently of their delivery time.

Important Files Changed

Filename Overview
crates/codspeed/src/instrument_hooks/mod.rs Adds process-wide serialization around Linux instrument-hooks FFI operations and keeps each timestamp pair adjacent.
crates/divan_compat/divan_fork/src/bench/mod.rs Moves marker transport out of worker threads and emits aggregate bounds after each parallel sample round.
crates/divan_compat/divan_fork/src/stats/sample.rs Extends internal raw samples with the monotonic benchmark bounds required for coordinator aggregation.
crates/divan_compat/benches/thread_example.rs Adds a four-worker benchmark on the native Divan path while retaining single-threaded simulation compatibility.

Sequence Diagram

sequenceDiagram
  participant C as Divan coordinator
  participant W as Worker threads
  participant H as InstrumentHooks
  participant R as Runner FIFO
  C->>W: Start parallel sampling round
  par Worker samples
    W->>W: Capture monotonic start
    W->>W: Run benchmark iterations
    W->>W: Capture monotonic end
  end
  W-->>C: Return samples and timestamp bounds
  C->>C: Select minimum start and maximum end
  C->>H: Add aggregate benchmark timestamps
  H->>H: Acquire process-wide transport lock
  H->>R: BenchmarkStart(minimum start)
  H->>R: BenchmarkEnd(maximum end)
  H->>H: Release transport lock
Loading

Reviews (1): Last reviewed commit: "fix(divan): support threaded walltime be..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant