Skip to content

FE-1314: Run experiment runs in parallel across workers - #9162

Open
kube wants to merge 2 commits into
cf/fe-1455-petrinaut-docs-site-tighter-chrome-and-more-room-for-contentfrom
cf/fe-1314-run-experiment-runs-in-parallel-across-workers
Open

FE-1314: Run experiment runs in parallel across workers#9162
kube wants to merge 2 commits into
cf/fe-1455-petrinaut-docs-site-tighter-chrome-and-more-room-for-contentfrom
cf/fe-1314-run-experiment-runs-in-parallel-across-workers

Conversation

@kube

@kube kube commented Aug 4, 2026

Copy link
Copy Markdown
Collaborator

🌟 What is the purpose of this PR?

Splits an experiment's runs across Web Workers: one worker per logical core minus one, capped at the run count. Measured on the SIR example with 2000 runs on a 10-core machine, 8 shards finish 4.1x faster than one.

This PR sits above #9268 in stack #9280, and #9177 builds on it.

🔗 Related links

  • FE-1314 (internal): this PR.
  • FE-948 (internal): sharding divides the quadratic enumeration cost across workers without removing it.

🔍 What does this change?

Three changes keep results reproducible at any shard count:

  • Per-run seeds derive from the run's global index, so run i gets the same seed whichever worker owns it.
  • A frame finalises once every shard that still has active runs has reported it, and a shard whose runs have all ended drops out of that requirement instead of blocking it.
  • Scalar metric frames carry their pre-reduction accumulator state, because frameValue is already reduced and the mean of per-shard means differs from the mean over all runs. The metric monoids (empty/merge) recombine the per-shard state.

Workers exchange plain messages: the app is not cross-origin isolated, so SharedArrayBuffer is unavailable, and independent runs need no shared memory. Hosts cap or pin parallelism with experimentShardCount on ExperimentsProvider, or shardCount on createMonteCarloExperiment.

Adds content/simulation/worker-sharding.mdx to the architecture docs.

Review fixes

  • The four shard helpers (createMonteCarloMetricShardMerger, planMonteCarloShards, getDefaultMonteCarloShardCount, MonteCarloShardPlanEntry) are no longer exported from the package barrels. The experiment runtime is their only consumer and imports them directly.
  • The changeset, the worker-sharding architecture page, and the user-facing experiments doc now scope the byte-identical claim: output is byte-identical while every shard has an active run. Once all of a shard's runs end early, its completed runs stop contributing samples to later frames, where a single simulator would keep sampling their frozen state. The divergence is real, and the docs record it as accepted behaviour.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • modifies an npm-publishable library and I have added a changeset file(s)

📜 Does this require a change to the docs?

The changes in this PR:

  • require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

  • Concurrent experiments each take the same number of workers, so they compete for cores.
  • Progress reports the slowest shard's position.

🛡 What tests cover this?

  • shard-plan.test.ts covers the split.
  • metrics/merge.test.ts covers monoid recombination.
  • experiment.test.ts covers frame finalisation and identical output at every shard count.
  • Benchmarks live in benchmarks/sharded-experiment*.mjs.

❓ How to test this?

  1. Run an experiment with a few thousand runs. CPU use spans several cores.
  2. Re-run with the same seed at experimentShardCount 1, then unset. The distributions match.

🤖 Generated with Claude Code

@kube kube self-assigned this Aug 4, 2026
@vercel

vercel Bot commented Aug 4, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Aug 22, 2026 3:04am
petrinaut Ready Ready Preview Aug 22, 2026 3:04am
petrinaut-docs Ready Ready Preview Aug 22, 2026 3:04am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Aug 22, 2026 3:04am

@github-actions github-actions Bot added area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team labels Aug 4, 2026
@vercel
vercel Bot temporarily deployed to Preview – petrinaut August 4, 2026 23:02 Inactive
@github-actions github-actions Bot added the area/deps Relates to third-party dependencies (area) label Aug 5, 2026
Comment thread libs/@hashintel/petrinaut-core/src/webgpu/emit-wgsl.ts Fixed
@github-actions github-actions Bot added the area/apps > hash.design Affects the `hash.design` design site (app) label Aug 5, 2026
@kube
kube force-pushed the cf/fe-1314-run-experiment-runs-in-parallel-across-workers branch from 986094b to be4731d Compare August 13, 2026 08:20
@kube
kube changed the base branch from main to cf/fe-1322-arch-docs-site August 13, 2026 08:20
@github-actions github-actions Bot added area/apps type/legal Owned by the @legal team type/eng > backend Owned by the @backend team and removed area/apps type/legal Owned by the @legal team labels Aug 13, 2026
@kube
kube deployed to pull-request August 13, 2026 08:21 — with GitHub Actions Active
@kube
kube deployed to pull-request August 13, 2026 08:21 — with GitHub Actions Active
@semgrep-code-hashintel

Copy link
Copy Markdown

Semgrep found 2 detect-non-literal-regexp findings:

RegExp() called with a overrides function argument, this might allow an attacker to cause a Regular Expression Denial-of-Service (ReDoS) within your application as RegExP blocks the main thread. For this reason, it is recommended to use hardcoded regexes instead. If your regex is run on user-controlled input, consider performing input validation or use a regex checking/sanitization library such as https://www.npmjs.com/package/recheck to verify that the regex does not appear vulnerable to ReDoS.

View Dataflow Graph
flowchart LR
    classDef invis fill:white, stroke: none
    classDef default fill:#e7f5ff, color:#1c7fd6, stroke: none

    subgraph File0["<b>libs/@local/petrinaut-arch-docs/src/scope.test.ts</b>"]
        direction LR
        %% Source

        subgraph Source
            direction LR

            v0["<a href=https://github.com/hashintel/hash/blob/be4731ded37213256e4c2f509ba31a8745a24dd7/libs/@local/petrinaut-arch-docs/src/scope.test.ts#L19 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 19] overrides</a>"]
        end
        %% Intermediate

        subgraph Traces0[Traces]
            direction TB

            v2["<a href=https://github.com/hashintel/hash/blob/be4731ded37213256e4c2f509ba31a8745a24dd7/libs/@local/petrinaut-arch-docs/src/scope.test.ts#L19 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 19] overrides</a>"]

            v3["<a href=https://github.com/hashintel/hash/blob/be4731ded37213256e4c2f509ba31a8745a24dd7/libs/@local/petrinaut-arch-docs/src/scope.test.ts#L56 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 56] pkg</a>"]
        end
            v2 --> v3
        %% Sink

        subgraph Sink
            direction LR

            v1["<a href=https://github.com/hashintel/hash/blob/be4731ded37213256e4c2f509ba31a8745a24dd7/libs/@local/petrinaut-arch-docs/src/scope.test.ts#L55 target=_blank style='text-decoration:none; color:#1c7fd6'>[Line: 55] new RegExp(<br>      sourceRootPattern([pkg({ path: &quot;libs/@scope/a.b&quot; })]),<br>      &quot;u&quot;,<br>    )</a>"]
        end
    end
    %% Class Assignment
    Source:::invis
    Sink:::invis

    Traces0:::invis
    File0:::invis

    %% Connections

    Source --> Traces0
    Traces0 --> Sink

Loading

@codspeed-hq

codspeed-hq Bot commented Aug 13, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 15.38%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 2 regressed benchmarks
✅ 96 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
bit_matrix/dense/iter_row[64] 140.8 ns 170 ns -17.16%
bit_matrix/dense/iter_row[200] 185.8 ns 215 ns -13.57%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing cf/fe-1314-run-experiment-runs-in-parallel-across-workers (64fbc46) with main (1f543d0)1

Open in CodSpeed

Footnotes

  1. No successful run was found on cf/fe-1414-arch-docs-support-python-packages-layers-and-docstring (e9619cf) during the generation of this report, so main (1f543d0) was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark results

@rust/hash-graph-benches – Integrations

policy_resolution_large

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2002 $$28.0 \mathrm{ms} \pm 227 \mathrm{μs}\left({\color{gray}-0.367 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.47 \mathrm{ms} \pm 25.4 \mathrm{μs}\left({\color{gray}0.475 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 1002 $$13.3 \mathrm{ms} \pm 120 \mathrm{μs}\left({\color{gray}3.25 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 3314 $$44.3 \mathrm{ms} \pm 323 \mathrm{μs}\left({\color{gray}1.16 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$14.6 \mathrm{ms} \pm 123 \mathrm{μs}\left({\color{gray}2.58 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 1527 $$25.1 \mathrm{ms} \pm 210 \mathrm{μs}\left({\color{gray}3.43 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 2078 $$29.5 \mathrm{ms} \pm 248 \mathrm{μs}\left({\color{gray}2.84 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.73 \mathrm{ms} \pm 20.4 \mathrm{μs}\left({\color{gray}0.127 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 1033 $$14.2 \mathrm{ms} \pm 124 \mathrm{μs}\left({\color{gray}1.54 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_medium

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 102 $$3.82 \mathrm{ms} \pm 25.9 \mathrm{μs}\left({\color{gray}0.126 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$3.00 \mathrm{ms} \pm 17.4 \mathrm{μs}\left({\color{gray}-1.029 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 52 $$3.39 \mathrm{ms} \pm 22.9 \mathrm{μs}\left({\color{gray}-1.716 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 269 $$5.14 \mathrm{ms} \pm 28.9 \mathrm{μs}\left({\color{gray}-1.726 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$3.57 \mathrm{ms} \pm 27.3 \mathrm{μs}\left({\color{gray}-0.033 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 108 $$4.14 \mathrm{ms} \pm 27.8 \mathrm{μs}\left({\color{gray}-0.532 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 133 $$4.36 \mathrm{ms} \pm 24.5 \mathrm{μs}\left({\color{gray}-2.312 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$3.44 \mathrm{ms} \pm 24.6 \mathrm{μs}\left({\color{gray}-0.687 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 63 $$4.13 \mathrm{ms} \pm 34.7 \mathrm{μs}\left({\color{gray}-0.512 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_none

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 2 $$2.71 \mathrm{ms} \pm 18.6 \mathrm{μs}\left({\color{gray}3.78 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.53 \mathrm{ms} \pm 13.8 \mathrm{μs}\left({\color{gray}0.772 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 2 $$2.68 \mathrm{ms} \pm 19.5 \mathrm{μs}\left({\color{gray}2.96 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 8 $$3.01 \mathrm{ms} \pm 22.4 \mathrm{μs}\left({\color{red}5.27 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.74 \mathrm{ms} \pm 17.0 \mathrm{μs}\left({\color{gray}3.76 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 3 $$3.02 \mathrm{ms} \pm 18.4 \mathrm{μs}\left({\color{red}6.00 \mathrm{\%}}\right) $$ Flame Graph

policy_resolution_small

Function Value Mean Flame graphs
resolve_policies_for_actor user: empty, selectivity: high, policies: 52 $$3.04 \mathrm{ms} \pm 22.3 \mathrm{μs}\left({\color{gray}-1.375 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: low, policies: 1 $$2.75 \mathrm{ms} \pm 17.0 \mathrm{μs}\left({\color{gray}0.086 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: empty, selectivity: medium, policies: 26 $$2.99 \mathrm{ms} \pm 19.5 \mathrm{μs}\left({\color{gray}2.53 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: high, policies: 94 $$3.44 \mathrm{ms} \pm 22.7 \mathrm{μs}\left({\color{gray}-1.454 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: low, policies: 1 $$2.97 \mathrm{ms} \pm 17.7 \mathrm{μs}\left({\color{gray}-1.651 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: seeded, selectivity: medium, policies: 27 $$3.30 \mathrm{ms} \pm 21.3 \mathrm{μs}\left({\color{gray}1.15 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: high, policies: 66 $$3.37 \mathrm{ms} \pm 21.6 \mathrm{μs}\left({\color{gray}-1.365 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: low, policies: 1 $$2.91 \mathrm{ms} \pm 17.5 \mathrm{μs}\left({\color{gray}-2.569 \mathrm{\%}}\right) $$ Flame Graph
resolve_policies_for_actor user: system, selectivity: medium, policies: 29 $$3.31 \mathrm{ms} \pm 20.3 \mathrm{μs}\left({\color{gray}-0.901 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_complete

Function Value Mean Flame graphs
entity_by_id;one_depth 1 entities $$44.1 \mathrm{ms} \pm 284 \mathrm{μs}\left({\color{gray}4.76 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 10 entities $$33.3 \mathrm{ms} \pm 211 \mathrm{μs}\left({\color{gray}-0.507 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 25 entities $$36.0 \mathrm{ms} \pm 243 \mathrm{μs}\left({\color{gray}1.34 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 5 entities $$34.1 \mathrm{ms} \pm 317 \mathrm{μs}\left({\color{red}6.46 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;one_depth 50 entities $$44.1 \mathrm{ms} \pm 291 \mathrm{μs}\left({\color{red}5.36 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 1 entities $$50.5 \mathrm{ms} \pm 391 \mathrm{μs}\left({\color{gray}1.86 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 10 entities $$40.6 \mathrm{ms} \pm 285 \mathrm{μs}\left({\color{gray}0.651 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 25 entities $$94.3 \mathrm{ms} \pm 529 \mathrm{μs}\left({\color{gray}3.90 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 5 entities $$34.2 \mathrm{ms} \pm 216 \mathrm{μs}\left({\color{gray}0.882 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;two_depth 50 entities $$308 \mathrm{ms} \pm 1.09 \mathrm{ms}\left({\color{gray}1.59 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 1 entities $$11.2 \mathrm{ms} \pm 80.1 \mathrm{μs}\left({\color{gray}2.48 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 10 entities $$11.2 \mathrm{ms} \pm 59.5 \mathrm{μs}\left({\color{gray}0.568 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 25 entities $$11.2 \mathrm{ms} \pm 67.5 \mathrm{μs}\left({\color{gray}1.54 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 5 entities $$11.4 \mathrm{ms} \pm 98.8 \mathrm{μs}\left({\color{gray}2.92 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id;zero_depth 50 entities $$11.2 \mathrm{ms} \pm 64.1 \mathrm{μs}\left({\color{gray}1.44 \mathrm{\%}}\right) $$ Flame Graph

read_scaling_linkless

Function Value Mean Flame graphs
entity_by_id 1 entities $$11.0 \mathrm{ms} \pm 66.9 \mathrm{μs}\left({\color{gray}-0.256 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10 entities $$11.1 \mathrm{ms} \pm 67.4 \mathrm{μs}\left({\color{gray}-0.514 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 100 entities $$11.2 \mathrm{ms} \pm 76.3 \mathrm{μs}\left({\color{gray}0.764 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 1000 entities $$11.6 \mathrm{ms} \pm 93.9 \mathrm{μs}\left({\color{gray}3.41 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id 10000 entities $$11.2 \mathrm{ms} \pm 71.3 \mathrm{μs}\left({\color{gray}-3.033 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity

Function Value Mean Flame graphs
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1 $$11.5 \mathrm{ms} \pm 71.1 \mathrm{μs}\left({\color{gray}-1.805 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1 $$11.6 \mathrm{ms} \pm 68.9 \mathrm{μs}\left({\color{gray}-1.861 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1 $$11.7 \mathrm{ms} \pm 86.5 \mathrm{μs}\left({\color{gray}0.087 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1 $$11.6 \mathrm{ms} \pm 72.1 \mathrm{μs}\left({\color{gray}-2.120 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2 $$11.8 \mathrm{ms} \pm 87.5 \mathrm{μs}\left({\color{gray}1.25 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1 $$11.6 \mathrm{ms} \pm 82.7 \mathrm{μs}\left({\color{gray}-0.270 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1 $$12.1 \mathrm{ms} \pm 82.5 \mathrm{μs}\left({\color{gray}2.99 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1 $$11.6 \mathrm{ms} \pm 66.4 \mathrm{μs}\left({\color{gray}-0.153 \mathrm{\%}}\right) $$ Flame Graph
entity_by_id entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1 $$11.5 \mathrm{ms} \pm 71.8 \mathrm{μs}\left({\color{gray}-3.697 \mathrm{\%}}\right) $$ Flame Graph

representative_read_entity_type

Function Value Mean Flame graphs
get_entity_type_by_id Account ID: bf5a9ef5-dc3b-43cf-a291-6210c0321eba $$8.62 \mathrm{ms} \pm 51.3 \mathrm{μs}\left({\color{gray}-1.300 \mathrm{\%}}\right) $$ Flame Graph

representative_read_multiple_entities

Function Value Mean Flame graphs
entity_by_property traversal_paths=0 0 $$57.0 \mathrm{ms} \pm 528 \mathrm{μs}\left({\color{lightgreen}-9.805 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$109 \mathrm{ms} \pm 634 \mathrm{μs}\left({\color{gray}-4.530 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$64.5 \mathrm{ms} \pm 589 \mathrm{μs}\left({\color{lightgreen}-6.838 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$72.9 \mathrm{ms} \pm 461 \mathrm{μs}\left({\color{lightgreen}-7.994 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$84.5 \mathrm{ms} \pm 601 \mathrm{μs}\left({\color{gray}-4.907 \mathrm{\%}}\right) $$
entity_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$87.4 \mathrm{ms} \pm 703 \mathrm{μs}\left({\color{lightgreen}-6.605 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=0 0 $$44.3 \mathrm{ms} \pm 285 \mathrm{μs}\left({\color{gray}-0.431 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=255 1,resolve_depths=inherit:1;values:255;properties:255;links:127;link_dests:126;type:true $$71.5 \mathrm{ms} \pm 480 \mathrm{μs}\left({\color{gray}-2.027 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:0;link_dests:0;type:false $$48.8 \mathrm{ms} \pm 326 \mathrm{μs}\left({\color{gray}-3.040 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:0;links:1;link_dests:0;type:true $$57.2 \mathrm{ms} \pm 388 \mathrm{μs}\left({\color{gray}-3.338 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:0;properties:2;links:1;link_dests:0;type:true $$61.8 \mathrm{ms} \pm 377 \mathrm{μs}\left({\color{gray}0.893 \mathrm{\%}}\right) $$
link_by_source_by_property traversal_paths=2 1,resolve_depths=inherit:0;values:2;properties:2;links:1;link_dests:0;type:true $$62.0 \mathrm{ms} \pm 525 \mathrm{μs}\left({\color{gray}0.330 \mathrm{\%}}\right) $$

scenarios

Function Value Mean Flame graphs
full_test query-limited $$121 \mathrm{ms} \pm 716 \mathrm{μs}\left({\color{red}7.09 \mathrm{\%}}\right) $$ Flame Graph
full_test query-unlimited $$133 \mathrm{ms} \pm 644 \mathrm{μs}\left({\color{red}6.20 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-limited $$20.4 \mathrm{ms} \pm 120 \mathrm{μs}\left({\color{lightgreen}-17.532 \mathrm{\%}}\right) $$ Flame Graph
linked_queries query-unlimited $$518 \mathrm{ms} \pm 1.41 \mathrm{ms}\left({\color{gray}2.21 \mathrm{\%}}\right) $$ Flame Graph

@kube
kube force-pushed the cf/fe-1314-run-experiment-runs-in-parallel-across-workers branch from be4731d to c78aad4 Compare August 13, 2026 16:39
@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (cf/fe-1455-petrinaut-docs-site-tighter-chrome-and-more-room-for-content@85722ed). Learn more about missing BASE report.

Additional details and impacted files
@@                                            Coverage Diff                                             @@
##             cf/fe-1455-petrinaut-docs-site-tighter-chrome-and-more-room-for-content    #9162   +/-   ##
==========================================================================================================
  Coverage                                                                           ?   59.71%           
==========================================================================================================
  Files                                                                              ?     1423           
  Lines                                                                              ?   138862           
  Branches                                                                           ?     6572           
==========================================================================================================
  Hits                                                                               ?    82915           
  Misses                                                                             ?    54881           
  Partials                                                                           ?     1066           
Flag Coverage Δ
apps.hash-api 14.66% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@cursor

cursor Bot commented Aug 21, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches the experiment runtime, seed derivation, and metric aggregation path, so bugs can silently change reported statistics. Covered by unit tests and a result-fingerprint benchmark, with a documented early-finish sampling caveat.

Overview
Experiments no longer run every seed in a single worker. createMonteCarloExperiment fans runs across one worker per logical core minus one (capped at run count), with optional shardCount / experimentShardCount to pin or cap that.

Seeds use a global runIndexOffset so run i is the same whichever shard owns it. Scalar frames now carry pre-reduction runAggregate (a mean of means is not a mean); the new shard merger recombines monoids, finalises a frame only after every still-running shard has reported it, and drops finished shards from that watermark. Progress follows the slowest live shard. One shard error or cancel tears the rest down.

Caveat: once every run on a shard ends early (e.g. deadlock), those runs stop contributing to later frames, unlike a single simulator that keeps sampling frozen state.

Also adds shard-plan/merge tests, user and architecture docs, and non-CI throughput/sharding benchmarks plus a long performance write-up. Concurrent experiments still each spawn their own worker set.

Reviewed by Cursor Bugbot for commit d316d8b. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3dc86d1. Configure here.

kube added 2 commits August 22, 2026 02:53
An experiment ran every run in a single worker, using one core however
many the machine had. Runs are independent, so they now split across
one worker per logical core minus one, capped at the run count —
measured at ~4x on 8 shards.

Sharding cannot change results: seeds derive from the run's global
index rather than its position in a shard, and per-frame statistics
recombine through the metric accumulator monoids, so output is
byte-identical at every shard count. Scalar frames carry their
pre-reduction accumulator state, since a mean of means is not a mean.
Replaces em dashes with commas, colons or full stops in
worker-sharding.mdx and the changeset. No content change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infra Relates to version control, CI, CD or IaC (area) area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

3 participants