FE-1408: Run a trial as multiple seeded simulations aggregated by mean

[kube]

🌟 What is the purpose of this PR?

The CLI scores an optimization trial on one simulation, so a stochastic net is judged on a single sample. This PR runs each trial as execution.seedsPerTrial seeded simulations and returns their mean.

• Runs execute sequentially. Parallel execution comes later through the shared experiment-backend interface (FE-1341), tracked as FE-1429, so the CLI and the editor share one worker implementation.
• The response keeps the { objective } shape, so the Python optimizer service needs no changes.
• Out of scope: aggregation-method selection (FE-1277) and Optuna-side per-seed handling (FE-1281).

Stack #9226: FE-1410 (contract) → FE-1411 (async protocol) → this PR → FE-1413 (arch-docs manual) → FE-1270 (Python bindings) → FE-1412 (Python timeout).

🔗 Related links

• FE-1408 (internal): this PR
• FE-1273 (internal): parent, M seeded runs per Optuna step
• FE-1341 (internal): the shared experiment-backend interface
• FE-1429 (internal): follow-up, run these replicates through that backend
• FE-1277 (internal): follow-up, user-selectable aggregation

🔍 What does this change?

@hashintel/petrinaut-cli only:

• optimization.evaluate runs the trial's seeds one after another and returns:
  • objective: the mean of the per-seed objectives;
  • replicates: [{ seed, objective }], present when seedsPerTrial > 1.
• optimization.describe reports study.seedsPerTrial.
• Seeds derive once per study and are identical for every trial (common random numbers):
  • replicate 0 uses execution.seed, so one seed per trial behaves bit-identically to today and one editor run still reproduces a replicate;
  • replicate i uses the Monte Carlo deriveRunSeed(seed, i) exported by FE-1410.
• Wall-clock time of a trial grows linearly with seedsPerTrial.
• New smoke test spawns the bundled dist/cli.js and checks a full describe/evaluate exchange, including a two-seed trial.

🧪 Example

The manifest's execution block asks for three seeds per trial:

// optimization.json (excerpt)
"execution": { "seed": 42, "dt": 1, "maxTime": 100, "seedsPerTrial": 3 }

Serve it and send one trial over stdio:

node libs/@hashintel/petrinaut-cli/dist/cli.js serve --optimization optimization.json --stdio

{ "id": 1, "method": "optimization.evaluate", "params": { "parameterValues": { "infected_ratio": 0.1 } } }

The trial runs seeds 42, 1013904268 and 1520856381 in that order and answers with their mean:

{
  "id": 1,
  "result": {
    "objective": 0.31,
    "replicates": [
      { "seed": 42, "objective": 0.28 },
      { "seed": 1013904268, "objective": 0.33 },
      { "seed": 1520856381, "objective": 0.32 }
    ]
  }
}

Every trial reuses those three seeds, so repeating the request returns identical values. With seedsPerTrial: 1 (or the field omitted) the response is { "objective": ... } alone, unchanged from today.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• modifies a workspace but not a publishable library (@hashintel/petrinaut-cli is private; the @hashintel/petrinaut-core changeset landed with FE-1410)

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR
  • OPTIMIZATION_INTEGRATION.md and the CLI README document the field, the seed derivation, and the response shape. The in-app user guide is untouched: the UI never sets seedsPerTrial, so in-app behaviour is unchanged until FE-1273's UI half lands.

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• affected the execution graph, and the turbo.json's have been updated to reflect this
  • petrinaut-cli's test:unit now depends on its own build, so the smoke test always runs against a fresh bundle in CI.

🐾 Next steps

• FE-1412 (top of stack): the Python client scales its per-response timeout by seedsPerTrial.
• FE-1429: run the seeded replicates through the shared experiment backend once FE-1341 lands.
• FE-1277 / FE-1281: aggregation selection and Optuna-side per-seed handling.

🛡 What tests cover this?

• optimization.test.ts: mean aggregation, per-seed replicates, identical seed list across trials, fail-fast on a non-finite replicate, extreme-value aggregation, deriveTrialSeeds range and stability.
• transports.test.ts: multi-seed evaluate over stdio, seedsPerTrial in describe.
• built-cli.test.ts: spawns the real bundled CLI for the full exchange with two seeds.

❓ How to test this?

1. turbo run test:unit --filter @hashintel/petrinaut-cli
2. Manual: add "seedsPerTrial": 4 to execution in libs/@hashintel/petrinaut-cli/examples/supply-chain-profit-optimization.json, run yarn workspace @hashintel/petrinaut-cli build, then node libs/@hashintel/petrinaut-cli/dist/cli.js serve --optimization <manifest> --stdio and send an optimization.evaluate request.
3. The response carries the mean objective plus four replicates; repeating the request returns identical values.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview	Aug 19, 2026 10:46am
petrinaut	Ready	Preview	Aug 19, 2026 10:46am
petrinaut-docs	Ready	Preview	Aug 19, 2026 10:46am

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Aug 19, 2026 10:46am

[cursor]

PR Summary

Medium Risk
Changes how optimization objectives are computed for stochastic models and multiplies per-trial runtime linearly with seedsPerTrial; backward compatible when seedsPerTrial is omitted or 1.

Overview
Each optimization trial can run execution.seedsPerTrial (default 1) simulations instead of a single run. optimization.evaluate runs those seeds sequentially, returns objective as the mean of per-seed results, and when seedsPerTrial > 1 adds replicates: [{ seed, objective }]. With one seed, the response stays { objective } only.

Seeds are fixed for the whole study via deriveTrialSeeds (first replicate uses execution.seed; later ones use core’s deriveRunSeed) so every trial shares the same sequence (common random numbers). optimization.describe now includes study.seedsPerTrial. Evaluation fails fast on non-finite replicate objectives and uses an online mean to avoid overflow.

Docs (OPTIMIZATION_INTEGRATION.md, README) describe the field, seed rule, and response shape. Tests cover aggregation, CRN, stdio transport, and a bundled dist/cli.js smoke test; turbo.json makes test:unit depend on build.

^{Reviewed by Cursor Bugbot for commit c44a97d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

Adds deterministic multi-seed optimization trials to the Petrinaut CLI, aggregating replicate objectives by mean.

Changes:

• Runs sequential simulations using reusable derived seeds.
• Returns mean objectives and per-seed replicate results.
• Adds protocol, bundle, and unit coverage plus documentation.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
turbo.json	Builds CLI before unit tests.
optimization.ts	Implements seeded trial evaluation and aggregation.
optimization.test.ts	Tests seeds, aggregation, and validation.
transports.test.ts	Tests multi-seed stdio responses.
built-cli.test.ts	Adds bundled CLI smoke test.
README.md	Documents multi-seed evaluation.
OPTIMIZATION_INTEGRATION.md	Documents protocol and seed derivation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated no new comments.