perf(core): retain workflow VM across inline steps (primitives-gated) - #3046
Merged
Conversation
Combines the retained-session architecture from #2984 with the env kill switch and loop-level single-VM test from #2966. - executeWorkflow with discriminated request/result types and a WorkflowSession state machine (running/suspended/failed/replay/completed) - EventsConsumer.append: only newly durable events feed the live VM - WORKFLOW_RETAINED_VM=0 kill switch (default on) - retained-vm-loop.test.ts: proves one VM per run and byte-identical output vs the from-scratch replay path
- executeWorkflow overloads: a fresh replay request can no longer return
{ type: 'replay' }, deleting the runtime invariant throw and
runWorkflow's dead branch
- isSameSuspensionBoundary reduced to the steps-array comparison (all
suspension counts are derived from steps in the constructor)
- runtime loop initializes workflowResult with a ternary
crypto.subtle.digest is the only sandbox API whose promise resolves on host timing rather than from the event log, so a workflow racing it against a step can advance while suspended and diverge from what replay reconstructs. A sticky usedHostAsync bit on the VM context makes canRetainWorkflowSession fall back to ordinary replay for such VMs; a quiescent step-only VM remains a pure function of the consumed event prefix and stays retainable.
Atomics.waitAsync (a wall-clock timer via SharedArrayBuffer) and the async WebAssembly compilation entry points resolve on host timing just like crypto.subtle.digest. Wrap every such intrinsic in createContext so usedHostAsync covers the complete set; dynamic import() settles within a microtask and cannot advance a suspended VM.
node:crypto createHash produces byte-identical values to WebCrypto and settles the digest promise on a deterministic microtask instead of host threadpool timing. A digest can therefore never advance a suspended workflow, so digest-using VMs stay retainable; only Atomics.waitAsync and async WebAssembly compilation remain host-timed. createHash is stable and undeprecated on Node 18-26 (DEP0179 only removed the direct Hash constructor).
GC observation depends on host GC timing that neither replay nor a retained VM can reconstruct from the event log. WeakMap/WeakSet stay available (they do not expose GC state).
Reject non-BufferSource digest input with TypeError like WebCrypto does, via the native ArrayBuffer.prototype.byteLength brand check (works across vm realms). Previously a plain number was treated as a Uint8Array length, turning a small input into a giant allocation.
…mness handleSuspension dehydrates step arguments with the live VM, and that serialization can execute user code (getters, WORKFLOW_SERIALIZE hooks). Randomness drawn there would desync the retained VM's future correlation IDs from what a fresh replay regenerates. Count every draw from the seeded stream at its single source in createContext and fall back to ordinary replay if handleSuspension consumed any.
Delete Atomics.waitAsync and the async WebAssembly entry points from the sandbox instead of tracking their use — with digest synchronous and GC intrinsics removed, no sandbox API settles a promise on host timing, so a suspended VM provably cannot advance. This deletes the trackHostAsync wrapper, the usedHostAsync bit and session method, the runtime gate clause, the session 'failed' state (unreachable), and the background-progress test scenarios (impossible by construction).
Replace the RNG draw-counter demotion with prevention: when a session is a retention candidate, new step inputs take a passive descriptor walk (never invoking getters; proxies, accessors, functions, custom classes, and platform wrappers decline) and safe values are structuredClone'd into the host realm before dehydration, so serialization never executes workflow-owned code against a retained VM. Unsafe inputs serialize the old way and the session falls back to ordinary replay.
- require enumerable on array index descriptors: structuredClone drops non-enumerable indices that devalue persists - read workflow globals and constructor prototypes via own-property descriptors only, so validation can never execute workflow-owned accessors on redefined globals
constructorPrototype reads both realms' constructors via own-property descriptors only and refuses proxies before any descriptor read, so a proxied redefined global can never observe validation.
- A mixed step batch (one unsafe sibling input) now serializes every input through the ordinary VM path: a clone snapshotted before an unsafe sibling's serialization runs its getters could otherwise durably capture stale sibling state. - crypto.subtle.digest rejects SharedArrayBuffer-backed views with TypeError, matching WebCrypto's BufferSource contract.
devalue serializes Map/Set through the realm's iterator protocol and Date/RegExp/typed arrays through prototype getters, all of which workflow code can mutate — so their serialization is not provably passive and their bytes could differ between retained and cold modes. The fast path now accepts only primitives, plain objects, and plain arrays, which devalue traverses exclusively via own-property reads. Slot-bearing exotics decline even with a swapped prototype. The sandbox digest now reads view metadata (buffer/byteOffset/ byteLength) through captured intrinsic getters, so own properties shadowing them cannot change which bytes are hashed or bypass the SharedArrayBuffer rejection.
instanceof dispatch (Symbol.hasInstance via the constructor, Function.prototype, and Object.prototype), the class reducer's value.constructor walk, and devalue's Object/Array traversal all consult intrinsics workflow code could redefine — legally and deterministically — which would make the durable step input depend on WORKFLOW_RETAINED_VM (spoofed values serialize as e.g. Maps on the cold path but as plain clones on the retained path). Freeze Object/Array/Function (constructors and prototypes), the VM collection constructors, and every reducer-referenced global binding (absent ones pinned to undefined) right before the workflow bundle evaluates, so the retained-input equivalence holds by construction. Host-realm constructor escapes (e.g. TextEncoder.constructor) remain out of the determinism contract: code scheduling host timers was never deterministic under ordinary replay either; documented on canRetainWorkflowSession.
Typed-array constructors (and their shared %TypedArray% parent), the Date wrapper, and the session-local AbortController/AbortSignal/ Request/Response bindings were pinned but not frozen, so workflow code could still add Symbol.hasInstance statics that diverge reducer dispatch between the retained clone (host constructors) and ordinary VM serialization. Freeze every binding value that is not the shared host intrinsic; shared host objects are dispatched identically by both paths, so mutations there cannot cause mode divergence.
Replace structuredClone with an explicit deep copy into an SDK-private realm: clones previously inherited host prototypes, which workflow code can reach (e.g. via structuredClone's return values) and vandalize with Symbol.toStringTag or constructor overrides, shifting devalue's classification of the clone relative to the ordinary VM path. The pristine realm is unreachable by any user code, and the explicit copy serializes exactly what devalue traverses (own indices, own enumerable string props). Arrays also now decline own constructor properties, which the class reducer reads even when non-enumerable.
Host intrinsics are shared with the whole process and cannot be frozen, but workflow code can reach them (structuredClone results, exposed host classes) and install Symbol.hasInstance predicates that distinguish the original from its clone — or WORKFLOW_SERIALIZE statics on host Object/Array that the class reducer reads for host-prototype originals (hydrated step results). prepareRetainedStepInput now verifies, via own-descriptor reads only, that every host dispatch point is pristine and declines retention before any clone exists — so a spoofed predicate can never observe or capture a pristine-realm object.
Reducers dispatch on symbol tags (e.g. the workflow abort-signal markers) that are non-enumerable and dropped by the pristine-realm copy, so a tagged object would serialize as an abort descriptor on the cold path but as plain data on the retained path.
Hidden own keys of any kind — non-enumerable properties, accessors, symbols — can be observed by serialization dispatch (reducer probes like .signal, thenable checks, the class reducer) while the pristine clone drops them. With no hidden own keys, every probe on an accepted object resolves deterministically through validated data or pristine prototypes.
Symbol.hasInstance dispatch walks the constructor's prototype chain, so the frozen Date wrapper still exposed the unfrozen original VM Date it delegates statics to. Freeze each non-shared binding's full chain (stopping at host Function/Object prototypes) and verify host Object.prototype carries no added hasInstance on the detection side.
… (v2) Serialize step inputs for retained boundaries through the one ordinary pipeline (original value, workflow global) instead of cloning into a pristine realm and serializing under the host global. With a single serialization event shared by every mode, durable bytes structurally cannot depend on WORKFLOW_RETAINED_VM; the only property retention needs is that serialization executes no workflow code, established by: - the passive walker (descriptor-only, unchanged in spirit), now also accepting Map/Set/Date/typed arrays/ArrayBuffer — the common built-in step arguments — via prototype-identity checks - vm/serialization-pins.ts: the 10 prototype members serialization executes for those built-ins (measured empirically), captured at context creation and identity-verified at each retained boundary; the 'touches only pinned members' test instruments every member and locks the list against serde drift - host-realm instances (hydrated step results) accepted without member verification: host members run host code, which cannot touch retained VM state Deletes the pristine clone realm, the host-dispatch pristineness checks, and the batch clone bookkeeping.
… (v3) Review found the pin approach's structural hole: the class reducer READS value.constructor through Map.prototype — a data property when pristine (so member instrumentation never listed it), but executable the moment workflow code redefines it as a getter. Pinning what serialization executes misses what it reads. Freeze the accepted built-ins' prototypes wholesale (Map/Set/Date + iterator prototypes, %TypedArray% + subclass prototypes, ArrayBuffer): reads and executes are both immutable, and a patch attempt now throws loudly at the patch site instead of silently degrading. Deletes vm/serialization-pins.ts; the walker requires Object.isFrozen on the realm prototype (also covering realms where the freeze never ran). Also restores the host-dispatch pristineness check the v2 cut lost: workflow code can reach shared host constructors (exposed classes, structuredClone results) and plant workflow-realm Symbol.hasInstance hooks or WORKFLOW_SERIALIZE statics that reducers would execute during retained serialization. Host-realm built-in instances decline for the same reason; host-realm plain data (hydrated results) stays retainable.
- Capture Map/Set forEach and the %TypedArray% buffer getter as module- load primordials: the checker previously invoked live host methods that workflow code can reach (structuredClone(new Map()).constructor) and replace with delegating workflow-realm closures. - Typed arrays must have one of the realm's real frozen subclass prototypes by identity — 'frozen and chains to %TypedArray%' admitted manufactured frozen hostile prototypes with delegating buffer getters.
…ializer statics - The walker resolved Object.getOwnPropertyDescriptor, Reflect.ownKeys, Array.isArray, Number/String helpers, and Object.getPrototypeOf/isFrozen from live host globals workflow code can reach and replace; all are now module-load captures, so the checker can never execute a planted delegate. - The class reducer reads cls[WORKFLOW_SERIALIZE]/cls.classId as inherited Gets, so isHostDispatchPristine now also verifies host Function.prototype and Object.prototype carry no serializer statics. Generic replacement of shared host statics (Object.keys, Array.from, …) via realm escape remains the documented host-reachability boundary, tracked by the realm-local intrinsics follow-up.
- Suspension signals capture ctx.suspensionGeneration when scheduled and no-op if the session resumed past that boundary. The harmful interleaving was already unreachable (queue items are deleted on consume, completion writes state synchronously, nextTick precedes timers) — the token turns those ordering facts into an explicit invariant. - The BigInt reducer calls .toString() on primitives from host code, which resolves on host BigInt.prototype: its identity joins the host dispatch check, and the VM BigInt.prototype is frozen besides.
…nly allow-list gate, drop prepareForRetention param
This was referenced Jul 27, 2026
# Conflicts: # packages/core/src/workflow.ts
…on suspend), whole predicate in canRetainWorkflowSession, lazy hook/wait scan, prewarm on resume path
Contributor
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 409957ms → this run 147089ms (Δ -262868ms, -64%) 1020 steps (queue-hop) Cumulative STSO time: main 6303ms → this run 2474ms (Δ -3829ms, -61%) 📜 Previous results (7)e7cfaa7Mon, 03 Aug 2026 18:51:08 GMT · run logs
e0dea14Mon, 03 Aug 2026 17:40:48 GMT · run logs
43a6347Sun, 02 Aug 2026 23:37:50 GMT · run logs
6084f8fSun, 02 Aug 2026 02:12:22 GMT · run logs
e1e5399Thu, 30 Jul 2026 17:01:05 GMT · run logs
ad78ad1Thu, 30 Jul 2026 00:36:46 GMT · run logs
8cea528Mon, 27 Jul 2026 20:31:30 GMT · run logs
ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
…h replayWorkflow/resumeWorkflow, drop low-value events-consumer tests, compact session and retained-loop tests
…ument the unguarded-signaler invariant
…umeResult), async resume(), rename runtime local to retainedSession
…), drop session telemetry test
# Conflicts: # packages/core/src/runtime.ts # packages/core/src/workflow.ts
Review finding (both panel reviewers): restartReplayInProcess — added on main by #3145 while this branch was in flight — reset the cached log but not the parked VM session. Any stale-snapshot continue then resumed a session belonging to the discarded log: after a run_completed 412 the completed session's resume() throws and the run is durably failed despite having completed; after a suspension-create 412 the session is resumed without ever passing the retention decision, bypassing both the WORKFLOW_RETAINED_VM kill switch and the step-input gate. A restart now always falls back to a fresh replay. Regression test injects a 412 on run_completed and proves fresh-replay completion (red without the fix).
…ts; correct the open-hook/wait scan comment (this suspension's writes are not merged into the cached log — non-step suspensions never reach the scan)
…it memo; drop optional-chaining that contradicted the surrounding guards
TooTallNate
approved these changes
Aug 3, 2026
TooTallNate
left a comment
TooTallNate
left a comment
Member
There was a problem hiding this comment.
Reviewed at eec9a09 (merge-base 2 days / 11 commits behind main — no overlap with the intervening serialization work; three-dot diff used throughout).
Verified locally:
- Full core unit suite green in both modes at head: 1797 passed / 3 expected fail with retention on, identical with
WORKFLOW_RETAINED_VM=0. (Oneroute-bundle-isolation.test.tsfailure under full-suite parallelism passes in isolation — suite flake, not PR-caused. The 6 e2e file collection errors are just missingDEPLOYMENT_URL.) - Session state machine: the 4-state machine + discriminated results hold up under adversarial reading.
resumeis only reachable fromsuspended; prefix divergence demotes permanently;failWorkflowcatches control-flow errors arriving via direct step-promise rejection (bypassingonWorkflowError) so every path converges onreplayinstead of a spuriousrun_failed. The suspension identity check (error === state.suspension) is sound becauseonWorkflowErrorsets the state before rejecting the interruption. - The retention type check is real: I confirmed
WorkflowSuspension.stepsis a full-queue snapshot ([...stepsInput.values()]at construction — every item type, despite the field name), soevery(item => item.type === 'step')genuinely excludes sleep/hook/attribute boundaries and is immune to later queue mutation. - Generation guard: the double bump (accept + resume) covers both same-boundary siblings and timers queued at boundary N firing after resume into N+1. The documented invariant — unguarded signalers (sleep/hook/attr) must be unretainable — is satisfied by the type check, and a late unguarded signal landing on a
suspendedsession demotes rather than corrupts. Good defensive layering. - Event array ownership:
EventsConsumercopying at construction plus tail-onlyappendmakes the strict-extension check meaningful against both the runtime's in-place growth and the wait-completion/stale-reload array replacement paths (the comment calling that out is accurate — I checked both call sites). - 412 hygiene:
retainedSession = nullon every precondition restart, including the run_completed 412 where a completed session must not be resumed — and the loop test covers exactly that. - Primitives gate is a provably-sound type prediction (serializing null/boolean/number/string executes nothing; BigInt and symbol rightly excluded). Conservative
thisVal/closureVarsexclusion. - Changeset: minor on both fixed packages ✓.
WORKFLOW_RETAINED_VMdocumented in runtime-tuning ✓. New e2e workflow lives inworkbench/exampleand reaches other workbenches via the existing symlink ✓.
Nice touches: the lazy once() open-hook/wait scan shared across all three gates, and the honest bimodality note on finalSchedulingReplay telemetry.
CI is green (only permission/path-gated skips). Ship it.
VaguelySerious
approved these changes
Aug 3, 2026
This was referenced Aug 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second of the 3-PR stack from #2990 — the retained-VM feature itself, stacked on #3045 (sandbox hardening, which guarantees a suspended VM cannot advance on host timing).
executeWorkflowwith discriminated request/result unions and a 4-state session machine (running/suspended/replay/completed); overloads make "fresh replay requests another replay" unrepresentableWORKFLOW_RETAINED_VM=0kill switch (default on); the off path is the always-exercised replay fallback, not a second implementationworkflow.runspans taggedworkflow.execution.mode=replay|retainedSerialization gate (intentionally narrow here): post-suspension argument serialization runs once and never replays, so it must execute no workflow code while the VM lives on. This PR retains only boundaries whose step arguments are primitive values (provably zero-code serialization). Everything else declines per boundary — one ordinary replay, then retention resumes. #3047 widens support to plain objects/arrays and standard built-ins via taint-reporting hardened serialization.
Performance
The 1,020-step STSO benchmark uses primitive args, so this PR alone lands the headline: STSO flat at ~102–140 ms regardless of history length vs. main's 300→625 ms growth (−46% early windows, −82…−84% at steps 1001–1020).
Review focus
The session state machine, the prefix check, and the runtime loop integration — the architecturally interesting part, with zero serialization-policy noise (that's #3047).
Validation
Full core suite green in both modes (default and kill switch): 1,537 passed, 3 expected failures each. Loop-level tests prove: one VM build per run (vs >1 under the kill switch) with byte-identical output, demotion for non-primitive args, retention for digest-using workflows.
Stack: #3045 → this → #3047. #2990 stays open as the reference implementation.