QuickJS engine: threshold-based VM-memory snapshotting (WORKFLOW_SNAPSHOT_THRESHOLD) - #3251
Open
TooTallNate wants to merge 3 commits into
Open
QuickJS engine: threshold-based VM-memory snapshotting (WORKFLOW_SNAPSHOT_THRESHOLD)#3251TooTallNate wants to merge 3 commits into
TooTallNate wants to merge 3 commits into
Conversation
🦋 Changeset detectedLatest commit: e294ed1 The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
🧪 E2E Test Results❌ Some tests failed ❌ Failed E2E Tests▲ Vercel Production (510 failed)astro-node (24 failed):
astro-quickjs (12 failed):
example-node (21 failed):
example-quickjs (41 failed):
express-node (17 failed):
express-quickjs (19 failed):
fastify-node (40 failed):
fastify-quickjs (27 failed):
hono-node (18 failed):
hono-quickjs (25 failed):
nextjs-turbopack-node (23 failed):
nextjs-turbopack-quickjs (14 failed):
nextjs-webpack-node (15 failed):
nextjs-webpack-quickjs (21 failed):
nitro-node (15 failed):
nitro-quickjs (27 failed):
nuxt-node (14 failed):
nuxt-quickjs (16 failed):
sveltekit-node (13 failed):
sveltekit-quickjs (25 failed):
vite-node (50 failed):
vite-quickjs (33 failed):
💻 Local Development (1 failed)astro-stable-node (1 failed):
📋 Other (84 failed)e2e-vercel-prod-nest-node (19 failed):
e2e-vercel-prod-nest-quickjs (23 failed):
e2e-vercel-prod-tanstack-start-node (22 failed):
e2e-vercel-prod-tanstack-start-quickjs (20 failed):
E2E Test SummarySummary
Details by Category❌ ▲ Vercel Production
❌ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
❌ 📋 Other
✅ vercel-multi-region
|
VaguelySerious
left a comment
VaguelySerious
left a comment
Member
There was a problem hiding this comment.
AI review: blocking issues found
TooTallNate
force-pushed
the
quickjs-vm-threshold-snapshots
branch
from
July 31, 2026 21:25
00d4d3e to
7f4d517
Compare
…KFLOW_SNAPSHOT_THRESHOLD)
…based PRNG fast-forward, unified host-callback list, lifecycle hardening - SnapshotMetadata gains eventCount, rngDraws and formatVersion. The max-events guard now compares restored total + delta (both at entry and per loop turn) — previously a run that kept snapshotting could never accumulate enough delta to trip the ceiling it exists for. - Correlation-id generation is position-based across snapshots: the runtime seeds from the BASE seed and fast-forwards the persisted draw count instead of mixing the snapshot cursor into the seed. Ids are now identical across snapshot generations AND identical to a no-snapshot run, so overlapping invocations straddling a snapshot save still collide on the world's dedup (new test pins restored ids == full-replay ids). Snapshots without a draw count fall back to full replay. - Host callbacks are declared in ONE list that drives both the fresh-boot install and the restore re-registration, so adding a callback can't silently skip the restore path. - Preloaded events are used again with snapshotting enabled (the first qualifying suspension skips its save — no cursor yet); short runs keep the zero-round-trip fast path. - Snapshot persist runs off the response path (waitUntil), with a 32MB plaintext size ceiling (skip + warn). Loads that fail format/shape checks warn instead of silently miming a miss. Terminal deletes are gated on a snapshot actually existing and now also fire on the runGone path; a server-side TTL remains the backstop for unobserved cancellations.
TooTallNate
force-pushed
the
quickjs-vm-threshold-snapshots
branch
from
July 31, 2026 23:09
81784aa to
c139c2a
Compare
Contributor
📊 Workflow Benchmarkscommit Backend:
📈 STSO distribution vs main (inline / queue-hop histograms)1020 steps (inline) Cumulative STSO time: main 152131ms → this run 126830ms (Δ -25301ms, -17%) 1020 steps (queue-hop) Cumulative STSO time: main 2084ms → this run 3050ms (Δ +966ms, +46%) 📜 Previous results (1)c139c2aFri, 31 Jul 2026 23:29:13 GMT · run logs
ℹ️ Metric definitions & methodologyThe collapsed STSO distribution section above buckets every step gap of the sequential-steps run (not a sampled window), split by whether the step ending the gap ran inline — in the same warm process as the step before it, so the gap is pure framework overhead — or after a queue-hop — the first step of a fresh process, which pays queue dispatch, client reinit and event-log replay. Bars overlay the two runs: Best/P75/P90/P99 deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body (in-deployment start() → first step body, deployment clocks) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored) · SL: stream latency (in-deployment write → read propagation, readAt - writtenAt) · SO: stream overhead (end-to-end write+consume time beyond the modelled generation window) Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · stream latency: parallel reader/writer steps on a dedicated stream; SL is the in-deployment write->read propagation (readAt - writtenAt) · stream overhead (text): writer streams 300 variable-length text token deltas paced at 100/s for 3s (a haiku-size LLM's token throughput) while a parallel reader drains the whole stream; SO is the end-to-end write+consume time beyond the 3s generation window (overhead/backpressure) · stream overhead (structured): same workload as stream overhead (text), but each delta is an AI-SDK-style structured object ({ type: 'text-delta', id, text }) instead of a raw string, so the SO gap vs the text scenario is the added serialization cost 🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · SO 250/500/1000 All metrics are measured from deployment-side timestamps only. Runs are triggered by an in-deployment route that stamps the anchor ( Cold starts are kept in the numbers on purpose — they are part of real bursty-workload latency. The workbench deployment cold-starts the |
pranaygp
reviewed
Aug 3, 2026
pranaygp
left a comment
pranaygp
left a comment
Contributor
There was a problem hiding this comment.
Reviewed the incremental diff (12 files, +926) and ran the stack top locally. Not merge-ready — headline finding inline at the load gate: snapshots never restore on world-postgres or world-vercel, and nothing in CI can tell, because a snapshot that never restores falls back to full replay, which is correct behavior. The quickjs-snapshot CI legs are currently red only on the inherited #3049 overflow bug; once that's fixed they'd go green while snapshots remain pure cost on both production worlds.
Local validation of what does work (world-local): snapshot lifecycle is real — 101 snapshot_saved / 283 snapshot_load diag checkpoints across a full e2e leg, 4 MB-ish .bin/.json pairs created and deleted on completion, restored: true resumptions observed. The position-based PRNG fast-forward design is correct: the restored-run-produces-identical-correlationIds property is the right one to pin, and I could not construct a divergence; the per-(runId, correlationId) dedup backstop can't wedge. Host-callback re-registration via the unified list is complete (no other newFunction sites). The eventsCursor frontier is sound across all three worlds (strictly exclusive cursors, no off-by-one). Compression is already workerd-safe.
One n=1 observation from a full snapshot-leg run worth your eyes: parallelStepsThenWebhookWorkflow - no hook_conflict from same-tick replay race failed once with a correctness assertion (harness posted body-<tokenA> for a token it listed from storage, workflow expected body-<tokenB>), i.e. a post-restore disagreement about the run's own webhook token. 10/10 passes in isolation afterwards and absent from a second full leg — but a restore-path token/PRNG-position determinism bug is what it would smell like if it recurs.
Non-blocking: first-invocation captures with no cursor are taken then dropped; the restore drain loop lacks the boot path's iteration-bound warning; deleteSnapshotIfAny early-returns when the threshold is 0, so flipping the env off strands stored snapshots (and a waitUntil save landing after a terminal delete re-creates one); __hookPayloadBuffer.__processedEventIds grows monotonically in-heap (snapshot size for long-lived reusable hooks only increases); __wdk_env isn't refreshed after restore; the docs say invalid threshold values "throw at startup" but getSnapshotThresholdFromEnv throws on first invocation. Changeset omits @workflow/world (this PR modifies packages/world/src/snapshots.ts).
Stack coordination: #3263 rebases under this per its own description — do that first. Three silent semantic breaks to check on the rebase: the inline __generateUlid registration won't be re-registered on restore (the branch's own comment says host callbacks must never be inline); the restore path's "serde survives in the heap" comment becomes false (serde is host-side after #3263 — createQuickJSSerde must run on restore); and the ULID monotonic factory's state moves host-side, so it's no longer captured by the snapshot and the rngDraws fast-forward desynchronizes.
| const version = loaded.metadata.formatVersion; | ||
| if ( | ||
| (version !== undefined && version !== SNAPSHOT_FORMAT_VERSION) || | ||
| loaded.metadata.rngDraws === undefined |
Contributor
There was a problem hiding this comment.
Blocking: this gate rejects every snapshot on world-postgres and world-vercel, so the feature never restores on either production world. Neither world persists the new metadata fields: world-postgres writes/reads only eventsCursor + createdAt (migration 0018 has no columns for eventCount/rngDraws/formatVersion), and world-vercel sends/parses only the two original headers (carrying the new fields also needs a workflow-server change). Only world-local round-trips them, which is why the local tests pass.
Net effect on prod worlds: every qualifying suspension pays session.snapshot() (two full heap copies) + compression + encryption + a 5–15 MB PUT, and every resume throws it away and full-replays — strictly worse than WORKFLOW_SNAPSHOT_THRESHOLD=0. Persisting the fields in both worlds is the right fix; tolerating missing rngDraws is not safe (a restored heap with rngDraws: 0 re-draws and collides correlationIds).
| // its next qualifying suspension instead.) | ||
| const snapshot = capturedSnapshot; | ||
| const totalEventCount = restoredEventCount + seenEventIds.size; | ||
| safeWaitUntil( |
Contributor
There was a problem hiding this comment.
"Off the response path" is false on the default codec: this IIFE runs synchronously up to its first real await, and compress with the default write codec is zlib.zstdCompressSync — synchronous compression of a multi-MB heap image blocks the event loop before the response flushes. The gzip fallback (CompressionStream) is genuinely async, so this only bites the default path. Defer past the current tick (Promise.resolve().then(...)) or use the async codec for snapshots.
| // (pre-snapshot count persisted in the metadata + delta) — otherwise a | ||
| // run that keeps snapshotting would never accumulate enough delta to | ||
| // trip the ceiling it exists to enforce. | ||
| const restoredEventCount = existingSnapshot?.metadata.eventCount ?? 0; |
Contributor
There was a problem hiding this comment.
restoredEventCount double-counts after a restore failure. The restore-failure catch resets existingSnapshot/lastEventsCursor to fall back to full replay but can't reset this const; the ceiling check then adds the stale count to a seenEventIds set that now holds the entire log, so MaxEventsExceededError can fire well below the real limit — and the next save stamps the inflated eventCount, compounding. Currently masked by the load-gate finding (no restores on prod worlds ⇒ no failures to fall back from), which is exactly the latent-bug shape that surfaces the day that's fixed.
| ) { | ||
| try { | ||
| capturedSnapshot = session.snapshot(); | ||
| if (capturedSnapshot.data.byteLength > MAX_SNAPSHOT_PLAINTEXT_BYTES) { |
Contributor
There was a problem hiding this comment.
Ceiling enforced after the expensive work: session.snapshot() has already made two full copies of the WASM heap by this check, and since WASM linear memory never shrinks, a run that once crossed 32 MB re-pays both copies at every subsequent suspension and discards the result every time. Gate on VM memory size before snapshotting, or latch a per-run "too big" flag on first rejection.
| * Current snapshot format version, bumped when the heap layout or the | ||
| * metadata contract changes incompatibly. | ||
| */ | ||
| export const SNAPSHOT_FORMAT_VERSION = 1; |
Contributor
There was a problem hiding this comment.
formatVersion covers the SDK's metadata envelope, not the heap image — the QJSS header that deserializeSnapshot validates is identical across quickjs-wasi builds, so bytes from one build restored by another pass validation and execute as undefined behavior in the interpreter. Real deploy-skew hazard (a quickjs-wasi bump mid-rollout has live snapshots from the old build), and a data-corruption-class failure. Cheap close: the library exposes vm.versions — persist it in metadata and reject a mismatch. Related smaller gap: the deterministic clock's high-water mark (vmNowMs) isn't persisted either, so Date.now() in a restored VM can regress until the first event advances it.
…-snapshots Carries the #3049 merge down the stack. Resolution notes: - inline loop's post-batch feed keeps BOTH sides: zero-event feeds raise pendingRequeueSignal before breaking (ack-without-requeue fix from #3049) and non-empty feeds advance eventsProcessedSinceSnapshot - quickjs-runtime.test.ts keeps both trailing suites (VM snapshot/restore + hook dispose-then-sleep replay) Validated: core 1899 passed; hook (26/26) and promiseRace e2e green under WORKFLOW_VM=quickjs with WORKFLOW_SNAPSHOT_THRESHOLD=5.
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
| // (pre-snapshot count persisted in the metadata + delta) — otherwise a | ||
| // run that keeps snapshotting would never accumulate enough delta to | ||
| // trip the ceiling it exists to enforce. | ||
| const restoredEventCount = existingSnapshot?.metadata.eventCount ?? 0; |
Contributor
There was a problem hiding this comment.
restoredEventCount is captured as a const before snapshot restore and cannot be reset when restore fails and the code falls back to full event-log replay, causing the pre-snapshot count to be added on top of the now-full events/seenEventIds.
| // its next qualifying suspension instead.) | ||
| const snapshot = capturedSnapshot; | ||
| const totalEventCount = restoredEventCount + seenEventIds.size; | ||
| safeWaitUntil( |
Contributor
There was a problem hiding this comment.
Snapshot compression claimed to run off the response path via safeWaitUntil actually runs synchronously on the response-path call stack for the default zstd codec, blocking the event loop while compressing a multi-MB heap.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Supersedes #3053, which GitHub auto-marked as merged (and auto-deleted the head branch) when a restacking mistake briefly force-pushed the head branch to the same commit as its base. Same content, freshly rebased on the stack.
Summary
PR 4 of the QuickJS VM roadmap: threshold-based VM-memory snapshotting — the middle ground that motivated reviving this effort (see #1298 / #1300 discussion). Instead of snapshotting at every suspension (the original branch's model, which cost ~25% on e2e wall clock), snapshots are taken only once
WORKFLOW_SNAPSHOT_THRESHOLDevents have been processed since the last one:How it works
WORKFLOW_SNAPSHOT_THRESHOLDenv var (default0= disabled) or per-runexecutionContext.snapshotThreshold, stamped atstart()for run affinity likeWORKFLOW_VM.session.snapshot()) → compress (zstd/gzip via the shared serialization pipeline; QuickJS heaps compress ~4×, measured 16.5 MB → 3.9 MB) → encrypt with the run's key when configured →world.snapshots.savewith the events cursor at the VM's feed frontier.world.snapshots.load→ decrypt → decompress →QuickJS.restoreover the cached WASM module, re-register host callbacks, fetch events from the snapshot's cursor and feed only the delta. Runs seamlessly through PR 2's inline continuation loop.Determinism model (restore + partial replay)
The threshold model's new mechanism vs. the original branch: a resumption may restore a snapshot older than the log head (suspensions since the snapshot weren't persisted) and must deterministically re-derive everything in between:
eventsCursor: the heap already consumed pre-snapshot draws, so re-seeding from the base would replay the first-N draws and collide with recorded correlationIds. The cursor is identical for every resume from the same snapshot (concurrent resumes still collide ids for the world's dedup) and advances only when a newer snapshot is taken.Validation
WORKFLOW_SNAPSHOT_THRESHOLD=1(maximum churn: snapshot on every qualifying suspension), wall clock within ~10% of the node baselinerestored: trueresumptions, save/restore/delete lifecycle, and threshold gating (threshold=100 short run ⇒ zero snapshots, pure replay)quickjs-snapshotmatrix leg (nextjs-turbopack, threshold=1) across local dev/prod/postgres e2e jobsNotes / follow-ups
quickjs-wasibuild that produced them. Per the deployment contract (runs continue on the version they started on — free on Vercel), this is a non-issue in production; environments without skew protection are covered by the restore-failure fallback to full replay.WORKFLOW_SNAPSHOT_THRESHOLDsection added to v5 Runtime Tuning.