Skip to content

Add world.snapshots storage interface (local, postgres, vercel) - #3050

Merged
TooTallNate merged 0 commit into
quickjs-vm-perffrom
quickjs-vm-snapshots
Jul 31, 2026
Merged

Add world.snapshots storage interface (local, postgres, vercel)#3050
TooTallNate merged 0 commit into
quickjs-vm-perffrom
quickjs-vm-snapshots

Conversation

@TooTallNate

Copy link
Copy Markdown
Member

Stacked PR — based on #3049 (quickjs-vm-perf), which is based on #3048 (quickjs-vm). Review only the top commit here until the bases merge.

Summary

PR 3 of the QuickJS VM roadmap: the world.snapshots storage interface and its implementations, ported from the original snapshot-runtime branch (#1300). Inert until PR 4 — nothing in the runtime calls these APIs yet.

What's included

  • @workflow/world: Storage.snapshots interface — save(runId, data, metadata) / load(runId) / delete(runId) — plus SnapshotMetadata (eventsCursor, createdAt). The cursor is storage-layer metadata stored alongside the opaque snapshot bytes so a restore only fetches events recorded after the snapshot.
  • @workflow/world-local: filesystem storage under {basedir}/snapshots/ ({runId}.bin + {runId}.json sidecar), with round-trip unit tests.
  • @workflow/world-postgres: workflow_snapshots table (migration 0016, renumbered from the original branch to follow main's 00120015).
  • @workflow/world-vercel: client for the workflow-server snapshot endpoints (PUT/GET/DELETE /v2/runs/:runId/snapshot, binary body + metadata headers — endpoints are live in workflow-server since vercel/workflow-server#339). Includes:
    • the undici.request() (not fetch()) PUT path so Buffer bodies survive RetryAgent retries (Retry Handler Fails When Request Has a Body nodejs/undici#3288),
    • explicit W3C trace-context injection on all three request paths (they route around makeRequest, per the repo's trace-propagation rule), covered by new trace-propagation.test.ts cases.

Notes

  • Bytes are transported opaquely: compression/encryption is @workflow/core's responsibility (PR 4's save pipeline); worlds must not add their own compression (ciphertext doesn't compress).
  • Adding a required member to Storage is a breaking change for community World implementations — appropriate for the v5 beta line; called out in the changeset.
  • The always-on WORLD_SNAPSHOT_DIAG warn logging from the original branch was demoted to console.debug.

@changeset-bot

changeset-bot Bot commented Jul 22, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: f43ee63

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 20 packages
Name Type
@workflow/world Minor
@workflow/world-local Minor
@workflow/world-postgres Minor
@workflow/world-vercel Minor
@workflow/cli Patch
@workflow/core Patch
@workflow/vitest Patch
@workflow/web-shared Patch
@workflow/web Patch
@workflow/world-testing Patch
workflow Patch
@workflow/builders Patch
@workflow/next Patch
@workflow/nitro Patch
@workflow/astro Patch
@workflow/nest Patch
@workflow/rollup Patch
@workflow/sveltekit Patch
@workflow/vite Patch
@workflow/nuxt Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview Jul 31, 2026 1:12am
example-nextjs-workflow-webpack Ready Ready Preview Jul 31, 2026 1:12am
example-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-astro-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-express-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-fastify-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-hono-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-nestjs-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-nitro-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-nuxt-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-sveltekit-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-tanstack-start-workflow Ready Ready Preview Jul 31, 2026 1:12am
workbench-vite-workflow Ready Ready Preview Jul 31, 2026 1:12am
workflow-docs Ready Ready Preview, v0 Jul 31, 2026 1:12am
workflow-swc-playground Ready Ready Preview Jul 31, 2026 1:12am
workflow-tarballs Ready Ready Preview Jul 31, 2026 1:12am
workflow-web Ready Ready Preview Jul 31, 2026 1:12am

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🧪 E2E Test Results

Some tests failed

❌ Failed E2E Tests

📦 Local Production (2 failed)

nextjs-turbopack-stable-node (1 failed):

  • webhookWorkflow | wrun_41KYTVK8V80GX7SNQ7TW7Y8XCN

nextjs-webpack-stable-node (1 failed):

  • webhookWorkflow | wrun_41KYTVK8V80GX7SNQ7TW7Y8XCN

E2E Test Summary

Summary
Passed Failed Skipped Total
✅ ▲ Vercel Production 1455 0 239 1694
✅ 💻 Local Development 2799 0 435 3234
❌ 📦 Local Production 3240 2 454 3696
✅ 🐘 Local Postgres 3242 0 454 3696
✅ 🪟 Windows 308 0 0 308
✅ 📋 Other 1788 0 368 2156
✅ vercel-multi-region 27 0 0 27
Total 12859 2 1950 14811
Details by Category

✅ ▲ Vercel Production

App Passed Failed Skipped
✅ astro-node 126 0 28
✅ example-node 126 0 28
✅ express-node 126 0 28
✅ fastify-node 126 0 28
✅ hono-node 126 0 28
✅ nextjs-turbopack-node 151 0 3
✅ nextjs-webpack-node 151 0 3
✅ nitro-node 126 0 28
✅ nuxt-node 126 0 28
✅ sveltekit-node 145 0 9
✅ vite-node 126 0 28

✅ 💻 Local Development

App Passed Failed Skipped
✅ astro-stable-node 128 0 26
✅ astro-stable-quickjs 128 0 26
✅ express-stable-node 128 0 26
✅ express-stable-quickjs 128 0 26
✅ fastify-stable-node 128 0 26
✅ fastify-stable-quickjs 128 0 26
✅ hono-stable-node 128 0 26
✅ hono-stable-quickjs 128 0 26
✅ nextjs-turbopack-canary-node 135 0 19
✅ nextjs-turbopack-canary-quickjs 135 0 19
✅ nextjs-turbopack-stable-node 154 0 0
✅ nextjs-turbopack-stable-quickjs 154 0 0
✅ nextjs-webpack-canary-node 135 0 19
✅ nitro-stable-node 128 0 26
✅ nitro-stable-quickjs 128 0 26
✅ nuxt-stable-node 128 0 26
✅ nuxt-stable-quickjs 128 0 26
✅ sveltekit-stable-node 147 0 7
✅ sveltekit-stable-quickjs 147 0 7
✅ vite-stable-node 128 0 26
✅ vite-stable-quickjs 128 0 26

❌ 📦 Local Production

App Passed Failed Skipped
✅ astro-stable-node 128 0 26
✅ astro-stable-quickjs 128 0 26
✅ express-stable-node 128 0 26
✅ express-stable-quickjs 128 0 26
✅ fastify-stable-node 128 0 26
✅ fastify-stable-quickjs 128 0 26
✅ hono-stable-node 128 0 26
✅ hono-stable-quickjs 128 0 26
✅ nextjs-turbopack-canary-node 135 0 19
✅ nextjs-turbopack-canary-quickjs 135 0 19
❌ nextjs-turbopack-stable-node 153 1 0
✅ nextjs-turbopack-stable-quickjs 154 0 0
✅ nextjs-webpack-canary-node 135 0 19
✅ nextjs-webpack-canary-quickjs 135 0 19
❌ nextjs-webpack-stable-node 153 1 0
✅ nextjs-webpack-stable-quickjs 154 0 0
✅ nitro-stable-node 128 0 26
✅ nitro-stable-quickjs 128 0 26
✅ nuxt-stable-node 128 0 26
✅ nuxt-stable-quickjs 128 0 26
✅ sveltekit-stable-node 147 0 7
✅ sveltekit-stable-quickjs 147 0 7
✅ vite-stable-node 128 0 26
✅ vite-stable-quickjs 128 0 26

✅ 🐘 Local Postgres

App Passed Failed Skipped
✅ astro-stable-node 128 0 26
✅ astro-stable-quickjs 128 0 26
✅ express-stable-node 128 0 26
✅ express-stable-quickjs 128 0 26
✅ fastify-stable-node 128 0 26
✅ fastify-stable-quickjs 128 0 26
✅ hono-stable-node 128 0 26
✅ hono-stable-quickjs 128 0 26
✅ nextjs-turbopack-canary-node 135 0 19
✅ nextjs-turbopack-canary-quickjs 135 0 19
✅ nextjs-turbopack-stable-node 154 0 0
✅ nextjs-turbopack-stable-quickjs 154 0 0
✅ nextjs-webpack-canary-node 135 0 19
✅ nextjs-webpack-canary-quickjs 135 0 19
✅ nextjs-webpack-stable-node 154 0 0
✅ nextjs-webpack-stable-quickjs 154 0 0
✅ nitro-stable-node 128 0 26
✅ nitro-stable-quickjs 128 0 26
✅ nuxt-stable-node 128 0 26
✅ nuxt-stable-quickjs 128 0 26
✅ sveltekit-stable-node 147 0 7
✅ sveltekit-stable-quickjs 147 0 7
✅ vite-stable-node 128 0 26
✅ vite-stable-quickjs 128 0 26

✅ 🪟 Windows

App Passed Failed Skipped
✅ nextjs-turbopack-node 154 0 0
✅ nextjs-turbopack-quickjs 154 0 0

✅ 📋 Other

App Passed Failed Skipped
✅ e2e-local-dev-nest-stable-node 128 0 26
✅ e2e-local-dev-nest-stable-quickjs 128 0 26
✅ e2e-local-dev-tanstack-start-node 128 0 26
✅ e2e-local-dev-tanstack-start-quickjs 128 0 26
✅ e2e-local-postgres-nest-stable-node 128 0 26
✅ e2e-local-postgres-nest-stable-quickjs 128 0 26
✅ e2e-local-postgres-tanstack-start-node 128 0 26
✅ e2e-local-postgres-tanstack-start-quickjs 128 0 26
✅ e2e-local-prod-nest-stable-node 128 0 26
✅ e2e-local-prod-nest-stable-quickjs 128 0 26
✅ e2e-local-prod-tanstack-start-node 128 0 26
✅ e2e-local-prod-tanstack-start-quickjs 128 0 26
✅ e2e-vercel-prod-nest-node 126 0 28
✅ e2e-vercel-prod-tanstack-start-node 126 0 28

✅ vercel-multi-region

App Passed Failed Skipped
✅ nextjs-turbopack 27 0 0

📋 View full workflow run

*/
export function createSnapshotsStorage(basedir: string) {
const snapshotsDir = path.join(basedir, 'snapshots');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

snapshots-storage.ts builds filesystem paths from an untrusted runId without validation, allowing path traversal outside the {basedir}/snapshots/ directory in save/load/delete.

Fix on Vercel

@TooTallNate
TooTallNate merged commit f64932e into quickjs-vm-perf Jul 31, 2026
@TooTallNate
TooTallNate deleted the quickjs-vm-snapshots branch July 31, 2026 02:15
@TooTallNate
TooTallNate force-pushed the quickjs-vm-snapshots branch from f43ee63 to f64932e Compare July 31, 2026 02:15
@VaguelySerious

VaguelySerious commented Jul 31, 2026

Copy link
Copy Markdown
Member

AI review: blocking issues found

Note on mechanics: this PR's diff renders empty (f64932e5 is also #3049's head) and it has no CI checks, so there is nothing to anchor inline comments to. Findings below are against the snapshots storage interface as it exists on the branch and as consumed by #3251.

AI Review: Blocking

snapshots is added as a required member of Storage on a minor changeset. Any third-party world implementation — the interface is public and community adapters exist — fails to typecheck on upgrade. Either make it optional with the consumer treating absence as "snapshotting unavailable", or ship it as a major. #3251 already tolerates every runtime failure of this interface by falling back to full replay, so an optional member costs it nothing.

AI Review: Note

No integrity binding between data and metadata. The payload and the eventsCursor that describes it are stored as two independent values, and the local world writes them as a separate .bin and .json pair with no atomicity. A crash between the two writes leaves a heap paired with a stale cursor, and the restore path in #3251 cannot detect it — it replays the wrong delta against a heap that already consumed those events. Storing the metadata inside the encrypted payload (or a checksum of the payload in the metadata) makes the mismatch detectable rather than silent.

Postgres table has no foreign key, cascade, or GC. Rows are only removed by an explicit delete call, which #3251 issues from the completed and failed branches only — cancelled and abandoned runs leak their snapshot permanently. A cascade on the run row, or a createdAt-based TTL sweep, makes cleanup independent of which client paths remember to call delete.

console.debug on every save and load. These fire per snapshot operation on the hot path and bypass the package's logger, so they cannot be filtered or leveled by operators. runtimeLogger.debug is already imported nearby.

load and delete bypass the retry/metrics helper that save uses. So a transient failure on the load path is a snapshot miss with no retry and no metric, and the resulting silent fallback to full replay looks identical to "no snapshot existed". Given the whole feature is invisible when it fails, the read path needs the instrumentation at least as much as the write path.

AI Review: Nit

Comments and the changeset name the internal backend service. In this repository that should be world-vercel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants