test: e2e coverage for run-idempotency conflict-handling strategies#2387
Merged
Conversation
Covers the patterns documented in foundations/idempotency: - claim-only hook mutex: token claimed and held with no payload data, duplicate identifies the owner, token released after completion - adopt the owner's result via conflict.returnValue - signal the owner: duplicate forwards its payload via resumeHook - supersede: duplicate cancels the owner and reclaims the token - route-side resume-or-start retry pattern reaching the started run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 0237347 The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Next.js (Turbopack) | Nitro | Express Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Next.js (Turbopack) | Nitro workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Next.js (Turbopack) | Express workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express | Nitro | Next.js (Turbopack) fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Nitro | Express | Next.js (Turbopack) SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds new end-to-end coverage for the documented run-idempotency conflict-handling strategies built on hook.getConflict(), including “claim-only mutex”, “adopt owner result”, “signal owner”, “supersede owner”, and a route-side resume-or-start retry pattern.
Changes:
- Add four new workflow fixtures demonstrating conflict-handling strategies (
hookClaimOnlyMutexWorkflow,hookAdoptOwnerResultWorkflow,hookSignalOwnerWorkflow,hookSupersedeOwnerWorkflow). - Add five new e2e tests that exercise those strategies against the runtime (including event-level assertions and route-side retry behavior).
- Wire default arguments for the new workflows into the workbench, and add a Changeset entry.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| workbench/nextjs-turbopack/app/workflows/definitions.ts | Adds default args for the new e2e strategy workflows so they can be triggered from the workbench. |
| workbench/example/workflows/99_e2e.ts | Introduces new workflow fixtures implementing the documented run-idempotency conflict-handling strategies. |
| packages/core/e2e/e2e.test.ts | Adds e2e tests covering each strategy plus the resume-or-start retry flow. |
| .changeset/idempotency-strategy-e2e.md | Adds a Changeset entry describing the new e2e coverage. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2077
to
+2079
| // The superseded owner ends up cancelled. | ||
| const { json: run1Data } = await cliInspectJson(`runs ${run1.runId}`); | ||
| expect(run1Data.status).toBe('cancelled'); |
Contributor
Author
There was a problem hiding this comment.
Fixed in 2e9d000: the test now awaits run1.returnValue and asserts WorkflowRunCancelledError.is(error) (matching the existing cancellation tests), alongside the status check. Note returnValue is a lazy getter so no pending promise existed before — but asserting the rejection both strengthens the test and guarantees nothing can leak if that changes.
… conflict On slow runtimes the duplicate's first invocation could land after the owner completed and released the token, making the duplicate a fresh owner that waits forever for a payload (90s timeout across CI matrices). Poll the duplicate's event log for hook_conflict before resuming the owner, and widen the test timeout for the added gate budget. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…edged in esbuild hang) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2e9d000 deployment's next build crashed in an esbuild hang but its task (70724907c9dd3a29) was recorded into the turbo remote cache anyway, so every subsequent build with the same input hash replays the broken artifact (missing routes-manifest). Change a build input to force a fresh execution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
TooTallNate
approved these changes
Jun 13, 2026
TooTallNate
left a comment
TooTallNate
left a comment
Member
There was a problem hiding this comment.
Approve — the documented strategies now have teeth
Test-only PR closing the docs↔coverage gap: every conflict-handling strategy from the run-idempotency docs now has an e2e test proving it end-to-end. I ran the full hook suite (25 tests) plus the resume-or-start test against a local nextjs-turbopack dev server — all pass.
What's notably good:
- The race-fix commit shows real understanding of the timing semantics. The adopt-owner-result test gates the owner's completion on run2 having observed the
hook_conflictevent — without it, a slow first invocation of run2 could land after the owner released the token, turning run2 into a fresh owner waiting forever. And the signal-owner test correctly omits that gate, because there the owner can only complete via the duplicate's forward — the dependency itself serializes the race. Gate where needed, not everywhere. - The claim-only mutex test asserts the negative: zero
hook_receivedevents on the owner — pinning "hooks as a pure run mutex" as a supported pattern rather than an accident, plus the full lifecycle (hold → duplicate identifies owner → release on completion → clean reclaim by a third run) with a poll-based release wait instead of a fixed sleep. - The supersede test handles the unhandled-rejection trap:
run1.returnValue.catch(...)+WorkflowRunCancelledError.is()assertion both verifies the cancellation semantics and prevents the rejected promise from leaking out of the test — and double-checks via CLI inspect. The reclaim assertion (waitForHookfiltered byrun2.runId) correctly handles the disposal-propagation window. - The resume-or-start test exercises the documented route pattern directly — including the pre-start
HookNotFoundErrorassertion, with a more forgiving retry budget (30s/250ms) than the docs' inline example. - Workflows live once in
workbench/exampleand reach other workbenches via symlink, per convention;definitions.tsupdated; empty changeset is right for test-only changes.
One process note for other reviewers: a two-dot diff against current main appears to delete #2385's reserved-attributes e2e test — that's the usual branch-point artifact (#2385 merged after this branch was cut), not a real deletion. The three-dot diff is purely additive (378+/0−) and the merge with current main is clean.
CI fully green. LGTM.
Contributor
|
Backport PR opened against |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds e2e coverage for the run-idempotency patterns and conflict-handling strategies documented in #2011 (built on
hook.getConflict()from #2373). Before this, e2e coverage proved claim-without-payload and conflict detection, but none of the documented strategies — and the purest use case (hooks purely as a run mutex, never carrying data) had no owner-side coverage.New tests (
packages/core/e2e/e2e.test.ts+ workflows inworkbench/example/workflows/99_e2e.ts)hookClaimOnlyMutexWorkflow— pure run mutex. The owner claims the token viagetConflict(), holds it through a slow step, and never awaits payload data (asserted via zerohook_receivedevents). A duplicate started during the hold identifies the owner; after the owner completes, the token is released and a third run claims it cleanly.hookAdoptOwnerResultWorkflow— adopt the owner's result. The duplicate awaitsconflict.returnValueand returns the owner's exact result, proving callers can't tell which run did the work.hookSignalOwnerWorkflow— signal the owner. The duplicate forwards its own input into the owner's hook viaresumeHook()from a step; the owner completes with the forwarded payload.hookSupersedeOwnerWorkflow— newest-wins. The duplicate cancels the owner viaconflict.cancel()and reclaims the released token through the documented retry loop; asserts the owner endscancelledand the new owner receives payloads on the reclaimed token.resumeHookfails withHookNotFoundErrorbefore any run exists, thenstart()+ retried resume reaches the started run without dropping the payload.Validation
cd workbench/example && pnpm build(workflows compile through the SWC plugin)nextjs-turbopackdev server: 25 tests pass, including all 5 new onesChangeset: patch for
workflow/@workflow/core(test-only; rides along on the next bump).