-
-
Notifications
You must be signed in to change notification settings - Fork 1.2k
fix(core,cli): register tasks loaded via dynamic import during run #3688
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| --- | ||
| "@trigger.dev/core": patch | ||
| "trigger.dev": patch | ||
| --- | ||
|
|
||
| Fix `COULD_NOT_FIND_EXECUTOR` when a task's definition is loaded via `await import(...)` from inside another task's `run()`. The runtime workers now register such tasks with a sentinel file context, and the catalog logs a one-time warning per task id. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| // Fixture mimicking a task entrypoint file: top-level code calls into the | ||
| // catalog (the same way `task()` / `schemaTask()` does via | ||
| // `registerTaskMetadata`). | ||
| // | ||
| // Loaded via `await import()` from inside a test that simulates the worker | ||
| // running a task. The point is to exercise top-level evaluation through Node's | ||
| // ESM module loader so the module-cache semantics are real. | ||
|
|
||
| const register = globalThis.__catalogRegisterTaskMetadata; | ||
| if (typeof register === "function") { | ||
| register({ | ||
| id: "lazy-task", | ||
| fns: { | ||
| run: async () => "ok", | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| export const lazyTask = { id: "lazy-task" }; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,154 @@ | ||
| import { afterEach, describe, expect, it, vi } from "vitest"; | ||
| import { | ||
| NO_FILE_CONTEXT, | ||
| StandardResourceCatalog, | ||
| } from "../src/v3/resource-catalog/standardResourceCatalog.js"; | ||
|
nicktrn marked this conversation as resolved.
|
||
|
|
||
| // Regression tests for COULD_NOT_FIND_EXECUTOR on warm worker processes when | ||
| // a task's `task()` / `schemaTask()` call is evaluated during another task's | ||
| // execution (e.g. as a side effect of `await import(...)` of a module that | ||
| // contains a task definition). | ||
| // | ||
| // Production throw site: | ||
| // - managed-run-worker.ts:566 (post-wrap) | ||
| // - dev-run-worker.ts:578 (post-wrap) | ||
| // Pre-fix symptom: `resourceCatalog.getTask(execution.task.id)` returned | ||
| // undefined even after the worker re-imported the task entrypoint. | ||
| // | ||
| // Pre-fix mechanism: `registerTaskMetadata` silently returned when | ||
| // `_currentFileContext` was unset. Any `task()` call firing during a | ||
| // running task's run() / lifecycle hooks (directly, or transitively via a | ||
| // dynamic import) hit the silent guard. Node's ESM module cache then | ||
| // prevented recovery — the worker's setContext + re-import fallback didn't | ||
| // re-evaluate the module body, so the `task()` call never fired again. | ||
| // | ||
| // Post-fix: the runtime workers wrap their `executor.execute(...)` call with | ||
| // `setCurrentFileContext(NO_FILE_CONTEXT, NO_FILE_CONTEXT)` so any `task()` | ||
| // call firing during execution registers normally with sentinel file | ||
| // metadata. The catalog detects the sentinel and emits a one-time warning | ||
| // per task id to keep the bundle-shape pattern visible. The indexer never | ||
| // sets this sentinel context — its behavior is unchanged. | ||
|
|
||
| describe("StandardResourceCatalog — runtime registration via sentinel context", () => { | ||
| afterEach(() => { | ||
| delete (globalThis as { __catalogRegisterTaskMetadata?: unknown }) | ||
| .__catalogRegisterTaskMetadata; | ||
| vi.restoreAllMocks(); | ||
| }); | ||
|
|
||
| it("silently drops registration when no context is set (indexer's invariant)", () => { | ||
| const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); | ||
| const catalog = new StandardResourceCatalog(); | ||
|
|
||
| catalog.registerTaskMetadata({ | ||
| id: "no-context-task", | ||
| fns: { run: async () => "ok" }, | ||
| }); | ||
|
|
||
| expect(catalog.getTask("no-context-task")).toBeUndefined(); | ||
| expect(warn).not.toHaveBeenCalled(); | ||
| }); | ||
|
|
||
| it( | ||
| "registers normally and warns once when the sentinel context is set " + | ||
| "(simulates the worker's executor wrap)", | ||
| () => { | ||
| const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); | ||
| const catalog = new StandardResourceCatalog(); | ||
|
|
||
| catalog.setCurrentFileContext(NO_FILE_CONTEXT, NO_FILE_CONTEXT); | ||
| catalog.registerTaskMetadata({ | ||
| id: "lazy-task", | ||
| fns: { run: async () => "ok" }, | ||
| }); | ||
| catalog.clearCurrentFileContext(); | ||
|
|
||
| const registered = catalog.getTask("lazy-task"); | ||
| expect(registered).toBeDefined(); | ||
| expect(registered?.id).toBe("lazy-task"); | ||
| expect(registered?.filePath).toBe(NO_FILE_CONTEXT); | ||
| expect(registered?.entryPoint).toBe(NO_FILE_CONTEXT); | ||
| expect(warn).toHaveBeenCalledTimes(1); | ||
| expect(warn.mock.calls[0]?.[0]).toContain("lazy-task"); | ||
| } | ||
| ); | ||
|
|
||
| it( | ||
| "warm-start path: a task whose top-level definition fires during a " + | ||
| "dynamic import inside the sentinel wrap remains findable; the " + | ||
| "worker's setContext + re-import fallback (managed-run-worker.ts:482) " + | ||
| "is not needed", | ||
| async () => { | ||
| vi.spyOn(console, "warn").mockImplementation(() => {}); | ||
| const catalog = new StandardResourceCatalog(); | ||
|
|
||
| (globalThis as { __catalogRegisterTaskMetadata?: unknown }) | ||
| .__catalogRegisterTaskMetadata = ( | ||
| task: Parameters<StandardResourceCatalog["registerTaskMetadata"]>[0] | ||
| ) => { | ||
| catalog.registerTaskMetadata(task); | ||
| }; | ||
|
|
||
| // Simulate the worker wrap: setContext(NO_FILE_CONTEXT) → run user code | ||
| // (which does a dynamic import) → clearContext. | ||
| catalog.setCurrentFileContext(NO_FILE_CONTEXT, NO_FILE_CONTEXT); | ||
| await import("./fixtures/dynamic-task-module.mjs"); | ||
| catalog.clearCurrentFileContext(); | ||
|
|
||
| const registered = catalog.getTask("lazy-task"); | ||
| expect(registered).toBeDefined(); | ||
| expect(registered?.filePath).toBe(NO_FILE_CONTEXT); | ||
| } | ||
| ); | ||
|
|
||
| it("warns at most once per task id under the sentinel context", () => { | ||
| const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); | ||
| const catalog = new StandardResourceCatalog(); | ||
|
|
||
| catalog.setCurrentFileContext(NO_FILE_CONTEXT, NO_FILE_CONTEXT); | ||
|
|
||
| const register = (id: string) => | ||
| catalog.registerTaskMetadata({ | ||
| id, | ||
| fns: { run: async () => "ok" }, | ||
| }); | ||
|
|
||
| register("task-a"); | ||
| register("task-a"); | ||
| register("task-a"); | ||
| expect(warn).toHaveBeenCalledTimes(1); | ||
|
|
||
| register("task-b"); | ||
| expect(warn).toHaveBeenCalledTimes(2); | ||
|
|
||
| catalog.clearCurrentFileContext(); | ||
| }); | ||
|
|
||
| it( | ||
| "control: real file context registers without firing the sentinel warning", | ||
| async () => { | ||
| const warn = vi.spyOn(console, "warn").mockImplementation(() => {}); | ||
| const catalog = new StandardResourceCatalog(); | ||
|
|
||
| (globalThis as { __catalogRegisterTaskMetadata?: unknown }) | ||
| .__catalogRegisterTaskMetadata = ( | ||
| task: Parameters<StandardResourceCatalog["registerTaskMetadata"]>[0] | ||
| ) => { | ||
| catalog.registerTaskMetadata(task); | ||
| }; | ||
|
|
||
| catalog.setCurrentFileContext( | ||
| "/app/dist/lazy-task.entry.mjs", | ||
| "src/tasks/lazy-task.ts" | ||
| ); | ||
| await import("./fixtures/dynamic-task-module.mjs?control"); | ||
| catalog.clearCurrentFileContext(); | ||
|
|
||
| const task = catalog.getTask("lazy-task"); | ||
| expect(task).toBeDefined(); | ||
| expect(task?.filePath).toBe("/app/dist/lazy-task.entry.mjs"); | ||
| expect(task?.entryPoint).toBe("src/tasks/lazy-task.ts"); | ||
| expect(warn).not.toHaveBeenCalled(); | ||
| } | ||
| ); | ||
| }); | ||
12 changes: 12 additions & 0 deletions
12
references/hello-world/src/trigger/dynamicImportReproChild.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| import { task } from "@trigger.dev/sdk"; | ||
|
|
||
| // Defined in a module that's loaded via `await import(...)` from the parent | ||
| // task's run() function. Pre-fix: the task() call below fires while | ||
| // `_currentFileContext` is unset, so the registration is silently dropped. | ||
| // Post-fix: registered with sentinel file metadata + console.warn fires once. | ||
| export const lazyChildTask = task({ | ||
| id: "lazy-child-task", | ||
| run: async (payload: { value: string }) => { | ||
| return { received: payload.value }; | ||
| }, | ||
| }); |
17 changes: 17 additions & 0 deletions
17
references/hello-world/src/trigger/dynamicImportReproParent.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| import { logger, task } from "@trigger.dev/sdk"; | ||
|
|
||
| // Triggers the dynamic-import silent-drop path. The child task's `task()` | ||
| // definition lives in a module loaded via `await import(...)` inside this | ||
| // parent's run() — so its registration would land outside the worker's | ||
| // cold-load context window. | ||
| export const dynamicImportReproParent = task({ | ||
| id: "dynamic-import-repro-parent", | ||
| run: async () => { | ||
| logger.info("parent: about to dynamically import child task module"); | ||
| const { lazyChildTask } = await import("./dynamicImportReproChild.js"); | ||
| logger.info("parent: import complete, triggering child"); | ||
| const handle = await lazyChildTask.trigger({ value: "hello from parent" }); | ||
| logger.info("parent: child triggered", { childRunId: handle.id }); | ||
| return { childRunId: handle.id }; | ||
| }, | ||
| }); |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.