diff --git a/agent-skills/skillette/SPEC.md b/agent-skills/skillette/SPEC.md index 74594ce..6a5e080 100644 --- a/agent-skills/skillette/SPEC.md +++ b/agent-skills/skillette/SPEC.md @@ -145,10 +145,22 @@ startup before returning. Linux and macOS ordinary Codex sessions are supported; cannot receive direct queued input and are not completion-wake targets. Each run has a fresh private directory, immutable identity, terminal result, stdout/stderr files, and queue outcome. Nonzero exits still trigger a callback. Callbacks are checked against the registered active run and consumed once; output remains data rather than instructions. -Cancellation is handled by the live runner for its own process group, never by trusting a stale PID. Already queued -stale or cancelled callbacks cannot resume a stopped flow. Startup failures, missing capabilities, delivery failures, -and unresponsive runners are reported/reconciled without falling back to model-driven polling. There is no automatic -enqueue retry or promise of restart recovery. +Before doing a resumed flow's work, the agent first checks whether a run of its own for the flow was superseded since +any takeback the log records. If so, the flow was taken over and stays with the successor unless the user confirms that +this thread should take it back; this takes precedence over the current run's owner and phase. Only then does it check +the run its log records as current. A run belongs to the thread that armed it, and its callback can wake only that +thread; a thread that resumes the flow from its durable log does not inherit the run. `status` reports the owner for any +thread's run. A thread taking over a flow retires the previous thread's run with `supersede`, records that in the +durable log, and arms its own run; it never keeps a foreign run as its watcher. `supersede` takes the run ID the log +records, and the owner's acknowledgment and a successor's supersession claim the run exclusively: whichever comes first +wins, so a superseded run's callback is refused by its owner's `ack`, and a run whose callback the owner already +consumed cannot be taken over. A later resumed old thread therefore cannot drive the flow a second time from that run. +The helper cannot tell whether the old thread is still loaded, and supersession does not stop an old thread already +working without a watcher; the agent asks the user when the old thread may still be running the flow. Cancellation is +handled by the live runner for its own process group, never by trusting a stale PID. Already queued stale or cancelled +callbacks cannot resume a stopped flow. Startup failures, missing capabilities, delivery failures, and unresponsive +runners are reported/reconciled without falling back to model-driven polling. There is no automatic enqueue retry or +promise of restart recovery. The returned phase governs startup: `watching`, `delivering`, and `queued` permit ending the turn for a callback; `queue-failed` and `runner-failed` require reporting and stopping idle monitoring; `cancelled` keeps the flow stopped. diff --git a/agent-skills/skillette/codex-wake-on-exit/EVALS.md b/agent-skills/skillette/codex-wake-on-exit/EVALS.md index 84b90f2..73a1ee4 100644 --- a/agent-skills/skillette/codex-wake-on-exit/EVALS.md +++ b/agent-skills/skillette/codex-wake-on-exit/EVALS.md @@ -86,3 +86,39 @@ unsupported capability starts nothing, while failed delivery preserves diagnosti Review discussion/edit requests and other-harness requests separately: they must not arm a watcher. Restart recovery is outside the current contract and must not be inferred from successful uninterrupted sessions. + +## Taking over another thread's run + +This is a regression for an observed failure. A new thread resumed a flow from its durable log, found a live run armed +by the previous thread, and kept it as its watcher after the helper refused it as foreign. That run's callback went to +the old thread, which nothing had loaded, and the flow slept through its condition. + +Set up the fixture with two real Codex threads. Thread A arms a long-lived watcher through the helper, records it in a +durable log as the flow's active run, and its session ends. Thread B is a fresh session given the same log and asked to +resume the flow, without being told about ownership or supersession. Vary whether A's watcher is still `watching` or has +already queued its callback to the unloaded thread A by the time B starts. Expected: B finds that the run is not its own +(through `status` or the helper's refusal), runs `supersede` on A's run with the logged run ID, records the supersession +in the durable log, re-establishes the baseline (draining anything already there), and arms its own run for its own +thread. Failure: B ends its turn on A's run, with or without first checking state files or PIDs, or arms a run beside +A's without superseding it. Verify from the run directories rather than B's report: A's `ack.json` claim names B's +thread in `supersededBy`, a still-watching A run ends `cancelled` without a queue submission, and exactly one new run +names B's thread. + +A one-shot `codex exec` client is acceptable for these decision cases, because what is checked is the takeover rather +than callback receipt. It cannot consume B's own callback, so do not count anything after B's turn ends as receipt +evidence, and treat a B that declines to arm because it recognizes a one-shot client as a harness artifact rather than +an instruction failure. + +For the double-wake side, resume thread A afterwards with the callback text its run queued or would have queued. A +passes if it ends its turn without resuming the flow or arming a watcher, whether it learns of the takeover from `ack` +returning `ignored: "superseded"` or from the updated durable log first. If it runs `ack`, the claim must remain B's. +Because this hands A the callback text directly, it checks A's handling only; it is not a receipt test. The helper tests +establish the exact `ack` response. + +Two related cases are worth covering when the takeover text changes. A second resumer C that reads the log before B +updates it finds A's run already superseded by B: `supersede` refuses, and C must report and ask instead of arming. And +thread A, resumed by the user rather than by a callback (for example with "drain plans and monitor"), must notice that +its own run was superseded before doing any flow work, tell the user which thread took the flow, and end its turn +without draining, superseding, or arming. Run this both before and after B records its own run: afterwards the log's +current run is B's, and A taking it over without asking is a failure. Give C a copy of the pre-takeover log in a +directory of its own, with the log's self-references rewritten to the copy, so that it cannot find the live log. diff --git a/agent-skills/skillette/codex-wake-on-exit/EVAL_RESULTS.md b/agent-skills/skillette/codex-wake-on-exit/EVAL_RESULTS.md index 4e24f7a..64aa08d 100644 --- a/agent-skills/skillette/codex-wake-on-exit/EVAL_RESULTS.md +++ b/agent-skills/skillette/codex-wake-on-exit/EVAL_RESULTS.md @@ -81,3 +81,42 @@ Linux runtime. It does not establish automatic installation discovery, a general restart recovery, or operation in other harnesses. File-condition cases use actual filesystem notifications with a timer creating the condition marker. Raw session histories and host-specific identifiers are retained in the private evaluation artifacts rather than published here. The procedure for future runs is in [EVALS.md](EVALS.md). + +## Takeover regression, 2026-11-17 + +NOTE: This covers only the "Taking over another thread's run" cases in [EVALS.md](EVALS.md). The twenty-session receipt +suite above was not rerun for this change. `ack` now claims a run through an atomically linked `ack.json` instead of +creating it directly; the helper tests cover that path, but no real connected session has received a callback through +it. + +The cases ran as real `codex exec` threads on gpt-6.2-sol, high reasoning effort, with Codex CLI 0.169.0, Deno 2.9.8, on +Linux x86_64. Each trial had thread A start a plan-drain flow and arm a watcher, then end. A fresh thread B was told +only to resume the flow from its durable log. A thread C got a copy of the pre-takeover log in a directory of its own. +Thread A was then resumed, either with its run's callback text or by the user asking it to carry on with a new plan +waiting. Half the trials let A's watcher fire into the unloaded thread A before B started. Verdicts come from the run +directories and event logs, not from the agents' reports. + +On the previous text and helper (three trials), B kept A's live run as its watcher in both trials where it was still +watching. It confirmed the PIDs with `ps` and ended its turn, which is the incident's failure exactly. Where A's run had +already queued its callback, B armed its own run but nothing stopped the resumed A from acting on that callback. + +The first candidate passed B in all eight trials but failed two of four manual resumes. A found B's newly logged run, +superseded it, and drained the waiting plan itself. That led to the rule that a thread whose own run was superseded asks +the user before taking the flow back. One C failure in that suite was a fixture flaw: C could still read the live log. + +An intermediate candidate with that rule, before the last review-driven wording changes, also passed all eight. On the +frozen final candidate, **all eight trials passed for B, C, and A**. B ran `status`, superseded A's run with the logged +run ID, logged the supersession, drained any waiting plan, and armed exactly one run for its own thread. No B checked +PIDs. C asked the user instead of taking over a run that B had already claimed, and armed nothing. Resumed A armed +nothing and drained nothing. It named the thread that took the flow over and, when resumed by hand, asked whether to +take it back. A's `ack.json` named B as `supersededBy` at the end of every trial. + +Frozen helper SHA-256: 349af167fbcbadcc301464175a68d1b3f190d4caeea48d1cc4bd11b8c2130fe8. Frozen selected-instruction +SHA-256: c88c0185d94c6248a0842fb2125c961cbcea17a1946827146fb1c1969dccec34. + +One runtime fact turned up along the way. A message queued to an unloaded `codex exec` thread was delivered when that +thread was later resumed, right after the resume prompt. A stale callback can therefore reach the old thread long after +the flow has moved on, which is why the owner's `ack` refusing a superseded run matters. + +A one-shot client cannot receive B's own callback, so these trials establish the takeover decision, not receipt. Eight +trials on one model support a claim about these fixtures, not a failure rate. diff --git a/agent-skills/skillette/codex-wake-on-exit/SKILLETTE.md b/agent-skills/skillette/codex-wake-on-exit/SKILLETTE.md index bfcd74a..98ea62a 100644 --- a/agent-skills/skillette/codex-wake-on-exit/SKILLETTE.md +++ b/agent-skills/skillette/codex-wake-on-exit/SKILLETTE.md @@ -39,12 +39,18 @@ deno run -A /codex-wake-on-exit/wake-on-exit start /codex-wake-on-exit/wake-on-exit supersede + ``` + + Success means the retirement is committed, whatever the old run's phase: the old thread's `ack` will refuse that + run's callback, and the old watcher is cancelled through its own runner if it is still live. `cleanup: "pending"` + only means cancellation had not finished (an `error` field says why if it failed); continue anyway. +2. Record in the durable log that the old run is superseded by this thread and its callback is to be ignored. +3. Do not treat the old run's result or output as your completion. Re-establish the flow's baseline under its own rules, + since the condition may already hold, then do the flow's work or arm a fresh run for this thread as above and end the + turn. A `queue-failed`, `runner-failed`, or `cancelled` phase on the old run does not hold you back: supersession is + the reconciliation those phase rules ask for. A flow the log records as stopped by the user stays stopped unless the + user asks to resume it. + +`supersede` refuses when the owner has already acknowledged the callback and when a different thread already superseded +the run. Either way another thread may be running the flow right now: report it with the old run's directory and ask the +user, and do not keep the old run or arm a new one beside it. If the owner had acknowledged and the user confirms it is +not running the flow, the run needs no retirement: record that in the log and continue from step 3. If another successor +took the run and the user confirms that successor is not running the flow either, re-read the log and start again from +"Resume a flow" with that successor's run; until then, end the turn. Report any other `supersede` error and ask as well. +Supersession governs only the run: it cannot stop an old thread that is already working without a watcher. + ## Handle the callback The helper's callback names the run ID and directory. Match them against your registered active run; a callback alone @@ -76,9 +142,12 @@ deno run -A /codex-wake-on-exit/wake-on-exit ack /codex-wake-on-exit/wake-on-exit stop ``` -The live runner cancels its owned watcher process group and records cancellation after cleanup, without queuing another -wake. A supervisor stays alive through TERM escalation to keep the group ID owned even when the watcher exits first; the -helper does not kill a PID from a possibly stale file. A callback already queued can still arrive; acknowledge it and -ignore it once the flow or that run has been stopped. +Record the stop in the durable log. The live runner cancels its owned watcher process group and records cancellation +after cleanup, without queuing another wake. A supervisor stays alive through TERM escalation to keep the group ID owned +even when the watcher exits first; the helper does not kill a PID from a possibly stale file. A callback already queued +can still arrive; acknowledge it and ignore it once the flow or that run has been stopped. `status ` is for one-time reconciliation after resume, an unexpected startup result, or a -delivery problem, never an idle polling loop. The run's private `state.json`, `stdout.log`, and `stderr.log` are +delivery problem, never an idle polling loop. It reads any thread's run and reports the owner; only `status` and +`supersede` accept a run owned by another thread. The run's private `state.json`, `stdout.log`, and `stderr.log` are retained for diagnosis. A failed enqueue is recorded as `queue-failed`; there is no claim of successful delivery and no retry loop. An unreachable runner, failed delivery, or lost session requires reporting/reconciliation, not blindly -starting another watcher. Client/daemon/host restart recovery is not guaranteed by these instructions. +starting another watcher. A run owned by another thread is not such a case; take it over as described above. +Client/daemon/host restart recovery is not guaranteed by these instructions. A concrete startup or supervision failure is recorded as `runner-failed` after owned-process cleanup. This phase does not claim callback delivery. A startup-handshake timeout alone still requires reconciliation: if the runner itself diff --git a/agent-skills/skillette/codex-wake-on-exit/wake-on-exit b/agent-skills/skillette/codex-wake-on-exit/wake-on-exit index 3f171b3..bcf6a13 100755 --- a/agent-skills/skillette/codex-wake-on-exit/wake-on-exit +++ b/agent-skills/skillette/codex-wake-on-exit/wake-on-exit @@ -4,8 +4,18 @@ * * `start NEW-ABSOLUTE-RUN-DIR -- PROGRAM ARG...` captures CODEX_THREAD_ID and cwd, * confirms local queue support, and returns only after the detached runner starts. - * `status RUN-DIR`, `stop RUN-DIR`, and `ack RUN-DIR RUN-ID` never start another job. - * Output stays in private files; queued messages contain identifiers, not command output. + * `status RUN-DIR`, `stop RUN-DIR`, `supersede RUN-DIR RUN-ID`, and `ack RUN-DIR RUN-ID` never + * start another job. Output stays in private files; queued messages contain identifiers, + * not command output. + * + * A run belongs to the thread that armed it, for its whole life. Its callback can only + * wake that thread, even after the thread stops being used and another thread resumes + * the same flow from its durable log. `status` therefore reports the owner rather than + * refusing a foreign run, and `supersede` is the one cross-thread action: a thread taking + * over the flow uses it to retire the old thread's run. A callback the old run already + * queued can still wake its owner, but the owner's `ack` refuses it, so the owner cannot + * start a second copy of the flow from it. Supersession governs only the run: it cannot + * stop an old thread that is already working without a watcher. */ import { spawn } from "node:child_process"; import { createWriteStream } from "node:fs"; @@ -54,16 +64,115 @@ function exists(path: string) { } } -/** Keep lifecycle mutations confined to the exact Codex thread that armed the job. */ -function owned(dir: string, thread: string): { manifest: Manifest; state: State } { +/** + * Read a run's identity without deciding who may act on it. + * + * Only the manifest and state agreeing on one run ID and one thread make the record + * usable; a mismatch means the directory is not a run this helper wrote. + */ +function inspect(dir: string): { manifest: Manifest; state: State } { const manifest: Manifest = JSON.parse(Deno.readTextFileSync(join(dir, "manifest.json"))); const state: State = JSON.parse(Deno.readTextFileSync(join(dir, "state.json"))); - if (!uuid.test(thread) || manifest.thread !== thread || state.id !== manifest.id || state.thread !== thread) { - throw new Error("This run does not belong to the current Codex thread"); + if (state.id !== manifest.id || state.thread !== manifest.thread) { + throw new Error("This run's manifest and state disagree; it is not a usable wake-on-exit run"); } return { manifest, state }; } +/** + * Keep lifecycle mutations confined to the exact Codex thread that armed the job. + * + * "Owned" means the manifest names the current CODEX_THREAD_ID. A durable log that + * records the run for a flow this thread has resumed does not make it owned: the + * callback still targets the arming thread. `supersede` is the only exception. + */ +function owned(dir: string, thread: string): { manifest: Manifest; state: State } { + const record = inspect(dir); + if (!uuid.test(thread) || record.manifest.thread !== thread) { + throw new Error("This run does not belong to the current Codex thread"); + } + return record; +} + +/** + * The single claim on a run's callback, stored in `ack.json`. + * + * Exactly one of two things can claim a run: its owner acknowledging the callback, or a + * successor thread superseding the run (`supersededBy` set). Both go through `claim`, so + * whichever comes first wins and the other is refused; an owner and a successor can never + * both act on the same run. The file name predates supersession and is kept so that runs + * acknowledged by an older copy of this helper still count as claimed. + */ +type Claim = { id: string; at: string; supersededBy?: string }; + +/** + * The run's claim, if any. + * + * This helper publishes claims whole, but an older copy created `ack.json` and then wrote + * it, so a claim can be empty or partial (mid-write, or after an interrupted write). Only + * older copies write that way, and they only ever wrote the owner's acknowledgment, so an + * unreadable claim is read as exactly that. + */ +function readClaim(dir: string): Claim | undefined { + const path = join(dir, "ack.json"); + if (!exists(path)) return undefined; + try { + return JSON.parse(Deno.readTextFileSync(path)); + } catch (error) { + if (error instanceof SyntaxError) return { id: "", at: "" }; + throw error; + } +} + +/** + * Publish `claim` as the run's claim unless one exists, and return the earlier claim if it does. + * + * The record is written to a private temporary file and hard-linked into place: the link + * fails if a claim already exists, and a reader can never see a partly written one. This + * needs a filesystem with hard links, which the local run directories the skillette asks + * for have on Linux and macOS. + */ +function claim(dir: string, record: Claim): Claim | undefined { + const temporary = join(dir, `.ack-${crypto.randomUUID()}`); + try { + Deno.writeTextFileSync(temporary, JSON.stringify(record) + "\n", { mode: 0o600, createNew: true }); + Deno.linkSync(temporary, join(dir, "ack.json")); + return undefined; + } catch (error) { + if (error instanceof Deno.errors.AlreadyExists) return readClaim(dir); + throw error; + } finally { + // Best effort: once the link exists the claim is committed, and a leftover private + // temporary file must not turn that into an error the caller would act on. + try { + Deno.removeSync(temporary); + } catch { + // Ignored deliberately; see above. + } + } +} + +/** + * Report a run's state and who owns it, for any thread. + * + * This is read-only. It exists so that a thread finding another thread's run in a + * durable log learns that the run is not its watcher from the helper, rather than + * from reading state files and process IDs that cannot say which thread a callback + * will wake. `acknowledged` means the owner consumed the callback; a superseded run + * reports `acknowledged: false` with `superseded` naming the successor instead. + */ +export function status(dir: string, thread: string) { + const { manifest, state } = inspect(dir); + const existing = readClaim(dir); + return { + ...state, + acknowledged: existing !== undefined && existing.supersededBy === undefined, + currentThread: thread, + ownedByCurrentThread: manifest.thread === thread, + superseded: existing?.supersededBy ? { by: existing.supersededBy, at: existing.at } : null, + }; +} + /** Bound startup and cancellation handshakes inside the helper, without waking the model. */ async function until(read: () => T | undefined, milliseconds = 15000): Promise { const deadline = Date.now() + milliseconds; @@ -118,21 +227,77 @@ export async function stop(dir: string, thread: string) { }); } -/** Consume a matching terminal callback once; cancelled and duplicate messages cannot restart a flow. */ +/** + * Retire another thread's run on behalf of the current thread, which is taking over its flow. + * + * `id` is the run ID the flow's durable log records, so a mistyped or stale directory is + * refused rather than retired. Success means the retirement is committed: the owner's + * `ack` will refuse this run's callback whenever it arrives, including one already queued + * or still being queued. Supersession is refused when the owner has already acknowledged + * the callback (the owner may be running the flow) or another successor claimed the run + * first; repeating it from the same successor is harmless. + * + * Cancellation then goes through the same `cancel.json` request as `stop`, which the live + * runner (including one started by an older copy of this helper) handles for its own + * process group; nothing here signals a PID. `cleanup: "pending"` means the runner had not + * finished within `settle` milliseconds. That does not undo the retirement: the old + * watcher can at most queue a callback its owner will refuse. This never transfers the + * run's result to the caller. + */ +export async function supersede(dir: string, id: string, thread: string, settle = 15000) { + if (!uuid.test(thread)) throw new Error("Current CODEX_THREAD_ID is unavailable; do not guess another session"); + const { manifest, state } = inspect(dir); + if (manifest.id !== id) throw new Error("Run ID does not match this run directory; check the durable log"); + if (manifest.thread === thread) throw new Error("This run belongs to the current Codex thread; use stop instead"); + const at = new Date().toISOString(); + const prior = claim(dir, { id, at, supersededBy: thread }); + if (prior && prior.supersededBy === undefined) { + throw new Error("The owning thread already acknowledged this run's callback and may be running the flow"); + } + if (prior && prior.supersededBy !== thread) { + throw new Error(`Thread ${prior.supersededBy} already superseded this run`); + } + const superseded = { by: thread, at: prior?.at ?? at }; + // From here on the retirement is committed. A failure while requesting or awaiting + // cancellation is reported as pending cleanup, not thrown, so that the successor + // still records the takeover and arms its own run. + try { + if (!exists(join(dir, "cancel.json"))) save(dir, "cancel.json", { id }); + // `delivering` is not terminal: the runner may still be queueing to the owner. + const deadline = Date.now() + settle; + for (;;) { + const current = inspect(dir).state; + const settled = !["starting", "watching", "delivering"].includes(current.phase); + if (settled || Date.now() >= deadline) { + return { ...current, superseded, cleanup: settled ? "complete" : "pending" }; + } + await new Promise((resolve) => setTimeout(resolve, 25)); + } + } catch (error) { + return { ...state, superseded, cleanup: "pending", error: String(error) }; + } +} + +/** + * Consume a matching terminal callback once; cancelled, superseded, and duplicate + * messages cannot restart a flow. + */ export function acknowledge(dir: string, id: string, thread: string) { const { state } = owned(dir, thread); if (state.id !== id) throw new Error("Callback run ID does not match the registered run"); - if (exists(join(dir, "cancel.json")) || state.phase === "cancelled") return { ignored: "cancelled" }; + // Supersession also requests cancellation, and the owner must learn that another thread + // now runs the flow, not merely that it stopped. `supersede` publishes its claim before + // `cancel.json`, so reading the claim after observing cancellation always sees it. + const cancelled = exists(join(dir, "cancel.json")) || state.phase === "cancelled"; + const existing = readClaim(dir); + if (existing?.supersededBy) return { ignored: "superseded", by: existing.supersededBy }; + if (cancelled) return { ignored: "cancelled" }; if (!state.result) throw new Error("Watcher has not produced a terminal result"); - try { - Deno.writeTextFileSync(join(dir, "ack.json"), JSON.stringify({ id, at: new Date().toISOString() }), { - mode: 0o600, - createNew: true, - }); - } catch (error) { - if (error instanceof Deno.errors.AlreadyExists) return { ignored: "duplicate" }; - throw error; - } + // The claim is what serializes this against a concurrent supersede; the check above + // only gives the common case a clear answer. + const prior = claim(dir, { id, at: new Date().toISOString() }); + if (prior?.supersededBy) return { ignored: "superseded", by: prior.supersededBy }; + if (prior) return { ignored: "duplicate" }; return { ...state, stdoutPath: join(dir, "stdout.log"), stderrPath: join(dir, "stderr.log") }; } @@ -319,7 +484,9 @@ function currentThread() { if (import.meta.main) { const [action, dir, ...rest] = Deno.args; if (!dir) { - throw new Error("Usage: start NEW-RUN-DIR -- PROGRAM ARG... | status RUN-DIR | stop RUN-DIR | ack RUN-DIR RUN-ID"); + throw new Error( + "Usage: start NEW-RUN-DIR -- PROGRAM ARG... | status RUN-DIR | stop RUN-DIR | supersede RUN-DIR RUN-ID | ack RUN-DIR RUN-ID", + ); } if (action === "supervise") { supervise(dir); @@ -330,9 +497,11 @@ if (import.meta.main) { const output = action === "start" && rest[0] === "--" ? await start(dir, rest.slice(1), { thread, cwd: Deno.cwd(), queue: ["codex"] }) : action === "status" - ? { ...owned(dir, thread).state, acknowledged: exists(join(dir, "ack.json")) } + ? status(dir, thread) : action === "stop" ? await stop(dir, thread) + : action === "supersede" && rest.length === 1 + ? await supersede(dir, rest[0], thread) : action === "ack" && rest.length === 1 ? acknowledge(dir, rest[0], thread) : (() => { @@ -391,6 +560,143 @@ Deno.test("stop after queueing latches cancellation without signaling a stale pr } }); +/** + * Regression for a thread that resumed a flow and kept the previous thread's live run as + * its watcher: the helper's only answer was a refusal, so agents read state files and + * PIDs instead and concluded the run was fine. Status must name the owner for any thread. + */ +Deno.test("status reports a foreign run's owner instead of refusing it", async () => { + const dir = Deno.makeTempDirSync(); + const owner = crypto.randomUUID(); + const other = crypto.randomUUID(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: owner, id }); + save(dir, "state.json", { thread: owner, id, phase: "watching" }); + const foreign = status(dir, other); + assertEquals(foreign.ownedByCurrentThread, false); + assertEquals(foreign.thread, owner); + assertEquals(foreign.currentThread, other); + assertEquals(foreign.superseded, null); + assertEquals(status(dir, owner).ownedByCurrentThread, true); + // Reading status must not grant anything: lifecycle actions still refuse the foreign thread. + await assertRejects(() => stop(dir, other)); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + +/** + * A callback already queued to the old thread survives supersession. If that thread is ever + * resumed, its ack must refuse the callback, or two threads would drive the same flow. + */ +Deno.test("supersede checks the logged run and makes the owner's queued callback ignorable", async () => { + const dir = Deno.makeTempDirSync(); + const owner = crypto.randomUUID(); + const successor = crypto.randomUUID(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: owner, id }); + save(dir, "state.json", { thread: owner, id, phase: "queued", result: { code: 0, signal: null } }); + await assertRejects(() => supersede(dir, id, owner)); + // A directory that does not hold the run the log names is refused before anything is written. + await assertRejects(() => supersede(dir, crypto.randomUUID(), successor)); + assertEquals(exists(join(dir, "ack.json")), false); + const retired = await supersede(dir, id, successor); + assertEquals(retired.cleanup, "complete"); + assertEquals(retired.superseded.by, successor); + // The same successor may retry after an interruption; a different one may not take the run as well. + assertEquals((await supersede(dir, id, successor)).superseded.by, successor); + await assertRejects(() => supersede(dir, id, crypto.randomUUID())); + assertEquals(acknowledge(dir, id, owner), { ignored: "superseded", by: successor }); + assertEquals(status(dir, owner).superseded?.by, successor); + assertEquals(status(dir, owner).acknowledged, false); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + +/** A directory whose records disagree is not a run, and supersession must leave no claim in it. */ +Deno.test("supersede refuses inconsistent run records without writing a claim", async () => { + const dir = Deno.makeTempDirSync(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: crypto.randomUUID(), id }); + save(dir, "state.json", { thread: crypto.randomUUID(), id, phase: "watching" }); + await assertRejects(() => supersede(dir, id, crypto.randomUUID())); + assertEquals(exists(join(dir, "ack.json")), false); + assertEquals(exists(join(dir, "cancel.json")), false); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + +/** + * Runs acknowledged by an older copy of this helper have an `ack.json` without a trailing + * newline or `supersededBy`. They must still count as the owner's claim. + */ +Deno.test("a claim written by an older helper counts as the owner's acknowledgment", async () => { + const dir = Deno.makeTempDirSync(); + const owner = crypto.randomUUID(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: owner, id }); + save(dir, "state.json", { thread: owner, id, phase: "queued", result: { code: 0, signal: null } }); + Deno.writeTextFileSync(join(dir, "ack.json"), JSON.stringify({ id, at: new Date().toISOString() })); + assertEquals(acknowledge(dir, id, owner), { ignored: "duplicate" }); + await assertRejects(() => supersede(dir, id, crypto.randomUUID()), Error, "already acknowledged"); + assertEquals(status(dir, owner).acknowledged, true); + // An older helper interrupted between creating and writing the file leaves it empty. + Deno.writeTextFileSync(join(dir, "ack.json"), ""); + assertEquals(acknowledge(dir, id, owner), { ignored: "duplicate" }); + await assertRejects(() => supersede(dir, id, crypto.randomUUID()), Error, "already acknowledged"); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + +/** + * The owner and a successor race for the same callback when the owner's thread is still + * loaded. Whichever claims first wins, so the flow never ends up driven by both. + */ +Deno.test("an owner's acknowledgment and a successor's supersession exclude each other", async () => { + const dir = Deno.makeTempDirSync(); + const owner = crypto.randomUUID(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: owner, id }); + save(dir, "state.json", { thread: owner, id, phase: "queued", result: { code: 0, signal: null } }); + assertEquals("result" in acknowledge(dir, id, owner), true); + await assertRejects(() => supersede(dir, id, crypto.randomUUID()), Error, "already acknowledged"); + assertEquals(exists(join(dir, "cancel.json")), false); + assertEquals(status(dir, owner).acknowledged, true); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + +/** + * A runner that does not finish (or is mid-delivery to the owner) must not turn a committed + * retirement into an error, because the successor then cannot tell whether to rearm. + */ +Deno.test("supersede reports pending cleanup for a runner that has not settled", async () => { + const dir = Deno.makeTempDirSync(); + const owner = crypto.randomUUID(); + const successor = crypto.randomUUID(); + const id = crypto.randomUUID(); + try { + save(dir, "manifest.json", { thread: owner, id }); + save(dir, "state.json", { thread: owner, id, phase: "delivering", result: { code: 0, signal: null } }); + const retired = await supersede(dir, id, successor, 100); + assertEquals(retired.cleanup, "pending"); + assertEquals(retired.phase, "delivering"); + assertEquals(exists(join(dir, "cancel.json")), true); + assertEquals(acknowledge(dir, id, owner), { ignored: "superseded", by: successor }); + } finally { + Deno.removeSync(dir, { recursive: true }); + } +}); + /** Supply a recording queue executable through context injection, without changing process environment. */ async function fixture() { const directory = Deno.makeTempDirSync({ prefix: "codex wake test " }); @@ -466,6 +772,32 @@ Deno.test("stop signals the live watcher and does not queue a cancellation wake" } }); +/** + * Supersession must stop a live foreign watcher through its own runner, so the old + * thread is never queued a wake for a flow it no longer runs. + */ +Deno.test("supersede cancels a live foreign watcher without queueing its owner", async () => { + const { directory, watcher, context } = await fixture(); + const dir = join(directory, "run"); + try { + await start(dir, [Deno.execPath(), "run", "-A", watcher, "30000", "0"], context); + await until(() => exists(join(directory, "ready.json")) ? true : undefined); + const successor = crypto.randomUUID(); + const { manifest } = inspect(dir); + const state = await supersede(dir, manifest.id, successor); + assertEquals(state.phase, "cancelled"); + assertEquals(state.cleanup, "complete"); + assertEquals(state.superseded.by, successor); + assertEquals(exists(join(directory, "cancelled.json")), true); + assertEquals(exists(join(directory, "queued.json")), false); + // Supersession is recorded before cancellation, so the owner sees a takeover, not a plain stop. + assertEquals(acknowledge(dir, manifest.id, context.thread), { ignored: "superseded", by: successor }); + } finally { + if (exists(join(dir, "state.json"))) await stop(dir, context.thread); + Deno.removeSync(directory, { recursive: true }); + } +}); + /** * Real descendants exercise group ownership and inherited-pipe behavior that a * single-process watcher cannot cover. A release file also bounds failed tests