Skip to content

refactor(core): settle steps lock-free by joining tool fibers first#38743

Open
kitlangton wants to merge 1 commit into
v2from
single-writer-step
Open

refactor(core): settle steps lock-free by joining tool fibers first#38743
kitlangton wants to merge 1 commit into
v2from
single-writer-step

Conversation

@kitlangton

@kitlangton kitlangton commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

What

Deletes all step-event locking from the runner — the v2 baseline's 12 serialized() semaphore sites become zero — and restructures callModel settlement so that every tool fiber is joined before any settlement write. Settlement becomes ~40 linear, contention-free lines. llm.ts lands net-negative against v2 (+105/−110).

flowchart LR
    P["provider loop — publish direct"] -->|tool-call, then fork| F[tool fiber]
    F -->|"ephemeral progress — direct"| U[UI]
    F -->|"own completion, masked, direct"| D[(durable history)]
    P --> D
    J["join ALL fibers (only interruptible line)"] --> S["settlement (masked, sole writer)"]
    S -->|"close unsettled calls, terminal event"| D
Loading

How

No lock, by invariant instead of mutex. The provider loop and each tool fiber publish durable events concurrently. This is safe for two reasons, now documented as a contract at the top of publish-llm-event.ts:

  1. Every publisher method commits its state marks synchronously before its first await — no yield between a check (tool.settled, stepStarted) and its mark, so check-and-mark is atomic under cooperative scheduling.
  2. Every required event order is per-source, and each source is one sequential fiber. A fiber's events are causally after its own Tool.Called (the fork happens below that publish). Cross-source order is unconstrained; either interleaving is a truthful history of concurrent work.

Join-first settlement. All owned fibers are awaited (all exits, not just the first failure) before any settlement logic. Afterwards no fiber is alive, the fold is final, and settlement is the step's only writer. The overflow-recovery, retry-escape, decline, sweep, and terminal-event logic is unchanged in substance — reordered after the join, with the thrown-LLM-failure branch (llmError) folded into the same linear sequence.

Fiber-owned completion, direct. Each tool fiber publishes its own durable completion under its existing uninterruptible mask, so a finished execution always reaches its settlement — including when the step is interrupted. Tool.Progress is Event.ephemeral (advisory UI state, not durable history) and publishes direct from the fiber.

Renames for truth. retryEvidenceoutputStarted (it is evidence the model already spoke, which bars retries and overflow recovery, not evidence for them). Three oracle getters (hasRetryEvidence/stepFailure/stepSettlement) collapse into one record(): StepRecord snapshot; needsContinuation is derived from the record at completion instead of accumulated mid-stream. Fixed en route: classifyToolExits typed its first failure non-optional via [0] indexing — now .at(0).

Why no lock (design history)

Two earlier revisions of this PR were built and superseded: a single-writer queue/feed (merged provider events and tool emissions into one ordered stream) and a two-permit semaphore. A requirements walk plus a survey of comparable agents (grok-build, pi) killed both: the queue's termination bookkeeping bought a slogan, and the semaphore protected interleavings between events that have no required order. The concrete tear both were guarding against cannot occur given the publisher's mutate-then-publish shape — so the synchronization became one documented invariant instead of a primitive. What we deliberately did not adopt from those codebases: both fabricate interrupted-tool state per client in the view layer (grok-build's reload silently drops in-flight calls). Eager per-call durable settlement stays; every projection reads the same settled truth.

Testing

From packages/core:

  • bun typecheck — clean
  • Recorded runner suites (exact durable event order) pass unmodified through both the lock removal and the settlement reorder
  • 255/255 across 12 focused suites; 760/760 with --rerun-each 5 on the order-pinning suites — the empirical check that lock-free publication does not destabilize durable order
  • Full suite 1441/1442: the one failure is the known pre-existing PluginSupervisor config flake, verified failing identically on clean v2

@kitlangton kitlangton changed the title refactor(core): route all step publishes through one writer refactor(core): single-writer step feed and pure settlement Jul 24, 2026
@kitlangton
kitlangton force-pushed the single-writer-step branch from d19acc5 to 6c1a76d Compare July 24, 2026 20:22
@kitlangton kitlangton changed the title refactor(core): single-writer step feed and pure settlement refactor(core): settle steps as a pure plan over the step record Jul 24, 2026
@kitlangton
kitlangton force-pushed the single-writer-step branch from 6c1a76d to 978787e Compare July 25, 2026 01:28
@kitlangton kitlangton changed the title refactor(core): settle steps as a pure plan over the step record refactor(core): settle steps lock-free by joining tool fibers first Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant