Support queued follow-up objectives so a run auto-starts the next task on completion

[proagentstore]

Feature request from a voice/chat operating session.

Problem: when a coding instance is mid-run, calling coding_loop_start again for that same instance/repo fails outright with an "already being worked on" error. Today the only options are to wait and manually re-poll for the run to finish, or stop the current run early, then remember to come back and issue the next objective myself. If I forget to check back, the follow-up request is just lost.

Requested behavior: let a caller attach one (or more) queued objective(s) to an instance while it's busy, so that the moment the active run reaches a terminal state (done, failed, or max iterations), the platform automatically starts the next queued objective without a human needing to catch the exact moment it frees up.

Suggested shape:

• A way to enqueue a follow-up objective for a specific instance (e.g. queue_instance_objective or an optional queue_if_busy flag on coding_loop_start) instead of erroring when the instance is occupied.
• Queue is per-instance, FIFO, visible via a list/check call so the caller can see what's queued and cancel/reorder before it starts.
• When the active run ends, the next queued objective is dequeued and started automatically, same semantics as a normal coding_loop_start (same max_iterations handling, same budget checks).
• If no queue and the instance is busy, keep today's current error behavior as the default (don't silently auto-queue unless asked).

Use case that surfaced this: reviewing a just-shipped feature with a self-review run, while wanting to line up a second, unrelated build task to start automatically right after — without babysitting the run to catch the free moment.

[serge-ivo]

Implemented and on main as fbfe809 — CI green, API/MCP/Host workers deployed, MCP 0.1.24 published to the registry.

What it does

coding_loop_start gains an opt-in queue_if_busy. When the repo is already being worked on, the objective is queued instead of refused, and the platform starts it itself the moment the active run reaches a terminal state — done, failed, or max iterations. Nobody has to catch the free moment.

coding_loop_start(instance_id, objective, queue_if_busy: true)
  → 202 {"queued": true, "entry": {"id": "objq-…", "status": "pending"}, "blocked": "<what it is waiting behind>"}

coding_loop_queue(instance_id [, repo_id])        → the pending entries, FIFO, top one starts next
coding_loop_queue_cancel(instance_id, entry_id)   → withdraw one before it starts

Over HTTP: POST /v1/instances/{id}/loop takes queueIfBusy; GET …/loop/queue and DELETE …/loop/queue/{entryId} are documented in store/openapi.yaml.

The four design calls, and why

Keyed (instance_id, repo_id), not per-instance. The lock this works around is the per-session-driver claim (#208), i.e. per repo — pickLoopRepo chooses the repo and claimSessionDriver claims that repo's session. A queue keyed only by instance would park a follow-up for repo B behind a run on repo A that never contended with it, on exactly the multi-repo Coder #374 exists for. repo_id NULL means "any", and is the only kind a chat-driver loop can drain.

Only the BUSY refusal queues. loop-drivers.ts now marks the single-flight refusal reason: "busy", and queue_if_busy turns on that field rather than on status === 409. The other 409s on the same path — no repository on the agent, no runner connected, a checkout that failed admission — are structural, and waiting fixes none of them; queueing behind one parks work that can never drain. Off by default throughout, per this issue's "don't silently auto-queue unless asked": a caller that did not ask still gets today's error.

Drains from the run's own terminal path. A step.do after closeDelegation in the Pilot (workflows/coding-session.ts) and after finish in the chat loop (workflows/agent-loop.ts) — last, so the session is closed and the driver claim released before another run is admitted, and inside if (!resuming) so a run Cloudflare is about to replay does not get a second run started against the engine it carries on driving. finishLoopRun stays what lib/status-domain.ts says it is: the single pure terminal write.

A drained entry gets its own budget. openBudget + resolveAccountCeilings, per entry. A queued objective is a new autonomous entry point and #184 says every one of those admits separately — inheriting the finished run's pool would let one coding_loop_start open a budget an unbounded chain of follow-ups then draws on. max_iterations is stored raw and clamped at START time, so an entry that sat in the queue across a limits change is bounded by the limit in force when it actually runs; an entry that named no number gets the same default a plain coding_loop_start gets.

Nothing is lost silently

Every way a start can fail writes a terminal status, because an entry that vanishes without one is this issue's complaint wearing a different hat:

outcome	entry becomes
started	started, with run_id — the run record is its account from then on
busy again (a race with another start)	back to pending, keeping its created_at — losing a race is not a reason to go to the back of the queue
budget refused	failed, reason recorded, plus an error_log row
no runner / no repo / bad checkout	failed with the driver's own sentence

The drain never throws and drains exactly one entry per call: the run it starts reaches its own terminal path and drains the next, so the queue empties in FIFO order at the rate work actually completes.

Not implemented: reorder

The issue mentions "cancel/reorder". Cancel is in; reorder deliberately is not. FIFO is the whole contract — "the moment the active run ends, the next one starts" — and a reorder verb turns the order into something every caller has to re-check rather than something it can rely on. Cancel-and-requeue expresses any reordering needed while keeping that invariant true. Happy to add it if the FIFO guarantee turns out not to be worth that.

Changes

migration 0149                        instance_objective_queue
workers/api/src/lib/objective-queue.ts        rows: enqueue, list, cancel, atomic claim, close, requeue
workers/api/src/lib/objective-queue-start.ts  the drain
workers/api/src/routes/loop-queue-routes.ts   the two reads — registered ABOVE /loop/:runId, because
                                              Hono matches in order and "queue" would otherwise be
                                              read as a run id (a test asserts that ordering)
workers/api/src/lib/loop-drivers.ts           reason: "busy" on the claim refusal
workers/api/src/routes/tools.ts               the queueIfBusy branch
workers/api/src/workflows/{coding-session,agent-loop}.ts   the drain hooks
workers/mcp/…                                 queue_if_busy + two tools; surface 0.1.23 → 0.1.24

55 new tests across four files, covering enqueue, the dequeue predicate and its atomic claim, cancel, route ordering, and every branch of the auto-start-on-completion decision table.