From d8434dfaa5cfa415b1e1767e61d7c76d05963b4a Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sat, 18 Jul 2026 15:01:34 +0200 Subject: [PATCH 1/7] Add Manager/Worker orchestration design decision entry Design/spike output for the coordinated Manager/Worker model: - Gate verdict (verified live, herdr 0.7.0): agent_status is pollable for agent-start tabs (idle|working|blocked|done), hook-integration-based; agent read returns clean TUI text. Resolves the previously-UNVERIFIED agent-status flag. - Cross-agent constraint: workers may be claude/codex/grok; only claude+codex are hook-integrated (status + session UUID), grok is TUI-scrape-only. git/PR state is the one uniform bus; the contract is agent-agnostic. - The .mailbox/ protocol: outbox+inbox (single-writer, single-sequencer), offset-drain, Claude Code hook push (Stop/SessionStart/Notification), herdr idle-wake as the only residual push. - Milestone-defined worker autonomy, deterministic merge sequencer, roadmap-as-derived-view co-equal with the GH board. Spawns 7 implementation task files (local backlog): add-lane-registry, add-lane-mailbox, add-manager-watch-loop, extend-worker-autonomy, add-merge-sequencer, add-roadmap-skill, add-mailbox-statusline. Co-Authored-By: Claude Opus 4.8 (1M context) --- .claude/knowledge/_index.md | 1 + .../manager-worker-orchestration.md | 137 ++++++++++++++++++ 2 files changed, 138 insertions(+) create mode 100644 .claude/knowledge/architecture/manager-worker-orchestration.md diff --git a/.claude/knowledge/_index.md b/.claude/knowledge/_index.md index 288bed5..17a2637 100644 --- a/.claude/knowledge/_index.md +++ b/.claude/knowledge/_index.md @@ -6,6 +6,7 @@ - `architecture/model-economics.md` — Which model per skill and why (Haiku/Sonnet/session-model) - `architecture/idempotent-scaffolding.md` — Scaffolding into shared user files: absorb unmarked sections, lazy dirs, never overwrite user content - `architecture/worktree-task-file-copy.md` — Why `/kickoff` copies the task file into the worktree (not a symlink): no skill simplification, avoids accidental-commit repo poisoning + cross-platform fragility +- `architecture/manager-worker-orchestration.md` — Design decisions for the coordinated Manager/Worker model: lane identity = worktree path, cross-agent (claude/codex/grok) git-as-uniform-bus, the `.mailbox/` protocol (outbox+inbox, hook-driven push), milestone worker autonomy, merge sequencer, roadmap-as-derived-view ## Features - `features/backfill-and-origin-metadata.md` — `/backfill-knowledge` significance bar + origin-reconstruction cascade diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md new file mode 100644 index 0000000..d326dac --- /dev/null +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -0,0 +1,137 @@ +--- +title: "Manager/Worker Orchestration (design)" +createdAt: 2026-07-18 +createdFrom: "session: design-manager-worker-orchestration 2026-07-18" +pluginVersion: 1.8.1 +prime: false +--- + +# Manager/Worker Orchestration (design) + +Design decisions for evolving work-system from fire-and-forget kickoffs into a +coordinated Manager/Worker model. This is the **decision record**; the +implementation is spawned across tasks `add-lane-registry`, `add-lane-mailbox`, +`add-manager-watch-loop`, `extend-worker-autonomy`, `add-merge-sequencer`, +`add-roadmap-skill`, `add-mailbox-statusline`. Full working notes: the design +task's `DESIGN-NOTES.md`. + +## The model +- **Manager** = the Claude Code session at the main repo root (herdr `◉` tab). A + *coordinator*, not a merge robot — the human stays merge authority unless + explicitly delegated at kickoff. +- **Worker** = one {claude|codex|grok} session per worktree, driving its task to a + reviewed, mergeable PR. +- **Lane** = `(worktree_path, task, branch)`. **Identity = worktree_path** — the one + key stable across agent types and restarts. herdr pane/tab, `agent_status`, + session UUID, PR state are live-attached attributes, **never identity**. Only the + task file persists; everything else is derived live (no lane registry file, no + stashed ids). + +## Gate verdict (herdr substrate, verified live 2026-07-16, herdr 0.7.0) +- An `agent start`-launched tab **does** expose a pollable `agent_status` ∈ + `idle|working|blocked|done|unknown`. `blocked` (worker on an input/permission + prompt) is the primary "needs the Manager" signal. Poll via `agent list`/`agent + get`; **bounded** block-wait via `agent wait --status … --timeout` (no busy loop). +- Status is produced by **installed hook integrations** merged with a TUI-scrape + rule engine (`agent explain`), NOT self-reported by the agent. +- `agent read --source recent` returns clean text incl. Claude's `※ recap:` line + + statusline. This RESOLVES the previously-UNVERIFIED agent-status flag in + [herdr-kickoff-automation](../features/herdr-kickoff-automation.md) / + [herdr-close-automation](../features/herdr-close-automation.md) and adds `blocked` + to their documented `idle|working|done` set. + +## Cross-agent constraint (workers may be claude, codex, OR grok) +The Manager is always Claude Code. Workers are tiered by herdr integration: +- **claude, codex** — installed hook (`herdr-agent-state.sh`) → authoritative status + **+ session UUID**; `agent read` carries branch+`PR#` (claude also `※ recap:`). +- **grok** — **no installable integration, no session UUID**, TUI-scrape-only status + (coarse; `blocked` unlikely), minimal `agent read`. +→ The orchestration **contract is expressed in git/PR terms**, the ONE channel +uniform across all three (every agent produces commits/branches/PRs). `agent_status` +is a coarse liveness layer; `agent read` is best-effort detail, parsed *per agent +type*, never format-required. Agent-specific skills (`/open`, `/rebase`) are each +worker's *implementation* of hitting agent-agnostic milestones — soft-coupling +([skill-composition](skill-composition.md)) applied across agent types. + +## Worker↔Manager mailbox (the coordination substrate) +A file mailbox carries judgment/intent signals that have no git artifact. +- **`/.mailbox/`** per participant (worker's worktree / Manager's main root), + **tool-agnostic name** (no herdr/claude). `outbox.jsonl` + `inbox.jsonl`, **JSONL + append-only**, **gitignored** (lane-local, never committed). +- **Envelope:** `{id, ts, from, to, type, body}`. `from`+`to` required. Types: + `ready-to-close`, `blocked-on-decision`, `coordination-request`, `needs-human`. +- **Topology — outbox + inbox, NOT inbox-only.** Each participant writes **only its + own outbox**; the Manager is the **only** writer of any inbox. Buys: single writer + per file (no clobber), no cross-worktree writes by a worker, and the + **single-sequencer invariant** — worker→worker messages are *addressable* but + **route through the Manager** (it drains outboxes, delivers to inboxes, may + reorder/veto/batch). Workers never write another lane's inbox. +- **Drain by offset, never mid-file delete.** Reader (Manager for outboxes, worker + for its inbox) tracks a last-read byte offset in a sidecar; files stay append-only, + archived/truncated only once fully consumed. Preserves single-writer. +- **`ready-to-close` is worker-authoritative.** Only the worker knows if a *second* + PR is open or a post-merge TODO remains — a single PR's `✓ merged` is a soft + "near-done" hint, NOT a close trigger. The worker emits `ready-to-close`; the + Manager/human then runs `/close`. + +## Push without polling — Claude Code hooks first +Files are pull; hooks make delivery push-like at turn boundaries, all gated by a +cheap `stat` (append-only ⇒ compare byte offset, read only the new tail): +- **Stop hook** (turn end): inbox grew → `decision:block` + `additionalContext` + injects the message so the worker continues. Respect `stop_hook_active` (exit 0 when + true) + block only on a real offset advance → no loop / 8-block cap. Injected text is + context; the worker still applies its own autonomy gates. +- **SessionStart(`resume`)**: pull what arrived while closed. **UserPromptSubmit**: + belt-and-suspenders prepend. +- **Notification hook** (`permission_prompt`/`idle_prompt`) auto-emits + `blocked-on-decision`/`needs-human` to the outbox — worker-stuck detection becomes + push, no Manager poll. +- **Manager** drains all outboxes at its own Stop/UserPromptSubmit boundaries. +- The **one gap**: a worker idle *at the prompt* (already Stopped). Only here does a + herdr **idle-wake ping** (`agent send` onto idle) earn its keep. Optional. +Hooks are Claude-Code-specific → reinforces **claude-first**; codex/grok stay +poll/Manager-inferred until they grow equivalents. + +## Worker autonomy arc (milestone-defined, agent-tiered) +Default arc = git/PR **milestones**: `commits → push → PR open → review pass → +ready-to-close`. claude self-drives via work-system/pr-flow/swarm; codex/grok drive +to *PR open* with `gh`, and the **Manager fills the review gap** (`/swarm:review --pr +N` — any agent's PR is a valid target). Kickoff **pre-authorizes** commit/push/PR/ +review/own-branch-rebase/agreed-fixes. **Stop-early gates** (→ `blocked`/escalate): +failing CI, scope drift, any destructive/irreversible step, **merge to main** (never +without explicit delegation), force-push beyond own-branch lease, any human decision. + +## Merge sequencing — deterministic decision helper +`merge-sequencer.sh` (a *decision* helper, not an actor) over the lanes view outputs: +mergeable-now (`◆` + CI green + no conflict) and, after a merge, rebase-due lanes +(behind main ∧ file overlap). One PR at a time; the human authorizes each; the +Manager sends rebase intents. The `marketplace.json`+`plugin.json` version-line +collision is baked in. Encodes today's ROADMAP prose rule as a script (prose-drift). + +## Roadmap — a derived view, co-equal with the GH board +Source of truth = task files + git/PR state. `ROADMAP.md` and the future GH board +(see task `design-github-projects-task-source`) are **two derived views**, not +competing sources — ROADMAP is the local/offline projection, the board the +shared/online one; both read the same `states`/`task-status.sh` substrate. Inside +ROADMAP.md, a marker-delimited **Manager-auto block** (per-lane state ○●◇◆✓, PR#, +version) is regenerated from the lanes view; the **human-curated block** (waves, +serialization policy, rationale) is never machine-touched (idempotent-scaffolding: +absorb-unmarked, never overwrite user content). Owned by a new `/roadmap` skill; +`/list` stays the ephemeral snapshot. + +## Safety envelope (cross-cutting) +Fail-closed pane id (reuse `worktree-tab-state` tri-state — never inject into an +unverified pane); no force-push beyond own-branch `--force-with-lease`, never +shared/main; no merge-to-main without delegation; bounded herdr waits, survey refresh +`--cached`; grok `always-approve` ⇒ Manager sends nothing destructive to a grok lane. + +## Reused existing substrate (build ON, don't reinvent) +- `ws-statusline.sh states
[--cached]` → `task\tstate\tglyph` = a ready-made + lane registry with states; the state machine ○●◇◆✓ + `◉` and the tab↔worktree join + already exist in [herdr-tab-glyphs](../features/herdr-tab-glyphs.md). +- The PR cache (`headRef\tstate\treviewDecision`) + sync/`--cached` refresh policy. + +Related: [skill-composition](skill-composition.md), +[idempotent-scaffolding](idempotent-scaffolding.md), +[herdr-kickoff-automation](../features/herdr-kickoff-automation.md), +[herdr-tab-glyphs](../features/herdr-tab-glyphs.md). From b710f1b303397497dbc8bf593c1e1f7c99f7d487 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sun, 19 Jul 2026 14:08:42 +0200 Subject: [PATCH 2/7] Extend orchestration design: central mailbox + handoff + broadcast MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two design additions (2026-07-19): - ready-to-close becomes a handoff report {summary, follow_ups, deploy, updates, learnings} so the worker's fresh context transfers to the Manager (→ /define, /curate, deploy-from-main-post-merge). - Mailbox moves to a central ~/.agent_messaging/ store (id = encoded absolute path): unlocks a broadcast channel + Manager presence registry for cross-Manager / system-wide coordination, and makes never-commit automatic. Broadcast ships as a follow-up task (add-agent-broadcast); aligns with plugin-settings-system's [related_projects] peering. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../manager-worker-orchestration.md | 41 ++++++++++++++----- 1 file changed, 30 insertions(+), 11 deletions(-) diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index d326dac..5a5527a 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -12,8 +12,8 @@ Design decisions for evolving work-system from fire-and-forget kickoffs into a coordinated Manager/Worker model. This is the **decision record**; the implementation is spawned across tasks `add-lane-registry`, `add-lane-mailbox`, `add-manager-watch-loop`, `extend-worker-autonomy`, `add-merge-sequencer`, -`add-roadmap-skill`, `add-mailbox-statusline`. Full working notes: the design -task's `DESIGN-NOTES.md`. +`add-roadmap-skill`, `add-mailbox-statusline`, and `add-agent-broadcast` (follow-up). +Full working notes: the design task's `DESIGN-NOTES.md`. ## The model - **Manager** = the Claude Code session at the main repo root (herdr `◉` tab). A @@ -55,24 +55,43 @@ worker's *implementation* of hitting agent-agnostic milestones — soft-coupling ## Worker↔Manager mailbox (the coordination substrate) A file mailbox carries judgment/intent signals that have no git artifact. -- **`/.mailbox/`** per participant (worker's worktree / Manager's main root), - **tool-agnostic name** (no herdr/claude). `outbox.jsonl` + `inbox.jsonl`, **JSONL - append-only**, **gitignored** (lane-local, never committed). +- **Central `~/.agent_messaging/` — NOT per-worktree** (decided 2026-07-19). Participant + id = **encoded absolute path**; a hook derives its dir from `cwd`: + `~/.agent_messaging/lanes//{outbox,inbox}.jsonl`. Central over `/.mailbox/` + because (a) it is the rendezvous for **broadcast / cross-Manager** (below), (b) + never-commit is automatic (outside every repo — no per-repo gitignore), (c) no + cross-worktree-boundary writes. Tool-agnostic name, JSONL append-only; survives `/close` + removing the worktree. - **Envelope:** `{id, ts, from, to, type, body}`. `from`+`to` required. Types: - `ready-to-close`, `blocked-on-decision`, `coordination-request`, `needs-human`. + `ready-to-close`, `blocked-on-decision`, `coordination-request`, `needs-human`, + `broadcast-request`. - **Topology — outbox + inbox, NOT inbox-only.** Each participant writes **only its own outbox**; the Manager is the **only** writer of any inbox. Buys: single writer - per file (no clobber), no cross-worktree writes by a worker, and the + per file (no clobber), no cross-participant writes by a worker, and the **single-sequencer invariant** — worker→worker messages are *addressable* but **route through the Manager** (it drains outboxes, delivers to inboxes, may reorder/veto/batch). Workers never write another lane's inbox. - **Drain by offset, never mid-file delete.** Reader (Manager for outboxes, worker for its inbox) tracks a last-read byte offset in a sidecar; files stay append-only, archived/truncated only once fully consumed. Preserves single-writer. -- **`ready-to-close` is worker-authoritative.** Only the worker knows if a *second* - PR is open or a post-merge TODO remains — a single PR's `✓ merged` is a soft - "near-done" hint, NOT a close trigger. The worker emits `ready-to-close`; the - Manager/human then runs `/close`. +- **`ready-to-close` is worker-authoritative + a handoff report.** Only the worker knows + if a *second* PR is open or a post-merge TODO remains — a single PR's `✓ merged` is a + soft "near-done" hint, NOT a close trigger. The message carries the worker's fresh + context as `{summary, follow_ups[], deploy?, updates[], learnings[]}`: `follow_ups` → + Manager `/define`; `deploy` runs **from main after merge** (Manager-at-main model); + `updates` = dependency/config recommendations (e.g. a plugin bump); `learnings` → + Manager `/curate`. The Manager acts on these, then it/human runs `/close`. + +## Broadcast + cross-Manager (follow-up phase; central location adopted now) +The central store unlocks system-wide coordination across the *multiple Managers* a +machine runs (one per project). `~/.agent_messaging/broadcast/global.jsonl` is the one +multi-writer append log every Manager reads (offset-tracked); `managers/.json` +presence files make Managers discoverable. Use: system-wide notices, or capability +queries ("who has Cloudflare access?") answered point-to-point into the asker's inbox. +**Single-sequencer up a level:** a worker emits `broadcast-request` and its **Manager** +decides/posts; **Managers peer-to-peer** (they are the coordinators). Aligns with +`plugin-settings-system`'s `[related_projects]` peering registry. Broadcast + registry +ship as a dedicated follow-up (`add-agent-broadcast`), not the Wave-1 core. ## Push without polling — Claude Code hooks first Files are pull; hooks make delivery push-like at turn boundaries, all gated by a From 4dd1074dd3912ee357c103d2ee19050f9f2338c3 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sun, 19 Jul 2026 14:25:14 +0200 Subject: [PATCH 3/7] Adopt agent-mail naming + Maildir/CloudEvents substrate, spike AMQ Prior-art research (2026-07-19): the mailbox is the recognized inbox/outbox pattern; AMQ (avivsinai/agent-message-queue) already implements our design (Maildir delivery, .agent-mail store, swarm/agent-teams, federation, presence). Decisions: - name the store ~/.agent-mail/ (community convention) - fix mechanics: Maildir (atomic rename, lockless, tmp/new/cur) + a CloudEvents-aligned envelope; align concepts with A2A for future interop - add spike-agent-mail-substrate (decide adopt-AMQ-adapter vs build-thin) before add-lane-mailbox; document the protocol in the work-system README Co-Authored-By: Claude Opus 4.8 (1M context) --- .../manager-worker-orchestration.md | 61 +++++++++++-------- 1 file changed, 34 insertions(+), 27 deletions(-) diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index 5a5527a..1d2229b 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -10,10 +10,10 @@ prime: false Design decisions for evolving work-system from fire-and-forget kickoffs into a coordinated Manager/Worker model. This is the **decision record**; the -implementation is spawned across tasks `add-lane-registry`, `add-lane-mailbox`, -`add-manager-watch-loop`, `extend-worker-autonomy`, `add-merge-sequencer`, -`add-roadmap-skill`, `add-mailbox-statusline`, and `add-agent-broadcast` (follow-up). -Full working notes: the design task's `DESIGN-NOTES.md`. +implementation is spawned across tasks `add-lane-registry`, `spike-agent-mail-substrate`, +`add-lane-mailbox`, `add-manager-watch-loop`, `extend-worker-autonomy`, +`add-merge-sequencer`, `add-roadmap-skill`, `add-mailbox-statusline`, and +`add-agent-broadcast` (follow-up). Full working notes: the design task's `DESIGN-NOTES.md`. ## The model - **Manager** = the Claude Code session at the main repo root (herdr `◉` tab). A @@ -55,25 +55,31 @@ worker's *implementation* of hitting agent-agnostic milestones — soft-coupling ## Worker↔Manager mailbox (the coordination substrate) A file mailbox carries judgment/intent signals that have no git artifact. -- **Central `~/.agent_messaging/` — NOT per-worktree** (decided 2026-07-19). Participant - id = **encoded absolute path**; a hook derives its dir from `cwd`: - `~/.agent_messaging/lanes//{outbox,inbox}.jsonl`. Central over `/.mailbox/` - because (a) it is the rendezvous for **broadcast / cross-Manager** (below), (b) - never-commit is automatic (outside every repo — no per-repo gitignore), (c) no - cross-worktree-boundary writes. Tool-agnostic name, JSONL append-only; survives `/close` - removing the worktree. -- **Envelope:** `{id, ts, from, to, type, body}`. `from`+`to` required. Types: - `ready-to-close`, `blocked-on-decision`, `coordination-request`, `needs-human`, - `broadcast-request`. +- **Central `~/.agent-mail/` — NOT per-worktree** (decided 2026-07-19; the community + "agent mail" name). Participant id = **encoded absolute path**; a hook derives its dir + from `cwd`: `~/.agent-mail/lanes//{outbox,inbox}/`. Central over + `/.mailbox/` because (a) it is the rendezvous for **broadcast / cross-Manager** + (below), (b) never-commit is automatic (outside every repo — no per-repo gitignore), + (c) no cross-worktree-boundary writes. Survives `/close` removing the worktree. +- **Prior art & substrate (researched 2026-07-19; spike-first decided).** Our design is + the recognized **inbox/outbox pattern**, and **AMQ** (`avivsinai/agent-message-queue`, + Go single binary) is startlingly close (Maildir delivery, `.agent-mail` store, + swarm/agent-teams mode, federation, presence, wake). `spike-agent-mail-substrate` + decides **adopt AMQ behind a soft-coupled adapter** (like swarm's codex/grok) vs. + **build thin**. Either way the mechanics are fixed: **Maildir** (atomic `rename()`, + no locking, `tmp/new/cur`, one file per message — supersedes the earlier append-JSONL + + byte-offset idea; unread = files in `new/`, read = moved to `cur/`) and a + **CloudEvents-aligned envelope** (`id, source(=from), type(reverse-DNS), time, subject/ + ext(=to), data(=body)`). Align concepts with **A2A** (its Task lifecycle maps to our + lane states) for a future HTTP bridge — don't adopt its transport now. +- **Message types:** `ready-to-close`, `blocked-on-decision`, `coordination-request`, + `needs-human`, `broadcast-request`. - **Topology — outbox + inbox, NOT inbox-only.** Each participant writes **only its own outbox**; the Manager is the **only** writer of any inbox. Buys: single writer - per file (no clobber), no cross-participant writes by a worker, and the - **single-sequencer invariant** — worker→worker messages are *addressable* but - **route through the Manager** (it drains outboxes, delivers to inboxes, may - reorder/veto/batch). Workers never write another lane's inbox. -- **Drain by offset, never mid-file delete.** Reader (Manager for outboxes, worker - for its inbox) tracks a last-read byte offset in a sidecar; files stay append-only, - archived/truncated only once fully consumed. Preserves single-writer. + per mailbox (Maildir already makes concurrent delivery lockless), no cross-participant + writes by a worker, and the **single-sequencer invariant** — worker→worker messages are + *addressable* but **route through the Manager** (it drains outboxes, delivers to + inboxes, may reorder/veto/batch). Workers never write another lane's inbox. - **`ready-to-close` is worker-authoritative + a handoff report.** Only the worker knows if a *second* PR is open or a post-merge TODO remains — a single PR's `✓ merged` is a soft "near-done" hint, NOT a close trigger. The message carries the worker's fresh @@ -84,7 +90,7 @@ A file mailbox carries judgment/intent signals that have no git artifact. ## Broadcast + cross-Manager (follow-up phase; central location adopted now) The central store unlocks system-wide coordination across the *multiple Managers* a -machine runs (one per project). `~/.agent_messaging/broadcast/global.jsonl` is the one +machine runs (one per project). `~/.agent-mail/broadcast/global.jsonl` is the one multi-writer append log every Manager reads (offset-tracked); `managers/.json` presence files make Managers discoverable. Use: system-wide notices, or capability queries ("who has Cloudflare access?") answered point-to-point into the asker's inbox. @@ -95,11 +101,12 @@ ship as a dedicated follow-up (`add-agent-broadcast`), not the Wave-1 core. ## Push without polling — Claude Code hooks first Files are pull; hooks make delivery push-like at turn boundaries, all gated by a -cheap `stat` (append-only ⇒ compare byte offset, read only the new tail): -- **Stop hook** (turn end): inbox grew → `decision:block` + `additionalContext` - injects the message so the worker continues. Respect `stop_hook_active` (exit 0 when - true) + block only on a real offset advance → no loop / 8-block cap. Injected text is - context; the worker still applies its own autonomy gates. +cheap check (Maildir ⇒ unread = files in `inbox/new/`, a `readdir`): +- **Stop hook** (turn end): new files in `inbox/new/` → `decision:block` + + `additionalContext` injects them so the worker continues, then moves them to `cur/`. + Respect `stop_hook_active` (exit 0 when true) + block only when genuinely new → no + loop / 8-block cap. Injected text is context; the worker still applies its own + autonomy gates. - **SessionStart(`resume`)**: pull what arrived while closed. **UserPromptSubmit**: belt-and-suspenders prepend. - **Notification hook** (`permission_prompt`/`idle_prompt`) auto-emits From 504abca7c5a9d8fc80617a4ed645b469a65e4cd3 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sun, 19 Jul 2026 16:22:35 +0200 Subject: [PATCH 4/7] Resolve agent-mail spike: adopt AMQ as transport only MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `amq swarm`'s self-reported claim/block/complete is NOT adopted as a task-state authority — it fights the derived-from-git/PR lanes model. AMQ owns message transport; herdr-derived lanes own state. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01LTZx9DqrXPXQFoTrCvcJmD --- .../architecture/manager-worker-orchestration.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index 1d2229b..9dec46b 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -65,8 +65,12 @@ A file mailbox carries judgment/intent signals that have no git artifact. the recognized **inbox/outbox pattern**, and **AMQ** (`avivsinai/agent-message-queue`, Go single binary) is startlingly close (Maildir delivery, `.agent-mail` store, swarm/agent-teams mode, federation, presence, wake). `spike-agent-mail-substrate` - decides **adopt AMQ behind a soft-coupled adapter** (like swarm's codex/grok) vs. - **build thin**. Either way the mechanics are fixed: **Maildir** (atomic `rename()`, + **decided (2026-07-19): adopt AMQ as *transport only*** behind a soft-coupled adapter + (like swarm's codex/grok; degrades to a thin in-house Maildir flow if absent). **`amq + swarm` is NOT adopted as a task-state authority** — its `claim/block/complete` is + *self-reported*, the opposite of our hook/TUI-scrape status grounded in git/PR artifacts; + AMQ owns transport, herdr-derived lanes own state. Either way the mechanics are fixed: + **Maildir** (atomic `rename()`, no locking, `tmp/new/cur`, one file per message — supersedes the earlier append-JSONL + byte-offset idea; unread = files in `new/`, read = moved to `cur/`) and a **CloudEvents-aligned envelope** (`id, source(=from), type(reverse-DNS), time, subject/ From 94f39ca03f61dcd34f3e883f082676f3414624a3 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sun, 19 Jul 2026 17:12:10 +0200 Subject: [PATCH 5/7] Sharpen agent-mail decision: AMQ-native, no thin fallback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - `amq swarm` explicitly out of scope: it bridges Claude Code's native Agent Teams (~/.claude/tasks/), a competing orchestration substrate — we use only AMQ core messaging (send/inbox-outbox/who/receipts/dlq/wake). - Drop the thin-Maildir fallback: AMQ is a prerequisite (brew install), absent degrades to the git/PR floor + manual relay (already the base). - Envelope is AMQ-native, not CloudEvents; CloudEvents/A2A stay conceptual alignment for a future HTTP bridge. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01LTZx9DqrXPXQFoTrCvcJmD --- .../manager-worker-orchestration.md | 25 +++++++++++-------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index 9dec46b..784436c 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -65,17 +65,22 @@ A file mailbox carries judgment/intent signals that have no git artifact. the recognized **inbox/outbox pattern**, and **AMQ** (`avivsinai/agent-message-queue`, Go single binary) is startlingly close (Maildir delivery, `.agent-mail` store, swarm/agent-teams mode, federation, presence, wake). `spike-agent-mail-substrate` - **decided (2026-07-19): adopt AMQ as *transport only*** behind a soft-coupled adapter - (like swarm's codex/grok; degrades to a thin in-house Maildir flow if absent). **`amq - swarm` is NOT adopted as a task-state authority** — its `claim/block/complete` is - *self-reported*, the opposite of our hook/TUI-scrape status grounded in git/PR artifacts; - AMQ owns transport, herdr-derived lanes own state. Either way the mechanics are fixed: - **Maildir** (atomic `rename()`, + **decided (2026-07-19): adopt AMQ core messaging as the transport** behind a soft-coupled + adapter (like swarm's codex/grok). **AMQ is a prerequisite, not reimplemented** — absent ⇒ + a `brew install` hint + degrade to the git/PR floor + manual relay (no thin-Maildir + fallback; that floor already exists, and a missing mailbox ≈ today's fire-and-forget, so no + regression). We use AMQ's own envelope + store (AMQ-native). **`amq swarm` is NOT used at + all**: despite the name it is a *bridge to Claude Code's native "Agent Teams"* (shared task + list at `~/.claude/tasks/{team}/`), a competing orchestration substrate (no worktrees, no + PR/CI grounding, grok can't join without the bridge). We call only `send`/inbox-outbox/ + `who`/`receipts`/`dlq`/`wake`. AMQ owns transport; herdr-derived lanes own state. The + mechanics are fixed: **Maildir** (atomic `rename()`, no locking, `tmp/new/cur`, one file per message — supersedes the earlier append-JSONL + - byte-offset idea; unread = files in `new/`, read = moved to `cur/`) and a - **CloudEvents-aligned envelope** (`id, source(=from), type(reverse-DNS), time, subject/ - ext(=to), data(=body)`). Align concepts with **A2A** (its Task lifecycle maps to our - lane states) for a future HTTP bridge — don't adopt its transport now. + byte-offset idea; unread = files in `new/`, read = moved to `cur/`) and **AMQ's own JSON + envelope** (`{schema, id, from, to[], thread, subject, created, priority, kind}`; our 5 + semantic types map onto `kind`+`subject`+body). **CloudEvents / A2A** stay *conceptual* + alignment (A2A's Task lifecycle maps to our lane states) for a future HTTP bridge — not the + on-disk format now that we're AMQ-native. - **Message types:** `ready-to-close`, `blocked-on-decision`, `coordination-request`, `needs-human`, `broadcast-request`. - **Topology — outbox + inbox, NOT inbox-only.** Each participant writes **only its From 0e3dc633e8efb865712fa74d84eb159ff6cbf941 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Sun, 19 Jul 2026 22:03:49 +0200 Subject: [PATCH 6/7] Add "Why not Agent Teams" rationale to orchestration ADR MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Answers the obvious reviewer question: CC's native Agent Teams is Claude-only, no worktree isolation, self-reported status, no PR/merge lifecycle — complementary, not competing. Note the shared Maildir envelope and the TeammateIdle / auto-unblock ideas worth reusing. Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01LTZx9DqrXPXQFoTrCvcJmD --- .../architecture/manager-worker-orchestration.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index 784436c..2238d8e 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -108,6 +108,22 @@ decides/posts; **Managers peer-to-peer** (they are the coordinators). Aligns wit `plugin-settings-system`'s `[related_projects]` peering registry. Broadcast + registry ship as a dedicated follow-up (`add-agent-broadcast`), not the Wave-1 core. +## Why not Claude Code Agent Teams (the native alternative) +CC's experimental **Agent Teams** (`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`; lead spawns +teammate processes, shared task list at `~/.claude/tasks/{team}/`, mailbox at +`~/.claude/teams/{team}/inboxes/`) solves a *different* problem: short-lived, **Claude-only**, +single-working-tree fan-out inside one session. Four blockers rule it out as our base: (1) +teammates ARE Claude processes → no codex/grok (our hard requirement); reaches externals only +via the AMQ bridge we use natively. (2) **No worktree isolation** → can't do N parallel +independent PRs. (3) Task status is **self-reported** (documented "teammate forgets +`completed`" bug) vs. our git/PR-artifact grounding. (4) No PR/review/merge/deploy lifecycle — +our four pillars. Also session-scoped/ephemeral (team config dies with the lead) vs. our +persistent worktree-lanes. **Complementary, not competing** — and both converge on the same +Maildir envelope (AMQ mirrors AT's `kind/priority/thread/receipts`), so a lane could later be +bridged to an AT team. Worth stealing now: AT's `TeammateIdle` hook validates our +Notification-hook auto-emit (T4); its dependency-aware auto-unblock informs the merge +sequencer (T5) / roadmap waves (T6). + ## Push without polling — Claude Code hooks first Files are pull; hooks make delivery push-like at turn boundaries, all gated by a cheap check (Maildir ⇒ unread = files in `inbox/new/`, a `readdir`): From bac00ff8aa77cafa91e24b943c56da63f6cf4175 Mon Sep 17 00:00:00 2001 From: Robert Gering Date: Thu, 23 Jul 2026 11:27:54 +0200 Subject: [PATCH 7/7] Address swarm review findings on orchestration ADR External-only swarm review (codex + grok) on PR #42. Apply the agreed findings to the decision record: - AMQ send vs single-sequencer: workers send only to the Manager handle; Manager relays. `amq send --to ` bypasses the sequencer (#1). - Broadcast: reuse AMQ fan-out/presence/federation instead of hand-rolled multi-writer global.jsonl (the model Maildir superseded) (#2). - Lane lifecycle: drain-on-/close + stale-mail guard so a reused path never reconsumes a prior occupant's undrained message (#3). - Soften the "RESOLVES" overclaim: this ADR supersedes the herdr docs' enum; their refresh is pending (#4). - Canonicalize the lane key via git rev-parse --show-toplevel, not raw cwd (#7). - Fix ws-statusline states flag order: `states [--cached] ` (#10). - _index.md blurb: central ~/.agent-mail/ mailbox, not the superseded .mailbox/ protocol (always-loaded surface) (#11). - Note the spike is decided, no open task file (#12). - Trust-model residuals (convention-based identity, home-dir readable store) as accepted for a single-user local tool (#5, #6). Co-Authored-By: Claude Opus 4.8 Claude-Session: https://claude.ai/code/session_01LTZx9DqrXPXQFoTrCvcJmD --- .claude/knowledge/_index.md | 2 +- .../manager-worker-orchestration.md | 48 ++++++++++++++----- 2 files changed, 36 insertions(+), 14 deletions(-) diff --git a/.claude/knowledge/_index.md b/.claude/knowledge/_index.md index 17a2637..35f57a3 100644 --- a/.claude/knowledge/_index.md +++ b/.claude/knowledge/_index.md @@ -6,7 +6,7 @@ - `architecture/model-economics.md` — Which model per skill and why (Haiku/Sonnet/session-model) - `architecture/idempotent-scaffolding.md` — Scaffolding into shared user files: absorb unmarked sections, lazy dirs, never overwrite user content - `architecture/worktree-task-file-copy.md` — Why `/kickoff` copies the task file into the worktree (not a symlink): no skill simplification, avoids accidental-commit repo poisoning + cross-platform fragility -- `architecture/manager-worker-orchestration.md` — Design decisions for the coordinated Manager/Worker model: lane identity = worktree path, cross-agent (claude/codex/grok) git-as-uniform-bus, the `.mailbox/` protocol (outbox+inbox, hook-driven push), milestone worker autonomy, merge sequencer, roadmap-as-derived-view +- `architecture/manager-worker-orchestration.md` — Design decisions for the coordinated Manager/Worker model: lane identity = worktree path, cross-agent (claude/codex/grok) git-as-uniform-bus, central `~/.agent-mail/` mailbox (Maildir/AMQ, outbox+inbox, hook-driven push), milestone worker autonomy, merge sequencer, roadmap-as-derived-view ## Features - `features/backfill-and-origin-metadata.md` — `/backfill-knowledge` significance bar + origin-reconstruction cascade diff --git a/.claude/knowledge/architecture/manager-worker-orchestration.md b/.claude/knowledge/architecture/manager-worker-orchestration.md index 2238d8e..081455f 100644 --- a/.claude/knowledge/architecture/manager-worker-orchestration.md +++ b/.claude/knowledge/architecture/manager-worker-orchestration.md @@ -10,7 +10,8 @@ prime: false Design decisions for evolving work-system from fire-and-forget kickoffs into a coordinated Manager/Worker model. This is the **decision record**; the -implementation is spawned across tasks `add-lane-registry`, `spike-agent-mail-substrate`, +implementation is spawned across tasks `add-lane-registry`, `spike-agent-mail-substrate` +(decided 2026-07-19 — see the mailbox section; no open task file), `add-lane-mailbox`, `add-manager-watch-loop`, `extend-worker-autonomy`, `add-merge-sequencer`, `add-roadmap-skill`, `add-mailbox-statusline`, and `add-agent-broadcast` (follow-up). Full working notes: the design task's `DESIGN-NOTES.md`. @@ -35,10 +36,11 @@ implementation is spawned across tasks `add-lane-registry`, `spike-agent-mail-su - Status is produced by **installed hook integrations** merged with a TUI-scrape rule engine (`agent explain`), NOT self-reported by the agent. - `agent read --source recent` returns clean text incl. Claude's `※ recap:` line + - statusline. This RESOLVES the previously-UNVERIFIED agent-status flag in - [herdr-kickoff-automation](../features/herdr-kickoff-automation.md) / - [herdr-close-automation](../features/herdr-close-automation.md) and adds `blocked` - to their documented `idle|working|done` set. + statusline. This entry **records the resolution** of the previously-UNVERIFIED + agent-status flag in [herdr-kickoff-automation](../features/herdr-kickoff-automation.md) / + [herdr-close-automation](../features/herdr-close-automation.md) and **supersedes** their + `idle|working|done` enum with `idle|working|blocked|done|unknown`. Those two entries still + carry the old enum (refresh pending) — this ADR is the current source until then. ## Cross-agent constraint (workers may be claude, codex, OR grok) The Manager is always Claude Code. Workers are tiered by herdr integration: @@ -56,11 +58,18 @@ worker's *implementation* of hitting agent-agnostic milestones — soft-coupling ## Worker↔Manager mailbox (the coordination substrate) A file mailbox carries judgment/intent signals that have no git artifact. - **Central `~/.agent-mail/` — NOT per-worktree** (decided 2026-07-19; the community - "agent mail" name). Participant id = **encoded absolute path**; a hook derives its dir - from `cwd`: `~/.agent-mail/lanes//{outbox,inbox}/`. Central over + "agent mail" name). Participant id = **encoded canonical worktree root** (`git rev-parse + --show-toplevel`, per [cwd-safety](../../rules/cwd-safety.md) — never raw `cwd`, which + differs for a subdirectory launch and would split a lane's mail across two dirs); a hook + derives its dir: `~/.agent-mail/lanes//{outbox,inbox}/`. Central over `/.mailbox/` because (a) it is the rendezvous for **broadcast / cross-Manager** (below), (b) never-commit is automatic (outside every repo — no per-repo gitignore), (c) no cross-worktree-boundary writes. Survives `/close` removing the worktree. + **Lifecycle:** `/close` **drains the lane's mailbox to the Manager before teardown**, and + a reader **guards against stale mail** (envelope `id`/`ts` + a worktree-exists check) so a + new task at a *reused* path never reconsumes a prior occupant's undrained message (e.g. a + late `rebase` intent). Undrained-after-teardown mail is caught by the Manager dead-letter + sweep (`add-manager-watch-loop`). - **Prior art & substrate (researched 2026-07-19; spike-first decided).** Our design is the recognized **inbox/outbox pattern**, and **AMQ** (`avivsinai/agent-message-queue`, Go single binary) is startlingly close (Maildir delivery, `.agent-mail` store, @@ -88,7 +97,11 @@ A file mailbox carries judgment/intent signals that have no git artifact. per mailbox (Maildir already makes concurrent delivery lockless), no cross-participant writes by a worker, and the **single-sequencer invariant** — worker→worker messages are *addressable* but **route through the Manager** (it drains outboxes, delivers to - inboxes, may reorder/veto/batch). Workers never write another lane's inbox. + inboxes, may reorder/veto/batch). Workers never write another lane's inbox. **Over AMQ** + this means a worker `amq send`s **only to the Manager handle** — never `--to `, + which delivers straight into a peer's Maildir and bypasses the sequencer; the Manager + re-sends to the target. `send` is the outbox-write + Manager-relay transport, not a + peer-delivery channel. - **`ready-to-close` is worker-authoritative + a handoff report.** Only the worker knows if a *second* PR is open or a post-merge TODO remains — a single PR's `✓ merged` is a soft "near-done" hint, NOT a close trigger. The message carries the worker's fresh @@ -99,10 +112,12 @@ A file mailbox carries judgment/intent signals that have no git artifact. ## Broadcast + cross-Manager (follow-up phase; central location adopted now) The central store unlocks system-wide coordination across the *multiple Managers* a -machine runs (one per project). `~/.agent-mail/broadcast/global.jsonl` is the one -multi-writer append log every Manager reads (offset-tracked); `managers/.json` -presence files make Managers discoverable. Use: system-wide notices, or capability -queries ("who has Cloudflare access?") answered point-to-point into the asker's inbox. +machine runs (one per project). Reuse **AMQ's native fan-out + presence/federation** here +too — a shared broadcast handle every Manager subscribes to, discovery via AMQ +`who`/presence — **not** a hand-rolled multi-writer `global.jsonl`, which would re-introduce +the append + byte-offset model Maildir already superseded for lane mail (concurrent-append +corruption, offset skew). Use: system-wide notices, or capability queries ("who has +Cloudflare access?") answered point-to-point into the asker's inbox. **Single-sequencer up a level:** a worker emits `broadcast-request` and its **Manager** decides/posts; **Managers peer-to-peer** (they are the coordinators). Aligns with `plugin-settings-system`'s `[related_projects]` peering registry. Broadcast + registry @@ -176,8 +191,15 @@ unverified pane); no force-push beyond own-branch `--force-with-lease`, never shared/main; no merge-to-main without delegation; bounded herdr waits, survey refresh `--cached`; grok `always-approve` ⇒ Manager sends nothing destructive to a grok lane. +**Trust model (accepted residuals, single-user local tool):** the envelope `from` is +self-declared and single-writer is a cooperative protocol — a *same-account* process could +forge a message, but every agent already runs under the user's account and consequential +actions stay human-gated, so it grants no new privilege. The central `~/.agent-mail/` is +home-dir readable across projects; accepted for a single-user machine (any home-read process +already sees SSH/cloud creds), not hardened for a shared host. + ## Reused existing substrate (build ON, don't reinvent) -- `ws-statusline.sh states
[--cached]` → `task\tstate\tglyph` = a ready-made +- `ws-statusline.sh states [--cached] ` → `task\tstate\tglyph` = a ready-made lane registry with states; the state machine ○●◇◆✓ + `◉` and the tab↔worktree join already exist in [herdr-tab-glyphs](../features/herdr-tab-glyphs.md). - The PR cache (`headRef\tstate\treviewDecision`) + sync/`--cached` refresh policy.