diff --git a/docs/specs/v0.20-agentic-turn.md b/docs/specs/v0.20-agentic-turn.md new file mode 100644 index 0000000..c6cfd68 --- /dev/null +++ b/docs/specs/v0.20-agentic-turn.md @@ -0,0 +1,672 @@ +# agentic-stack v0.20 — "The Agentic Turn" + +**Status:** Draft spec +**Author:** codejunkie99 +**Target release:** v0.20.0 +**Baseline:** v0.19.1 +**Scope:** Convert agentic-stack from a *static portable brain* into an *active multi-agent runtime*. + +--- + +## 0. Goals and non-goals + +### Goals +1. Make `.agent/` a runtime, not just a config surface. +2. Let multiple agents coordinate across harnesses on the same project without double-work. +3. Turn accepted lessons into testable, regression-gated behavior contracts. +4. Plan, retrieve, evaluate, and execute speculatively — all local-first. +5. Preserve harness-agnosticism. Nothing here may require a specific LLM, harness, or cloud. + +### Non-goals +- Training models. (Flywheel still exports JSONL only.) +- Hosted infra in this release. Cloud is downstream. +- Replacing harnesses' own task loops. We coordinate them; we don't supplant them. + +### Design invariants +- **Local-first, file-based.** Every new subsystem is a directory of JSONL + small Python tools. No daemons required for the MVP. File watchers are optional. +- **Append-only audit.** Every state change writes a record. Nothing is destructive except explicit `compact` commands. +- **Permissioned.** Every new tool goes through the existing `protocols/permissions.md` pre-tool-call hook. +- **Cross-platform.** POSIX + Windows. `fcntl` on POSIX, `msvcrt` on Windows, mirroring the v0.9 harness manager pattern. +- **No content in journals.** Runtime records carry whitelisted metadata and *references* to artifacts. Never inline prompts, commands, or output. This is already enforced by `harness_manager/loops/storage.py`; every new journal inherits it. + +--- + +## 0.1 What v0.19 already shipped, and what it means here + +This spec was first drafted against a v0.18 baseline. v0.19.0 then shipped the +bounded loop supervisor, which already implements several primitives the +original draft proposed to build from scratch. **Nothing below may reimplement +them.** Concretely, `harness_manager/loops/` already provides: + +| Primitive | Where it lives | Consequence for this spec | +|---|---|---| +| Owned Git worktrees, creation + `changed_paths` + cleanup | `loops/worktrees.py` | §2.7 fans out existing loop runs. It does **not** create a second worktree manager. | +| Finite budgets (attempts, runtime, output chars, token estimate) | contract `limits`, `loops/policy.py` | §2.7/§2.8 express budgets as loop `limits`, not new budget files. | +| Deny-path gates on changed files | `policy.check_changed_paths` | §2.8's `disallowed.any_write_outside` is a constraints file, not new gate code. | +| Approval before mutating / external-write runs | contract `approval` | §2.8's human-approval step is this gate, not a parallel one. | +| Stagnation breaker, atomic resumable checkpoints | `policy.evaluate_breaker`, `loops/storage.py` | Reused as-is by every long-running subsystem. | +| Append-only, **content-free** event journal | `storage.append_event`, `EVENT_FIELDS` whitelist | §2.2's bus adopts the same whitelist discipline (see §2.2). | +| Deterministic verification command per contract | contract `verification.command` | §2.3's `shell` judge *is* this mechanism. | +| Autonomy tiers L1/L2/L3 | contract `autonomy` | Report-only subsystems are L1 loops; mutating ones are L2/L3. | + +Subsystems **unaffected** by v0.19.0 and still specced as originally written: +plans (§2.1), retriever (§2.4), skill graph (§2.5), federation (§2.6). + +--- + +## 1. New `.agent/` layout + +``` +.agent/ +├── AGENTS.md +├── loops/ # EXISTING (v0.19) — bounded loop contracts +├── runtime/loops/ # EXISTING (v0.19) — checkpoints + events.jsonl +├── plans/ # NEW — fifth memory layer +│ ├── active/.json +│ ├── archive/.json +│ └── INDEX.md # human-readable rollup +├── bus/ # NEW — multi-agent coordination +│ ├── messages.jsonl # append-only journal (metadata + refs only) +│ ├── claims.jsonl # who owns what subgoal +│ ├── locks/.lock # advisory file locks +│ └── inbox// # per-agent unread pointer +├── evals/ # NEW — behavior contracts +│ ├── cases/.yaml +│ ├── runs/.jsonl +│ ├── runner.py +│ └── REGRESSIONS.md +├── retriever/ # NEW — hybrid context +│ ├── index/ # fastembed + bm25 indices (gitignored) +│ ├── pack.py # context_pack assembler +│ └── retriever.py +├── trials/ # NEW — speculative execution (see §2.7) +│ └── runs/.json # approaches, scores, cost — worktrees owned by loops/ +├── act/ # NEW — background autonomous actions +│ ├── policies.yaml # what act may attempt +│ ├── proposals/.json # awaiting human review +│ └── act.py +├── memory/ # EXISTING +├── skills/ # EXISTING, frontmatter extended +├── protocols/ # EXISTING, schemas extended +└── tools/ # EXISTING, plus new CLIs + +~/.agentic-stack/global/ # NEW — cross-project federation +├── personal/PREFERENCES.md +├── semantic/lessons.jsonl +├── skills/ +└── federation.log +``` + +`trials/`, not `spec/`: the v0.19 loop contract already uses "spec" vocabulary +for schemas, and a `.agent/spec/` directory reads as "the spec for `.agent`". + +--- + +## 2. Subsystem specs + +### 2.1 Plans layer (`.agent/plans/`) + +**Purpose.** A fifth memory layer holding *intent* — what the agent suite is trying to accomplish — separately from working/episodic/semantic/personal memory. + +**Plan object (JSON).** +```json +{ + "plan_id": "plan_2026_08_07_a1b2", + "intent": "Ship v0.20 with Windows hook parity", + "created_at": "2026-08-07T12:00:00Z", + "created_by": "human|agent_id", + "status": "active|blocked|done|abandoned", + "subgoals": [ + { + "id": "sg_001", + "summary": "Port pre_tool_call.py hook to PowerShell", + "owner": "agent_id|null", + "depends_on": [], + "status": "todo|claimed|in_progress|blocked|done", + "blockers": [], + "skills_hint": ["git-proxy", "debug-investigator"], + "evidence": [".agent/episodic/2026-08-07.jsonl#L42"] + } + ], + "context_refs": ["semantic/LESSONS.md#L120-L140"], + "deadline": null, + "parent_plan_id": null +} +``` + +**CLI.** +``` +plan.py new --intent "" [--from-template ] +plan.py decompose [--llm ] # expands subgoals; the only LLM path +plan.py show +plan.py next [--agent ] # next unblocked subgoal +plan.py update --subgoal --status +plan.py block --subgoal --reason "" +plan.py archive +``` + +**Invariants.** +- `next` is deterministic given the same state. No LLM in the path. +- `decompose` is the only path that calls an LLM and is always optional. +- Plans may reference but never overwrite memory layers. + +**Acceptance.** `plan.py new --intent "X"` then `decompose` yields a subgoal tree with skill hints. `plan.py next` from two terminals returns two different subgoals once the bus is wired. + +--- + +### 2.2 Multi-agent bus (`.agent/bus/`) + +**Purpose.** Let N agents across N harnesses coordinate on one project without stepping on each other. + +**Message shape (`messages.jsonl`, append-only).** +```json +{ + "msg_id": "m_2026_08_07_001", + "ts": "2026-08-07T12:01:03Z", + "from": "claude-code:host_a", + "kind": "claim|release|request|result|notice|heartbeat", + "subject": "plan:plan_2026_08_07_a1b2:sg_001", + "payload_ref": ".agent/working/results/m_2026_08_07_001.json", + "correlates_with": "m_2026_08_07_000", + "ttl_seconds": 3600 +} +``` + +**Privacy rule (changed from the v0.18-era draft).** The original draft carried +an inline `payload` object capped at 4 KB. v0.19 shipped the opposite rule for +`.agent/runtime/loops/events.jsonl`: a field whitelist, with task, prompt, +command, and output content excluded *by construction*. Two append-only +journals in the same `.agent/` with contradictory privacy guarantees is a +footgun — the weaker one becomes the leak. So the bus adopts the loop rule: +messages carry whitelisted metadata plus `payload_ref`, a path to an artifact. +`bus.py post --payload @file.json` writes the artifact and records the +reference. Reuse `storage.append_event`'s whitelist helper rather than copying it. + +**Kinds.** +- `claim` — "I'm taking subgoal X for T seconds." Writes to `claims.jsonl`. +- `release` — "I'm done with, or giving up, X." +- `request` — "I need skill/result Y from any agent." +- `result` — paired response; always a `payload_ref`, never inline. +- `notice` — non-blocking broadcast ("test suite green on main"). +- `heartbeat` — liveness ping every 60 s while a claim is held. Two missed heartbeats auto-release the claim. + +**Claims file (`claims.jsonl`, append-only with periodic compaction).** +```json +{"claim_id":"c_001","subject":"plan:...:sg_001","holder":"claude-code:host_a","granted_at":"...","expires_at":"...","status":"active|released|stolen|expired"} +``` + +**Locks (`locks/.lock`).** Optional advisory mutexes for shared resources (e.g. `package.json`), holding holder + PID + timestamp, respecting OS file locking. + +**CLI.** +``` +bus.py listen [--agent ] [--kind ] # tails messages.jsonl +bus.py post --kind --subject [--payload @file.json] +bus.py claim [--ttl 1800] +bus.py release +bus.py status # active claims, last 20 messages +bus.py gc # expire stale claims, compact +``` + +**Transport.** MVP: append-only JSONL + inotify/`ReadDirectoryChangesW` for `listen`, single machine, zero deps. v0.21+: optional NATS or Redis sink behind `--transport`, same message shape. + +**Invariants.** +- The bus is a *journal*, not a queue. Replayability is the point. +- Every claim expires. No infinite locks. +- `bus.py gc` is safe to run from cron. + +**Acceptance.** Two terminals run `plan.py next` and get different subgoals because `claim` prevents double-pick. Killing one expires its claim within 3 heartbeats; the other picks it up. + +--- + +### 2.3 Eval runner (`.agent/evals/`) + +**Purpose.** Turn accepted lessons into regression-tested behavior contracts. A lesson that breaks its own eval is auto-quarantined. + +**Case shape (`cases/.yaml`).** +```yaml +lesson_id: lsn_2026_04_12_payments_idempotency +title: "All payment writes must be idempotent" +contract: + given: "a function that writes to the payments table" + must: "include an idempotency_key argument and a SELECT-before-INSERT guard" + must_not: "perform a naked INSERT without the guard" +fixtures: + - kind: code_snippet + path: fixtures/payments_good.py + expected: pass + - kind: code_snippet + path: fixtures/payments_bad.py + expected: fail +judge: + kind: rubric_llm | regex | ast_check | shell + spec: "..." +budget: + max_tokens: 4000 + max_seconds: 30 +``` + +**Runner.** +``` +evals/runner.py run [--lesson ] [--changed-only] +evals/runner.py status +evals/runner.py quarantine --reason "" +evals/runner.py release +``` + +- Triggered by a git pre-commit hook on `.agent/memory/semantic/lessons.jsonl`. +- A failing eval flips the lesson's `recall_status` to `quarantined`. Quarantined lessons are excluded from `recall.py` until released — the same exclusion path v0.19.1 added for superseded lessons (`render_lessons.superseded_by_map`), so retrieval keeps exactly one filter chain. +- Run history in `runs/.jsonl`; regression deltas rendered into `REGRESSIONS.md`. + +**Judges.** `regex` (deterministic), `ast_check` (Python AST or tree-sitter), `shell` (exit code = pass/fail — the same contract as a loop's `verification.command`, so a judge and a loop verifier are interchangeable), `rubric_llm` (fallback, costs tokens, rubric checked in). + +**Invariants.** +- Evals never call the network unless `judge.kind == rubric_llm` and a model is configured. +- Fixtures live in `cases/fixtures/` and are git-tracked. +- A new lesson without an eval warns but does not block graduation. After 7 days without one, it auto-quarantines. +- Quarantine threshold is 2 consecutive failures on *different* commits, not 1. + +**Acceptance.** `graduate.py` accepts a lesson → runner auto-generates a stub case → human fills fixtures → `runner run` passes → lesson appears in `recall.py` results. + +--- + +### 2.4 Hybrid retriever (`.agent/retriever/`) + +**Purpose.** Replace FTS-only memory search with hybrid BM25 + dense embeddings + reranking, and emit *context packs* sized to a harness's context window. + +**Index layout (gitignored).** +``` +.agent/retriever/index/ +├── bm25/ # tantivy or simple inverted index +├── embeddings/ # fastembed output, sqlite-vec storage +└── manifest.json +``` + +**Components.** Embedder: `fastembed` with `BAAI/bge-small-en-v1.5` (CPU, ~130 MB, ~10 ms/chunk), model swappable. Vector store: `sqlite-vec` (zero-server, file-backed). BM25: existing FTS5 path stays a backend option; default is pure-Python `rank-bm25` for cross-platform simplicity. Reranker: optional cross-encoder via fastembed, off by default to keep cold start small. + +**API.** +```python +from agent.retriever import retrieve, context_pack + +results = retrieve( + query="why does deploy fail on windows runners", + k=20, + filters={"layer": ["semantic", "episodic"]}, + rerank=True, +) + +pack = context_pack( + task="implement: port pre_tool_call.py to PowerShell", + budget_tokens=8000, + must_include=["semantic/LESSONS.md#L120-L140"], +) +# Returns: {"path": ".agent/memory/working/context_pack_.md", "manifest": [...]} +``` + +**Pack composition heuristic (default).** 30 % top-k semantic lessons · 30 % recent episodic entries touching the same files · 20 % `SKILL.md` bodies for skills the task hints at · 20 % explicit `must_include` plus overflow buffer. + +**CLI.** +``` +retriever.py index [--rebuild] [--watch] +retriever.py search "" [--k 20] [--layer semantic] +retriever.py pack --task "" --budget 8000 [--out ] +retriever.py status +``` + +**Invariants.** +- All embedding is local. Zero network calls. +- Index rebuild is incremental by default (mtime + content hash). +- A context pack is a **file**, not a stream. Harnesses ingest it through their existing file-include mechanism. +- Quarantined and superseded lessons are excluded at index time, not at query time. + +**Acceptance.** After `index --rebuild`, `search "deploy failure"` returns the FTS lesson cluster plus 2+ semantically related results FTS missed. `pack` emits markdown under the token budget, verified with `tiktoken`. + +--- + +### 2.5 Skill graph (`skills/` extension) + +**Purpose.** Make skills composable via typed dependencies and contracts, so one skill can call another deterministically. + +**Extended SKILL.md frontmatter.** +```yaml +--- +name: data-pipeline +version: 0.1.0 +triggers: [...existing...] +requires: + - skill: git-proxy + version: ">=0.3" + capabilities: [safe_commit] + - skill: data-layer + version: ">=0.2" +provides: + - capability: pipeline_run + schema: protocols/tool_schemas/pipeline_run.json +contract: + inputs: + pipeline_id: { type: string, required: true } + window: { type: string, default: "7d" } + outputs: + artifact_path: { type: string } + metrics: { type: object } +side_effects: [writes:.agent/data-layer/exports] +permission_class: medium +--- +``` + +**Resolver (`tools/skill_graph.py`).** +``` +skill_graph.py resolve # prints DAG +skill_graph.py validate # checks manifests for missing deps +skill_graph.py call --input @in.json --output out.json +``` + +`call` is the typed invocation path: look up `provides`, validate inputs against schema, invoke the skill (Python entry point or templated agent instruction), validate outputs. Two skills providing one capability conflict at `validate` time. + +**Invariants.** +- Backward compatible: skills without `requires`/`provides` keep working. +- Cycles fail `validate` loudly. +- `permission_class` is enforced by the existing pre-tool-call hook. +- `sync-manifest` must round-trip the new fields into `.agent/skills/_manifest.jsonl`. + +**Acceptance.** A `data-pipeline` skill declares deps on `git-proxy` and `data-layer`; `resolve` prints a 3-node DAG; `call` runs end to end and produces an artifact. + +--- + +### 2.6 Cross-project federation (`~/.agentic-stack/global/`) + +**Purpose.** Personal preferences and reusable lessons live above any single project. Project memory inherits read-only from global; global accepts writes only through explicit promotion. + +**Promotion rule.** A project lesson is eligible when it is `accepted` in **3+ distinct projects** (content hash × project_id) and its eval passes in each. + +**CLI.** +``` +federate.py status # what's global, what's eligible +federate.py promote # one-shot, requires confirmation +federate.py demote # remove from global, history retained +federate.py sync # refresh this project's view of global +federate.py diff # project vs global +``` + +**Recall integration.** `recall.py` queries `~/.agentic-stack/global/semantic/` and `.agent/memory/semantic/`, tagging each result `[global]` or `[project]`. Project results win on conflict. Global lessons pass through the same supersession and quarantine filters as project ones. + +**Invariants.** +- Global memory is per-machine and unsynced by default; users may opt into git-syncing it themselves. +- No project writes to global except via `federate.py promote`. +- A `permissions.md` rule blocks agents from writing `~/.agentic-stack/global/`. +- `transfer_bundle.py`'s secret scanner runs before any promotion write. + +**Acceptance.** Accept one lesson (by content hash) in 3 projects → `status` lists it eligible → `promote` moves it → a fresh project clone sees it on first `recall.py` with no setup. + +--- + +### 2.7 Speculative execution (`.agent/trials/`) + +**Purpose.** Try N approaches to one task, score them, present the winner. + +**Built on loops, not beside them.** v0.19 already owns worktree creation and +teardown, per-attempt budgets, deny-path gates, checkpointing, and the +stagnation breaker. A trial is therefore **N loop runs of one contract with N +different initial instructions**, plus a scoreboard. `trials/` stores only the +scoreboard; the worktrees stay under the loop runtime and are cleaned up by +`loop cleanup`. + +**Trial object.** +```json +{ + "trial_id": "trial_2026_08_07_x", + "task": "fix flaky test in tests/test_upgrade.py", + "loop": "ci-sweeper", + "approaches": [ + {"id": "a", "strategy": "retry with backoff", "run_id": "ci-sweeper-0001"}, + {"id": "b", "strategy": "deterministic seed", "run_id": "ci-sweeper-0002"}, + {"id": "c", "strategy": "isolate fixture", "run_id": "ci-sweeper-0003"} + ], + "scoreboard": { + "a": {"evals_passed": 4, "evals_failed": 1, "tokens": 12000, "seconds": 90}, + "b": {"evals_passed": 5, "evals_failed": 0, "tokens": 8000, "seconds": 60}, + "c": {"evals_passed": 3, "evals_failed": 2, "tokens": 15000, "seconds": 110} + }, + "winner": "b", + "status": "running|scored|merged|aborted" +} +``` + +**CLI.** +``` +agentic-stack trial new --loop --task "" --approaches @strategies.yaml +agentic-stack trial score +agentic-stack trial merge --pick b # fast-forwards the winner onto a branch +agentic-stack trial keep-losers # archives losing diffs as flywheel signal +agentic-stack trial prune # delegates to loop cleanup +``` + +**Execution model.** MVP runs each approach sequentially in its own loop run on the local machine — one bounded command at a time, matching the v0.19 scheduler guidance. v0.21: optional Modal/E2B adapter to fan out in parallel. + +**Invariants.** +- Worktree lifetime and TTL are the loop supervisor's, not a second policy. +- Losing approaches are never lost: written to `.agent/flywheel/approved-runs.jsonl` with `verdict=lost` so the flywheel learns from them too. +- A trial never auto-merges. Human approval required. +- Scoring uses the eval runner (§2.3), so "winner" means "passed more checked-in contracts", not "looked better". + +**Acceptance.** `trial new --loop ci-sweeper --task "..." --approaches @three.yaml` produces 3 scored runs; `merge --pick b` yields a branch ready for PR; losing diffs appear in flywheel exports. + +--- + +### 2.8 Background autonomous actions (`.agent/act/`) + +**Purpose.** Between sessions, let a sandboxed agent take low-risk actions (re-run tests, propose skill rewrites, refresh caches) under tight policy. + +**Built on loops, not beside them.** Each allowed action is an **L1 report-only +loop contract** whose output is a proposal file. Budgets are the contract's +`limits`; write-scoping is a `constraints.json` deny-path set; human approval is +the contract's `approval` gate. `act.py` is a scheduler and proposal store, not +a second supervisor. + +**Policy file (`policies.yaml`).** +```yaml +allowed_actions: + - id: rerun_failed_tests + loop: daily-triage # an existing L1 contract + trigger: cron:hourly + permission_class: low + budget: { max_tokens: 2000, max_seconds: 120, max_runs_per_day: 24 } + - id: propose_skill_rewrite + loop: skill-critic + trigger: on_failure:>=3_in_14d + permission_class: medium + requires_human_approval: true +disallowed: + - any_network_call_outside_allowlist + - any_write_outside: [.agent/act/proposals, .agent/memory/episodic] +network_allowlist: + - "api.github.com" +sandbox: + kind: local|modal|e2b + spec: { ... } +``` + +**Proposal shape.** +```json +{ + "proposal_id": "prop_2026_08_07_001", + "action_id": "propose_skill_rewrite", + "target": "skills/debug-investigator/SKILL.md", + "diff_path": ".agent/act/proposals/prop_2026_08_07_001.patch", + "rationale": "Failed 4x in 14 days on the 'reproduce' step", + "supporting_evidence": ["memory/episodic/2026-08-01.jsonl#L88"], + "status": "pending|approved|rejected|expired", + "ttl": "7d" +} +``` + +**CLI.** +``` +act.py run-once [--action ] # safe to crontab; one bounded run at a time +act.py list-proposals +act.py approve +act.py reject --reason "" +act.py simulate # dry run, prints what would happen +``` + +**Invariants.** +- `act.py` **never** edits source directly. It writes patches to `.agent/act/proposals/`. +- Every action emits an episodic record; every approval/rejection emits a dream-cycle candidate. +- A proposal that hits its TTL without a decision auto-rejects with `reason=expired`. +- The supervisor bounds and audits child processes; it is **not an operating-system sandbox**. Use harness-native sandboxes for stronger isolation. (Same caveat as v0.19 loops — `sandbox.kind` selects an executor, not a security boundary.) + +**Acceptance.** `act.py simulate propose_skill_rewrite` prints the patch it would create; `run-once` creates it; `approve ` applies it via `git-proxy`. The whole loop replays from `messages.jsonl` plus the loop event journal. + +--- + +## 3. Protocol additions (`protocols/`) + +### 3.1 New permission classes +Extend `permissions.md`: +- `bus_post` — any agent may post `notice`, `heartbeat`, `result`. `claim`/`release` require an active plan and a valid agent_id. +- `plan_write` — only a top-level harness or human creates/archives plans; sub-agents may update subgoal status. +- `act_apply` — only human approval flips a proposal `pending` → `approved`. +- `global_write` — only `federate.py` may write `~/.agentic-stack/global/`. + +### 3.2 New tool schemas +Under `protocols/tool_schemas/`: `plan_object.json`, `bus_message.json`, `eval_case.json`, `context_pack_request.json`, `skill_contract.json`, `trial_run.json`, `act_proposal.json`. All JSON Schema Draft 2020-12, validated by `jsonschema` in CI, versioned with a `schema_version` integer like the v0.19 loop contracts. + +### 3.3 Delegation contract +Extend `delegation.md`: (1) parent posts `claim` on the subgoal; (2) parent posts `request` referencing the claim with `payload_ref.subgoal_id`; (3) sub-agent posts `result` referencing the request; (4) parent validates outputs against `skill_contract` and posts `release`. + +--- + +## 4. Adapter changes (harness shims) + +Each adapter gets one addition: announce identity to the bus. + +```bash +# at session start, every adapter calls: +python3 .agent/tools/bus.py announce \ + --agent "claude-code:$(hostname):$$" \ + --capabilities "code_edit,git,tests" +``` + +- **Claude Code:** wire into the existing hook setup. +- **Copilot CLI / Gemini / Cursor / Codex / Autohand / Antigravity:** a startup instruction in `AGENTS.md` / the rules file telling the agent to announce on its first turn. +- **Standalone Python:** call directly from `run.py`. + +No adapter must *consume* the bus. Those that do (claude-code, copilot-cli, standalone-python) get a `bus.py listen --kind request` companion for delegated work. + +--- + +## 5. CLI surface summary + +New top-level verbs on `install.sh` / `install.ps1` / `agentic-stack`, all routed through the existing `harness_manager/cli.py` dispatcher alongside `loop`: + +``` +agentic-stack plan ... # plans layer +agentic-stack bus ... # multi-agent bus +agentic-stack eval ... # evals +agentic-stack retrieve ... # hybrid retriever +agentic-stack skill ... # graph: resolve, validate, call +agentic-stack federate ... # cross-project +agentic-stack trial ... # speculative execution +agentic-stack act ... # background actions +``` + +--- + +## 6. Storage, performance, and dependencies + +### Footprint +- `.agent/retriever/index/` grows ~5–20 MB per 10k chunks. Gitignored. +- Trial worktrees are full repo copies; TTL and pruning belong to `loop cleanup`. +- `.agent/bus/messages.jsonl` rotates daily; `bus.py gc` compacts after 30 days. + +### Dependencies (additive; every optional feature fails soft) +`fastembed` (CPU embeddings) · `sqlite-vec` (vector storage, single .so/.dll) · `rank-bm25` (pure-Python BM25 default; tantivy optional) · `jsonschema` (schema validation) · `watchdog` (optional watcher for `bus.py listen`, falls back to polling) · `tiktoken` (token budgeting). + +No GPU. No daemon. The Python floor stays wherever v0.19 leaves it — this spec does not raise it, and any subsystem needing a newer floor must fail soft instead. + +### Performance targets +`plan.py next` < 50 ms cold · `bus.py post` < 20 ms · `retriever.py search` over 10k chunks < 200 ms · eval runner over 50 cases < 30 s without LLM judges. + +--- + +## 7. Migration and compatibility + +### From v0.19.x → v0.20 +- `agentic-stack upgrade --yes` creates the new directories empty, exactly as it already adds loop starters — add-only, never overwriting authored contracts or runtime state. +- Existing memory, skills, adapters, and loop contracts untouched. +- Nothing is added to git unless opted into (`upgrade --enable plans,bus,evals`). +- Feature toggles in `.agent/memory/.features.json`: + ```json + { + "plans.enabled": true, + "bus.enabled": true, + "evals.enabled": true, + "retriever.enabled": false, + "skill_graph.enabled": false, + "federation.enabled": false, + "trials.enabled": false, + "act.enabled": false + } + ``` +- Onboarding gains one step: "Enable agentic runtime features (beta)?" + +### Backward compatibility guarantees +- Every existing CLI, including all `loop` verbs, keeps working unchanged. +- Skills without the new frontmatter still load. +- With every new feature disabled, a repo behaves identically to v0.19.1. + +--- + +## 8. Documentation deliverables + +Under `docs/`: `agentic-runtime.md` (how the eight subsystems compose, and how they sit on the v0.19 loop supervisor), plus `plans.md`, `bus.md`, `evals.md`, `retriever.md`, `skill-graph.md`, `federation.md`, `trials.md`, `act.md`. + +A `docs/diagram-runtime.svg` showing: +`intent → plan → bus → [loop runs in worktrees] → context_pack → skill_graph → eval → dream → federate → act`. + +README gains one "New in v0.20" section and nothing more — per-version history stays in `CHANGELOG.md`. + +--- + +## 9. Rollout plan + +| Phase | Duration | Content | +|---|---|---| +| 1 — Foundation | 1 week | Plans layer + bus (file transport), new permission classes, new schemas, `bus.py announce` in claude-code and standalone-python. | +| 2 — Quality gates | 1 week | Eval runner + auto-quarantine, pre-commit hook on `lessons.jsonl`, stub auto-generation in `graduate.py`. | +| 3 — Context and composition | 1 week | Hybrid retriever, context-pack assembler, skill graph resolver + typed `call`. | +| 4 — The bold pieces | 2 weeks | Federation, trials on top of the loop supervisor, act proposals workflow. | +| 5 — Polish and launch | 1 week | The eight docs, dashboard surfaces plans/claims/proposals, Mission Control "Runtime" tab, launch post. | + +Total: ~6 weeks of focused work. Each phase lands as its own PR series; this document is the design surface, not a merge candidate for implementation. + +--- + +## 10. Risks and open questions + +| Risk | Mitigation | +|---|---| +| File-bus contention on Windows | `msvcrt.locking`, as `harness_manager` already does; fall back to polling `listen`. | +| Eval judges drift (LLM rubrics non-deterministic) | Prefer regex/ast/shell judges. LLM judge requires `temperature=0`, fixed seed, golden snapshot. | +| Auto-quarantine causes lesson churn | Threshold is 2 consecutive failures on different commits. | +| Federation leaks secrets across projects | Reuse `transfer_bundle.py`'s secret scanner before any promotion write. | +| Trial worktrees fill disk | Owned by the loop supervisor's TTL + `loop cleanup` in nightly cron. | +| Bus replay confuses agents on restart | `bus.py announce` always emits a fresh agent_id; replay is forensics, not state. | +| Two journals drift apart | Bus and loop events share one whitelist helper; a new field must be added in one place. | + +**Open questions.** +1. Should `plan.py decompose` use the host harness's LLM or a project-pinned default? Leaning host's, with an override flag. +2. Do agents need `subscriptions.jsonl` to declare interest, or is `bus.py listen --kind X` enough? MVP says the latter. +3. Should the retriever index `.agent/flywheel/` exports? Probably yes, behind a config flag. +4. Should a trial's losing worktrees survive until `keep-losers` runs, or should the diff be captured at scoring time? Capturing at scoring time is cheaper on disk and is the current lean. + +--- + +## 11. Definition of done for v0.20 + +- All eight subsystems present, each behind a feature flag, defaulting to **off** except plans, bus, and evals. +- `agentic-stack upgrade --yes` from v0.19.1 leaves a working project that opts into the three default-on subsystems, with existing loop contracts and runtime state untouched. +- `agentic-stack dashboard` shows live plan, claim, and proposal counts beside the existing loop health. +- Two design-partner projects run a full cycle: human posts intent → plan decomposes → two harnesses claim subgoals via the bus → context packs assembled → evals pass → lesson federated → act proposes a skill rewrite → human approves. +- No new journal carries prompt, command, or output content. + +— end of spec —