diff --git a/CLAUDE.md b/CLAUDE.md index 7ef88de4..76aa0ffd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -165,7 +165,7 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph | **AI** | `src/ai-checker-base.ts`, `ai-idle-checker.ts`, `ai-plan-checker.ts` | | | **Tasks** | `src/task.ts`, `task-queue.ts`, `task-tracker.ts` | | | **State** | `src/state-store.ts`, `run-summary.ts`, `session-lifecycle-log.ts`, `intent-store.ts`, `tab-layout.ts` (pure model) + `-service` (sole mutation boundary) + `-persistence` + `-legacy-order` | | -| **Infra** | `src/hooks-config.ts`, `push-store`, `tunnel-manager`, `image-watcher`, `file-stream-manager`, `remote-hosts` + `remote-reconnect` (pure), `docker-hosts` + `docker-export` | Remote/docker case overlays; see Key Patterns | +| **Infra** | `src/hooks-config.ts`, `push-store`, `tunnel-manager`, `image-watcher`, `file-stream-manager`, `remote-hosts` + `remote-reconnect` + `remote-wake` (pure), `docker-hosts` + `docker-export` | Remote/docker case overlays; see Key Patterns | | **Web tabs** | `src/webview-store.ts`, `webview-capabilities.ts`, `src/web/webview-proxy.ts` (pure), `src/web/routes/webview-routes.ts` | Dashboard URLs as tabs; NOT a SessionMode | | **Search** | `src/search-service.ts` | Pure in-memory core for `GET /api/search` | | **Attachments** | `src/attachment-registry.ts`, `attachment-magic`, `generated-artifact-attachments`, `session-attachment-history`, `document-preview-cache`, `document-thumbnailer`, `document-conversion-limiter`, `config/attachment-guard` | See Key Patterns | @@ -217,6 +217,8 @@ Codeman is a Claude Code session manager with web interface and autonomous Ralph **Remote sessions + remote SSH cases**: a case can point at a remote host. The agent runs inside a durable remote `tmux -L codeman-remote` (session name `codeman-ssh-`, deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so an instance on the target host never adopts it), fronted by a LOCAL tmux pane running `ssh`. Attached (`owned:false`) sessions **detach, never kill** on tab close; owned ones propagate `kill-session`. A bounded-backoff watcher auto-reconnects dropped sessions (`remoteAutoReconnect`, default ON). ⚠️ **It revives ONLY when the durable remote tmux session is verifiably still alive** (`remoteTmuxSessionAlive()`, a `has-session` probe over ssh, #355): a clean agent exit (Ctrl-C, Ctrl-D, `exit`) tears that session down, and `isPaneDead()` cannot tell it from a transport drop, so the watcher used to relaunch a FRESH agent after every clean exit (claude only looked fine because its `|| --resume` fallback masked it). An unreachable host answers `undefined`, which also means do not revive. ⚠️ `has-session` prints NOTHING on success, so the probe is classified by EXIT STATUS (`classifyRemoteAliveExit`: 0 alive, ssh's 255 or a timeout unknown, anything else gone); reading stdout classified every live session as gone and silently disabled transport-drop reconnects. The answer is cached per session and forgotten whenever the pane is seen alive again, or a stale `true` from one transport drop would revive the next clean exit. ⚠️ **File reads in a remote case are the second ssh surface** (#415, `src/remote-files.ts`): they go through `buildSshConnectionArgs()` as well, a browser-supplied path is only ever a `shellescape`d token, an unreachable host answers 502 (never 404), the size cap uses the REMOTE size, and no remote file is ever copied onto the server's disk — which is why writes, office previews and thumbnails are deliberately unsupported over ssh (the `PUT` guard sits BEFORE the local path validation, or a same-named local directory such as an sshfs mount takes the write). The probe's symlink resolution FAILS CLOSED (a path it cannot canonicalize is a 404, never its own unresolved string: the directory-only fallback let a `notes.txt -> ~/.ssh/id_rsa` link pass containment), and ssh children are BOUNDED by `src/remote-ssh-limiter.ts` plus one batched probe per attachment-history listing, because terminal output in a remote session is written on the remote host and a prompt-injected agent can print hundreds of `codeman://attach` links. The ATTACHMENT routes (a clicked path outside the case dir) go through the same layer, and which host a record is read from follows the SESSION, never the path string. ⚠️ **Command-injection surface: every ssh command line must flow through `buildSshConnectionArgs()`**, which `shellescape`s every user field. Never hand-build an ssh line elsewhere. ⚠️ Run flows must route remote cases through `POST /api/quick-start`, not `POST /api/sessions` (which stat-validates `workingDir` locally and has no `caseName`). → [architecture-invariants#remote-sessions-over-ssh](docs/architecture-invariants.md#remote-sessions-over-ssh), [#remote-ssh-cases](docs/architecture-invariants.md#remote-ssh-cases), `docs/remote-sessions.md` +**Wake-on-LAN (`remote-wake.ts`)**: an optional `RemoteHost.wakeMac` (Codeman builds the magic packet itself) or `RemoteHost.wakeCommand` (single executable path, run without a shell, takes precedence) lets the INPUT route and `POST /api/sessions/:id/wake` wake a sleeping host instead of writing into a stalled ssh pane. ⚠️ Only user input or an explicit wake request may wake: the auto-reconnect watcher, `handleRemoteSessionDropped` and boot recovery have no access to the registry (a wake there would re-wake the host seconds after every suspend), which `test/remote-wake.test.ts` asserts as a wiring guard; `GET /api/sessions/:id/reachability` merely probes and never wakes. Detection is a throttled bare TCP probe — deliberately no `ServerAliveInterval`, because keepalives move bytes into an idle connection every interval and that is what a byte-threshold idle detector must not read as activity. Input arriving during a wake is buffered and flushed in order after `reattachRemote()`; send-and-wait blocks instead. The wake fields are re-read from `remote-hosts.json` on recovery and, throttled+cached via `RemoteWakeDeps.resolveRemote`, for a LIVE session, since the persisted `remote` snapshot never sees a field added later. UI: the amber `#hostWakeBanner` (`host-wake-ui.js`) with Wake / "Configure WoL" → `#wakeConfigModal`. + **Docker cases**: a case can point at a **container**, with any of the CLI run modes running inside it. Like remote-SSH this is a **LOCATION OVERLAY on cases, never a `SessionMode` of its own**. Exactly one long-lived container **per case**, shared by all its sessions, so killing a session kills only that session's in-container tmux and **never** `docker stop` while siblings remain. The workspace is a real host dir bind-mounted at the **same absolute path**, which is what keeps file-routes/watchers on real host bytes and makes the in-container transcript projHash match the host. Credentials are **seeded** (RO mount, copied into the container once) rather than shared RW, so in-container CLIs never write refreshed tokens back to the host, and bind mounts are excluded from `docker commit` so exports stay secret-free. **NEVER a create-time `-e` for secrets, NEVER `--privileged`, NEVER the docker socket.** Config drift is detected via a label hash and a drifted launch is REFUSED rather than silently launched with stale config. ⚠️ A case may instead **ADOPT** a container the user already runs (`DockerCase.owned === false`, mirror of remote-SSH's `owned:false`): Codeman only `exec`s into it and never creates, starts, stops, restarts or removes it, so a missing or stopped container FAILS CLOSED with an actionable message instead of being fixed. Absent = owned, so existing cases are byte-identical. ⚠️ An ADOPTED container may back SEVERAL cases at different in-container directories (`classifyAdoptContainerConflict` in `docker-hosts.ts`: an exact twin on the same container AND directory is refused, an owned container still backs exactly one case, and a container another user adopted is refused), which is what the Add Case panel's "copy an existing case" picker relies on; the wire carries `CaseInfo.docker.owned` ONLY when false, so the picker tests `=== false`, never truthiness. The guarantee is enforced at four independent layers because it cannot be observed by using the feature: `buildDockerStopCommand`/`buildDockerRemoveCommand` throw during pure STRING CONSTRUCTION, `removeDockerContainer` refuses again, drift reports "none" (an adopted container carries no `codeman.confighash` label, so a real comparison would 409 the launch forever), and the boot reaper skips it. ⚠️ Two lifecycle touches the original design missed and that are easy to re-introduce: the full-image export `docker commit`s the container (refused for an adopted case) and the workspace export `docker pause`s it first (skipped — it freezes the owner's processes for the length of the tar). ⚠️ `owned` is applied AFTER `dockerConfigHash`, which takes an explicit field list, or every pre-existing case would trip the drift gate at once. ⚠️ Run modes for a container case come from the CONTAINER (`availableModes`, live-probed): gating the run menu on HOST CLIs (#201) is right for local sessions and wrong here, since a host with no `claude` may run a container that ships one. ⚠️ **A failed probe means opposite things per ownership** — for an ADOPTED case it is a fault worth reporting, for an OWNED one it is the NORMAL state before the first session (the launch chain creates the container), so treating it as a fault hid every agent mode on every freshly linked Docker case behind "start it yourself first". That is why `CaseInfo.docker.owned` is on the wire. ⚠️ Claude is launched WITHOUT `--dangerously-skip-permissions` when the container's exec user is root (Claude Code refuses the flag as root and the refusal is visible only inside the container); which flag to drop is a per-CLI fact, so it is the registry's `overlays.docker.rootCommand`, never a branch. ⚠️ Adoption is **admin-only in multi-user mode**, unlike `docker-link`: linking creates OUR container, whose one bind mount `isWorkingDirAllowed` has already confined, while an adopted container's mounts belong to its owner and one mounting `/` hands the adopter the host. The same reasoning admin-gates the container listing and the in-container directory browser; the preflight instead admits a non-admin for a container already linked to a case they own, because the run menu probes it for every docker case. ⚠️ On the loopback-only prod bind a container cannot reach 127.0.0.1, so in-container hooks need `CODEMAN_DOCKER_BRIDGE_HOOKS=1`; otherwise idle detection falls back to output-based. → [architecture-invariants#docker-cases](docs/architecture-invariants.md#docker-cases), `docs/docker-cases.md` (user guide), `docs/docker-cases-plan.md` (design) **Docker Compose deployment** (`docker/`, contributed): Codeman itself runs in a container and spawns Docker cases as **SIBLING** containers through the mounted host socket (Docker-outside-of-Docker), never nested. That inverts one assumption the bare-host path takes for granted: the daemon no longer shares Codeman's filesystem, so a bind source valid *inside* Codeman means nothing to it. `resolveDockerDaemonMountSource()` translates sources under HOME into the daemon's namespace via `CODEMAN_DOCKER_HOST_HOME`, and `CODEMAN_CASES_PATH` points the cases dir at a host-absolute bind mount so a workspace resolves to the SAME absolute path on both sides (which is what keeps the transcript projHash matching, per Docker cases above). ⚠️ **`CODEMAN_CASES_PATH` must move every consumer or none**: it is resolved once in `config/cases-dir.ts` because `src/cli.ts` resolves case paths too, and when only the server's `CASES_DIR` learned the override, `codeman skill install --case ` reported "Case not found" on exactly the deployment the override exists for. ⚠️ **`.dockerignore` patterns match the WHOLE context-relative path**, so a bare `.env` line excludes only the ROOT file: `docker/.env` (which holds `CODEMAN_PASSWORD` and any provider keys) rode `COPY . .` into the image until `**/.env` was added — verified in both directions with a real build context. ⚠️ A Compose LONG-form bind (`type: bind`) **creates a missing host source directory ROOT-OWNED** rather than refusing. `Start-Codeman.sh` pre-creates both `CODEMAN_APPDATA_PATH` and `CODEMAN_CASES_PATH` on the host before `up`, which is what keeps the daemon from ever having to materialise either as root in the first place; the container ALSO starts as root (`cap_add: [CHOWN, DAC_OVERRIDE, KILL, SETGID, SETUID]` against the base `cap_drop: ALL`; `test/docker-entrypoint.test.ts` pins that list) so `docker/entrypoint.sh` can correct a bind source that turns up root-owned anyway (a restored backup, a cleared directory, plain `docker compose up` run without the script) before dropping to `PUID:PGID` via `setpriv` — a directory owned by neither root nor `PUID:PGID` is never re-owned, since that ownership is not this container's to reassign; it is PROBED for writability as the runtime account (`setpriv ... test -w`, so ACLs, group-writable trees and CIFS/NFS mounts pass) and refused with a message naming path, owner and PUID:PGID if that fails. ⚠️ `KILL` is in that list for tini, not the entrypoint: `init: true` keeps tini as root while the server runs as PUID, and without CAP_KILL its SIGTERM forward fails and the server is SIGKILLed on every `compose down`/`restart` instead of flushing state. ⚠️ `/opt/codeman-cli` (the runtime-owned CLI prefix) is APPENDED to `PATH`, never prepended, and the entrypoint pins its own `PATH` to the system dirs: the root part of the start resolves `setpriv` by bare name, and a prefix ahead of `/usr/bin` let a planted `setpriv` run as uid 0 (measured). `CODEMAN_DOCKER_DISABLE_SWAP_LIMIT=1` drops `--memory-swap` (and filters only that one kernel warning) for hosts without swap accounting; `--memory` still applies. ⚠️ The deployment ALSO self-updates in place (the repo bind mount at `/opt/codeman` + a restart-by-exiting supervisor) — see Self-update below and `docs/docker-self-update.md` before touching `server.Dockerfile`, the compose file or `.env.example`, since each is an input to the updater's environment gate. `docs/docker-compose.md` + `docker/README.md` (user guides) diff --git a/docs/architecture-invariants.md b/docs/architecture-invariants.md index eb9b772a..8753fea8 100644 --- a/docs/architecture-invariants.md +++ b/docs/architecture-invariants.md @@ -1,6 +1,6 @@ # Architecture invariants -Implementation detail extracted from `CLAUDE.md` so that file stays small enough to load into every session cheaply. Most sections are the original paragraphs, verbatim, including the version history and PR references that explain *why* each rule exists; newer ones are written here first and summarized back into `CLAUDE.md` as a short rule plus a pointer. +Implementation detail extracted from `CLAUDE.md` so that file stays small enough to load into every session cheaply. Most sections are the original paragraphs, verbatim, including the version history and PR references that explain _why_ each rule exists; newer ones are written here first and summarized back into `CLAUDE.md` as a short rule plus a pointer. `CLAUDE.md` keeps the short form of each rule plus a pointer to the section here. Read the pointer first; come here when you need the mechanism, the file names, or the history behind a constraint. @@ -54,6 +54,8 @@ Model is NOT a session field: it is a composition entry in the profile's config ### Remote SSH cases +**Remote host wake-on-LAN from user input**: an optional `RemoteHost.wakeMac` (magic packet built and broadcast by Codeman) or `RemoteHost.wakeCommand` (a single executable path, run WITHOUT a shell, and the explicit override) lets the input route — and an explicit `POST /api/sessions/:id/wake` — wake a SLEEPING host instead of writing into a stalled ssh pane; `tmux send-keys` succeeds against a stalled pane, so the bytes used to vanish silently. The wake flow lives in `src/remote-wake.ts` and is reachable **only** from an EXPLICIT user request: `POST /api/sessions/:id/input`, that explicit wake route, and the create/attach path (`POST /api/quick-start` for a remote case, `POST /api/sessions` with `attachRemoteSession`, via `ensureHostAwake`), because "the user pressed Run on a sleeping host" is the same kind of request and the tmux probe would otherwise fail with a misleading "needs tmux installed". Everything TIMER-driven must never wake a host: the COD-108 auto-reconnect watcher, `Server.handleRemoteSessionDropped` and boot recovery have no access to the registry, or a host would be re-woken seconds after each suspend and could never stay asleep (asserted by wiring guards in `test/remote-wake.test.ts`, not just documented — including that `ensureHostAwake` is called from the HTTP route only, since `cron-service.ts` builds sessions through the shared service with nobody waiting on the answer). `GET /api/sessions/:id/reachability` only ASKS — it never wakes — and feeds the amber "host unreachable" banner (`host-wake-ui.js`) whose action is either Wake or, with no target configured, "Configure WoL" → `#wakeConfigModal` (saved via `PUT /api/remote-hosts/:id`). Detection is a throttled bare TCP probe (no ssh, no `ServerAliveInterval` — keepalives would move bytes into an idle connection every interval), input is buffered and flushed in order after `reattachRemote()` (the send-and-wait path blocks instead, as does the create path, with a shorter request budget), and the wake fields are re-read from `remote-hosts.json` on recovery AND (throttled, cached) live for a running session, because the persisted `remote` snapshot would never see a field added later (`rehydrateRemoteHostFields` + `RemoteWakeDeps.resolveRemote`). Design + invariants: `docs/remote-sessions.md` §Wake-on-LAN from user input. + **Remote SSH cases** (COD-94/#145): cases can point at a **remote host** (`~/.codeman/remote-hosts.json` + `remote-cases.json` via `src/remote-hosts.ts`; CRUD under `/api/cases` — cases route file). A remote session launches a LOCAL tmux pane running `ssh ` that creates a durable REMOTE tmux session on a **dedicated socket** `-L codeman-remote` with name `codeman-ssh-` — deliberately failing the remote Codeman's `SAFE_MUX_NAME_PATTERN` so a Codeman instance on the target host never adopts it; no `-g` global tmux options are set remotely. `remotePath`/`identityFile` are schema-guarded against shell injection (backticks/`$` rejected — same approach as `extraSshOptions`); remote tmux availability is probed via `checkRemoteTmuxAvailable()` in quick-start (ssh args carry `-o ConnectTimeout=10`). Remote claude defaults to an idempotent `claude --session-id || claude --resume ` pair under a login shell, so a respawn or reattach continues the SAME conversation rather than starting a fresh one (remote omp gets the same treatment via `--continue`; ⚠️ because the claude arm is an `a || b` pair under `-c`, that pane's PID is the login shell, not the agent); per-host `commands.*` override. Session kill best-effort kills the remote tmux too. `SessionState.remote`/`MuxSession.remote` round-trip through recovery (`restoreMuxSessions` passes `remote` back into the Session constructor). ⚠️ Run flows must route remote cases through `POST /api/quick-start` (which resolves the remote case and skips LOCAL CLI availability gates) — `POST /api/sessions` stat-validates `workingDir` locally and has no `caseName`. `envOverrides`/`effort`/`modelOverride`/`codexConfig`/`geminiConfig` are rejected for remote quick-starts (not silently dropped). UI: Create Case modal → Remote tab. Tests: `test/remote-hosts.test.ts`, `test/remote-ssh-options.test.ts`. ⚠️ **Reading a file in a remote case goes over ssh too** (#415): `src/remote-files.ts` is the single remote-READ layer (`buildRemoteFileCommand` = `buildSshConnectionArgs` + one shellescaped remote command; `remoteProbePaths` returns remote realpath + stat; `remoteCreateReadStream` streams a `Range` via `tail -c +N | head -c L` and its `close()` must be wired to the response's `close` or the ssh child outlives an aborted download). The guard order matches the local path exactly (`validateSessionFilePathLexical` → remote realpath of BOTH file and workspace root → containment → sensitive-path → size cap on the REMOTE size), a request path arrives from the browser and is only ever interpolated as a `shellescape`d token, and an unreachable host answers **502**, never a 404. ⚠️ The probe's symlink resolution FAILS CLOSED: `readlink -f` where it exists, otherwise a `cd -P`/`pwd -P` directory walk plus a bounded plain-`readlink` loop over the last component, and anything it cannot fully resolve is reported unresolvable (404), never as the unresolved string — the first version resolved the directory chain only, so on a host without `readlink -f` a `ws/notes.txt -> ~/.ssh/id_rsa` link passed containment under its own path while `cat` served the key. Records are NUL-separated and index-keyed so a newline in a filename cannot shift the mapping. ⚠️ ssh children are BOUNDED: probes and buffered reads go through `src/remote-ssh-limiter.ts` (a `document-conversion-limiter`-shaped semaphore, default 4), the attachment-history list probes its whole history in ONE batched call (`probeRemoteAttachmentHistory`, threaded into `registerExternalAttachment({remoteProbes})`), and probes chunk at 40 paths — a prompt-injected agent printing `codeman://attach` links in a remote session used to fork one `ssh` per link. `describeExecError` never returns Node's `Command failed: ` message (identity path + probe script in a 502 body). The `PUT /file-content` guard sits AHEAD of `validateSessionFilePath`, which resolves LOCALLY, or a same-named local directory (an sshfs mount) takes the write. Under `VITEST` the three IO functions refuse rather than connect. This covers the ATTACHMENT routes too, which is the half a clicked path needs when the file is OUTSIDE the case directory (`_isExternalPreviewPath` sends it to `POST …/attachments`): registration, by-id `raw`, metadata and the history list all resolve over ssh (`registerExternalAttachment({remote})`, `resolveServableRemoteAttachment`), and what decides the host is the SESSION, never the path string — the same absolute path means a different file on each host. Deliberately NOT supported over ssh: writes (`edit=1`/`PUT` answer 400, `editable` is always false), office previews/thumbnails, the file tree/picker, `tail-file`. Tests: `test/remote-files.test.ts`, `test/routes/file-routes-remote.test.ts`. ### Docker cases @@ -104,7 +106,7 @@ Tests: `test/docker-hosts.test.ts`, `test/docker-exec-options.test.ts`, `test/do ### Session lineage lines (tab → tab it spawned) -**The relationship did not exist before this** (1.17.0): `SessionState` had no `parentSessionId`, `quick-start` recorded only the multi-user *human* owner, and an agent's spawn call is plain `curl` from a tmux pane, so nothing in the request identifies the caller (`SO_PEERCRED` needs a unix socket; the API is TCP). The caller therefore supplies it — every managed pane already gets `CODEMAN_SESSION_ID` from `session-cli-builder.ts`. Two equivalent inputs, body wins: a `parentSessionId` field on `POST /api/sessions` / `POST /api/quick-start`, or the `X-Codeman-Parent-Session` header, which exists so the agent skill can set it ONCE on its shared curl invocation and have every present and future spawn recipe carry it. +**The relationship did not exist before this** (1.17.0): `SessionState` had no `parentSessionId`, `quick-start` recorded only the multi-user _human_ owner, and an agent's spawn call is plain `curl` from a tmux pane, so nothing in the request identifies the caller (`SO_PEERCRED` needs a unix socket; the API is TCP). The caller therefore supplies it — every managed pane already gets `CODEMAN_SESSION_ID` from `session-cli-builder.ts`. Two equivalent inputs, body wins: a `parentSessionId` field on `POST /api/sessions` / `POST /api/quick-start`, or the `X-Codeman-Parent-Session` header, which exists so the agent skill can set it ONCE on its shared curl invocation and have every present and future spawn recipe carry it. **Resolved, not trusted** (`resolveParentSessionId()`, route-helpers.ts): exact id first, then a UNIQUE prefix of ≥8 chars (ids reach agents truncated — mux names and a Docker export's `$CODEMAN_SESSION_ID` both carry 8), and an ambiguous prefix resolves to NOTHING rather than to a guess. The parent must be a live session the caller can already see (`canAccessOwned`) AND carry the same owner as the session being created, so a multi-user caller cannot staple their session under someone else's tab. ⚠️ **Everything unresolvable is DROPPED, never a 400**: a stale id from a cached skill preamble must cost a decorative line, not a worker. ⚠️ It is decoration at every layer — never an ownership, permission or lifecycle signal; a child outlives its parent, and the Session ctor refuses a self-parent (reachable only via recovery, where both values come off disk). It rides `toState()` into `session_created` / `session_updated`, so there is **no new SSE event**, and `server.ts`'s recovery path restores it so lineage survives a restart. @@ -164,7 +166,7 @@ A file path an agent prints is a link on both surfaces it can appear on, and cli **Media is single-sourced across the two preview paths.** `VIDEO_ATTACHMENT_EXTENSIONS` / `AUDIO_ATTACHMENT_EXTENSIONS` live in `attachment-registry.ts` and are imported by `file-content`'s media classification, so a clip plays identically whether it is in the workspace or reached by id from outside it. They diverged first: the workspace path had its own inline sets and the registry allowlist had no media at all, so a video an agent wrote to `/tmp` was refused as an unsupported type while the same file inside the repo played. ⚠️ Three things have to line up for a player rather than a dead frame: the extension in the allowlist, a **real MIME entry** in `MIME_TYPES` (a `