diff --git a/.abcd/development/brief/04-surfaces/08-import.md b/.abcd/development/brief/04-surfaces/08-import.md new file mode 100644 index 0000000..9b92fc5 --- /dev/null +++ b/.abcd/development/brief/04-surfaces/08-import.md @@ -0,0 +1,92 @@ +# `testimony import` + +Normalises an operator-recorded terminal session — an asciinema recording, in +either asciicast format — into a session's `interactions.jsonl` on the shared +session clock, and keeps the raw `.cast` in the session as an archival +`terminal.cast`. It is `transcribe -audio`'s peer for the terminal: an artefact +the CLI never produced, an anchor read out of that artefact's own metadata, an +explicit `-offset` that always wins, a mandatory printed provenance line, an +idempotent re-run, and an all-or-nothing write. Nothing here calls a model, +spawns a process, allocates a pty, or touches the network. + +`record` is deliberately untouched: no `-terminal` flag, no wrapped recorder, +no second way for a session to end. The operator runs their own recorder in the +window where the work happens, and hands the file over afterwards — the pattern +`demo` already uses for QuickTime and external audio. + +## Flags + +| Flag | Default | Meaning | +|---|---|---| +| `-session` | (required) | session directory | +| `-cast` | (optional) | asciicast file to import; omit to re-import the session's own `terminal.cast` | +| `-offset` | derived | cast→session clock offset in seconds | + +## Behaviour + +- Accepts **asciicast v2 and v3**, told apart by the header's `version` field: + v2 event times are absolute seconds since recording start, v3 event times are + intervals since the previous event, reconstructed by a running sum. Any other + version is refused by name. The difference is confined to one accumulator, and + that accumulator runs on an exact integer grain — microseconds, finer than + either format writes — so the same recording in either format yields + byte-identical records. A `float64`-seconds clock does not give that: v2 rounds + a stated time while v3 rounds a running sum, and at a half-millisecond tie the + two land on different milliseconds, which is enough to flip a coalescing cut. +- Resolves the cast→session offset in `transcribe`'s order: an explicit + `-offset` wins; otherwise the offset is derived from the header's `timestamp` + minus the manifest's `t0_epoch_ms`, in exact integer arithmetic; otherwise 0. + The offset and its provenance are always printed, and the derived provenance + carries the `(whole seconds, ±1s)` caveat, because the header field is an + integer in both formats and the report's default join window is 2.5 s. No + sidecar is persisted: the archived cast keeps its own header, so a re-import + re-derives the identical offset from the identical bytes. +- Requires a usable `t0` on every path, unlike `transcribe`: the records are + epoch-millisecond-timed and `-offset` is defined against the session clock, + and `merge` already refuses a session with interactions and no anchor. +- Keeps only `o` (output) events. `i` (input), `r` (resize), `m` (marker), `x` + (exit), and any unrecognised code are dropped and counted by code, with the + counts printed. The tally is bounded against a cast carrying a different code + on every line, but the input count is exempt from that bound: it is a privacy + disclosure rather than a scoping note, and must not be suppressible. Dropping input is a privacy requirement, not a + simplification: a cast recorded with input capture still cannot put + keystrokes into the derived text. An unrecognised code is dropped rather than + refused, so a future asciicast revision does not make its casts unimportable. +- Coalesces adjacent output into one record per line the terminal displayed, + closed at the first of a newline, a 250 ms inter-event gap, a 1 s span cap + measured from the record's first event, or the encoded JSONL line budget. A + record's time is the instant its first rune arrived. Carriage returns are kept + and are not a boundary, so a redrawn progress line is one record rather than + one per frame. A record whose text renders empty is dropped and counted. +- Splits a single oversized output event across consecutive records at rune + boundaries, budgeted against the **encoded** length of the timeline entry + `merge` will wrap the record in — escaping is what consumes the budget, an + escape byte costing six bytes — and every finished record is then measured for + real, so a wrong assumption about the encoder costs a refusal rather than a + session no command can read back. +- Writes `interactions.jsonl` whole and atomically: records from an earlier + import (identified by `kind: "terminal_output"`, and nothing else) are + dropped, every other line is kept byte-for-byte in file order, and the new + records are appended. So a re-import is byte-identical, a `-demo` session's + clicks survive untouched, and an import that would yield zero records refuses + rather than erase — naming which of the two cases it hit, no output events at + all or output that all rendered empty. Before writing, the assembly is checked + against the line and file limits `session.ReadJSONL` enforces, and the merged + timeline the import implies is measured against the file limit too — the case + only an offline importer can compute rather than estimate — with each entry + charged the id growth `merge` adds past the thousandth interaction, so a + session this pre-flight passes is one `merge` can still read back. +- Archives the cast in two phases: the copy is staged into a temp file beside + `terminal.cast`, the records are written, and the staged copy is renamed into + place last. A failure anywhere before that rename leaves the session exactly + as it was; the one residual state is records with no archival copy, which is + the less misleading of the two. With `-cast` omitted there is no copy phase. +- Keeps ANSI escape sequences raw in the record, because the record is + evidence and a hand-written escape-sequence parser would corrupt it rather + than merely litter it. `report`'s existing sink strips the escape byte, so no + terminal control sequence reaches `report.md`; the printable residue is + answered by recording guidance (`NO_COLOR=1`), not by code. +- Requires no change to `merge`, `report`, `analyze`, or `review`: the records + carry only `t`, `kind`, and `text`, `kind` is an open set, and the timeline + learns no new `src` value. Output schema: + [`../05-internals/02-schemas.md`](../05-internals/02-schemas.md). diff --git a/.abcd/development/brief/05-internals/01-packages.md b/.abcd/development/brief/05-internals/01-packages.md index f07b8c3..fe1a8fb 100644 --- a/.abcd/development/brief/05-internals/01-packages.md +++ b/.abcd/development/brief/05-internals/01-packages.md @@ -4,9 +4,9 @@ Everything lives under `internal/`; `cmd/testimony/main.go` is a thin entrypoint that calls `cli.Run` and exits with its return code. - **`internal/cli`** — the command-line interface: usage text, one - `flag.FlagSet` per subcommand (`demo`, `record`, `transcribe`, `merge`, - `report`, `analyze`, `draft-tests`, `review`, `version`, `help`), and dispatch - into the other packages. Holds the `Version` variable stamped by the release + `flag.FlagSet` per subcommand (`demo`, `record`, `transcribe`, `import`, + `merge`, `report`, `analyze`, `draft-tests`, `review`, `version`, `help`), and + dispatch into the other packages. Holds the `Version` variable stamped by the release process. Errors print as `testimony: ` and map to exit codes (1 failure, 2 usage). - **`internal/demo`** — the instrumented demo app: an embedded single-page @@ -40,6 +40,14 @@ entrypoint that calls `cli.Run` and exits with its return code. engines' JSON output files into engine-neutral segments, and the mapping of segments to the `Utterance` schema. Fixture-tested against golden JSONL files in `testdata/`. +- **`internal/cast`** — the terminal import path: a streaming asciicast reader + that resolves v2's absolute times and v3's interval sums into one absolute + recording clock, the offset resolution from the cast header's own timestamp, + the coalescer that turns adjacent output events into one record per displayed + line, and the all-or-nothing rewrite of `interactions.jsonl` that replaces + only the records this importer wrote. Named for the artefact it parses because + `import` is a Go keyword. Fixture-tested against a v2/v3 pair describing one + recording, plus a golden JSONL file, in `testdata/`. - **`internal/report`** — Markdown rendering of a merged timeline: the event↔utterance attachment pass (the same window test as `timeline.EventsNear`, inlined and keyed by position so the join does not diff --git a/.abcd/development/brief/05-internals/02-schemas.md b/.abcd/development/brief/05-internals/02-schemas.md index 2349ba7..91df9d1 100644 --- a/.abcd/development/brief/05-internals/02-schemas.md +++ b/.abcd/development/brief/05-internals/02-schemas.md @@ -10,6 +10,7 @@ sessions// audio.offset.json # audio→session offset for an external recording (written by transcribe; local only) screen.mp4 # screen capture (written by record -video; local only) events.rrweb.jsonl # raw rrweb events (archival; web sessions only) + terminal.cast # raw asciicast (archival; written by import; local only) interactions.jsonl # normalised interaction events (epoch ms) transcript.jsonl # word-aligned utterances (session-relative seconds) timeline.jsonl # merged, session-relative timeline @@ -61,6 +62,23 @@ schema changes update code, sample, and tests together | `value` | string | optional input value | | `route` | string | optional | +`kind` is an open set with one reserved value: `terminal_output`, written only +by [`import`](../04-surfaces/08-import.md), which identifies its own earlier +records by that value alone. Such a record carries `t`, `kind`, and `text` only; +one record is one line the terminal displayed, its `text` keeping carriage +returns and ANSI escape sequences verbatim. + +## `terminal.cast` — one asciicast, as recorded + +Not a session schema this project defines: the file is an +[asciicast](https://docs.asciinema.org/manual/asciicast/v2/) v2 or v3 recording, +copied into the session byte-for-byte by `import` and read back by a later +`import` that omits `-cast`. Archival only — nothing downstream reads it — and +local only, since it holds every event the recorder captured, keystrokes +included. `import` reads only the header's `version` and `timestamp`, ignoring +every other header field, and bounds a read at 16 MiB per line and 64 MiB per +file. + ## `timeline.jsonl` — one `Entry` per line | Field | Type | Notes | diff --git a/.abcd/development/brief/README.md b/.abcd/development/brief/README.md index dc93957..ca02362 100644 --- a/.abcd/development/brief/README.md +++ b/.abcd/development/brief/README.md @@ -19,7 +19,7 @@ note is preserved as drafted in - [`02-dependencies.md`](02-constraints/02-dependencies.md) — zero Go dependencies; external capability means a subprocess. - [`03-invariants.md`](02-constraints/03-invariants.md) — the one-wall-clock rule, the privacy boundary, schema discipline. - [`04-ethics.md`](02-constraints/04-ethics.md) — validity limits and the privacy/ethics posture. -- [`04-surfaces/`](04-surfaces/) — one file per command: [`demo`](04-surfaces/01-demo.md), [`transcribe`](04-surfaces/02-transcribe.md), [`merge`](04-surfaces/03-merge.md), [`report`](04-surfaces/04-report.md), [`record`](04-surfaces/05-record.md), [`analyze`](04-surfaces/06-analyze.md), [`review`](04-surfaces/07-review.md). +- [`04-surfaces/`](04-surfaces/) — one file per command: [`demo`](04-surfaces/01-demo.md), [`transcribe`](04-surfaces/02-transcribe.md), [`merge`](04-surfaces/03-merge.md), [`report`](04-surfaces/04-report.md), [`record`](04-surfaces/05-record.md), [`analyze`](04-surfaces/06-analyze.md), [`review`](04-surfaces/07-review.md), [`import`](04-surfaces/08-import.md). - [`05-internals/`](05-internals/) - [`01-packages.md`](05-internals/01-packages.md) — the package map under `internal/`. - [`02-schemas.md`](05-internals/02-schemas.md) — the JSON schemas of the session artefacts. diff --git a/.abcd/development/intents/drafts/itd-11-terminal-cast-import.md b/.abcd/development/intents/drafts/itd-11-terminal-cast-import.md deleted file mode 100644 index ebd419e..0000000 --- a/.abcd/development/intents/drafts/itd-11-terminal-cast-import.md +++ /dev/null @@ -1,74 +0,0 @@ ---- -id: itd-11 -slug: terminal-cast-import -spec_id: null -kind: null -suggested_kind: null -reclassification_history: [] -builds_on: [] -severity: major ---- - -# A Terminal Recording Arrives the Way a Voice Recording Already Does - -## Press Release - -> **Testimony turns an operator-recorded terminal session into timeline evidence.** The operator records their shell themselves — `asciinema rec session.cast` in the terminal where they work, ended the way asciinema always ends, while `testimony record` captures narration in another window exactly as it does today. Afterwards they hand the `.cast` file to an import step, the same hand-off `transcribe -audio` already performs for an external voice recording: the cast's own header timestamp anchors every command and line of output to the session clock, the records land in the session's ordinary interaction stream, and `merge`, `report`, `analyze`, and `review` run byte-identical. A spoken "I have no idea what that error is telling me" lands next to the command that produced it and the text it printed. -> -> "I already start my own voice recorder when I want one — Testimony just tells me where to hand the file," said Alice, the maintainer. "Recording my own terminal the same way means nothing about how my shell or my session ends had to change. One extra command afterwards, and the stumble, the command, and my exact words sit on one line of the timeline." - -## Why This Matters - -The evidence need is the one itd-6 identified, unchanged: the pipeline's model wants a structured, timestamped, text-searchable interaction stream for terminal targets, the codebase-mapping intent's acceptance criteria (itd-3) already assume a cast stream exists, and asciinema's asciicast formats already *are* that stream — no instrumentation of the tool under test. Without it, CLI sessions degrade to the weakest and most expensive evidence channel. - -There are now two asciicast formats in the wild, and the import must accept both. asciinema 3.0 (September 2025, a Rust rewrite) made asciicast **v3** the default output format: its event times are *intervals since the previous event*, where v2's are *absolute seconds since recording start* — a file-corrupting difference if a v3 cast is read with v2 semantics, since every event after the first drifts steadily earlier than reality. Homebrew — the recommended install on macOS, the target platform — ships the 3.x line (3.2.1 at the time of writing), so a fresh `brew install asciinema` produces v3 casts by default; meanwhile PyPI's latest release is still 2.4.0 (October 2023) and Debian packages the same, so v2-only recorders remain widespread indefinitely. The 3.x CLI can be told to write v2 (`--output-format asciicast-v2`), but the 2.x CLI rejects that flag outright, so no single recommended invocation works across both lines. The decoupled delivery shape absorbs this cleanly: the operator records with whatever asciinema they have, and the import step sniffs the header's `version` field and anchors each format by its own semantics — the churn is a parsing concern solved once, in one function, rather than a binary-version matrix inside `record`. - -What this intent changes is the delivery shape. The wrap-the-shell design (spc-3) pulled the whole recorder lifecycle inside `record`: a pty hijack of the operator's interactive shell, a conditional end-of-session gesture — Ctrl-D on one flag, Ctrl+C otherwise — signal-handling reasoning about Ctrl+C mid-command, an acknowledged SIGTERM/SIGHUP forwarding gap, and a runtime binary dependency, all landing in the one command that must never lose a session. Yet the pipeline already has a normalised pattern for external capture it never manages: `demo`'s printed instructions tell the operator to start QuickTime themselves and hand the file to `transcribe -audio` afterwards, and `record`/`demo` never touch that recorder's lifecycle at all. Applying the same pattern to the terminal removes every item on that list — `record` gains no flag, no wrapped process, no second way to end. - -The trade is honest but favourable. The cost is per-session ceremony: the operator starts and stops one more recorder and runs one more command, the same ceremony the external-audio path already asks of them. The wrap-the-shell design demanded operator ceremony too — knowing that one flag silently changes how a session ends — and paid for it again in standing failure modes. The decoupled hand-off is anchored from data inside the artefact rather than from file creation time: an asciicast header carries an absolute Unix timestamp in both v2 and v3. That anchor is honest to a bound, not exact — the header field is an *integer*, whole seconds, in both formats, so the reconstructed clock can sit up to a second adrift of `t0`'s millisecond precision, which is material against `report`'s 2.5-second default join window. The spoken start marker therefore stays recommended for the terminal path as the calibration cross-check, and the explicit `-offset` override stays for correcting a skewed or absent header. - -Alternatives considered and set aside: plain `script(1)` is universally available where asciinema is not, but its timing capture splits across two files, carries no absolute timestamp to anchor against `t0`, and diverges between the BSD/macOS and util-linux implementations — buying availability at the cost of exactly the anchoring this evidence needs. Shell history with timestamps records commands but never output, which is half the evidence. A screen recording of the terminal (`record -video`, which works today) is not text-searchable and cannot serve as a mapping anchor. A bespoke pty wrapper would trade a format-parsing concern for owning pty allocation, raw-mode handling, and resize plumbing across platforms — far more surface than a dual-format parser. Asciicast, produced by a recorder the operator runs themselves, keeps the format's strengths without inheriting its process-management costs. - -## What's In Scope - -- An import step in `transcribe -audio`'s mould: given a session directory and an asciicast file, normalise the cast's output events into the session's interaction stream on the shared clock, and keep the raw `.cast` in the session directory as an archival artefact alongside `events.rrweb.jsonl`. -- Accepting **both asciicast v2 and v3**, distinguished by the header's `version` field: v2 event times are absolute seconds since recording start; v3 event times are intervals since the previous event, reconstructed by a running sum. A cast declaring any other version is refused with a message naming the file and the version found — never guessed at. -- Clock anchoring from the cast header's absolute timestamp (an optional integer in both formats), an explicit `-offset` override, and a printed offset-provenance line matching `transcribe`'s existing pattern — including the whole-second quantisation caveat in the printed line, so the operator knows the anchor's honest precision. -- Output events larger than a JSONL line split across consecutive records at safe boundaries, never truncated — evidence is not silently dropped. -- Output-only capture guidance: the recommended invocation is plain `asciinema rec session.cast` on either CLI line, which records what the terminal displays, never raw keystrokes — input capture is opt-in on both lines (`--stdin` on 2.x; `--capture-input`/`-I`, with `--stdin` kept as an alias, on 3.x) and the guidance says to opt out, so a password typed at a suppressed-echo prompt cannot land in the evidence. Any `i` (input) events present in a handed-over cast are dropped at import, never normalised into the interaction stream. -- Documenting the terminal path: the archival cast in the session-directory reference, a how-to for the two-terminal session (`record` in one, `asciinema rec` in the one where the work happens), and guidance-only install pointers for asciinema as the suggested recorder. -- A privacy warning in the terminal how-to: terminal output routinely carries usernames, hostnames, absolute paths, environment values, and occasionally secrets printed by tools, and `analyze` sends timeline text to a model — the how-to tells the operator to review or redact the session before running `analyze`, in exactly the way the browser path never had to, because a shell shows more of the machine than a demo app does. -- `merge`, `report`, `analyze`, and `review` unchanged — the timeline schema learns no new source type. - -## What's Out of Scope - -- `record` wrapping, spawning, or supervising any terminal recorder — no `-terminal` flag, no pty, no change to how a session ends. This is the fence that distinguishes this intent from itd-6's spec. -- Keystroke capture (`--stdin` on the 2.x line, `--capture-input`/`-I` on 3.x); the suppressed-echo password hazard identified in spc-3 carries forward unchanged. -- Accepting formats other than asciicast v2 and v3 — asciicast v1, `script(1)` timing pairs, shell history — noted above as considered and set aside. -- Forcing or converting between cast format versions at record time (`--output-format`); the operator records with whatever their asciinema writes, and the importer meets the file where it is. -- Resolving cast-stream anchors to source locations; that is the codebase-mapping step (itd-3), which this intent unblocks rather than performs. -- TUI redraw handling beyond preserving the raw cast; line-oriented CLI sessions are the target. -- The installer behaviour change to `whisper.cpp` that spc-3 bundled — unrelated scope, deliberately not carried into this intent. -- Replaying a `.cast` as video; the stream is evidence and analysis input, not a playback surface. - -## Acceptance Criteria - -- **Given** a narrated session and an asciicast v2 file recorded alongside it, **when** the import step and then `merge` run, **then** `timeline.jsonl` interleaves the spoken utterances and the cast's commands and output on one session-relative clock derived from the same `t0`, with no separate clock for the terminal stream. -- **Given** the same session recorded as asciicast v3 (a stock `brew install asciinema` recorder), **when** the import step runs, **then** each event's absolute time is reconstructed by summing intervals from the recording start, and the resulting timeline is identical to what the equivalent v2 cast produces — the operator never states, or needs to know, which format their recorder wrote. -- **Given** a cast whose header declares a version other than 2 or 3, **when** the import step runs, **then** it refuses with a message naming the file and the version found, and writes nothing — never a silently misread clock. -- **Given** an imported terminal session, **when** `report` runs, **then** each utterance renders with the commands and output that fall inside its join window, through the existing event rendering unchanged. -- **Given** the import step has run, **when** `merge`, `report`, or `analyze` execute, **then** they behave identically to today for the same inputs — no new source type, flag, or schema field is required of them. -- **Given** a cast whose header timestamp precedes the session's `t0` (the recorder was started early), **when** the timeline is built, **then** its early events carry negative session-relative times and render exactly as an early-started audio recording's utterances already do. -- **Given** a single output event larger than the readable JSONL line limit, **when** it is imported, **then** it is split across multiple records within the limit with every byte preserved, and no record is truncated. -- **Given** a `record` session with a terminal recording underway in another window, **when** the operator presses Ctrl+C in `record`'s terminal, **then** the session finalises exactly as an audio-only session does — nothing about terminal capture appears in `record`'s lifecycle. - -## Open Questions - -- Command surface: a new verb, or a flag on an existing command? The `transcribe -audio` analogy suggests a peer command; the name should not imply it transcribes speech. -- Re-run and mixing semantics: a second import of the same session should be idempotent like a re-run of `transcribe`, but a session that also holds browser interactions (a `-demo` session) shares the interaction stream — does import append, refuse, or replace only records it previously wrote? -- Should `install.sh` mention asciinema with the guidance-only pattern (explain and print the install command, never run it), or is the how-to page alone the right home? -- Chunking granularity: a shell echoes typed commands back one keystroke at a time, so the raw `o` stream around a command is a run of one-character events interleaved with prompt redraws — one timeline entry per raw event would fragment a single typed command across dozens of near-empty records. Does the importer coalesce adjacent output events below an inter-event gap threshold into one record (the natural fix, tuned against a real session), and is the coalesced record still a faithful "what the terminal displayed" claim? This is the spec-level question the "commands appear via shell echo" guidance rests on. - -## Audit Notes - -_Empty. Populated by intent-fidelity-reviewer when intent moves to shipped/._ diff --git a/.abcd/development/intents/shipped/itd-11-terminal-cast-import.md b/.abcd/development/intents/shipped/itd-11-terminal-cast-import.md new file mode 100644 index 0000000..b6cc93f --- /dev/null +++ b/.abcd/development/intents/shipped/itd-11-terminal-cast-import.md @@ -0,0 +1,213 @@ +--- +id: itd-11 +slug: terminal-cast-import +spec_id: spc-2609120417486971 +kind: standalone +suggested_kind: null +reclassification_history: [] +builds_on: [] +severity: major +--- + +# A Terminal Recording Arrives the Way a Voice Recording Already Does + +## Press Release + +> **Testimony turns an operator-recorded terminal session into timeline evidence.** The operator records their shell themselves — `asciinema rec session.cast` in the terminal where they work, ended the way asciinema always ends, while `testimony record` captures narration in another window exactly as it does today. Afterwards they hand the `.cast` file to an import step, the same hand-off `transcribe -audio` already performs for an external voice recording: the cast's own header timestamp anchors every command and line of output to the session clock, the records land in the session's ordinary interaction stream, and `merge`, `report`, `analyze`, and `review` run byte-identical. A spoken "I have no idea what that error is telling me" lands next to the command that produced it and the text it printed. +> +> "I already start my own voice recorder when I want one — Testimony just tells me where to hand the file," said Alice, the maintainer. "Recording my own terminal the same way means nothing about how my shell or my session ends had to change. One extra command afterwards, and the stumble, the command, and my exact words sit on one line of the timeline." + +## Why This Matters + +The evidence need is the one itd-6 identified, unchanged: the pipeline's model wants a structured, timestamped, text-searchable interaction stream for terminal targets, the codebase-mapping intent's acceptance criteria (itd-3) already assume a cast stream exists, and asciinema's asciicast formats already *are* that stream — no instrumentation of the tool under test. Without it, CLI sessions degrade to the weakest and most expensive evidence channel. + +There are now two asciicast formats in the wild, and the import must accept both. asciinema 3.0 (September 2025, a Rust rewrite) made asciicast **v3** the default output format: its event times are *intervals since the previous event*, where v2's are *absolute seconds since recording start* — a file-corrupting difference if a v3 cast is read with v2 semantics, since every event after the first drifts steadily earlier than reality. Homebrew — the recommended install on macOS, the target platform — ships the 3.x line (3.2.1 at the time of writing), so a fresh `brew install asciinema` produces v3 casts by default; meanwhile PyPI's latest release is still 2.4.0 (October 2023) and Debian packages the same, so v2-only recorders remain widespread indefinitely. The 3.x CLI can be told to write v2 (`--output-format asciicast-v2`), but the 2.x CLI rejects that flag outright, so no single recommended invocation works across both lines. The decoupled delivery shape absorbs this cleanly: the operator records with whatever asciinema they have, and the import step sniffs the header's `version` field and anchors each format by its own semantics — the churn is a parsing concern solved once, in one function, rather than a binary-version matrix inside `record`. + +What this intent changes is the delivery shape. The wrap-the-shell design (spc-3) pulled the whole recorder lifecycle inside `record`: a pty hijack of the operator's interactive shell, a conditional end-of-session gesture — Ctrl-D on one flag, Ctrl+C otherwise — signal-handling reasoning about Ctrl+C mid-command, an acknowledged SIGTERM/SIGHUP forwarding gap, and a runtime binary dependency, all landing in the one command that must never lose a session. Yet the pipeline already has a normalised pattern for external capture it never manages: `demo`'s printed instructions tell the operator to start QuickTime themselves and hand the file to `transcribe -audio` afterwards, and `record`/`demo` never touch that recorder's lifecycle at all. Applying the same pattern to the terminal removes every item on that list — `record` gains no flag, no wrapped process, no second way to end. + +The trade is honest but favourable. The cost is per-session ceremony: the operator starts and stops one more recorder and runs one more command, the same ceremony the external-audio path already asks of them. The wrap-the-shell design demanded operator ceremony too — knowing that one flag silently changes how a session ends — and paid for it again in standing failure modes. The decoupled hand-off is anchored from data inside the artefact rather than from file creation time: an asciicast header carries an absolute Unix timestamp in both v2 and v3. That anchor is honest to a bound, not exact — the header field is an *integer*, whole seconds, in both formats, so the reconstructed clock can sit up to a second adrift of `t0`'s millisecond precision, which is material against `report`'s 2.5-second default join window. The spoken start marker therefore stays recommended for the terminal path as the calibration cross-check, and the explicit `-offset` override stays for correcting a skewed or absent header. + +Alternatives considered and set aside: plain `script(1)` is universally available where asciinema is not, but its timing capture splits across two files, carries no absolute timestamp to anchor against `t0`, and diverges between the BSD/macOS and util-linux implementations — buying availability at the cost of exactly the anchoring this evidence needs. Shell history with timestamps records commands but never output, which is half the evidence. A screen recording of the terminal (`record -video`, which works today) is not text-searchable and cannot serve as a mapping anchor. A bespoke pty wrapper would trade a format-parsing concern for owning pty allocation, raw-mode handling, and resize plumbing across platforms — far more surface than a dual-format parser. Asciicast, produced by a recorder the operator runs themselves, keeps the format's strengths without inheriting its process-management costs. + +## What's In Scope + +- An import step in `transcribe -audio`'s mould: given a session directory and an asciicast file, normalise the cast's output events into the session's interaction stream on the shared clock, and keep the raw `.cast` in the session directory as an archival artefact alongside `events.rrweb.jsonl`. +- Accepting **both asciicast v2 and v3**, distinguished by the header's `version` field: v2 event times are absolute seconds since recording start; v3 event times are intervals since the previous event, reconstructed by a running sum. A cast declaring any other version is refused with a message naming the file and the version found — never guessed at. +- Clock anchoring from the cast header's absolute timestamp (an optional integer in both formats), an explicit `-offset` override, and a printed offset-provenance line matching `transcribe`'s existing pattern — including the whole-second quantisation caveat in the printed line, so the operator knows the anchor's honest precision. +- Output events larger than a JSONL line split across consecutive records at safe boundaries, never truncated — evidence is not silently dropped. +- Output-only capture guidance: the recommended invocation is plain `asciinema rec session.cast` on either CLI line, which records what the terminal displays, never raw keystrokes — input capture is opt-in on both lines (`--stdin` on 2.x; `--capture-input`/`-I`, with `--stdin` kept as an alias, on 3.x) and the guidance says to opt out, so a password typed at a suppressed-echo prompt cannot land in the evidence. Any `i` (input) events present in a handed-over cast are dropped at import, never normalised into the interaction stream. +- Documenting the terminal path: the archival cast in the session-directory reference, a how-to for the two-terminal session (`record` in one, `asciinema rec` in the one where the work happens), and guidance-only install pointers for asciinema as the suggested recorder. +- A privacy warning in the terminal how-to: terminal output routinely carries usernames, hostnames, absolute paths, environment values, and occasionally secrets printed by tools, and `analyze` sends timeline text to a model — the how-to tells the operator to review or redact the session before running `analyze`, in exactly the way the browser path never had to, because a shell shows more of the machine than a demo app does. +- `merge`, `report`, `analyze`, and `review` unchanged — the timeline schema learns no new source type. + +## What's Out of Scope + +- `record` wrapping, spawning, or supervising any terminal recorder — no `-terminal` flag, no pty, no change to how a session ends. This is the fence that distinguishes this intent from itd-6's spec. +- Keystroke capture (`--stdin` on the 2.x line, `--capture-input`/`-I` on 3.x); the suppressed-echo password hazard identified in spc-3 carries forward unchanged. +- Accepting formats other than asciicast v2 and v3 — asciicast v1, `script(1)` timing pairs, shell history — noted above as considered and set aside. +- Forcing or converting between cast format versions at record time (`--output-format`); the operator records with whatever their asciinema writes, and the importer meets the file where it is. +- Resolving cast-stream anchors to source locations; that is the codebase-mapping step (itd-3), which this intent unblocks rather than performs. +- TUI redraw handling beyond preserving the raw cast; line-oriented CLI sessions are the target. +- The installer behaviour change to `whisper.cpp` that spc-3 bundled — unrelated scope, deliberately not carried into this intent. +- Replaying a `.cast` as video; the stream is evidence and analysis input, not a playback surface. + +## Acceptance Criteria + +- **Given** a narrated session and an asciicast v2 file recorded alongside it, **when** the import step and then `merge` run, **then** `timeline.jsonl` interleaves the spoken utterances and the cast's commands and output on one session-relative clock derived from the same `t0`, with no separate clock for the terminal stream. +- **Given** the same session recorded as asciicast v3 (a stock `brew install asciinema` recorder), **when** the import step runs, **then** each event's absolute time is reconstructed by summing intervals from the recording start, and the resulting timeline is identical to what the equivalent v2 cast produces — the operator never states, or needs to know, which format their recorder wrote. +- **Given** a cast whose header declares a version other than 2 or 3, **when** the import step runs, **then** it refuses with a message naming the file and the version found, and writes nothing — never a silently misread clock. +- **Given** an imported terminal session, **when** `report` runs, **then** each utterance renders with the commands and output that fall inside its join window, through the existing event rendering unchanged. +- **Given** the import step has run, **when** `merge`, `report`, or `analyze` execute, **then** they behave identically to today for the same inputs — no new source type, flag, or schema field is required of them. +- **Given** a cast whose header timestamp precedes the session's `t0` (the recorder was started early), **when** the timeline is built, **then** its early events carry negative session-relative times and render exactly as an early-started audio recording's utterances already do. +- **Given** a single output event larger than the readable JSONL line limit, **when** it is imported, **then** it is split across multiple records within the limit with every byte preserved, and no record is truncated. +- **Given** a `record` session with a terminal recording underway in another window, **when** the operator presses Ctrl+C in `record`'s terminal, **then** the session finalises exactly as an audio-only session does — nothing about terminal capture appears in `record`'s lifecycle. + +## Scope Conditions + +- **The operator's asciinema writes asciicast v2 or v3**, and the file reaches the import step as that recorder wrote it. Any other format — asciicast v1, a `script(1)` timing pair, a converted or hand-edited cast declaring another version — is refused by name, so the claim covers the two formats in the wild and nothing else. +- **The cast header carries an integer `timestamp`, or the operator states the anchor with `-offset`.** Both formats make the field optional. With neither, the terminal stream is placed at offset 0 — the assumption that the recorder started at `t0` — and the spoken start marker is the only cross-check the operator has. +- **The anchor is honest to a whole second, not to a millisecond.** The header field is an integer in both formats, so the reconstructed clock can sit up to a second adrift of `t0`, against a 2.5-second default join window. +- **The session directory holds a `manifest.json` with a positive `t0_epoch_ms`.** Interaction times are epoch milliseconds anchored against it; a session without a usable `t0` cannot place any interaction on the session clock, terminal or otherwise. +- **The narration and the terminal recording belong to one wall-clock session**, so both streams resolve against the same `t0`. Two recordings made at different times do not interleave by being imported into one directory. +- **The recorded work is line-oriented shell output.** A full-screen TUI, a pager, or a progress bar redrawing over itself keeps its evidence in redraw sequences: they are preserved verbatim, and are not reconstructed into what the screen finally displayed. +- **One terminal recording per session.** Records from a second, different cast replace the first's, because both are identified as this importer's own. +- **Input capture is left off at record time**, the recommended invocation on both CLI lines. The importer drops `i` events regardless, but only recording without input capture keeps keystrokes out of the artefact itself. +- **The operator reviews or redacts the derived timeline before `analyze` runs.** Terminal output routinely carries usernames, hostnames, absolute paths, environment values, and occasionally secrets printed by tools — more of the machine than a demo app ever shows. + +## Open Questions + +- Command surface: a new verb, or a flag on an existing command? The `transcribe -audio` analogy suggests a peer command; the name should not imply it transcribes speech. +- Re-run and mixing semantics: a second import of the same session should be idempotent like a re-run of `transcribe`, but a session that also holds browser interactions (a `-demo` session) shares the interaction stream — does import append, refuse, or replace only records it previously wrote? +- Should `install.sh` mention asciinema with the guidance-only pattern (explain and print the install command, never run it), or is the how-to page alone the right home? +- Chunking granularity: a shell echoes typed commands back one keystroke at a time, so the raw `o` stream around a command is a run of one-character events interleaved with prompt redraws — one timeline entry per raw event would fragment a single typed command across dozens of near-empty records. Does the importer coalesce adjacent output events below an inter-event gap threshold into one record (the natural fix, tuned against a real session), and is the coalesced record still a faithful "what the terminal displayed" claim? This is the spec-level question the "commands appear via shell echo" guidance rests on. + +## Audit Notes + + +Fidelity review — receipt rcp-fcf0bc47445c (verifier abcd:intent-auditor claude-opus-5[1m]). + +Provenance: abcd:intent-auditor@claude-opus-5[1m] · rubric_hash sha256:43133dfce85f90e6462ffcc449daa41b27c8f4fcc1975710222a5703d3d57946 · prompt_hash sha256:17b9a757f3fcc8c565c184165fadbdb48810c4c4869ddde98b9a683694b87fdb +Input attestations: diff:origin/main..working tree (4e81f718ac5db206ebc4df960a18044331d0396c..4c3f8433e7274010ee305a4b6571e1a18df90236 plus the uncommitted working tree)@sha256:e84082ba9ba36bd43bc91d39d3a02f344c807466ab1c1a3ff049c896c1745c78; review-request:.abcd/.work.local/reviews/rcp-fcf0bc47445c.request.md@sha256:43133dfce85f90e6462ffcc449daa41b27c8f4fcc1975710222a5703d3d57946; intent:.abcd/development/intents/shipped/itd-11-terminal-cast-import.md@sha256:17b9a757f3fcc8c565c184165fadbdb48810c4c4869ddde98b9a683694b87fdb; note:.abcd/.work.local/reviews/rcp-fcf0bc47445c.request.md:1@-; gates:go.mod:1@-; + +Acceptance rollup: MET 7 · MET_WITH_CONCERNS 1 · NOT_MET 0 · INCONCLUSIVE 0 + +Per-criterion verdicts: +- ac-1 — MET: A v2 cast plus a two-utterance transcript merges into one session-relative clock: TestImportThenMergeInterleaves asserts the exact interleaved entry order and times (-2 event, -1.5 speech, -1.48 event, …) and the records carry only epoch-ms t derived from the same manifest t0 through recordTime, with no per-source offset anywhere in timeline.jsonl. + evidence: internal/cast/cast_test.go:962 + evidence: internal/cast/coalesce.go:189 + evidence: internal/cast/cast.go:274 +- ac-2 — MET: The v2/v3 difference is confined to one accumulator in parseEvent (v3 adds each interval to a running float64 sum), and TestV2AndV3Agree asserts the two fixtures describing the same recording produce a byte-identical interactions.jsonl, while TestImportThenMergeInterleaves runs the identical golden entry table over both fixtures; the operator states nothing about the format, since -cast carries no format flag. + evidence: internal/cast/scan.go:233 + evidence: internal/cast/cast_test.go:210 + evidence: internal/cast/testdata/v3.cast:1 +- ac-3 — MET: parseHeader refuses any version other than 2 or 3 naming both the file and the version as written, the refusal fires in scan step 3 before any of Run's write steps, and TestUnsupportedVersionRefuses (versions 1, "2", absent) plus TestVersion4FixtureRefuses each assert the message names the file and that assertSessionUnchanged holds a full-directory hash snapshot. + evidence: internal/cast/scan.go:186 + evidence: internal/cast/cast_test.go:255 + evidence: internal/cast/cast_test.go:246 +- ac-4 — MET: TestReportRendersTerminalEvents imports, merges and renders, asserting the utterance line at [-00:02] is accompanied by `- [-00:02] terminal\_output` and `[00:00] terminal\_output` bullets carrying the command text, and internal/report has an empty diff against origin/main, so the rendering path is the existing eventLine unchanged. + evidence: internal/cast/cast_test.go:993 + evidence: internal/report/report.go:1 + evidence: .github/workflows/ci.yml:196 +- ac-5 — MET: internal/timeline, internal/report, internal/analyze and internal/review are all absent from the delivered diffstat, the records carry only the already-documented t/kind/text fields (no new src value, no new flag), and every record is put through the exported timeline.CheckInteraction before it is written so import cannot persist anything merge would refuse. + evidence: internal/cast/cast.go:288 + evidence: internal/cast/testdata/golden.interactions.jsonl:2 + evidence: internal/timeline/timeline.go:1 +- ac-6 — MET: TestEarlyStartedCastGoesNegative sets the header timestamp 30 s before t0 and asserts merged entry times of -30 and 0 plus `[-00:30]` in report.Render's output; the record's t stays a positive epoch-ms value (t0 + offsetMS + ms) so timeline.CheckInteraction's positivity rule is satisfied and report.clock signs it exactly as it signs an early-started audio utterance. + evidence: internal/cast/cast_test.go:1035 + evidence: internal/cast/cast_test.go:1041 + evidence: .github/workflows/ci.yml:197 +- ac-7 — MET_WITH_CONCERNS: A 6 MiB output event does split: coalescer.add closes the record when the next rune would breach a per-record encoded budget and TestOversizedEventSplits asserts more than one record, every wrapped timeline entry inside session.MaxJSONLLine, a successful Merge, and that concatenating the records' text reproduces the event's data exactly (utf8.ValidString per record in the coalescer unit) — but the concern is that 'every byte preserved' is honoured at rune granularity over the string encoding/json decoded, not over the cast's raw bytes: invalid UTF-8 becomes U+FFFD in the decoder before the importer sees it, and the byte-exact artefact is the archived terminal.cast (TestCastArchivedVerbatim). The spec records this as the one place it deliberately narrows a criterion. + evidence: internal/cast/coalesce.go:94 + evidence: internal/cast/cast_test.go:816 + evidence: internal/cast/coalesce_test.go:210 + evidence: .abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md:841 +- ac-8 — MET: internal/record is absent from the delivered diffstat entirely — no flag, no recorder, no subprocess, no signal path touched — so Ctrl+C finalises a session exactly as before, and TestUsageListsImport asserts the usage text offers no -terminal flag anywhere while naming the new hand-off verb. + evidence: internal/cli/cli_test.go:895 + evidence: internal/record/record.go:1 + evidence: internal/cli/cli.go:36 + +Gap audit: +- honoured: + - An import step in transcribe -audio's mould: a session directory plus an asciicast file, normalised into the session's interaction stream, with the raw .cast kept as an archival artefact alongside events.rrweb.jsonl + evidence: internal/cast/cast.go:61 + evidence: internal/session/session.go:55 + evidence: docs/reference/session-directory.md:103 + - Both asciicast v2 and v3 accepted and distinguished by the header's version field; any other version refused by name + evidence: internal/cast/scan.go:221 + evidence: internal/cast/scan.go:186 + - Clock anchoring from the header timestamp, an explicit -offset override, and a printed offset-provenance line carrying the whole-second quantisation caveat + evidence: internal/cast/cast.go:137 + evidence: internal/cli/cli_test.go:797 + - Output events larger than a JSONL line split across consecutive records at safe boundaries, never truncated + evidence: internal/cast/coalesce.go:178 + evidence: internal/cast/cast_test.go:800 + - Input (i) events dropped at import and never normalised into the interaction stream, with the drop counted and printed + evidence: internal/cast/cast.go:122 + evidence: internal/cast/cast.go:319 + evidence: .github/workflows/ci.yml:187 + - Documenting the terminal path: the archival cast in the session-directory reference, a two-terminal how-to, and guidance-only asciinema install pointers + evidence: docs/how-to/record-a-terminal-session.md:10 + evidence: docs/how-to/record-a-terminal-session.md:30 + evidence: docs/reference/session-directory.md:12 + - A privacy warning in the terminal how-to: review or redact the session before running analyze, and record with input capture off + evidence: docs/how-to/record-a-terminal-session.md:81 + evidence: docs/how-to/record-a-terminal-session.md:65 + - merge, report, analyze and review unchanged — the timeline schema learns no new source type + evidence: internal/cast/testdata/golden.interactions.jsonl:1 + evidence: internal/timeline/timeline.go:1 + - record wraps, spawns or supervises no terminal recorder — no -terminal flag, no pty, no change to how a session ends (the intent's fence) + evidence: internal/record/record.go:1 + evidence: internal/cli/cli_test.go:895 + - 'One extra command afterwards' — the hand-off is a single peer verb the operator runs after the fact, with an idempotent re-run + evidence: internal/cli/cli.go:317 + evidence: internal/cli/cli_test.go:841 + evidence: .github/workflows/ci.yml:190 +- diverged: + - 'Every byte preserved' on an oversized split: delivered as rune-exact preservation of the string encoding/json decoded, with byte-exactness held by the archived terminal.cast instead + evidence: internal/cast/cast_test.go:816 + evidence: internal/cast/cast_test.go:653 + evidence: .abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md:836 + - The unsupported-version refusal was specified as `asciicast version %d`; delivered as the JSON literal (`version "2"`), neutralised and clipped, so a non-integer version is named as written rather than as a decode failure + evidence: internal/cast/scan.go:186 + evidence: .abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md:239 + - A header `timestamp` field present but not an integer is refused with -offset guidance rather than treated as the absent case the intent's scope condition describes + evidence: internal/cast/scan.go:192 +- missing: + - The live two-terminal validation the spec names as part of done for the implementing change — a real record + asciinema session on both the 2.x and 3.x CLI lines, against which the 250 ms gap and 1 s span constants were to be tuned — leaves no artefact in the delivered change (no note in DECISIONS.md, CONTEXT.md, or the CI smoke, which is hermetic and runs no recorder) + evidence: .abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md:1036 + evidence: .abcd/work/DECISIONS.md:1620 + +Scope-condition dispositions: +- cond-2609120432406223 — survived: The importer accepts exactly v2 and v3 as the header declares and refuses every other version by name before anything is written, so the claim covers the two formats in the wild and nothing else. + evidence: internal/cast/scan.go:180 + evidence: internal/cast/testdata/bad-version.cast:1 +- cond-2609120432405024 — narrowed: An absent (or null) header timestamp does default to offset 0 with the provenance printed, and -offset always wins — but a timestamp that is present and unusable is refused rather than defaulted, so the offset-0 fallback covers less ground than the condition states. + narrowing: The 'placed at offset 0' fallback holds only when the header timestamp is absent or JSON null; a present but non-positive timestamp (internal/cast/cast.go:263) or a present non-integer one (internal/cast/scan.go:192) refuses the run with -offset guidance instead of assuming the recorder started at t0. + evidence: internal/cast/cast.go:255 + evidence: internal/cast/cast.go:263 + evidence: internal/cast/testdata/v3-nots.cast:1 +- cond-2609120432402714 — survived: The derived anchor is integer arithmetic over a whole-second header field, and the ±1 s bound is printed to the operator on every derived run rather than only documented. + evidence: internal/cast/cast.go:274 + evidence: internal/cli/cli_test.go:797 +- cond-2609120432402423 — survived: Run resolves t0 through session.Manifest.T0 before any other work and refuses an absent or negative anchor on every path including explicit -offset, with three table cases asserting the session is left byte-identical. + evidence: internal/cast/cast.go:86 + evidence: internal/cast/cast_test.go:307 +- cond-2609120432402550 — survived: Both streams are placed against the one manifest t0 with no per-source clock, which the interleaving test demonstrates end to end; nothing in the delivery contradicts the one-wall-clock-session assumption, and nothing tries to reconcile two unrelated recordings. + evidence: internal/cast/cast_test.go:940 + evidence: internal/cast/coalesce.go:189 +- cond-2609120432404128 — survived: Carriage returns are kept and are explicitly not a record boundary, so a progress line's redraw frames are preserved as they arrived rather than reconstructed into the final rendering — visible verbatim in the golden record. + evidence: internal/cast/coalesce.go:107 + evidence: internal/cast/testdata/golden.interactions.jsonl:4 + evidence: docs/reference/session-directory.md:67 +- cond-2609120432407170 — survived: A re-import drops every line whose kind is the reserved terminal_output and keeps every other line byte-for-byte, so a second, different cast's records replace the first's — asserted by the replaced count and the byte-identical re-import, and stated in both the reference and the how-to. + evidence: internal/cast/write.go:129 + evidence: internal/cli/cli_test.go:841 + evidence: docs/how-to/record-a-terminal-session.md:117 +- cond-2609120432408324 — survived: The how-to tells the operator to record output only and names both opt-in input flags with the suppressed-echo hazard, while the importer drops i events unconditionally and prints the count so an operator learns their recorder captured keystrokes. + evidence: docs/how-to/record-a-terminal-session.md:65 + evidence: internal/cast/cast_test.go:749 + evidence: docs/reference/session-directory.md:107 +- cond-2609120432400428 — untested: Whether an operator actually reviews or redacts the timeline before analyze is an assumption about human behaviour that the delivery neither exercises nor contradicts; it restates the instruction in docs/how-to/record-a-terminal-session.md:85 and docs/explanation/privacy.md, and adds no code-level check, so nothing in the delivered reality tests it. +## Grounds + +- pursued: an operator-recorded asciicast (v2 or v3), anchored by its header timestamp to the session t0 and coalesced per displayed line, gives merge and report a terminal interaction stream good enough to sit a spoken stumble beside the command and output that caused it, with no change to record; what would show it wrong is real sessions where the whole-second header anchor or the line coalescing leaves output misaligned with speech beyond the join window diff --git a/.abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md b/.abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md new file mode 100644 index 0000000..2c40313 --- /dev/null +++ b/.abcd/development/specs/closed/spc-2609120417486971-terminal-cast-import.md @@ -0,0 +1,1195 @@ +--- +id: spc-2609120417486971 +slug: terminal-cast-import +intent: itd-11 +origin: researcher-authored +production_mode: hand-written +--- +# terminal-cast-import + +## Summary + +`testimony import` is the terminal path's hand-off step: one command that takes a +session directory and an asciicast file the operator recorded themselves, and +normalises the cast's **output** events into that session's ordinary +`interactions.jsonl` on the shared clock. It is `transcribe -audio`'s peer in +every respect that matters — an artefact the CLI never produced, an anchor read +out of that artefact's own metadata, an explicit `-offset` that always wins, a +mandatory printed provenance line, an idempotent re-run, and an all-or-nothing +write — and it is nothing like `record`: no flag on `record`, no pty, no +subprocess, no change to how a session ends. + +Both asciicast formats are accepted and distinguished by the header's `version` +field: v2 event times are absolute seconds since recording start, v3 event times +are intervals since the previous event, reconstructed by a running sum. Every +other version is refused by name. Output events are coalesced into one record per +line the terminal displayed — cut at a newline, at a 250 ms inter-event gap, at a +one-second span cap, and at the JSONL line limit, whichever comes first — so a +command echoed back one keystroke at a time becomes one record rather than +dozens, and a single oversized output event becomes several records with every +rune preserved. Input (`i`) events are dropped and counted, never normalised. + +Downstream is untouched: the records carry only the fields +`docs/reference/session-directory.md` already documents for an interaction, so +`merge` reads them through `checkedInteractions`/`BuildEntries` with no change, +`report` renders them through `eventLine` with no change, and `analyze` and +`review` see one more `event`-source entry each. The timeline schema learns no new +source type. The raw `.cast` is kept in the session as `terminal.cast`, the +archival counterpart of `events.rrweb.jsonl`. + +New package `internal/cast` — one exported entry point, `cast.Run`, plus the +`cast.OutputKind` constant the reserved kind is named by — one new `session` +file-name constant (`TerminalCastFile`), and one new `cli` verb. Standard library +only; no new dependency, no network call, no subprocess. + +## Design + +### Command surface + +``` +testimony import -session DIR [-cast FILE] [-offset SECONDS] +``` + +| Flag | Default | Meaning | +|---|---|---| +| `-session` | *(required)* | session directory | +| `-cast` | *(optional)* | asciicast file to import; omit to re-import the session's own `terminal.cast` | +| `-offset` | derived | cast→session clock offset in seconds | + +The verb is `import`: a peer command, not a flag on `transcribe` (which is +speech-only and would have to grow a second, mutually-exclusive input mode) and +not a flag on `record` (the fence this intent exists to hold). The name says what +the step does — it imports an artefact the operator already holds — and says +nothing about speech. + +Flag names follow `transcribe` exactly. `-session` is the same required flag every +pipeline command takes. `-cast` is `-audio`'s twin: the operator-named external +artefact, optional in the same way and for the same reason — `transcribe` reuses +the session's own `audio.wav` when `-audio` is omitted, so `import` reuses the +session's own `terminal.cast`, which makes "re-run with a corrected `-offset`" a +one-liner (`testimony import -session DIR -offset -12.4`) instead of an +instruction to keep the original file findable. A `-cast` that resolves to the +session's own `terminal.cast` (`os.SameFile`, the `transcribe.sameFile` precedent) +is treated as the omitted case, so the archival copy is never copied onto itself. +`-offset` keeps `transcribe`'s meaning verbatim — seconds added to every +cast-clock time to place it on the session clock — and reuses +`transcribe.CheckOffset` for validation, so the rule that an offset must be finite +and within ±10⁹ seconds keeps one home. + +`-cast` carries **no** extension check, unlike `-audio`'s closed `.m4a/.mov/.wav` +set. That set exists because ffmpeg accepts only those containers; here the +header's `version` field is the authority on whether a file is importable, and a +name rule would refuse a legitimately-named cast (`asciinema rec` writes whatever +name the operator gives, and a redirected recording may carry none). + +`-session` obeys the shared inference rule itd-12 established for every pipeline +command: it is resolved through `cli.resolveSession(fs, dir)` — the explicit +flag wins; otherwise the current directory when it holds a Testimony session +`manifest.json`; a usage error otherwise — and the resolution is the **last** of +the command's invocation checks, so a run refused for any other flag never first +announces a session it did not use. `import` is a pipeline command like the +rest, and two rules for one flag would be the defect; the synopsis is therefore +`[-session DIR]`, as on its five siblings. + +CLI-layer refusals, all exit 2 (the `usageErr` path), all before any work starts: +`-session` missing with no session manifest in the current directory; +`-session` or `-cast` explicitly empty (the +`transcribe: -audio must not be empty` precedent — an unset shell variable spliced +into the flag, which would otherwise silently select the in-place branch); a +stray positional (`rejectArgs`); an `-offset` that fails +`transcribe.CheckOffset`. Everything else is a runtime failure at exit 1. + +The exact strings, which `internal/cli`'s tables pin: + +``` +import: -session is required (no -session flag, and the current directory holds no regular manifest.json file) +import: -session must not be empty +import: -cast must not be empty +import: unexpected argument "junk" (the command takes no positional arguments) +import: -offset must be a finite number of seconds, got NaN +import: -offset 1e+10 exceeds 1e+09 seconds in magnitude; no recording→session offset is that large +``` + +Reusing `transcribe.CheckOffset` generalises its magnitude message from +"no audio→session offset is that large" to "no recording→session offset", so the +one home for the rule reads correctly for both callers rather than gaining a +second copy of the bound. + +The usage text gains one block, placed after `transcribe` (its analogue) and +before `merge` (its consumer), with the continuation line `transcribe`'s own +block already uses: + +``` + testimony import [-session DIR] [-cast FILE] import an asciinema recording's output into interactions.jsonl (reuses the session's terminal.cast when -cast is omitted) + [-offset SECONDS] +``` + +and the footer that lists the inferring commands gains `import`. + +### Package layout + +`internal/cast`, entered through one exported function shaped like +`transcribe.Run`: + +```go +// Package cast reads an asciinema recording (asciicast v2 or v3) and +// normalises its terminal-output events into a session's interaction stream. +// The package is named for the artefact it parses rather than for the verb it +// implements, because `import` is a Go keyword. +package cast + +// OutputKind is the interaction kind every record this package writes carries. +// It is the importer's own marker: a re-run replaces exactly the records +// carrying it and leaves every other interaction untouched. +const OutputKind = "terminal_output" + +type Options struct { + SessionDir string // session directory + Cast string // asciicast file; "" reuses the session's terminal.cast + Offset float64 // cast→session clock offset in seconds + OffsetSet bool // true when -offset was given explicitly + Log io.Writer // status sink; defaults to os.Stderr when nil +} + +// Run performs the import and returns the number of records written to +// interactions.jsonl. +func Run(opts Options) (int, error) +``` + +Unexported internals, each independently testable and all but two of them pure: + +- `scanCast(r io.Reader, name string, onHeader func(castHeader) error, fn func(castEvent) error) (castHeader, error)` + — the streaming reader: header, then one callback per event line. `onHeader` + runs after the header is decoded and **before the first event**, which is what + lets the caller resolve the offset — the thing the header's timestamp anchors, + and the thing every record's `t` needs — without buffering the events or + reading the file twice. A header-first callback is the whole reason the + signature carries two functions rather than one. +- `castHeader{Version *int; Timestamp *int64}` — a pointer version and timestamp + so an absent field stays distinguishable from a genuine `0`, the + `timeline.rawInteraction.T` / `transcribe.offsetSidecar.OffsetSeconds` + precedent. Every other header field (`width`, `height`, `term`, `env`, `theme`, + `command`, `title`, `duration`, `idle_time_limit`) is ignored: the importer + needs the version and the anchor and nothing else, and unknown fields must not + be rejected, since both formats are extensible. +- `castEvent{Line int; US int64; Code, Data string}` — `US` is always **absolute + microseconds since recording start**, so the v2/v3 difference is resolved + inside `scanCast` and nothing downstream of it knows which format was read. + This is the single seam behind the "identical timeline from either format" + criterion, and the grain is what makes that criterion literally true rather + than approximately so (see *Reading the cast*). +- `resolveOffset(name string, opts Options, man session.Manifest, hdr castHeader) (offsetMS int64, provenance string, err error)` + — pure given its arguments; the `transcribe.resolveOffset` twin. `name` is the + cast's display name, which the implausible-timestamp refusal below names. +- `coalescer` — the record builder (`add(castEvent) error`, `flush() error`), + pure over its inputs, holding one pending record's runes at a time. +- `rewriteInteractions(dir, castName string, t0 int64, records []timeline.Interaction) (replaced int, err error)` + — the all-or-nothing write. `castName` is named by the size refusal, and `t0` + anchors the merged-timeline pre-flight. +- `stageCast(dir, name string, src *os.File) (tmpPath string, err error)` and + `commitCast(tmpPath string) error` — the two-phase archival copy, staged from + the descriptor the scan already holds. + +`Run`'s order of operations is `transcribe.Run`'s, for `transcribe.Run`'s stated +reason — **every refusal fires before anything on disk changes**, so a refused +import leaves the session byte-for-byte as it found it: + +1. Load `manifest.json`; resolve `t0` through `session.Manifest.T0`. +2. Resolve the cast input: `-cast` (stat-and-regular-file guard) or the session's + `terminal.cast` (no-follow guard). Refuse if neither is usable. +3. Scan the cast, building records as events arrive. Refuse on any malformed + line, unsupported version, or implausible time, naming the line. +4. Validate the record set: each record through `timeline.CheckInteraction`, and + the assembled `interactions.jsonl` (plus the merged `timeline.jsonl` it + implies) against `session.MaxJSONLBytes`. Refuse the run, not the record. + Each wrapped timeline entry is measured against `session.MaxJSONLLine` as its + record is **closed** (see *Splitting an oversized event*) rather than in a + separate pass: the check exists to catch a wrong encoded-length table, so it + belongs beside the budget it verifies. Both fire before step 5, so the + nothing-written guarantee is the same either way. +5. Stage the archival cast copy into a temp file beside `terminal.cast`. +6. Rewrite `interactions.jsonl` atomically. +7. Rename the staged copy over `terminal.cast`. + +### Reading the cast (both formats) + +`scanCast` reads with a `bufio.Scanner` bounded exactly as `session.ReadJSONL` +bounds a JSONL file, since a cast is the same kind of input — line-oriented, +operator-supplied, and possibly received rather than recorded here: + +- per line: `sc.Buffer(make([]byte, 0, 64*1024), maxCastLine)` with + `maxCastLine = 16 << 20` (16 MiB). Rationale: one `o` event can legitimately be + a single large write (a `cat` of a file), and JSON-escaping inflates it — an + ESC byte encodes to six bytes (verified against `encoding/json`) — so the + 4 MiB `session.MaxJSONLLine` would refuse casts whose *records* this importer + can split and persist perfectly well. 16 MiB matches + `session.MaxJSONLBytes`'s scale, and a line at the bound still splits into + records that fit. `bufio.ErrTooLong` is reported as + `%s:%d: line exceeds %d bytes; refusing to read`, with the line number the + scanner reached. +- per file: `maxCastBytes = 64 << 20` (64 MiB), counted as lines are scanned + (including the newline, and counted before any blank-line skip, exactly as + `ReadJSONL` counts). A cast large enough to matter cannot become records that + fit `interactions.jsonl`'s own 16 MiB cap anyway, so the file cap exists only + to bound the scan itself: `%s: exceeds %d bytes across %d lines; refusing to + read`. + +Line 1 is the header, decoded into `castHeader`: + +- not a JSON object (including an event line, so a cast with no header is caught + here) → `%s:1: not an asciicast header (expected a JSON object with a "version" field)`; +- no `version` → `%s:1: asciicast header carries no "version" field`; +- `version` other than 2 or 3 → `%s: asciicast version %d is not supported (import reads version 2 and 3)`. Naming the file + and the version found is the criterion's exact wording, and because this fires + in step 3 nothing has been written. + +Subsequent lines are events, `[time, code, data]`, decoded into +`[]json.RawMessage` and then element-wise (`float64`, `string`, `string`). Blank +lines are skipped, matching every other line reader in the repository. Any of +these is a refusal naming the line, with nothing written: + +- not a 3-element JSON array, or an element of the wrong type → + `%s:%d: malformed asciicast event (expected [time, code, data])`. An + out-of-range numeric literal (`1e400`) lands here too: `encoding/json` refuses + it into `float64` rather than yielding `+Inf`. +- v2, time decreasing → + `%s:%d: event time %g precedes the previous event's %g; asciicast v2 times must not decrease`. +- v3, negative interval → + `%s:%d: event interval %g is negative; asciicast v3 intervals must not be negative`. +- either format, absolute time beyond `maxCastSeconds = 1e9` → + `%s:%d: event time %gs exceeds %g seconds; that is no recording clock`. The + bound mirrors `timeline.maxUtteranceSeconds` and `transcribe.maxOffsetSeconds`, + so a time this importer accepts is a time `merge` accepts. + +The version difference is confined to one accumulator, and that accumulator runs +on an **exact integer grain**: microseconds (`castTimeGrain = 1e6`). Each event's +time is rounded onto the grain as it is parsed — `us := int64(math.Round(t*1e6))` +— and from there v2 sets the clock to it (refusing a decrease) while v3 adds it +(refusing a negative). The rounding from the grain to the millisecond a record +records happens once more, downstream, in one place (`microsToMillis`). + +The grain is what makes "identical records from either format" true rather than +nearly true. A `float64`-seconds clock does not give it: v2 rounds a **stated** +absolute time while v3 rounds a **running sum**, and at a half-millisecond tie +the two land on different milliseconds. Seven 0.0015 s intervals sum to +0.010499999999999999 and round to 10 ms, where v2's stated 0.0105 rounds to 11 — +and one millisecond is enough to put a following event on either side of the +250 ms coalescing gap, so the same recording becomes one record read as v2 and +two read as v3. On the grain both formats reach 10500 µs and the question does +not arise. A microsecond is exact for every time either format writes (both cap a +time at six decimal places), and 1e9 seconds on the grain is 1e15, three orders +of magnitude inside `int64`. + +The bound is applied **before** the conversion as well as after it, because +`int64(math.Round(x))` has no defined answer for a float past the integer range: +a 1e300 time or interval is refused where it is read rather than converted. + +Event codes: only `o` (output) becomes records. `i` (input), `r` (resize), `m` +(marker), `x` (exit), and any code this importer does not recognise are dropped +and **counted by code**, with the counts printed (see *Printed output*) — the +intent's "evidence is not silently dropped" applied to whole event classes, not +only to oversized ones. Dropping `i` is a privacy requirement, not a +simplification (below). Dropping `r`/`m`/`x` is scope: a resize and an exit +status are terminal-session facts with no place in an interaction stream whose +schema is `kind`/`selector`/`text`/`value`/`route`, and a marker is a navigation +aid for a player, not an observed action. An unrecognised code is dropped rather +than refused so a future asciicast revision that adds one does not turn every +cast it writes into an unimportable file; the printed count is what keeps that +tolerance honest. + +The tally is bounded, because a code is a single character in both formats but +nothing in the file format enforces that: a crafted cast can carry a different +code on every line, and an unbounded map would grow — along with the line it +prints — in step with the file. At most `maxDropCodes = 16` distinct codes are +named individually; everything past that is counted together and reported as +`%d under further codes`. Each named code is passed through `session.SafeText` +and clipped to 8 runes, the same rule the header literals above obey, and the +summary is ordered **by code** rather than by its formatted string, so a +three-digit count cannot sort a code above one that precedes it. + +Two details of the bound are load-bearing: + +- **`i` is exempt from it.** The input count is the one tally entry that is a + privacy disclosure rather than a scoping note — it is how an operator learns + their recorder captured keystrokes — and a cast carrying sixteen junk codes + before its first `i` would otherwise swallow that line into the anonymous + overflow and never print it. Exempting one fixed key cannot unbound the tally: + the map holds at most `maxDropCodes+1` entries. +- **The overflow is a counter, not a map entry under a reserved key.** `""` is a + legitimate event code a cast can carry, so a sentinel key would report a real + empty-coded event as overflow and overflow as a real event. + +Rune fidelity: `encoding/json` replaces invalid UTF-8 in a JSON string with +U+FFFD when it decodes, so what reaches `castEvent.Data` is already a valid Go +string. The preservation guarantee this spec makes is therefore stated at rune +granularity over the decoded string, and the byte-exact record is the archived +`terminal.cast` — the one place a byte-for-byte claim can honestly be made. See +the acceptance-criteria mapping, where this is called out as a refinement of the +criterion's wording. + +### Clock anchoring + +An interaction's `t` is epoch milliseconds, so the record's time is computed in +integers: + +``` +t = t0 + offsetMS + round(eventSeconds * 1000) +``` + +`offsetMS` is the cast→session offset in milliseconds, resolved by +`resolveOffset` — which reads `t0` through `session.Manifest.T0` itself, so it +stays a pure function of its arguments and its table test can drive the +usable/absent-anchor axis directly — in the order `transcribe.resolveOffset` +uses: + +| Condition | `offsetMS` | Printed provenance | +|---|---|---| +| `-offset` given | `round(opts.Offset * 1000)` | `from -offset flag` | +| header `timestamp` present and positive | `*hdr.Timestamp*1000 − t0` | `derived: cast header timestamp − manifest t0 (whole seconds, ±1s)` | +| header carries no `timestamp` | `0` | `default 0: cast header carries no timestamp` | + +The derived case is exact integer arithmetic — no float enters it — because both +operands are integers: the header's `timestamp` is whole Unix seconds in both +formats, and `t0_epoch_ms` is whole milliseconds. Only an explicit `-offset` +introduces a rounding step, to the nearest millisecond. + +Two refusals, both the `transcribe` twin: + +- `t0` is obtained through `session.Manifest.T0`, never the raw field, so an + absent (`0`) or negative anchor refuses the run: + `anchoring the terminal cast: %w`. Unlike `transcribe`, `import` needs `t0` + even on the explicit-`-offset` path, because the artefact it writes is + epoch-millisecond-timed and `-offset` is defined relative to the session clock. + This is not a gap: `merge` already refuses a session whose `interactions.jsonl` + is non-empty and whose manifest carries no usable `t0`, so importing into such + a session would persist records no command could ever read back — the + write-before-read invariant `session.WriteJSONL` and `SaveManifest` both + enforce. +- a present-but-non-positive header `timestamp` is refused rather than defaulted: + `%s: header timestamp %d is not a recording instant; pass -offset SECONDS to anchor the cast explicitly`. + `Manifest.T0`'s reasoning applies unchanged — no recorder produces a capture + instant at or before 1 January 1970 — and `transcribe` likewise refuses a + *present but implausible* creation time (its derived-offset magnitude bound) + while defaulting only when the metadata is *absent*. +- a derived offset beyond `1e9` seconds in magnitude is refused in + `transcribe.resolveOffset`'s words: + `derived cast offset %+.2fs exceeds %g in magnitude; the cast's header timestamp or the manifest t0 is implausible — pass -offset SECONDS to state it explicitly`. + +**Absent-timestamp policy: default 0, mirroring `transcribe` exactly.** When +`transcribe` cannot read an external recording's `creation_time` it prints +`default 0: audio creation time unavailable` and continues; `import` prints +`default 0: cast header carries no timestamp` and continues. Refusing instead +would make the terminal path stricter than the audio path for the same class of +missing metadata, and the remedy is identical in both: the spoken "session start" +marker as the cross-check, then `-offset` to correct. The provenance line is +printed on **every** run, so "offset 0 because the header said nothing" is never +a silent assumption. + +**No offset sidecar.** `transcribe` persists `audio.offset.json` because +converting an external recording into `audio.wav` destroys the `creation_time` it +derived from, leaving the session unable to tell external audio from +record-origin audio. Nothing analogous happens here: the archived +`terminal.cast` keeps its own header, so a re-import re-derives the identical +offset from the identical bytes. An explicit `-offset` must be repeated on a +re-import, and `-cast` being optional is what makes repeating it cheap. + +**The quantisation caveat is in the printed line, not only in the docs.** The +derived provenance string carries `(whole seconds, ±1s)` because the header field +is an integer in both formats: the reconstructed clock can sit up to a second +adrift of `t0`'s millisecond precision, which is material against `report`'s +2.5-second default join window. The operator reads the bound at the moment they +read the offset. + +### From output events to interaction records + +The problem the intent names: a shell echoes a typed command back roughly one +keystroke at a time, so the raw `o` stream around a command is a run of +one-character events. One record per raw event fragments a single typed command +across dozens of near-empty records. The fix is a single accumulator with four +boundaries. + +`coalescer.add(ev)` appends `ev.Data`'s runes to the pending record and closes +that record at the first of: + +1. **a newline.** The `\n` is included in the record, and the next rune starts a + new record. This is the primary boundary and the one that makes the stream + legible: one record per line the terminal displayed. It also solves the + keystroke-echo problem outright — the echo of a typed command carries no + newline until Enter, so the whole command arrives as one record. +2. **an inter-event gap of `coalesceGap = 250 ms` or more** to the next `o` + event. This closes a record whose output never ends in a newline: a bare + prompt (`$ `), a `read` prompt, a progress line. 250 ms sits above a human's + typical inter-keystroke interval (~100–200 ms), so a typed command still + coalesces; three orders of magnitude above the gaps inside a program's output + burst, so it never merges across a genuine pause; and an order of magnitude + below `report`'s 2.5 s join window, so coalescing alone can never pull content + across a window boundary. It is a named constant, not a flag: an operator has + no way to know what value to pass, and the value interacts with `-window`, + which is already a flag on the command that needs it. +3. **a span cap of `maxCoalesceSpan = 1 s`** measured from the record's first + event. A record states one time — its first event's — so unbounded coalescing + would attribute minutes of output to a single early instant. One second keeps + the attribution error well inside `report`'s 2.5 s default window, so a record + still joins to the utterance spoken over it. +4. **the JSONL line budget** (below). + +`coalescer.flush()` closes the pending record at end of stream. A record's `t` is +always the time of the event that contributed its **first** rune — never +fabricated, never averaged — so a record cut by any of the four boundaries is +honestly timed, and continuation records after a split carry the time of the +event whose data they open with. + +A record whose text renders empty — `strings.TrimSpace(session.SafeText(text)) == ""`, +the `transcribe.mapSegments` test, so a blank line or a lone carriage return does +not become a timeline bullet showing only the word `terminal_output` — is +dropped and counted, and the count is printed. The archived `terminal.cast` holds +those bytes verbatim, which is what makes the drop a rendering decision rather +than a loss of evidence. + +Carriage returns inside a record are kept, and `\r` is **not** a boundary: a +progress bar emits many `\r`-separated frames for one displayed line, and one +record per frame would flood the stream. A record holding several frames renders +as their concatenation (`report`'s `SafeText` strips the `\r` itself), which is +the acknowledged cost of the intent's "TUI redraw handling beyond preserving the +raw cast is out of scope". + +**Splitting an oversized event.** The binding limit is not the record's own line +length but the size of the **timeline entry `merge` wraps it in** — the +`transcribe.checkEntriesFit` and `demo.tooLongOnceWrapped` invariant. The budget +is computed once **per record**, at the moment the record opens, from that +record's own time: + +``` +probe := timeline.Interaction{T: t0 + offsetMS + castMS, Kind: OutputKind, Text: "x"} +envelope := session.EncodedLen(timeline.EventEntry(probe, t0)) - 1 +budget := session.MaxJSONLLine - envelope - castEntryIDMargin +``` + +Two details of that expression are load-bearing. The probe carries a one-rune +text (whose single byte the `- 1` removes) because `timeline.BuildEntries` omits +an empty `text` from the payload entirely, so an envelope measured without one +under-counts by the whole `,"text":""` scaffolding. And the budget is per record +rather than per run because the entry's session-relative `t` varies in encoded +length across a session — `0` is one byte, `-1000000000.123` is fifteen, and +float64 division of an integer millisecond count produces the decimal form +either way — so a single run-wide envelope would have to guess at the widest +case. A record's `t` is known the instant the record opens, which makes +measuring it exact and costs one small struct encode per record. + +`castEntryIDMargin = 32` is `demo.eventIDGrowthMargin`'s twin, for the same +reason: `timeline.EventEntry` stamps the placeholder id `ev-001`, while the real +ordinal depends on the record's position among every interaction in the session, +so the measured entry is a lower bound and the margin covers the ordinal's +growth. (`demo`'s constant is unexported, so the value is restated here with the +citation rather than reached across the package boundary.) + +The accumulator tracks the pending text's **JSON-encoded** length, not its raw +length, because escaping is what actually consumes the budget: an ESC byte costs +six bytes encoded, and ANSI-coloured output is dense in them. Per rune +(verified against `encoding/json` with `SetEscapeHTML(false)`, which is how +`session.WriteJSONL` encodes): + +| rune | encoded bytes | +|---|---| +| `"`, `\` | 2 | +| `\n`, `\r`, `\t` | 2 | +| any other C0 control (ESC 0x1b included) | 6 | +| U+2028, U+2029 | 6 | +| everything else, DEL and `<`/`>`/`&` included | `utf8.RuneLen(r)` | + +The table is an **upper bound**, not an equality: `encoding/json` has written +backspace and form feed as a six-byte u-escape in some Go versions and as a +two-byte short escape in others, and the table takes the larger for both. That +is the safe direction — an over-count spends a few of a 4 MiB budget's bytes, +while an under-count over-fills the budget and costs a false refusal on the +measured check below. The property test asserts the bound (never under, equal +everywhere the two agree) rather than exact equality. + +A rune is appended only if it keeps the running total within `budget`; otherwise +the record closes and the rune opens the next one. A rune is split off only from +a **non-empty** pending record, so a rune that could not fit even an empty +record's budget is appended anyway — and caught by the measured check — rather +than closing and opening records for ever on one it can never hold. Splitting therefore never +occurs inside a rune, and concatenating a split's records reproduces the event's +decoded data exactly. Because the table is an assumption about another package's +encoder, every finished record is additionally measured for real — +`session.EncodedLen(timeline.EventEntry(rec, t0)) + castEntryIDMargin <= session.MaxJSONLLine` +— and a failure is a refusal, not a truncation: +`record %d encodes to a %d-byte timeline entry, over the %d-byte JSONL line limit; this is an importer bug — please report it with the cast that triggered it`. +The check is unreachable if the table is right, and it is the difference between a +wrong table costing a refused run and a wrong table costing a session no command +can read back. + +No continuation marker is written on a split record. A field outside the +documented interaction schema would be dropped by `timeline.rawInteraction` at +merge and so could never reach the report anyway, and overloading `value` or +`route` to carry one would pollute a documented schema for a cosmetic gain. +Consecutive records with identical or adjacent `t` values, rendered as +consecutive bullets, are what a split looks like — and the whole-line boundary +means splits are rare in practice. + +### The interaction record shape + +Exactly the fields `docs/reference/session-directory.md` already documents, and +no others: + +```json +{"t":1784300424100,"kind":"terminal_output","text":"$ ls -la\n"} +``` + +| Field | Value | +|---|---| +| `t` | epoch milliseconds, computed as above | +| `kind` | `"terminal_output"` — the constant `cast.OutputKind` | +| `text` | the record's accumulated output, runes exactly as the cast's JSON decoded them | +| `selector`, `value`, `route` | never set (omitted by `omitempty`) | + +**`kind` is the source marker.** A re-run identifies its own prior records by +`kind == cast.OutputKind` and nothing else. This is a deliberate deviation from +adding a `source: "cast"` field: an extra field would be dropped by every reader +in the repository (`timeline.rawInteraction` decodes six fields and ignores the +rest), so it would be dead weight in an exchanged artefact while still needing a +row in a schema table whose readers ignore it — whereas `kind` is already the +documented discriminator for *what happened*, is already the field `report` +renders first, and already survives into `timeline.jsonl` where it tells a reader +that this entry came from a terminal. The cost is a collision hazard: a +third-party instrumented app posting `kind:"terminal_output"` to +`POST /api/interactions` would have its record replaced by a later import. It is +closed by documentation — `docs/reference/session-directory.md` names +`terminal_output` a reserved kind — rather than by a new refusal in `demo`'s +endpoint, which would change an accept set for a hazard with no realistic +attacker payoff (the "attack" substitutes genuine terminal evidence for a forged +record) and is outside this intent's fence. + +`report` reads `kind` (`mdOrDash`) and `text` (`mdInline`, in straight quotes) +from an event payload, and nothing else this record carries, so a terminal record +renders as `- [01:23] terminal_output "$ ls -la"` through `eventLine` +**unchanged**. Two consequences of that existing sink are worth stating plainly +rather than discovering later: + +- `session.SafeText` strips `\n`, so a record's trailing newline is invisible in + `report.md` — which is why the record is one displayed line: were records + multi-line, the report would run their lines together. +- `SafeText` strips ESC but not the rest of a CSI sequence, so coloured output + renders with residue (`[0;34m`). See *ANSI escape sequences*. + +### Writing: all-or-nothing, idempotent re-run + +`rewriteInteractions` replaces `interactions.jsonl` whole: + +1. Read the existing file, if any, through `session.OpenFileNoFollowRead` (a + FIFO or symlink planted at the name in a received session is refused, not + followed or blocked on), bounded by `session.MaxJSONLLine` per line and + `session.MaxJSONLBytes` for the file — the same pair `ReadJSONL` enforces. A + missing file is zero lines, not an error. The read is whole rather than + scanned, and the split is on `\n` by hand: `bufio.Scanner` strips a trailing + carriage return along with the newline, which would silently rewrite a CRLF + line the step below promises to keep byte-for-byte. The replacement is + assembled in memory anyway for the atomic write, so reading the 16 MiB-capped + original whole costs nothing extra. +2. Classify each line by decoding a probe struct (`{Kind string}` only): a line + whose `kind` equals `cast.OutputKind` is a record from an earlier import and + is **dropped**; every other line — a `demo` click, an `input`, a line this + importer cannot decode at all, a blank line — is **kept byte-for-byte**. + Lines are never re-encoded, which is why this write cannot go through + `session.WriteJSONL[T]` (it encodes values, and would silently rewrite a + foreign record's field order, number formatting, or an unknown field it cannot + model). +3. Assemble kept lines, in file order, followed by the new records encoded with + `session.EncodedLen`'s encoder settings (`SetEscapeHTML(false)`, so a + measured size and a written line cannot disagree). One byte is added and only + one: a final kept line carrying no terminating newline gets one, because + without it the first imported record would be appended onto that line and + neither would survive a read back. +4. Pre-flight the assembly with the checks `WriteJSONL` would have applied, since + step 2 is why they cannot be delegated: every line within + `session.MaxJSONLLine`, and the total within `session.MaxJSONLBytes` — + `importing %s would take %s past its %d-byte limit (%d bytes across %d lines); record shorter terminal sessions, or start a fresh session`. +5. Pre-flight the **merged timeline** the assembly implies, which `demo`'s + endpoint can only estimate and an offline importer can measure: the sum of + `session.EncodedLen(timeline.EventEntry(rec, t0))` over every decodable + interaction line plus, when `transcript.jsonl` is present, + `session.EncodedLen(timeline.SpeechEntry(u))` over every utterance, against + `session.MaxJSONLBytes`. Each event entry is charged its **id growth** — + `demo.idGrowth`'s arithmetic, restated as `castIDGrowth` — because + `timeline.EventEntry` sizes every entry with the placeholder id `ev-001` + while `merge` assigns `ev-%03d` by position, so past the thousandth + interaction the real id is longer than the measured one. Without the charge a + session sitting just under the cap with a few thousand interactions passes + this pre-flight and is then refused by `merge` — the exact "import succeeds, + merge can never read it back" state the pre-flight exists to prevent. The + ordinal is the position across the prior records and then the new ones, which + is the order they are written in and so the order `merge` numbers them in. + The message is — + `the imported records would take the merged %s past its %d-byte limit; record shorter terminal sessions, or start a fresh session`. + An interaction line that does not decode is not sized: `merge` will refuse + that session for its own, pre-existing reason, and `import` must not be + blamed for it. Like `demo`'s estimate, the guarantee is "as of import time" — + a `transcribe` run afterwards adds speech entries this pass could not see. +6. Write with `session.WriteFileAtomicNoFollow` (temp file plus rename, symlink + refused up front, an existing file's mode preserved exactly). A failure at any + point leaves the prior `interactions.jsonl` untouched. + +Consequences, all intended: + +- **Idempotent.** Importing the same cast twice yields a byte-identical + `interactions.jsonl`: the second run drops exactly what the first wrote and + writes exactly the same records back. +- **Mixed sessions are safe.** A `record -demo` session's clicks and inputs are + preserved byte-for-byte, in their original order, ahead of the terminal + records. The file need not be time-sorted — `merge` sorts, and `report` sorts + again — so appending is correct without reordering anything. +- **One terminal recording per session.** Importing a *different* cast into a + session replaces the first cast's records (and its `terminal.cast`), because + both are identified by the same reserved kind. Two concurrent terminals in one + session are out of scope; the reference says so, and the remedy is one terminal + per session. +- **Zero records is a refusal, not an erasure.** A cast holding no importable + output would otherwise silently delete a prior import's records — the hazard + `transcribe`'s zero-utterance guard and `merge`'s zero-entry guard both refuse. + The two ways to reach it are named apart, because they call for different + remedies: a cast with no output events at all is the wrong file (or one + recorded with input capture only) — + `%s holds no output events; refusing to rewrite %s` — while a cast whose output + all rendered empty is a real recording of a terminal that displayed nothing + legible — + `%s holds no importable output (%d record(s) rendered empty); refusing to rewrite %s`. + +### The archival copy + +The raw cast is kept in the session directory as `terminal.cast`, a new +`session.TerminalCastFile` constant beside `AudioFile`, `ScreenFile`, and +`RawEventsFile`, and documented in the `session` package doc comment and the +session-directory reference in the same pass (the schema-move invariant spc-1 +states). It is the archival counterpart of `events.rrweb.jsonl`: nothing +downstream reads it, and it exists so the byte-exact record survives and so a +re-import needs no external file. + +`import` **copies** the operator's file rather than requiring it to be in place +already. This mirrors `transcribe -audio`, which normalises an external recording +into the session as `audio.wav`: the hand-off pattern's whole point is that the +operator hands over a file and the session becomes self-contained. Requiring the +operator to place the file themselves would add a step whose only failure mode is +a wrong name. + +The copy is two-phase, so the ordering question ("which artefact is left behind +if the other write fails?") has a defensible answer rather than a rollback: + +- `stageCast(dir, name string, src *os.File)` streams the source into + `.terminal.cast.tmp-*` beside the target with `os.CreateTemp` and `io.Copy` + over an `io.LimitReader(maxCastBytes+1)`. `src` is the descriptor the scan + already read, **rewound** rather than re-opened by path: the archive must hold + the bytes the records were derived from, and between two opens of a path the + operator's file can be replaced or rewritten, which would leave the session + asserting a byte-for-byte archive of something else. The copy is refused if it + reaches the limit reader's extra byte — the bound is read one byte past so a + file that grew is refused rather than archived truncated, which is the one + place this design makes a byte-for-byte claim. Otherwise the + `transcribe.atomicConvert` shape, including + the prior-mode preservation rule (an existing `terminal.cast`'s own mode is reapplied; a new file takes + `0o644 &^ umask`, so a privacy-conscious operator's umask is honoured). + A non-regular or symlinked `terminal.cast` is refused before the temp is + created, `transcribe.checkPlainOutput`'s rule. +- `rewriteInteractions` runs next. +- `commitCast` renames the temp into place last, and the temp is removed by a + `defer` on every failure path. + +So a failure anywhere before the final rename leaves the session exactly as it +was, and the only residual window is a same-directory rename failing after the +records landed — leaving records with no archival copy, which is the less +misleading of the two possible residual states: the records are the evidence, the cast +is the archive. With `-cast` omitted (the in-place re-import), there is no copy +phase at all. + +### Input events and the privacy boundary + +`i` events are dropped in `scanCast` and never reach a record, under any flag. +The recommended invocation (`asciinema rec session.cast`, on either CLI line) +does not capture input at all; input capture is opt-in (`--stdin` on 2.x, +`--capture-input`/`-I` on 3.x), and the how-to tells the operator to leave it off, +because a password typed at a suppressed-echo prompt is exactly the thing that +would land in the evidence. The importer's unconditional drop is the second +layer: a cast that *was* recorded with input capture — by an operator who did +not read the guidance, or handed over by someone else — still cannot put +keystrokes into `interactions.jsonl`. The printed count says how many were +dropped, so the operator learns their recorder captured input. + +Note the honest boundary: the dropped keystrokes remain in the archived +`terminal.cast`, a local-only file exactly like `audio.wav`. That belongs in the +privacy documentation, not in a silent deletion — the importer does not rewrite +the operator's evidence. + +### ANSI escape sequences + +Kept raw in `text`. The record is evidence, and stripping escape sequences would +mean the importer applying a lossy transform to the one copy every downstream +reader consumes — with a hand-written ANSI parser as the new surface, whose bugs +would corrupt evidence rather than merely litter it. + +`report` renders them through `eventLine` → `mdInline` → `session.SafeText`, +which strips the ESC byte (and every other C0 control) but leaves the printable +tail of a CSI sequence, so a coloured `ls` renders as +`- [01:23] terminal_output "[0;34mdocs[0m"`. That is deliberate hardening in the +render sink — a raw ANSI sequence must never reach `report.md` or the `review` +terminal — and this spec does not change it: there is no code fence, no raw +passthrough, and `report` stays untouched. The remedy is guidance, not code: the +how-to tells the operator to record with colour disabled (`NO_COLOR=1`, or +`TERM=dumb` for tools that ignore it), which also makes the records more +searchable and cheaper to hand to an analysis model. Stripping CSI/OSC sequences +at import is recorded here as a deliberate follow-up option, with the archived +`terminal.cast` as the safety net that would make it reversible. + +### Printed output + +Everything `import` says about its own run goes to `opts.Log`, which the CLI +wires to **stderr** — beside `resolveSession`'s inference line, and for the same +reason. `stdout` carries exactly one line, the summary the CLI itself prints, so +a script reads one line rather than parsing diagnostics out of a stream. (This +is a deliberate split from `transcribe`, which puts its own progress on stdout: +`transcribe` predates the inference line, and `import` has more to say.) + +In order, and all but the first printed only when they are non-zero: + +``` +offset: %+.2fs (%s) the provenance table above; every run +dropped %d input (i) event(s): keystrokes are never imported +dropped %d other event(s): 1 marker (m), 1 resize (r), 1 exit (x) +dropped %d record(s) that render empty +replaced %d terminal_output record(s) from an earlier import +``` + +then, from the CLI: + +``` +imported %d records → /interactions.jsonl +``` + +The offset line is `transcribe`'s format verbatim, and it prints on **every** +run, so "offset 0 because the header said nothing" is never a silent +assumption. Input is named on its own line because its drop is a privacy +guarantee rather than a scoping decision, and the count is how an operator +learns their recorder captured keystrokes — and the count is exempt from the +tally's bound, so a cast full of junk codes cannot suppress the disclosure. The +other codes are listed together, ordered by code so the line is deterministic +whatever order the tally iterates in, with any overflow last. + +The offset and drop lines print before the zero-records refusal, so an operator +whose cast held nothing importable still learns what was in it. + +### Downstream: merge, report, analyze unchanged + +Confirmed by reading, not assumed: + +- `timeline.rawInteraction` decodes `t`, `kind`, `selector`, `text`, `value`, + `route`. A terminal record sets `t`, `kind`, `text`, all three of which it + already handles. +- `timeline.checkInteraction` requires `t` non-nil, `t > 0`, `|rel| ≤ 1e9`, and a + non-empty `kind`. Every record satisfies all four by construction, and each one + is checked through the exported `timeline.CheckInteraction` before it is + written — the same guard `demo`'s endpoint applies, so `import` cannot persist a + record `merge` would refuse. +- `kind` is an **open** set: nothing in `timeline`, `report`, or `analyze` + validates it against an enum (`docs/reference/session-directory.md` says + `e.g. "click", "input"`). `terminal_output` therefore adds no schema field and + no new `src` value — `BuildEntries` emits `src:"event"`, which `CheckSrc`, + `report`'s bucket switch, and `analyze`'s indexer all already accept. +- `BuildEntries` assigns `ev-%03d` by position; more events widen the ordinal + (`ev-1234`) and stay unique, which `merge`'s duplicate-id scan confirms. +- `report`'s join, `end()`, `clock()`, and `eventLine` need nothing new; negative + session-relative times already render with a leading `-`. +- `analyze` indexes event ids and emits the timeline inline; a terminal record is + citable evidence like any other event. + +**No change to `merge`, `report`, `analyze`, or `review` is required.** One +non-behavioural consequence is worth naming: `analyze` emits the whole timeline +inline, so a verbose terminal session makes a much larger analysis request. That +is a documentation matter (keep terminal sessions short; review before +analysing), not a code change, and it is where the privacy warning lands too. + +### Failure modes and exit statuses + +| Situation | Status | Message shape | +|---|---|---| +| `-session` missing with no session manifest in the current directory, empty `-session`/`-cast`, stray positional, bad `-offset` | 2 | the existing `usageErr` shapes | +| no `manifest.json`, or no usable `t0` | 1 | `anchoring the terminal cast: %w` | +| neither `-cast` nor `terminal.cast` | 1 | `no %s in session %s and no -cast given: record a terminal with asciinema, then pass -cast FILE` | +| `-cast` names a missing or non-regular file | 1 | `cast file: %w` / `refusing to read %s: it is not a regular file` | +| unsupported version, malformed header or event, over-long line or file | 1 | the `%s:%d:`-prefixed shapes above | +| implausible header timestamp or derived offset | 1 | the `-offset SECONDS` guidance shapes above | +| no importable output | 1 | `%s holds no output events…`, or `%s holds no importable output (%d record(s) rendered empty)…` | +| a size limit reached | 1 | the limit shapes above | +| a write failure | 1 | wrapped `session`/`os` error | + +Every exit-1 path above fires before any file in the session changes, except a +write failure, which is left all-or-nothing by the atomic writers. + +## Acceptance-criteria mapping + +Each bullet is the intent's criterion, then the mechanism, then the test. + +1. **v2 cast + narrated session → one interleaved clock from the same `t0`.** + Mechanism: `scanCast` yields absolute seconds; `resolveOffset` derives + `header_ts*1000 − t0`; records carry epoch-ms `t`; `merge` rebases them + through the same `t0` it rebases nothing else with — there is no second clock + anywhere in the design, and no per-source offset in `timeline.jsonl`. + Test: `TestImportThenMergeInterleaves` — a `t.TempDir()` session with + `manifest.json`, a two-utterance `transcript.jsonl`, and a v2 fixture; run + `cast.Run` then `timeline.Merge`; assert the entry order and each entry's `t` + against golden values. +2. **The same session as v3 yields an identical timeline.** + Mechanism: the v2/v3 difference is confined to `scanCast`'s accumulator, and + `castEvent.T` is absolute in both cases, so every stage after it is + format-blind. + Test: `TestV2AndV3Agree` — a table over two fixture pairs, each describing one + recording in both formats (identical header timestamp, event codes, and + absolute times; v3's intervals are the differences): the ordinary recording, + and the half-millisecond tie pair above, which is the case a `float64` clock + splits apart. Assert the two runs produce byte-identical `interactions.jsonl`, + and the expected record count, so the pair cannot pass by both formats being + wrong the same way; `TestTiesCoalesceIntoOneRecord` states that count + independently. Two more pin the reconstruction itself: + `TestScanCastV3RunningSum` over a table of interval sequences (compared + exactly, on the grain, ties included) and `TestMicrosToMillis` over the one + rounding step from the grain to a record's millisecond. +3. **A version other than 2 or 3 is refused, naming file and version, writing + nothing.** + Mechanism: the header switch refuses before step 4, and every write is in + steps 5–7. + Test: `TestUnsupportedVersionRefuses` — `version` 1, 4, `"2"`, and absent; + assert the message contains the file name and the version found, and that a + hash of every file in the session directory is unchanged (a shared + `assertSessionUnchanged` helper reused by every refusal test). +4. **`report` renders utterances with the terminal output in their window, + through the existing event rendering unchanged.** + Mechanism: records carry only `kind` and `text`, which `eventLine` already + renders; one record per displayed line is what keeps that rendering legible + under `SafeText`'s newline stripping. + Test: `TestReportRendersTerminalEvents` — import into a session with one + utterance, `timeline.Merge`, `report.Render`; assert the utterance line is + followed by indented `- [MM:SS] terminal_output "…"` bullets, and that + `internal/report` carries no change (no new test in that package). +5. **`merge`, `report`, `analyze` behave identically for the same inputs.** + Mechanism: no new `src`, no new schema field, `kind` is an open set. + Test: the existing `internal/timeline`, `internal/report`, and + `internal/analyze` suites stay green with no edits — the evidence is the empty + diff in those packages, asserted by the review, plus + `TestImportedRecordsPassCheckInteraction`, which runs every record every + fixture produces through `timeline.CheckInteraction`. +6. **A header timestamp preceding `t0` yields negative session-relative times + that render as an early-started audio recording's do.** + Mechanism: `offsetMS` is negative, `t` stays a positive epoch-ms value + (`t0 + offsetMS` is still ~1.7e12), so `CheckInteraction`'s positivity rule is + met and `merge` produces a negative `rel`; `report.clock` already signs it. + Test: `TestEarlyStartedCastGoesNegative` — header timestamp 30 s before `t0`; + assert entry times around `-30`, and that `report.Render`'s output contains + `[-00:30]`. +7. **A single output event larger than the line limit is split, within the limit, + nothing truncated.** + Mechanism: the encoded-length budget, the per-rune append test, and the + measured `EncodedLen` assertion per record. + Test: `TestOversizedEventSplits` — one `o` event of mixed ASCII, + multi-byte runes, and ESC bytes, sized as the test plan explains (~6 MiB at + the package boundary, the full ~12 MiB in the `coalescer` unit, which writes + nothing); assert (a) more than one record, (b) every + wrapped entry within `session.MaxJSONLLine`, (c) concatenating the records' + `text` reproduces the event's decoded data exactly, (d) no record boundary + falls inside a rune (implied by (c), asserted directly with `utf8.ValidString` + on each record). **Refinement of the criterion's wording:** the criterion says + "every byte preserved"; what is preserved at rune granularity is the string + `encoding/json` decodes, since the decoder itself replaces invalid UTF-8 with + U+FFFD before this package sees a byte. The byte-exact artefact is + `terminal.cast`, asserted byte-identical to the input by + `TestCastArchivedVerbatim`. This is the one place the spec narrows a criterion, + and it narrows it to something true rather than leaving a claim no + JSON-reading importer can honour. +8. **Ctrl+C in `record`'s terminal finalises exactly as an audio-only session + does; nothing about terminal capture appears in `record`'s lifecycle.** + Mechanism: `internal/record` is not touched — no flag, no recorder, no + subprocess, no signal path. + Test: the existing `internal/record` suite stays green with no edits; the + evidence is the empty diff in that package. `TestUsageListsImport` asserts the + new verb appears in the usage text (so the surface change is visible), that + the inferring-commands footer names it, and that no `-terminal` flag is + offered anywhere in that text — the fence, asserted rather than assumed. + +Additional criteria the intent states in scope rather than as a Given/When/Then, +each with its test: input events dropped (`TestInputEventsDropped`, asserting no +record and a non-zero printed count); the printed provenance line and its +quantisation caveat (`TestOffsetProvenance`, a table over the three cases); +idempotent re-run and preservation of `demo` records +(`TestReimportIsIdempotent`, `TestForeignRecordsPreserved`). + +## Decisions on open questions + +The intent's four open questions, and the two this spec had to add. + +1. **Command surface: a new verb.** `testimony import -session DIR [-cast FILE] + [-offset SECONDS]`, a peer of `transcribe` with `transcribe`'s flag names and + `transcribe`'s optional-input rule. A flag on `transcribe` would give one + command two unrelated input modes; a flag on `record` is the fence this intent + exists to hold. `import` names the step without implying speech. +2. **Re-run and mixing: replace only what this importer wrote, identified by + `kind: "terminal_output"`.** The write is a whole-file atomic rewrite that + drops prior terminal records, keeps every other line byte-for-byte, and + appends the new records — so a re-import is byte-identical, a `-demo` + session's clicks are untouched, and an import that would yield zero records + refuses rather than erase. The marker is the `kind` value rather than a new + `source` field because every reader in the repository drops fields outside the + documented six, so a `source` field would be dead weight that still needed + documenting; `kind` is already the discriminator, already rendered, and + already carried into the timeline. +3. **`install.sh`: no. The how-to alone.** The installer's guidance-only pattern + covers dependencies the CLI itself executes — ffmpeg (`record`'s capture, + `transcribe`'s conversion) and an ASR engine (`transcribe`'s engine). The CLI + never executes asciinema: `import` reads a file, and reads it just as happily + from any other producer of asciicast v2/v3. Listing it among the installer's + dependencies would assert a runtime dependency that does not exist, and would + put a third-party recorder's install path on the critical path of installing + a binary that does not need it. The how-to prints the install line where the + operator is already deciding to record a terminal. +4. **Chunking granularity: coalesce, and cut at the line the terminal displayed.** + Adjacent `o` events accumulate into one record, closed by a newline, a 250 ms + gap, a 1 s span cap, or the encoded line budget — values justified in the + design above, all named constants, none exposed as flags. Is the coalesced + record still a faithful "what the terminal displayed" claim? Yes, with the + boundaries stated: the record holds the runes the terminal received, in order, + with nothing inserted; its `t` is the instant its first rune arrived, and the + span cap bounds how stale that instant can be (≤ 1 s, inside `report`'s + window). The two places the claim is weaker than "what was displayed" are + named rather than hidden — a `\r`-redrawn line records every frame rather than + the final rendering, and an ANSI sequence is data in the record that a + terminal would have consumed as a command. Both are the intent's out-of-scope + TUI boundary, and both are exactly why the raw cast is archived. +5. **Added: the absent-anchor policy is `transcribe`'s, verbatim.** No header + `timestamp` → offset 0 with the provenance printed; a present but + non-positive timestamp, or a derived offset beyond ±10⁹ s → refuse with + `-offset` guidance. Mirroring the audio path keeps one rule for one class of + missing metadata, and the always-printed provenance line is what makes the + default non-silent. +6. **Added: ANSI sequences stay raw in the record.** The report sink already + neutralises them (`SafeText` strips ESC), the archived cast keeps the bytes, + and the remedy for the printable residue is recording guidance + (`NO_COLOR=1`) rather than an ANSI parser in the importer. Stripping at + import is recorded as a reversible follow-up. +7. **Settled: `-session` follows itd-12's shared inference rule.** The open + question in the command-surface section is closed by itd-12 having landed: + `import` resolves `-session` through `cli.resolveSession`, last among its + invocation checks, and carries no required-flag refusal of its own. One flag, + one rule, on all six pipeline commands. +8. **Settled: the budget is measured per record, and `cast` reuses + `transcribe.CheckOffset`.** Two numeric rules that could each have been + copied are not: the encoded-length budget is measured from the record's own + time through `session.EncodedLen` (rather than assumed once per run), and the + offset's finiteness and ±10⁹ s bound stay in `transcribe.CheckOffset`, called + by both the CLI (for exit 2) and `cast.Run` (so a direct caller cannot pass a + non-finite offset into an integer conversion). `cast` importing `transcribe` + for one predicate is the cheaper of the two costs. + +## Test plan + +Everything below is hermetic: `t.TempDir()` session directories, fixture casts +under `internal/cast/testdata/`, no network, no subprocess, no ffmpeg, no TTY. CI +runs it via the existing `go test -race ./...` gate, with `gofmt -l .` and +`go vet ./...` as before. + +**Fixtures** (`internal/cast/testdata/`): `v2.cast` and `v3.cast` (the same +recording in both formats, with a header timestamp, output, input, resize, +marker, and exit events); `v2-ties.cast` and `v3-ties.cast` (the same recording +again, built so the clock's rounding is the thing under test: seven 1.5 ms +keystrokes put the seventh at exactly 10.5 ms — a half-millisecond tie that +rounds one way from a stated absolute time and the other from a running sum — +with the eighth event 249.5 ms later, so a one-millisecond disagreement falls on +either side of the 250 ms gap and the recording becomes one record or two — on a +`float64` clock this pair yields 1 record read as v2 and 2 read as v3, and on the +microsecond grain 1 either way); `v3-nots.cast` (no header `timestamp`); +`bad-version.cast`, `bad-header.cast`, `bad-event.cast`, `decreasing.cast`, +`negative-interval.cast`; `golden.interactions.jsonl` (the expected records for +`v2.cast`, the `whisperx.golden.jsonl` precedent). Oversized inputs are generated +in-test rather than committed. + +**Pure units** +- `scanCast`: v2 absolute times; v3 running sum (table of interval sequences, + including `0` intervals and a long tail); every code delivered to the callback + in order; blank-line skipping (with the line numbers still counting the blanks, + so a refusal names a line the operator can find); every malformed-line refusal, + each asserting the line number in the message; the per-line and per-file bounds + (the file bound driven by a repeating reader rather than a 64 MiB fixture); + both callbacks' errors propagating unchanged. The per-code drop counts are + asserted at `Run`, where the classification lives, along with the tally's + 16-code bound and the clipping of an over-long code. +- `resolveOffset`: table over `-offset` set/unset × header timestamp + present/absent/non-positive × `t0` usable/absent → `(offsetMS, provenance, + error)`, asserting the three provenance strings verbatim (they are a documented + contract) and the two refusal messages. +- `coalescer`: table of synthetic event sequences → expected record boundaries + and times, one case per boundary (newline, 250 ms gap, 1 s span, budget), plus + the interaction of two boundaries falling together, plus the + renders-empty drop and its count. +- the per-rune encoded-length table: property test asserting the table is never + *under* `session.EncodedLen`'s real cost for every rune in a sampled set (all + of ASCII, a handful of multi-byte runes, U+2028/9, U+FFFD, the invisible Cf + runes), and equal wherever the two agree — so a stdlib change that invalidates + the table fails here rather than as a refused import, while the deliberate + over-count on backspace and form feed stays legal. +- the measured entry-size refusal, which is unreachable while the table holds: + driven white-box, by falsifying a `coalescer`'s budget so an over-long record + reaches `close`, and asserting the importer-bug message and that no record is + kept. + +**Package-level `Run`** +- happy path: records written, return value, printed lines (a `bytes.Buffer` + `Log`), `terminal.cast` archived byte-identically. +- idempotence: two runs → byte-identical `interactions.jsonl`. +- mixing: a pre-existing `interactions.jsonl` of `demo` records plus a prior + import's records → foreign lines byte-for-byte preserved in order, terminal + records replaced; a foreign line that does not decode at all is preserved too. +- in-place re-import (`-cast` omitted) and the `os.SameFile` case (`-cast` + pointing at the session's own `terminal.cast`): no copy, same result. +- refusals, each with `assertSessionUnchanged`: unsupported version; malformed + header/event; over-long line and file; missing manifest; unusable `t0`; + non-positive header timestamp; implausible derived offset; zero output events; + neither `-cast` nor `terminal.cast`; `-cast` naming a directory or a FIFO; a + symlink or FIFO at `terminal.cast` or at `interactions.jsonl`. +- size limits: an assembled `interactions.jsonl` over 16 MiB; a merged-timeline + total over 16 MiB with `interactions.jsonl` itself under it (the case only an + offline importer can measure). +- oversized single event: the four assertions in criterion 7, plus a `Merge` over + the result, since fitting the line limit is only worth anything if merge then + accepts it. The event is ~6 MiB of mixed ASCII, multi-byte runes, and ESC + bytes rather than the ~12 MiB the criterion's prose suggests: escaping inflates + those records by about a third, so a 12 MiB event is refused for + `interactions.jsonl`'s own 16 MiB file cap before the split can be observed. + 6 MiB is comfortably past the 4 MiB line limit, which is all the split needs. + The `coalescer` unit test, which writes nothing, uses the full 12 MiB. +- the file-mode rules: an existing `interactions.jsonl`'s mode preserved across + the rewrite, and an existing `terminal.cast`'s preserved across the archive. + +**Integration (still hermetic)** +- `import` → `timeline.Merge` → `report.Render` for the v2 and the v3 fixture: + interleaving, negative times, the rendered bullets, and golden `report.md` + fragments. +- `TestImportedRecordsPassCheckInteraction` over every fixture. + +**`internal/cli`** — `import` joins the four shared tables that already state +these contracts for its siblings, rather than growing a table of its own: +`TestStrayPositionalIsAUsageError`, `TestInvalidFlagValuesExitTwo` (empty +`-cast`, non-finite and out-of-bound `-offset`), `TestMissingSessionIsAUsageError` +and `TestEmptySessionIsAUsageErrorNotInference` (the two `-session` refusals), +and `TestRefusedInvocationAnnouncesNoSession` (a run refused for `-offset` +announces no inferred session). Six cases are its own: +- `TestImportWritesTerminalRecords` — a well-formed run over a cast fixture + written in-test: exit 0, the summary line on stdout, the records and their + times in `interactions.jsonl`, no keystroke among them, and the cast archived + byte-identically. +- `TestImportDiagnosticsStayOffStdout` — the offset provenance and the drop + counts on stderr, and stdout carrying nothing but the one summary line. +- `TestImportInfersSessionAndReImportsInPlace` — the offset-correction recipe + end to end: import with `-cast`, then a bare `import -offset -12.4` from inside + the session, asserting the inference line, the explicit-offset provenance, the + replaced-record count, and that the correction replaced the derived offset + rather than compounding it. +- `TestImportRefusesUnreadableCastAtRuntime` — an absent `-cast` is exit 1, not + exit 2, so a script can tell a mistyped flag from a missing file. +- `TestUsageListsImport` — the usage block, the inferring-commands footer, and + the absence of any `-terminal` flag. + +**CI** — one hermetic smoke step (`Terminal import smoke test`) builds a +throwaway session around `internal/cast/testdata/v2.cast` in a temp directory and +asserts what the unit tests cannot reach through the CLI: the stdout/stderr +split, the archived cast being byte-identical (`cmp`), a bare re-import leaving +`interactions.jsonl` byte-for-byte unchanged (`cmp`), and the records merging and +rendering with a signed clock. `examples/sample-session/` is deliberately +untouched, so the quickstart's golden report is unaffected. + +**Live verification** (not a CI gate, and stated here for exactly what it does +and does not cover). + +*Verified against a real recorder.* An asciinema **2.4.0** recording — the PyPI +line, so asciicast **v2** — of a shell session running `printf hello`, `ls /`, a +three-frame `\r` progress line, and three ticks 300 ms apart, imported into a +scratch session whose `t0` sits 1.5 s before the cast header's timestamp. The +derived offset is `+1.50s`, matching the constructed skew; the run produces 8 +records; each `ls` line is its own record; the three `\r` frames coalesce into +one record; the three ticks stay separate, which is the 250 ms gap behaving on +real shell timing rather than on fixture timing. `merge` and `report` then render +them under the utterance as designed. The one thing the live run showed that the +fixtures only asserted is the cost this spec already accepts: a coloured `ls` +renders as CSI residue in `report.md` (open ledger issue +`iss-2609120520334220`), which is what the `NO_COLOR=1` guidance is for. + +*Not verified against a real recorder.* The **3.x (v3)** line is not installed on +the machine the verification ran on, so asciicast v3 is exercised by fixtures +only — including the tie pair, which is the case the two formats could diverge +on. The two-terminal procedure itself (`testimony record` narrating in one window +while `asciinema rec` records in another) was not run end to end with live audio, +so the spoken-marker cross-check for the whole-second anchor remains a documented +procedure rather than a measured one. Both gaps are named here rather than in a +claim the tests do not carry. + +## Docs plan + +`docs/` is user-facing, one Diátaxis type per page, present tense, British +English in prose. + +- **`docs/reference/cli.md`** — a new `## testimony import` section between + `## testimony transcribe` and `## testimony merge`: the synopsis, the + three-row flag table, the behaviour (what is read, both formats and how they + differ, which event codes are kept and which dropped, the coalescing + boundaries and their values, the split rule, the archival copy, the re-run + semantics, the one-cast-per-session limit), the three provenance strings + verbatim with the quantisation caveat, the printed lines, and the refusals that + exit 2 versus 1. The `transcribe` section gains nothing; the `merge` section + gains one sentence noting that terminal records merge like any other + interaction. +- **`docs/reference/session-directory.md`** — `terminal.cast` in the layout + block (archival, written by `import`, local only) and its own + `## terminal.cast` section modelled on `## events.rrweb.jsonl`: what it is, + that nothing downstream reads it, that `import` re-reads it when `-cast` is + omitted, and the 64 MiB read bound. The `## interactions.jsonl` section gains: + `terminal_output` as a **reserved** kind written only by `import` (with the + collision consequence stated), that its `text` may hold multi-line output and + ANSI bytes, and that one record is one displayed line. +- **`docs/how-to/record-a-terminal-session.md`** (new) — the two-terminal + procedure: `testimony record -app …` in window one, `asciinema rec session.cast` + in window two (the window where the work happens), say "session start" aloud, + work, end the cast the way asciinema always ends it, `Ctrl+C` in window one, + then `testimony import -session sessions/ -cast session.cast`, + `transcribe`, `merge`, `report`. Plus: the install pointer + (`brew install asciinema`, or `pipx install asciinema`) with the note that + either version works and no `--output-format` flag is needed; the + **do not capture input** warning naming `--stdin` and `--capture-input`/`-I` + and the suppressed-echo password hazard; the colour guidance (`NO_COLOR=1`); + the **privacy warning** — terminal output routinely carries usernames, + hostnames, absolute paths, environment values, and occasionally secrets printed + by tools, and `analyze` sends timeline text to a model, so review or redact + `timeline.jsonl` before running `analyze`, and keep terminal sessions short + because the whole timeline goes into the request; and the offset-correction + recipe, cross-referencing + [fix a wrong clock offset](transcribe-a-recording.md) rather than restating it + (`testimony import -session DIR -offset -12.4`, no `-cast` needed). +- **`docs/README.md`** — the new how-to in the how-to list. +- **`docs/explanation/privacy.md`** — one short paragraph in *The privacy + boundary*: a terminal recording shows more of the machine than a demo app + does; keystrokes are never imported into the derived text; `terminal.cast` is + raw local evidence of the same class as `audio.wav`. +- **`README.md`** — `import` in the *Status and roadmap* working-today list, and + `terminal.cast` in the session-directory block. The pipeline diagram gains one + row (`terminal ──► asciinema ──► terminal.cast ──► interactions.jsonl`). +- **`AGENTS.md`** (`CLAUDE.md` is a symlink to it, so one edit serves both) — + the *Current state* paragraph's command inventory goes from seven pipeline + commands to eight, pairing `import` with `transcribe` as the two hand-off + commands for a recording the CLI never made; and the *Build, test, and checks* + block gains the import smoke line beside the `merge`/`report` one. +- **`.github/workflows/ci.yml`** — one hermetic `Terminal import smoke test` + step (see the test plan) and the workflow header comment that enumerates what + it runs. +- **`CHANGELOG.md`** — one entry under the unreleased heading, per the + changelog-driven release gate. +- **`install.sh`** — **no change**, for the reason recorded under decision 3: the + CLI never executes asciinema, so it is not a dependency the installer's + guidance-only pattern is for. +- **`examples/sample-session/`** — no change. Adding terminal records to the + bundled sample would change the quickstart's golden report for every reader to + demonstrate a path the how-to already walks; a fixture in + `internal/cast/testdata/` carries the same weight for tests without touching + the published example. diff --git a/.abcd/work/CONTEXT.md b/.abcd/work/CONTEXT.md index fc24859..46b20d6 100644 --- a/.abcd/work/CONTEXT.md +++ b/.abcd/work/CONTEXT.md @@ -9,7 +9,9 @@ useful. Short and pointer-heavy; durable design truth lives in Testimony captures usability evidence, on the record. A Go CLI (`testimony`, standard library only) with `record` (managed capture), -`demo`, `transcribe`, `merge`, and `report` working end-to-end, plus the +`demo`, `transcribe`, `import` (an operator-recorded asciinema terminal +session joins the interaction stream on the shared clock), `merge`, and +`report` working end-to-end, plus the first-pass analysis layer — `analyze` (emit a host-delegated analysis request, then validate the answer into `findings.jsonl`) and `review` (record human verdicts, appended non-destructively) — and the diff --git a/.abcd/work/DECISIONS.md b/.abcd/work/DECISIONS.md index be6bad5..fa89237 100644 --- a/.abcd/work/DECISIONS.md +++ b/.abcd/work/DECISIONS.md @@ -1642,3 +1642,36 @@ Architecture-shaping decisions graduate to an ADR under not the model's to choose, and requiring the quote byte for byte makes the drafting step structurally incapable of introducing evidence the human never vouched for. +- 2026-09-12 — A terminal recording arrives by hand-off, not by wrapping: + `testimony import -session DIR [-cast FILE] [-offset SECONDS]` reads an + operator-recorded asciicast (v2 or v3, told apart by the header's `version`) + into `interactions.jsonl`, and `record` gains no flag, no pty, and no second + way to end. The offset comes from the cast header's own integer timestamp, + with the `(whole seconds, ±1s)` caveat in the printed provenance line. +- 2026-09-12 — Imported terminal records are marked by `kind: + "terminal_output"` — a reserved kind, documented as such — rather than by a + new `source` field every reader in the repository would drop; a re-import + rewrites `interactions.jsonl` whole, replacing exactly those records and + keeping every other line byte-for-byte, and refuses rather than erase when a + cast yields none. +- 2026-09-12 — Output events coalesce into one record per displayed line (cut + at a newline, a 250 ms gap, a 1 s span, or the encoded JSONL line budget), + input events are dropped unconditionally as a privacy layer, and ANSI + sequences stay raw in the record with the raw `.cast` archived as + `terminal.cast`; stripping CSI/OSC at import is recorded as a reversible + follow-up. +- 2026-09-12 — The recording clock is carried on an exact integer grain + (microseconds), not `float64` seconds: v2 rounds a stated absolute time while + v3 rounds a running sum, so at a half-millisecond tie the two land on + different milliseconds and flip a 250 ms coalescing cut — the same recording + becoming one record in one format and two in the other. The `v2-ties`/ + `v3-ties` fixture pair is the case that fails if the grain goes. +- 2026-09-12 — `import` live-verified against a real asciinema 2.4.0 recorder + (PyPI line, asciicast v2, run via `uvx` with nothing installed): a shell + session of `printf hello`, `ls /`, a three-frame `\r` progress line and three + ticks 300 ms apart, imported into a session whose `t0` sat 1.5 s before the + cast header's timestamp. Derived offset `+1.50s`; 8 records; each `ls` line + its own record; the `\r` frames one record; the ticks separate; `merge` and + `report` rendered them under the utterance. Coloured `ls` output rendered as + CSI residue, as designed and documented — logged as `iss-2609120520334220`. + The 3.x (v3) recorder is not installed, so v3 stays fixture-verified only. diff --git a/.abcd/work/issues/open/iss-2609120520334220-imported-terminal-output-keeps-ansi-csi-sequences-raw-report.md b/.abcd/work/issues/open/iss-2609120520334220-imported-terminal-output-keeps-ansi-csi-sequences-raw-report.md new file mode 100644 index 0000000..8fd95a5 --- /dev/null +++ b/.abcd/work/issues/open/iss-2609120520334220-imported-terminal-output-keeps-ansi-csi-sequences-raw-report.md @@ -0,0 +1,14 @@ +--- +schema_version: 1 +id: "iss-2609120520334220" +slug: "imported-terminal-output-keeps-ansi-csi-sequences-raw-report" +severity: "minor" +category: "observation" +source: "user-observation" +found_during: "itd-11-live-verification" +origin: researcher-authored +production_mode: hand-written +found_at: "internal/cast/coalesce.go" +--- + +Imported terminal output keeps ANSI CSI sequences raw; report's SafeText strips only the ESC byte, so a coloured ls line renders as '[1m[34mApplications[39;49m[0m …' in report.md and the same residue reaches analyze's request text — a live asciinema 2.4.0 session showed every ls line unreadable. The spec chose raw evidence plus NO_COLOR=1 guidance and named stripping as a reversible follow-up; a small CSI state machine applied at report/emit time (not on the archived cast) would keep the evidence and fix the rendering diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2b9c775..bc62a1e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -3,7 +3,8 @@ name: ci # Lightweight checks on every push and pull request: format, build, vet, and # test on Linux, a cross-compile check for the other release platforms, a # version-stamp ldflags check, a smoke test of the merge → report pipeline on -# the bundled sample, installer syntax and flag-handling checks, full-history +# the bundled sample, a smoke test of the terminal import hand-off against the +# v2 cast fixture, installer syntax and flag-handling checks, full-history # secret scanning, and a workflow audit. Also runs on `merge_group` so the # same jobs gate each entry the merge queue builds — the queue tests every PR # against the exact main it will land on, and its required checks are these @@ -162,6 +163,52 @@ jobs: # utterance. grep -q '^ - \[00:19\] click `\[data-testid=save-btn\]`' examples/sample-session/report.md + # The terminal hand-off has no bundled sample session of its own: adding + # terminal records to examples/sample-session would change the + # quickstart's golden report for every reader, to demonstrate a path the + # how-to already walks. So the smoke builds a throwaway session in a temp + # directory around the v2 fixture the unit tests already carry — hermetic, + # no network, no asciinema, no recorder of any kind, since `import` only + # ever reads a file. + # + # The assertions are the three things the unit tests cannot reach through + # the CLI: the stdout/stderr split (a script reads one summary line while + # the offset provenance and drop counts go to stderr), the archived cast + # being byte-identical to what was handed over, and a bare re-import — + # the in-place path, with no -cast to name — leaving interactions.jsonl + # byte-for-byte unchanged. + - name: Terminal import smoke test + run: | + set -euo pipefail + s="$(mktemp -d)" + trap 'rm -rf "$s"' EXIT + printf '{"session":"smoke","app":"a shell","participant":"P1","t0_epoch_ms":1784300400000}\n' > "$s/manifest.json" + ./testimony import -session "$s" -cast internal/cast/testdata/v2.cast 2> "$s/stderr.txt" > "$s/stdout.txt" + grep -q "imported 5 records" "$s/stdout.txt" + grep -q "derived: cast header timestamp" "$s/stderr.txt" + grep -q "dropped 2 input (i) event(s)" "$s/stderr.txt" + # Diagnostics stay off the stream a caller pipes. + ! grep -q "offset:" "$s/stdout.txt" + # The archive is the one place the byte-for-byte claim is made. + cmp internal/cast/testdata/v2.cast "$s/terminal.cast" + grep -q '"kind":"terminal_output"' "$s/interactions.jsonl" + # The keystroke-echoed command is one record, not one per keystroke. + grep -q 'ls --color' "$s/interactions.jsonl" + # An input event must never reach the derived text. + ! grep -q '"text":"l"' "$s/interactions.jsonl" + # A bare re-import reads the archived cast and rewrites the same bytes. + cp "$s/interactions.jsonl" "$s/first.jsonl" + ./testimony import -session "$s" > /dev/null 2>&1 + cmp "$s/first.jsonl" "$s/interactions.jsonl" + # Downstream is untouched: the records merge and render through the + # existing event path, and the cast's header timestamp sits before t0, + # so the early records carry a signed clock. + ./testimony merge -session "$s" > /dev/null + ./testimony report -session "$s" > /dev/null + grep -qF 'terminal\_output' "$s/report.md" + grep -qF '[-00:02]' "$s/report.md" + grep -q '\*\*Events:\*\* 5' "$s/report.md" + # install.sh is served live from main as the documented install path, so # a merged syntax error breaks `curl | sh` for every new user with no # gate having run. Parse it with both shells an operator realistically diff --git a/AGENTS.md b/AGENTS.md index 565465b..5c8e076 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -50,13 +50,14 @@ Testimony captures usability evidence, on the record. ## Current state -v0.4.0. A Go CLI (`testimony`, standard library only) whose eight pipeline +v0.4.0. A Go CLI (`testimony`, standard library only) whose nine pipeline commands are all implemented and dispatched from `internal/cli` behind the `cmd/testimony` entry point: `record` and `demo` -(capture), `transcribe`, `merge`, `report`, the analysis layer `analyze` -and `review`, and the regression-test drafting layer `draft-tests` (with -`review -kind tests` for its human pass) — plus `version` and `help`. The model -work is host-delegated — +(capture), `transcribe` and `import` (hand-off of a recording the CLI never +made — a voice recording, or an operator-recorded asciinema terminal session), +`merge`, `report`, the analysis layer `analyze` and `review`, and the +regression-test drafting layer `draft-tests` (with `review -kind tests` for its +human pass) — plus `version` and `help`. The model work is host-delegated — the CLI never calls a model, holds no keys, and adds no network dependency. The user-facing documentation is [`docs/README.md`](docs/README.md); the exact command and file contracts are [`docs/reference/cli.md`](docs/reference/cli.md) and @@ -79,6 +80,8 @@ go test -run TestEventsNearWindow ./internal/timeline/ # a single test ./testimony merge -session examples/sample-session # pipeline smoke: ./testimony report -session examples/sample-session # writes timeline.jsonl + report.md ./testimony draft-tests -session examples/sample-session # emit the drafting request (after merge) +S=$(mktemp -d) && cp examples/sample-session/manifest.json "$S" \ + && ./testimony import -session "$S" -cast internal/cast/testdata/v2.cast # terminal hand-off smoke sh -n install.sh && bash -n install.sh # installer syntax ``` diff --git a/CHANGELOG.md b/CHANGELOG.md index 4519ca6..23ec339 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,25 @@ break an existing invocation is called out in the entry that records it. message does change — a failure while writing `findings.jsonl` is now prefixed `write findings.jsonl:` rather than `write findings:`, so every failure on that path names the file the same way. +- `testimony import -session DIR [-cast FILE] [-offset SECONDS]` brings a + terminal session into the evidence record: the operator records their own + shell with `asciinema rec` in the window where the work happens, then hands + the `.cast` file over, exactly as `transcribe -audio` already takes an + externally recorded voice. Both asciicast formats are read and told apart by + the header's `version` field — v2's absolute event times and v3's intervals + reconstruct onto one exact integer clock, finer than either format writes, so + the same recording in either format yields byte-identical records — and the + cast's own header timestamp anchors it to the + session's `t0`, with an explicit `-offset` always winning and the offset's + provenance (including its whole-second precision) printed on every run. + Output coalesces into one `terminal_output` interaction per line the terminal + displayed, an oversized event splits across records with every rune + preserved, input events are dropped so keystrokes cannot reach the derived + text, and the raw cast is archived in the session as `terminal.cast`. A + re-import is byte-identical and leaves a `-demo` session's own records + untouched. `record` is unchanged: no flag, no wrapped recorder, no second way + for a session to end. New guide: [record a terminal + session](docs/how-to/record-a-terminal-session.md). - `transcribe` prints an elapsed-time status line every 5 seconds while the ASR engine is still running, instead of staying silent between the offset line and completion — a CPU-only `whisperx`/`whisper-cli` run can take diff --git a/README.md b/README.md index 0bb064f..2b12644 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,13 @@ timestamped interaction stream, rendered as a report that shows what was said ne to what was done. ``` - voice ──► local Whisper ──► transcript.jsonl ─────┐ - ├─► timeline.jsonl ─► report.md - clicks ──► capture hooks ──► interactions.jsonl ───┘ + voice ──► local Whisper ──► transcript.jsonl ────┐ + ├─► timeline.jsonl ─► report.md + clicks ──► capture hooks ──┐ │ + terminal ──► asciinema ──────┴─► interactions.jsonl ┘ + (the raw terminal.cast is kept alongside, archival) - page ──► rrweb ──► events.rrweb.jsonl (archival only; nothing downstream reads it) + page ──► rrweb ──► events.rrweb.jsonl (archival only; nothing downstream reads it) ``` Raw audio and video never leave your machine; only derived text is analysed. See @@ -78,7 +80,8 @@ The demo app contains at least one intentional usability flaw. Find it by talkin - [Tutorials](docs/tutorials/getting-started.md) — your first session, end to end. - [How-to guides](docs/how-to/) — [transcribe a recording](docs/how-to/transcribe-a-recording.md) - (engines, languages, offsets), [analyse a session](docs/how-to/analyse-a-session.md) + (engines, languages, offsets), [record a terminal session](docs/how-to/record-a-terminal-session.md) + (asciinema, two windows, one import), [analyse a session](docs/how-to/analyse-a-session.md) (findings and verdicts), [draft regression tests](docs/how-to/draft-regression-tests.md) (drafts and decisions), [instrument your own app](docs/how-to/instrument-your-own-app.md). - [Reference](docs/reference/) — the [command line](docs/reference/cli.md) and the @@ -97,6 +100,7 @@ sessions// audio.offset.json # audio→session offset for an external recording (local only) screen.mp4 # screen capture, with record -video (local only) events.rrweb.jsonl # raw rrweb stream (archival) + terminal.cast # raw asciicast, as recorded (archival; local only) interactions.jsonl # normalised interaction events transcript.jsonl # time-aligned utterances timeline.jsonl # merged, session-relative timeline @@ -111,16 +115,17 @@ Exact schemas: [session directory reference](docs/reference/session-directory.md Working today: `record` (managed capture — one command starts the recorders and stamps the session), `demo` (instrumented capture), `transcribe` (local WhisperX -or whisper.cpp), `merge`, `report`, the first-pass analysis layer — `analyze` -(emit an analysis request, then validate the answer into findings) and `review` -(record human verdicts) — and the regression-test drafting layer, `draft-tests` -(turn a confirmed finding into a proposed test case, then render the accepted -ones as a Markdown test plan) with `review -kind tests` for the accept / edit / -reject pass. `record` captures the microphone by default; screen video is opt-in -with `-video`. The model work is host-delegated — the CLI never calls a model, -holds no keys, and adds no network dependency — every finding is *unverified* -until you confirm or reject it, and every drafted test is a *proposal* until you -accept it. +or whisper.cpp), `import` (an asciinema terminal recording joins the session's +interaction stream on the shared clock), `merge`, `report`, the first-pass +analysis layer — `analyze` (emit an analysis request, then validate the answer +into findings) and `review` (record human verdicts) — and the regression-test +drafting layer, `draft-tests` (turn a confirmed finding into a proposed test +case, then render the accepted ones as a Markdown test plan) with +`review -kind tests` for the accept / edit / reject pass. `record` captures the +microphone by default; screen video is opt-in with `-video`. The model work is +host-delegated — the CLI never calls a model, holds no keys, and adds no network +dependency — every finding is *unverified* until you confirm or reject it, and +every drafted test is a *proposal* until you accept it. Coming next, in user terms: diff --git a/docs/README.md b/docs/README.md index f3021bd..f5261d0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,6 +1,6 @@ # Testimony documentation - **[Tutorials](tutorials/getting-started.md)** — learn by doing: capture, transcribe, and report on your first session in about five minutes. -- **[How-to guides](how-to/)** — recipes for specific tasks: [transcribe a recording](how-to/transcribe-a-recording.md), [instrument your own app](how-to/instrument-your-own-app.md), [analyse a session](how-to/analyse-a-session.md), [draft regression tests](how-to/draft-regression-tests.md). +- **[How-to guides](how-to/)** — recipes for specific tasks: [transcribe a recording](how-to/transcribe-a-recording.md), [record a terminal session](how-to/record-a-terminal-session.md), [instrument your own app](how-to/instrument-your-own-app.md), [analyse a session](how-to/analyse-a-session.md), [draft regression tests](how-to/draft-regression-tests.md). - **[Reference](reference/)** — exact descriptions of the [command line](reference/cli.md) and the [session directory](reference/session-directory.md). - **[Explanation](explanation/)** — background and reasoning: [how alignment works](explanation/how-alignment-works.md), [privacy](explanation/privacy.md). diff --git a/docs/explanation/privacy.md b/docs/explanation/privacy.md index d192eb4..1fa68ce 100644 --- a/docs/explanation/privacy.md +++ b/docs/explanation/privacy.md @@ -13,6 +13,8 @@ The rule is simple: **raw recordings stay local; only derived text is ever analy The distinction matters because the derived text is a much narrower disclosure than the recording it came from. A transcript contains what was said; the audio contains a voiceprint. An event stream says a button was clicked; a screen recording shows everything else that was visible at the time. When an analysis layer (local or cloud) enters the picture, it sits on the far side of this boundary: it sees only the text you choose to give it, never the raw audio or video. If your setting demands it, a fully local analysis path keeps even the derived text on the machine. +A terminal recording sits at the widest point of that boundary. A shell shows far more of the machine than a demo app does: its output routinely carries usernames, hostnames, absolute paths, environment values, and occasionally a secret a tool prints. Keystrokes never reach the derived text — `import` drops every input event a cast holds, so a password typed at a prompt that suppresses echo cannot enter the interaction stream — and the raw `terminal.cast` is local evidence of the same class as `audio.wav`. What does travel outward is the derived text, so a terminal session asks one thing of you that a browser session never had to: read or redact `timeline.jsonl` before running `analyze`. [Record a terminal session](../how-to/record-a-terminal-session.md) sets out the practice. + One caveat deserves emphasis: anything extracted *from* a recording inherits its sensitivity. A video frame can show personal data on screen — treat stills and clips with the same care as the recording itself. ## Participant pseudonyms diff --git a/docs/how-to/record-a-terminal-session.md b/docs/how-to/record-a-terminal-session.md new file mode 100644 index 0000000..f12c0e6 --- /dev/null +++ b/docs/how-to/record-a-terminal-session.md @@ -0,0 +1,117 @@ +# Record a terminal session + +Testimony reads a terminal recording the same way it reads a voice recording made outside the tool: you record it yourself, then hand the file over. The recorder is [asciinema](https://asciinema.org), which writes an *asciicast* file; `testimony import` normalises that file's terminal output into the session's interaction stream, on the session clock, so a spoken stumble lands beside the command that caused it. + +`record` is untouched by any of this. It gains no flag, starts no recorder, and ends exactly as it does for an audio-only session. + +## Install asciinema + +```sh +brew install asciinema # or: pipx install asciinema +``` + +Either version works. Homebrew ships the 3.x line, which writes asciicast v3; PyPI ships 2.4.0, which writes asciicast v2. `import` reads both and tells the two apart from the file's own header, so no `--output-format` flag is needed and you never have to state which format you have. + +## Record the session + +You need two terminal windows: one for Testimony, one for the work. + +1. **Window one — start the session.** This is the window that owns the session directory and the narration. + + ```sh + testimony record -app "my CLI" -participant P1 -task "Build the project and read the output" + ``` + + Note the session directory it prints. + +2. **Window two — start the terminal recording.** This is the window where the work happens. + + ```sh + asciinema rec session.cast + ``` + +3. **Say "session start" aloud.** The spoken marker is the cross-check for the clock (see *Fix a wrong clock offset* below), and it matters more here than for audio: an asciicast header timestamp is a whole number of seconds, so the reconstructed clock can sit up to a second away from the session anchor. + +4. **Do the work in window two, thinking aloud.** Speak into window one's recording as you go — what you expect, what surprises you, what you cannot tell from the output. + +5. **End the cast** the way asciinema always ends one: exit the recorded shell (`exit`, or `Ctrl-D`). + +6. **End the session** with `Ctrl+C` in window one. + +## Import the cast + +```sh +testimony import -session sessions/ -cast session.cast +``` + +`import` copies the cast into the session as `terminal.cast`, normalises its output into `interactions.jsonl`, and prints the clock offset it used and its provenance. Then finish the pipeline as usual: + +```sh +testimony transcribe -session sessions/ +testimony merge -session sessions/ +testimony report -session sessions/ +``` + +Each line the terminal displayed becomes one interaction record, which the report renders beside the utterance it falls next to: + +``` +**[00:22] P1:** “Wait, it says the build succeeded, but there's no binary.” + - [00:21] terminal_output "make build" + - [00:22] terminal_output "build finished in 1.2s" +``` + +## Do not capture input + +Record output only, which is what plain `asciinema rec` does. Input capture is opt-in on both CLI lines — `--stdin` on 2.x, `--capture-input` (or `-I`) on 3.x — and the reason to leave it off is a password typed at a prompt that suppresses echo: the characters never appear on screen, so they are absent from the output stream, but input capture records them as keystrokes. + +`import` drops every input event it finds, under any invocation, so keystrokes cannot reach the derived text even in a cast someone else handed you. It prints how many it dropped, which is how you learn that a recorder captured them. Those keystrokes do remain in the archived `terminal.cast`, exactly as the raw voice recording remains in `audio.wav` — `import` normalises the operator's evidence rather than rewriting it. + +## Turn colour off + +Record with colour disabled: + +```sh +NO_COLOR=1 asciinema rec session.cast +``` + +Some tools ignore `NO_COLOR`; `TERM=dumb` persuades most of the rest. + +Escape sequences are kept verbatim in the interaction records, because a record is evidence. The report's rendering strips the escape byte itself — no terminal control sequence ever reaches `report.md` — but the printable tail of a colour sequence survives, so a coloured `ls` reads as `terminal_output "[0;34mdocs[0m"`. Recording without colour avoids the litter, and it makes the records shorter, cheaper to hand to an analysis model, and easier to search. + +## Privacy: read the timeline before you analyse it + +A shell shows far more of the machine than a demo app does. Terminal output routinely carries usernames, hostnames, absolute paths, environment values, and occasionally a secret a tool prints. `testimony analyze` emits the whole timeline as part of its request, so: + +- **Read or redact `timeline.jsonl` before running `analyze`.** It is a small, line-oriented file; skim it the way you would a document you are about to send someone. +- **Keep terminal sessions short.** The whole timeline goes into the request, so a verbose session makes a large one. +- **Treat `terminal.cast` as raw local evidence**, of the same class as `audio.wav`: it holds the bytes the terminal emitted, and it never leaves your machine. + +See [privacy](../explanation/privacy.md) for the boundary the whole pipeline holds. + +## Fix a wrong clock offset + +`import` prints the offset it used and where it came from, for example: + +``` +offset: -2.00s (derived: cast header timestamp − manifest t0 (whole seconds, ±1s)) +``` + +Three provenance forms appear: + +| Printed provenance | Meaning | +|---|---| +| `from -offset flag` | you stated the offset; it always wins | +| `derived: cast header timestamp − manifest t0 (whole seconds, ±1s)` | taken from the cast's own header, to the nearest whole second | +| `default 0: cast header carries no timestamp` | the cast carries no anchor, so the recording is assumed to start at the session anchor | + +If the report shows terminal output clearly misaligned with the speech, correct it from the spoken marker exactly as [fix a wrong clock offset](transcribe-a-recording.md#fix-a-wrong-clock-offset) describes for audio, then re-import. `-cast` is not needed the second time — the session already holds `terminal.cast`: + +```sh +testimony import -session sessions/ -offset -12.4 +``` + +Re-run `testimony merge` and `testimony report` afterwards to rebuild the timeline and the report. + +## One terminal recording per session + +A session holds one terminal recording. Importing a different cast into the same session replaces the first one's records and its `terminal.cast`, because both are identified as this importer's own. Two terminals recorded side by side belong in two sessions. diff --git a/docs/reference/cli.md b/docs/reference/cli.md index e9ccefd..61df47e 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -16,7 +16,7 @@ Running `testimony` with no command, or with an unknown command, prints the usag ## Session directory inference -The six pipeline commands — `transcribe`, `merge`, `report`, `analyze`, `draft-tests`, and `review` — take their session directory from `-session DIR`. When `-session` is omitted and the current directory itself holds a Testimony session `manifest.json`, that directory is the session: the command operates on it exactly as `-session .` does, and prints one line to stderr naming what it inferred (`merge: using session . (inferred from the current directory)`) before it starts work, so the implicit choice is visible in the output of the run. The line goes to stderr, never stdout, so `analyze`'s emitted request stays a clean pipe. An explicit `-session` always wins, is used verbatim, and prints no such line — the current directory is not consulted at all. +The seven pipeline commands — `transcribe`, `import`, `merge`, `report`, `analyze`, `draft-tests`, and `review` — take their session directory from `-session DIR`. When `-session` is omitted and the current directory itself holds a Testimony session `manifest.json`, that directory is the session: the command operates on it exactly as `-session .` does, and prints one line to stderr naming what it inferred (`merge: using session . (inferred from the current directory)`) before it starts work, so the implicit choice is visible in the output of the run. The line goes to stderr, never stdout, so `analyze`'s emitted request stays a clean pipe. An explicit `-session` always wins, is used verbatim, and prints no such line — the current directory is not consulted at all. The marker is a session manifest, not merely the file name. `manifest.json` is one of the most common file names in software, so the file must be a regular file (a directory or a symlink at that name is not a marker) and, when it parses, must carry the `session` field every `testimony` session has (see [`manifest.json`](session-directory.md#manifestjson)). A `manifest.json` that belongs to something else leaves the command refusing rather than writing into a directory that is not a session: @@ -92,6 +92,44 @@ Behaviour: reads `manifest.json` (required). With `-audio`, requires ffmpeg on P It then prints `transcribed N utterances → `. With an `-audio` that names a file other than the session's own `audio.wav`, the offset in force is written to `audio.offset.json`; without it (including `-audio audio.wav`), the sidecar is rewritten only when an explicit `-offset` is given and the session already has one. A later bare run reuses the persisted value. +## `testimony import` + +Imports an operator-recorded terminal session — an [asciinema](https://asciinema.org) recording — into the session's `interactions.jsonl` on the shared session clock, and keeps the raw cast in the session as `terminal.cast`. It is `transcribe -audio`'s peer for the terminal: the CLI never runs the recorder, so the operator records their own shell and hands the file over afterwards. Nothing here spawns a process, allocates a pty, or touches the network, and `record` is unaffected — there is no `-terminal` flag and no change to how a session ends. + +``` +testimony import [-session DIR] [-cast FILE] [-offset SECONDS] +``` + +| Flag | Default | Meaning | +|---|---|---| +| `-session` | *(inferred)* | session directory; when omitted, the current directory if it holds a Testimony session `manifest.json` (see [session directory inference](#session-directory-inference)) | +| `-cast` | *(optional)* | asciicast file to import. Omit to re-import the session's own `terminal.cast`, which is what makes correcting an offset a one-line command. A `-cast` resolving to that same file is the omitted case, so the archive is never copied onto itself. Unlike `-audio`, no extension is required: the cast's own header decides whether a file is importable | +| `-offset` | derived | cast-to-session clock offset in seconds — the value added to every cast-clock time to place it on the session clock. A non-finite value, or one beyond ±10⁹ seconds, is a usage error | + +**What is read.** Both asciicast formats are accepted and told apart by the header's `version` field: **v2** event times are absolute seconds since recording start, **v3** event times are intervals since the previous event, reconstructed by a running sum. Both are reconstructed onto one exact integer clock — microseconds, which is finer than either format writes — so the same recording in either format yields byte-identical records, and the operator never has to state which their recorder wrote. Any other version is refused by name. The header's `version` and `timestamp` are the only fields read; every other one is ignored, since both formats are extensible. A cast is read within two bounds: 16 MiB for a single line and 64 MiB for the whole file. + +**Which events are kept.** Only `o` (output) becomes records. `i` (input), `r` (resize), `m` (marker), `x` (exit), and any code this importer does not recognise are dropped and counted by code, with the counts printed. Dropping input is unconditional: a cast recorded with input capture still cannot put keystrokes into the derived text. An unrecognised code is dropped rather than refused, so a later asciicast revision does not make its casts unimportable. + +**How output becomes records.** A shell echoes a typed command back roughly one keystroke at a time, so adjacent output accumulates into one record per line the terminal displayed. A record closes at the first of: a **newline** (included in the record); an inter-event **gap of 250 ms** or more, which closes output that never ends in a newline, such as a bare prompt; a **span of 1 second** measured from the record's first event, so a record's stated time is never more than a second stale against `report`'s 2.5-second join window; or the JSONL line budget. A record's `t` is the instant its first rune arrived. Carriage returns are kept and are **not** a boundary, so a progress line redrawing over itself is one record holding its frames rather than one record per frame. A record whose text would render empty — a blank line, a lone carriage return — is dropped and counted. + +A single output event too large for the line limit is split across consecutive records at rune boundaries, so concatenating their text reproduces the event exactly and nothing is truncated. The budget is measured against the **encoded** length of the timeline entry `merge` will wrap the record in, because escaping is what consumes it: an escape byte costs six bytes encoded, and coloured output is dense in them. + +Records carry `t`, `kind`, and `text` and nothing else, with `kind` always `terminal_output` — a reserved kind (see [`interactions.jsonl`](session-directory.md#interactionsjsonl)). ANSI escape sequences are kept verbatim, because the record is evidence; `report` strips the escape byte at its own boundary, so recording with colour disabled (`NO_COLOR=1`) is the remedy for the printable residue. See [record a terminal session](../how-to/record-a-terminal-session.md). + +**The archival copy.** The cast is copied into the session as `terminal.cast`, staged into a temp file beside it before the records are written and renamed into place afterwards, so a failure anywhere before that rename leaves the session exactly as it was. An existing `terminal.cast`'s own file mode is preserved; a new one is created honouring the umask. With `-cast` omitted there is no copy at all. + +**Re-running.** `interactions.jsonl` is rewritten whole and atomically: records from an earlier import (identified by their `terminal_output` kind, and nothing else) are dropped, every other line is kept byte-for-byte in its original order, and the new records are appended. So importing the same cast twice yields a byte-identical file, a `record -demo` session's clicks and inputs survive untouched, and a line this importer cannot decode at all is preserved as it stands. A session holds **one** terminal recording: importing a different cast replaces the first one's records and its `terminal.cast`. + +**Printed output.** The offset in force and its provenance are printed on every run, so the default is never a silent assumption — one of: + +- `from -offset flag` — the explicit flag, which always wins; +- `derived: cast header timestamp − manifest t0 (whole seconds, ±1s)` — taken from the cast header's own timestamp. The caveat is part of the line because the field is an integer in both formats, so the reconstructed clock can sit up to a second adrift of `t0`'s millisecond precision — material against the 2.5-second default join window, which is why the spoken "session start" marker stays the cross-check; +- `default 0: cast header carries no timestamp` — the cast carries no anchor, so the recording is taken to start at `t0`. + +The dropped-event counts, the count of records that render empty, and the number of earlier records replaced follow it. Dropped input events are counted on a line of their own, because that count is how you learn a recorder captured keystrokes; the other codes share a line, ordered by code. All of these go to **stderr**, beside the session-inference line; **stdout** carries only `imported N records → `, so a script reads one line. + +**Failure modes.** Exit 2 (a wrong invocation, refused before any work): a missing `-session` with no session manifest in the current directory, an explicitly empty `-session` or `-cast`, a stray positional argument, or an unusable `-offset`. Exit 1 (a runtime failure): no `manifest.json`, or one with no usable `t0_epoch_ms` — required on every path, including with an explicit `-offset`, since the records are epoch-millisecond-timed; neither `-cast` nor a `terminal.cast` to fall back on; a `-cast` naming a missing or non-regular file; an unsupported version, a malformed header or event, or a line or file past its bound, each naming the line; a header timestamp at or before the epoch, or a derived offset beyond ±10⁹ seconds, both asking for an explicit `-offset`; a cast holding no importable output — either no output events at all, or output that all rendered empty, each named as such — which refuses rather than erase an earlier import's records; and any size limit the assembled `interactions.jsonl` — or the merged `timeline.jsonl` it implies — would cross. Every exit-1 path above fires before any file in the session changes. + ## `testimony merge` Merges the transcript and interaction stream into `timeline.jsonl`. @@ -104,6 +142,8 @@ testimony merge [-session DIR] |---|---|---| | `-session` | *(inferred)* | session directory; when omitted, the current directory if it holds a Testimony session `manifest.json` (see [session directory inference](#session-directory-inference)) | +Terminal records written by [`import`](#testimony-import) merge exactly like any other interaction: they carry no new source type, flag, or schema field. + Behaviour: reads `manifest.json` (required), `transcript.jsonl`, and `interactions.jsonl`; converts interaction epoch-millisecond times to session-relative seconds via `t0_epoch_ms`; writes the time-sorted `timeline.jsonl`; prints `merged N utterances + M events → `. A missing `transcript.jsonl` or `interactions.jsonl` counts as zero records rather than an error, so a default audio-only `record` session (which never writes `interactions.jsonl`) still merges to a speech-only timeline. If the two sources together yield zero entries — missing, empty, or both — and a `timeline.jsonl` from an earlier merge already exists and is non-empty, merge refuses rather than truncate it to zero entries; a session with no timeline yet, or one already empty, still merges to an empty one. When interactions are present, `t0_epoch_ms` is required: without it their epoch-millisecond times cannot be placed on the session clock, so merge fails rather than write a corrupt timeline. ## `testimony report` diff --git a/docs/reference/session-directory.md b/docs/reference/session-directory.md index 13c6b89..4c5d91d 100644 --- a/docs/reference/session-directory.md +++ b/docs/reference/session-directory.md @@ -9,6 +9,7 @@ sessions// audio.offset.json # audio→session offset for an external recording (written by transcribe; local only) screen.mp4 # screen capture, H.264, 30 fps with cursor (written by record -video; local only) events.rrweb.jsonl # raw rrweb stream, archival (written by demo and record -demo) + terminal.cast # raw asciicast, archival (written by import; local only) interactions.jsonl # normalised interaction events (written by demo and record -demo) transcript.jsonl # time-aligned utterances (written by transcribe) timeline.jsonl # merged, session-relative timeline (written by merge) @@ -17,7 +18,7 @@ sessions// report.md # human-readable aligned record (written by report) ``` -All `.jsonl` files are JSON Lines: one JSON value per line, blank lines ignored. `timeline.jsonl`, `transcript.jsonl`, `interactions.jsonl`, `findings.jsonl`, and `tests.jsonl` each carry a 16 MiB total-size limit: a write that would push one over the cap is refused, and a load of one already over it is refused, so a session that reaches it needs a fresh session directory to continue in. `events.rrweb.jsonl` is archival and carries no such limit. +All `.jsonl` files are JSON Lines: one JSON value per line, blank lines ignored. `timeline.jsonl`, `transcript.jsonl`, `interactions.jsonl`, `findings.jsonl`, and `tests.jsonl` each carry a 16 MiB total-size limit: a write that would push one over the cap is refused, and a load of one already over it is refused, so a session that reaches it needs a fresh session directory to continue in. `events.rrweb.jsonl` is archival and carries no such limit, and `terminal.cast` is not a JSON Lines file at all — it carries its own read bound, described below. ## `manifest.json` @@ -62,6 +63,14 @@ One normalised interaction event per line, as posted by the instrumented app. Ti {"t":1784300419200,"kind":"click","selector":"[data-testid=save-btn]","text":"Save","route":"#general"} ``` +`kind` is an open set, with one exception: **`terminal_output` is reserved for `testimony import`**. Records carrying it are written only by `import`, and `import` identifies its own earlier records by that value alone — so a re-import replaces every `terminal_output` record in the file and leaves every other line byte-for-byte as it was. An instrumented app that posts `kind: "terminal_output"` to the demo capture endpoint therefore has its record replaced by a later import; use any other kind. + +A `terminal_output` record carries `t`, `kind`, and `text` and nothing else. One record is one line the terminal displayed, so its `text` ends in a newline when the line was complete, may hold several carriage-return-separated frames of a progress line, and keeps any ANSI escape sequences the terminal received verbatim. A single output event too large for the JSONL line limit is split across consecutive records, each carrying the time of the event whose data it opens with, so concatenating their `text` reproduces the output exactly. `report` renders the records through the same event rendering as any other interaction, with control bytes — the ANSI escape byte included — stripped at that boundary. + +```json +{"t":1784300398520,"kind":"terminal_output","text":"ls --color\r\n"} +``` + ## `transcript.jsonl` One utterance per line. Times are session-relative seconds (audio time plus the transcription offset), rounded to two decimal places. @@ -92,6 +101,14 @@ Written by `transcribe` only when the audio came from an external recording (a ` One raw [rrweb](https://github.com/rrweb-io/rrweb) event per line, exactly as emitted by the recorder (DOM snapshots, incremental mutations, pointer movement). Archival only: nothing downstream reads it; it exists so full session replay stays possible later. The demo page loads the rrweb recorder from a public CDN, so on a machine without network access (or with the CDN blocked) the file is created but stays empty — the session still captures `interactions.jsonl`, which carries the evidence the pipeline consumes. +## `terminal.cast` + +One [asciicast](https://docs.asciinema.org/manual/asciicast/v2/) recording, exactly as the operator's asciinema wrote it — asciicast v2 or v3, as its header's `version` field declares. Archival only: nothing downstream reads it, and it exists so the byte-exact record of the terminal survives alongside the derived records, as `events.rrweb.jsonl` does for a web session. + +`import` writes it by copying the file named with `-cast`, and reads it back when `-cast` is omitted, which is what makes re-importing with a corrected `-offset` a one-line command. A cast is read within two bounds: 16 MiB for a single line, and 64 MiB for the whole file; a file past either is refused rather than read in part. + +The file is local only, and it holds more than the derived records do: every event the recorder captured, including any keystrokes a recorder run with input capture recorded, which `import` drops rather than normalising. Treat it with the same care as `audio.wav`. + ## `timeline.jsonl` The merged record — one entry per line, speech and interface events on the shared session-relative clock, stably sorted by `t`. This is the single artefact the report (and any later analysis) consumes. diff --git a/internal/cast/cast.go b/internal/cast/cast.go new file mode 100644 index 0000000..bc35861 --- /dev/null +++ b/internal/cast/cast.go @@ -0,0 +1,398 @@ +// Package cast reads an asciinema recording (asciicast v2 or v3) and +// normalises its terminal-output events into a session's interaction stream. +// The package is named for the artefact it parses rather than for the verb it +// implements, because `import` is a Go keyword. +// +// It is transcribe's peer for the terminal: an artefact the CLI never +// produced, an anchor read out of that artefact's own metadata, an explicit +// -offset that always wins, a mandatory printed provenance line, an idempotent +// re-run, and an all-or-nothing write. Nothing here spawns a process, opens a +// pty, or touches the network — the operator records their own terminal and +// hands the file over afterwards. +package cast + +import ( + "errors" + "fmt" + "io" + "math" + "os" + "path/filepath" + "sort" + "strings" + + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" + "github.com/REPPL/Testimony/internal/transcribe" +) + +// OutputKind is the interaction kind every record this package writes carries. +// It is the importer's own marker: a re-run replaces exactly the records +// carrying it and leaves every other interaction untouched. It is a reserved +// kind (docs/reference/session-directory.md) for exactly that reason — an +// instrumented app posting it to the demo capture endpoint would have its +// record replaced by a later import. +const OutputKind = "terminal_output" + +// Options configures one import run. +type Options struct { + SessionDir string // session directory (docs/reference/session-directory.md) + Cast string // asciicast file; "" reuses the session's terminal.cast + Offset float64 // cast→session clock offset in seconds + OffsetSet bool // true when -offset was given explicitly + Log io.Writer // status sink; defaults to os.Stderr when nil +} + +// Run performs the import and returns the number of records written to +// interactions.jsonl. +// +// The order of operations is transcribe.Run's, for transcribe.Run's stated +// reason — every refusal fires before anything on disk changes, so a refused +// import leaves the session byte-for-byte as it found it: +// +// 1. load manifest.json and resolve t0 through session.Manifest.T0; +// 2. resolve the cast input (-cast, or the session's own terminal.cast); +// 3. scan the cast, resolving the offset from its header and building records +// as events arrive; +// 4. validate the record set against what merge will later demand of it; +// 5. stage the archival cast copy into a temp file beside terminal.cast; +// 6. rewrite interactions.jsonl atomically; +// 7. rename the staged copy over terminal.cast. +func Run(opts Options) (int, error) { + // Default the log sink as transcribe.Run does, so a caller that leaves Log + // nil gets progress on stderr instead of a panic at the first status line. + if opts.Log == nil { + opts.Log = os.Stderr + } + // An explicit offset is validated here as well as at the CLI boundary (where + // it is a usage error, exit 2): Run is also called directly, and a non-finite + // Offset would otherwise reach an int64 conversion with no defined answer. + // transcribe.CheckOffset is the one home for the rule. + if opts.OffsetSet { + if err := transcribe.CheckOffset(opts.Offset); err != nil { + return 0, err + } + } + man, err := session.LoadManifest(opts.SessionDir) + if err != nil { + return 0, err + } + // Unlike transcribe, import needs t0 even on the explicit -offset path: the + // artefact it writes is epoch-millisecond-timed and -offset is defined + // relative to the session clock. Refusing here is not a gap — merge already + // refuses a session whose interactions.jsonl is non-empty and whose manifest + // carries no usable t0, so importing into such a session would persist + // records no command could ever read back. + t0, err := man.T0() + if err != nil { + return 0, fmt.Errorf("anchoring the terminal cast: %w", err) + } + + src, name, external, err := resolveCast(opts.SessionDir, opts.Cast) + if err != nil { + return 0, err + } + + f, err := openCast(src, external) + if err != nil { + return 0, err + } + defer f.Close() + + co := coalescer{t0: t0} + var drops dropTally + var offsetMS int64 + var provenance string + _, err = scanCast(f, name, func(h castHeader) error { + // The offset is resolved between the header and the first event, because + // the header's timestamp is what anchors it and every record's t needs it. + off, prov, rerr := resolveOffset(name, opts, man, h) + if rerr != nil { + return rerr + } + offsetMS, provenance = off, prov + co.offsetMS = off + return nil + }, func(ev castEvent) error { + // Only output becomes records. Everything else is dropped and counted by + // code — the intent's "evidence is not silently dropped" applied to whole + // event classes. Dropping i (input) is unconditional and has no flag: a + // cast recorded with input capture still cannot put keystrokes into + // interactions.jsonl, whatever the operator's recorder did. + if ev.Code != codeOutput { + drops.add(ev.Code) + return nil + } + return co.add(ev) + }) + if err != nil { + return 0, err + } + if err := co.flush(); err != nil { + return 0, err + } + + // The provenance line is printed on every run, so "offset 0 because the + // header said nothing" is never a silent assumption. + fmt.Fprintf(opts.Log, "offset: %+.2fs (%s)\n", float64(offsetMS)/1000, provenance) + drops.report(opts.Log, co.dropped) + + if len(co.records) == 0 { + // A cast holding no importable output would otherwise silently delete a + // prior import's records — the hazard transcribe's zero-utterance guard + // and merge's zero-entry guard both refuse. The two ways to get here are + // named apart, because they call for different remedies: a cast with no + // output events at all is the wrong file (or one recorded with only input + // capture), while a cast whose output all rendered empty is a real + // recording of a terminal that displayed nothing legible. + if co.dropped > 0 { + return 0, fmt.Errorf("%s holds no importable output (%d record(s) rendered empty); refusing to rewrite %s", + name, co.dropped, session.InteractionsFile) + } + return 0, fmt.Errorf("%s holds no output events; refusing to rewrite %s", name, session.InteractionsFile) + } + if err := checkRecords(co.records, t0); err != nil { + return 0, err + } + + // The copy is staged before the records are written and committed after, so + // a failure anywhere before the final rename leaves the session exactly as + // it was. The only residual window is a same-directory rename failing after + // the records landed — records with no archival copy, the less misleading of + // the two possible residual states: the records are the evidence, the cast + // is the archive. + tmpPath := "" + if external { + // Copied from the descriptor the scan already read, rewound, rather than + // by re-opening the path: between the two opens the operator's file can be + // replaced, and the archive would then hold bytes the records did not come + // from — silently contradicting the one byte-for-byte claim this design + // makes. + if tmpPath, err = stageCast(opts.SessionDir, name, f); err != nil { + return 0, err + } + defer os.Remove(tmpPath) + } + replaced, err := rewriteInteractions(opts.SessionDir, name, t0, co.records) + if err != nil { + return 0, err + } + if replaced > 0 { + fmt.Fprintf(opts.Log, "replaced %d %s record(s) from an earlier import\n", replaced, OutputKind) + } + if tmpPath != "" { + if err := commitCast(tmpPath); err != nil { + return 0, err + } + } + return len(co.records), nil +} + +// resolveCast decides which file this run reads. -cast names an external +// artefact the operator holds; omitting it — or pointing it at the session's +// own terminal.cast, which os.SameFile settles — reuses the archival copy in +// place, so "re-run with a corrected -offset" is a one-liner and the archival +// copy is never copied onto itself. name is the display name for messages. +func resolveCast(dir, flagCast string) (src, name string, external bool, err error) { + archive := filepath.Join(dir, session.TerminalCastFile) + if flagCast != "" && !sameFile(flagCast, archive) { + // -cast carries no extension check, unlike -audio's closed .m4a/.mov/.wav + // set: that set exists because ffmpeg accepts only those containers, while + // here the header's version field is the authority on whether a file is + // importable, and a name rule would refuse a legitimately-named cast. + fi, serr := os.Stat(flagCast) + if serr != nil { + return "", "", false, fmt.Errorf("cast file: %w", serr) + } + if !fi.Mode().IsRegular() { + return "", "", false, fmt.Errorf("refusing to read %s: it is not a regular file", flagCast) + } + return flagCast, flagCast, true, nil + } + return archive, archive, false, nil +} + +// openCast opens the resolved cast. The session's own terminal.cast goes +// through the no-follow guard every other session-artefact read uses (a FIFO or +// symlink planted at the name in a received session is refused, not followed or +// blocked on); an operator-named -cast is opened plainly, the transcribe -audio +// precedent — that path the operator named themselves rather than received. +func openCast(src string, external bool) (*os.File, error) { + if external { + f, err := os.Open(src) + if err != nil { + return nil, fmt.Errorf("cast file: %w", err) + } + return f, nil + } + f, err := session.OpenFileNoFollowRead(src) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return nil, fmt.Errorf("no %s in session %s and no -cast given: record a terminal with asciinema, then pass -cast FILE", + session.TerminalCastFile, filepath.Dir(src)) + } + return nil, err + } + return f, nil +} + +// resolveOffset picks the cast→session offset in milliseconds, in +// transcribe.resolveOffset's order: an explicit -offset wins; otherwise the +// header's own timestamp anchors the cast; otherwise 0, with the provenance +// printed so the default is never silent. +// +// The derived case is exact integer arithmetic — no float enters it — because +// both operands are integers: the header's timestamp is whole Unix seconds in +// both formats, and t0_epoch_ms is whole milliseconds. Only an explicit -offset +// introduces a rounding step, to the nearest millisecond. +// +// name is the cast's display name, which the implausible-timestamp refusal +// must carry (the spec's message shape names the file). +func resolveOffset(name string, opts Options, man session.Manifest, hdr castHeader) (offsetMS int64, provenance string, err error) { + // t0 is obtained through session.Manifest.T0, never the raw field, so an + // absent (0) or negative anchor refuses the run rather than placing every + // record about fifty-seven years into the session. + t0, err := man.T0() + if err != nil { + return 0, "", fmt.Errorf("anchoring the terminal cast: %w", err) + } + if opts.OffsetSet { + return int64(math.Round(opts.Offset * 1000)), "from -offset flag", nil + } + if hdr.Timestamp == nil { + // transcribe's absent-metadata policy verbatim: default 0 and print why. + // Refusing instead would make the terminal path stricter than the audio + // path for the same class of missing metadata, and the remedy is identical + // in both — the spoken "session start" marker as the cross-check, then + // -offset to correct. + return 0, "default 0: cast header carries no timestamp", nil + } + ts := *hdr.Timestamp + if ts <= 0 { + // Manifest.T0's reasoning applies unchanged: no recorder produces a capture + // instant at or before 1 January 1970. A present-but-implausible anchor is + // refused rather than defaulted, exactly as transcribe refuses a present + // but implausible creation time while defaulting only on absence. + return 0, "", fmt.Errorf("%s: header timestamp %d is not a recording instant; pass -offset SECONDS to anchor the cast explicitly", name, ts) + } + // Bound the magnitude in float before the multiplication, so an astronomical + // header timestamp cannot overflow int64 on its way to the refusal. + if off := float64(ts) - float64(t0)/1000; math.Abs(off) > maxCastSeconds { + return 0, "", fmt.Errorf("derived cast offset %+.2fs exceeds %g in magnitude; the cast's header timestamp or the manifest t0 is implausible — pass -offset SECONDS to state it explicitly", off, maxCastSeconds) + } + // The caveat rides in the printed line, not only in the docs: the header + // field is an integer in both formats, so the reconstructed clock can sit up + // to a second adrift of t0's millisecond precision — material against + // report's 2.5-second default join window. + return ts*1000 - t0, "derived: cast header timestamp − manifest t0 (whole seconds, ±1s)", nil +} + +// checkRecords validates the assembled record set against what merge will later +// demand of it: every record through timeline.CheckInteraction, the same guard +// demo's capture endpoint applies, so import cannot persist a record merge +// would refuse. The wrapped timeline entry's own size is checked as each record +// is closed (see coalescer.close), next to the budget that decision rests on. +func checkRecords(records []timeline.Interaction, t0 int64) error { + for i, rec := range records { + line, err := encodeRecord(rec) + if err != nil { + return fmt.Errorf("record %d: %w", i+1, err) + } + if err := timeline.CheckInteraction(line, t0); err != nil { + return fmt.Errorf("record %d %v", i+1, err) + } + } + return nil +} + +// maxDropCodes bounds how many distinct event codes the drop tally names. A +// code is a single character in both formats, but a crafted cast can carry a +// different one on every line, which would otherwise grow the tally — and the +// line it prints — in step with the file. +const maxDropCodes = 16 + +// dropTally counts the events this run did not import, by code. +// +// The bound above is on the map's growth, and the overflow is a plain counter +// rather than an entry under a reserved key: "" is a legitimate event code a +// cast can carry, so a sentinel key would report a real empty-coded event as +// overflow, and overflow as a real event. +type dropTally struct { + byCode map[string]int + overflow int // events whose code arrived past the bound +} + +// add counts one dropped event. +// +// The input count is exempt from the bound. It is the one code whose count is a +// privacy disclosure rather than a scoping note — it is how an operator learns +// their recorder captured keystrokes — and a cast carrying sixteen junk codes +// before its first `i` would otherwise swallow that disclosure into the +// anonymous overflow and never print it. Exempting it cannot unbound the tally: +// it is a single fixed key, so the map holds at most maxDropCodes+1 entries. +func (d *dropTally) add(code string) { + if d.byCode == nil { + d.byCode = make(map[string]int, maxDropCodes) + } + if _, seen := d.byCode[code]; !seen && code != codeInput && len(d.byCode) >= maxDropCodes { + d.overflow++ + return + } + d.byCode[code]++ +} + +// report prints what this run did not import. Input events are named on their +// own line because their drop is a privacy guarantee rather than a scoping +// decision, and the count tells an operator their recorder captured keystrokes. +func (d *dropTally) report(log io.Writer, blank int) { + if n := d.byCode[codeInput]; n > 0 { + fmt.Fprintf(log, "dropped %d input (i) event(s): keystrokes are never imported\n", n) + } + // Ordered by code, so the line is deterministic whatever order the map + // iterates in — and ordered by the thing the operator reads it by, rather + // than by the formatted string, whose leading count would otherwise sort + // "10 resize" before "2 exit". + codes := make([]string, 0, len(d.byCode)) + total := d.overflow + for code, n := range d.byCode { + if code == codeInput { + continue + } + total += n + codes = append(codes, code) + } + if total > 0 { + sort.Strings(codes) + others := make([]string, 0, len(codes)+1) + for _, code := range codes { + // The code is echoed from the cast, so it is neutralised and clipped: it + // is attacker-authorable and this line reaches a terminal. + others = append(others, fmt.Sprintf("%d %s (%s)", d.byCode[code], codeName(code), clip(session.SafeText(code), 8))) + } + if d.overflow > 0 { + others = append(others, fmt.Sprintf("%d under further codes", d.overflow)) + } + fmt.Fprintf(log, "dropped %d other event(s): %s\n", total, strings.Join(others, ", ")) + } + if blank > 0 { + fmt.Fprintf(log, "dropped %d record(s) that render empty\n", blank) + } +} + +// sameFile reports whether a and b resolve to the same on-disk file, so a +// -cast flag pointing at the session's own terminal.cast is treated as the +// in-place case rather than copying the archive onto itself. It is +// transcribe.sameFile's twin; the two packages share no code because the +// helper is three lines of os.Stat and copying it costs less than exporting a +// filesystem predicate from an ASR package. +func sameFile(a, b string) bool { + fa, err := os.Stat(a) + if err != nil { + return false + } + fb, err := os.Stat(b) + if err != nil { + return false + } + return os.SameFile(fa, fb) +} diff --git a/internal/cast/cast_test.go b/internal/cast/cast_test.go new file mode 100644 index 0000000..539e51e --- /dev/null +++ b/internal/cast/cast_test.go @@ -0,0 +1,1179 @@ +package cast + +import ( + "bytes" + "crypto/sha256" + "encoding/json" + "fmt" + "math" + "os" + "path/filepath" + "strings" + "syscall" + "testing" + + "github.com/REPPL/Testimony/internal/report" + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" +) + +// The fixture pair describes one recording in both formats: the same header +// timestamp, event codes, and absolute times, with v3's intervals as the +// differences. Its header timestamp sits two seconds before testT0, so the +// session-relative clock starts negative and crosses zero. +const ( + fixtureV2 = "v2.cast" + fixtureV3 = "v3.cast" + + // The tie pair is the same recording in both formats too, built so that every + // way a float64 clock could split the two formats apart is exercised: seven + // 1.5 ms keystrokes put the seventh at exactly 10.5 ms — a half-millisecond + // tie that rounds one way from a stated absolute time and the other from a + // running sum — and the eighth event sits 249.5 ms after it, so a + // one-millisecond disagreement falls on either side of the 250 ms coalescing + // gap. One record, or two. + fixtureV2Ties = "v2-ties.cast" + fixtureV3Ties = "v3-ties.cast" +) + +// newSession makes a hermetic session directory holding just a manifest. +func newSession(t *testing.T, t0 int64) string { + t.Helper() + dir := t.TempDir() + m := session.Manifest{Session: "cast-test", App: "a shell", Participant: "P1", T0EpochMS: t0} + if t0 == 0 { + // SaveManifest writes whatever it is given; an absent anchor is the point + // of the caller's case. + m.T0EpochMS = 0 + } + if err := session.SaveManifest(dir, m); err != nil { + t.Fatalf("SaveManifest: %v", err) + } + return dir +} + +// snapshot hashes every file in dir, so a refusal can be shown to have left the +// session byte-for-byte as it found it. +func snapshot(t *testing.T, dir string) map[string]string { + t.Helper() + out := map[string]string{} + entries, err := os.ReadDir(dir) + if err != nil { + t.Fatalf("ReadDir: %v", err) + } + for _, e := range entries { + path := filepath.Join(dir, e.Name()) + // Lstat, and only regular files are read: a FIFO a test planted in the + // session would block os.ReadFile in open(2) for ever, and a symlink's own + // target is not part of the session. + fi, err := os.Lstat(path) + if err != nil || !fi.Mode().IsRegular() { + out[e.Name()] = fmt.Sprintf("non-regular %v", e.Type()) + continue + } + b, err := os.ReadFile(path) + if err != nil { + out[e.Name()] = "unreadable" + continue + } + out[e.Name()] = fmt.Sprintf("%x", sha256.Sum256(b)) + } + return out +} + +func assertSessionUnchanged(t *testing.T, dir string, before map[string]string) { + t.Helper() + after := snapshot(t, dir) + if len(before) != len(after) { + t.Fatalf("session file set changed: %v -> %v", keys(before), keys(after)) + } + for name, sum := range before { + if after[name] != sum { + t.Errorf("%s changed: %s -> %s", name, sum, after[name]) + } + } +} + +func keys(m map[string]string) []string { + out := make([]string, 0, len(m)) + for k := range m { + out = append(out, k) + } + return out +} + +func fixture(t *testing.T, name string) string { + t.Helper() + return filepath.Join("testdata", name) +} + +func mustRun(t *testing.T, opts Options) (int, string) { + t.Helper() + var log bytes.Buffer + opts.Log = &log + n, err := Run(opts) + if err != nil { + t.Fatalf("Run: %v", err) + } + return n, log.String() +} + +func readLines(t *testing.T, path string) []string { + t.Helper() + b, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + s := strings.TrimSuffix(string(b), "\n") + if s == "" { + return nil + } + return strings.Split(s, "\n") +} + +func decodeRecords(t *testing.T, path string) []timeline.Interaction { + t.Helper() + var out []timeline.Interaction + for i, line := range readLines(t, path) { + var rec timeline.Interaction + if err := json.Unmarshal([]byte(line), &rec); err != nil { + t.Fatalf("%s:%d: %v", path, i+1, err) + } + out = append(out, rec) + } + return out +} + +// TestRunImportsFixture is the happy path: the records written, the return +// value, the printed lines, and the archived cast. +func TestRunImportsFixture(t *testing.T) { + dir := newSession(t, testT0) + n, log := mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + if n != 5 { + t.Fatalf("imported %d records, want 5", n) + } + recs := decodeRecords(t, filepath.Join(dir, session.InteractionsFile)) + if len(recs) != 5 { + t.Fatalf("interactions.jsonl holds %d records, want 5", len(recs)) + } + // The header timestamp is two seconds before t0, so the cast's first record + // lands two seconds before the session clock's zero. + wantRel := []int64{-2000, -1480, -498, 0, 1000} + for i, rec := range recs { + if got := rec.T - testT0; got != wantRel[i] { + t.Errorf("record %d: session-relative t = %d ms, want %d", i+1, got, wantRel[i]) + } + if rec.Kind != OutputKind { + t.Errorf("record %d: kind = %q, want %q", i+1, rec.Kind, OutputKind) + } + } + if got := recs[1].Text; got != "ls --color\r\n" { + t.Errorf("the echoed command coalesced to %q, want %q", got, "ls --color\r\n") + } + if got := recs[3].Text; !strings.Contains(got, "building 0%\rbuilding 50%\rbuilding 100%\r\n") { + t.Errorf("the progress frames coalesced to %q", got) + } + for _, want := range []string{ + "offset: -2.00s (derived: cast header timestamp − manifest t0 (whole seconds, ±1s))", + "dropped 2 input (i) event(s): keystrokes are never imported", + // Ordered by code — m, r, x — not by the formatted string, whose leading + // count would sort "10 resize" above "2 exit". + "dropped 3 other event(s): 1 marker (m), 1 resize (r), 1 exit (x)", + "dropped 1 record(s) that render empty", + } { + if !strings.Contains(log, want) { + t.Errorf("printed output %q does not contain %q", log, want) + } + } +} + +// TestImportMatchesGolden pins the exact records v2.cast produces. +func TestImportMatchesGolden(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + got, err := os.ReadFile(filepath.Join(dir, session.InteractionsFile)) + if err != nil { + t.Fatal(err) + } + want, err := os.ReadFile(fixture(t, "golden.interactions.jsonl")) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(got, want) { + t.Errorf("interactions.jsonl does not match the golden file\n got: %s\nwant: %s", got, want) + } +} + +// TestV2AndV3Agree is criterion 2: the operator never states, or needs to know, +// which format their recorder wrote. +// +// The tie pair is the case a float64 clock could not hold. v2 rounds a stated +// absolute time; v3 rounds a running sum of intervals; at a half-millisecond tie +// the two land on different milliseconds, and one millisecond is enough to flip +// a coalescing cut. Carrying the clock on the microsecond grain is what makes +// the two byte-identical, and this pair is what fails if that goes. +func TestV2AndV3Agree(t *testing.T) { + pairs := []struct { + name string + v2, v3 string + wantRecords int + }{ + {"the fixture recording", fixtureV2, fixtureV3, 5}, + {"half-millisecond ties", fixtureV2Ties, fixtureV3Ties, 1}, + } + for _, pair := range pairs { + t.Run(pair.name, func(t *testing.T) { + var out [2][]byte + for i, name := range []string{pair.v2, pair.v3} { + dir := newSession(t, testT0) + n, _ := mustRun(t, Options{SessionDir: dir, Cast: fixture(t, name)}) + if n != pair.wantRecords { + t.Errorf("%s produced %d records, want %d", name, n, pair.wantRecords) + } + b, err := os.ReadFile(filepath.Join(dir, session.InteractionsFile)) + if err != nil { + t.Fatal(err) + } + out[i] = b + } + if !bytes.Equal(out[0], out[1]) { + t.Errorf("v2 and v3 of the same recording produced different records\n v2: %s\n v3: %s", out[0], out[1]) + } + }) + } +} + +// TestTiesCoalesceIntoOneRecord states what the tie pair is supposed to produce, +// so the agreement test above cannot be satisfied by both formats being wrong in +// the same way. +func TestTiesCoalesceIntoOneRecord(t *testing.T) { + for _, name := range []string{fixtureV2Ties, fixtureV3Ties} { + t.Run(name, func(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, name)}) + recs := decodeRecords(t, filepath.Join(dir, session.InteractionsFile)) + if len(recs) != 1 { + t.Fatalf("got %d records, want 1: %+v", len(recs), recs) + } + if got := recs[0].Text; got != "abcdefgh\r\n" { + t.Errorf("record text = %q, want the whole line", got) + } + // A record's time is its FIRST rune's: 1.5 ms on the recording clock, + // which rounds to 2, placed by the header's -2000 ms offset. + if got, want := recs[0].T-testT0, int64(-1998); got != want { + t.Errorf("session-relative t = %d ms, want %d", got, want) + } + }) + } +} + +// TestUnsupportedVersionRefuses is criterion 3: a message naming the file and +// the version found, and nothing written. +func TestUnsupportedVersionRefuses(t *testing.T) { + cases := []struct { + name string + header string + want string + }{ + {"version 1", `{"version":1,"timestamp":1784300398}`, "asciicast version 1 is not supported"}, + {"version as a string", `{"version":"2"}`, "asciicast version \"2\" is not supported"}, + {"version absent", `{"timestamp":1784300398}`, "carries no \"version\" field"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + dir := newSession(t, testT0) + castPath := filepath.Join(t.TempDir(), "session.cast") + if err := os.WriteFile(castPath, []byte(tc.header+"\n[0,\"o\",\"x\\r\\n\"]\n"), 0o644); err != nil { + t.Fatal(err) + } + before := snapshot(t, dir) + _, err := Run(Options{SessionDir: dir, Cast: castPath, Log: &bytes.Buffer{}}) + if err == nil { + t.Fatal("want a refusal, got nil") + } + if !strings.Contains(err.Error(), tc.want) { + t.Errorf("error %q does not contain %q", err, tc.want) + } + if !strings.Contains(err.Error(), "session.cast") { + t.Errorf("error %q does not name the file", err) + } + assertSessionUnchanged(t, dir, before) + }) + } +} + +// TestVersion4FixtureRefuses uses the committed fixture, so the refusal is +// exercised against a file on disk as well as against a generated header. +func TestVersion4FixtureRefuses(t *testing.T) { + dir := newSession(t, testT0) + before := snapshot(t, dir) + _, err := Run(Options{SessionDir: dir, Cast: fixture(t, "bad-version.cast"), Log: &bytes.Buffer{}}) + if err == nil || !strings.Contains(err.Error(), "asciicast version 4 is not supported") { + t.Fatalf("error = %v, want an unsupported-version refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +func TestRefusalsLeaveTheSessionUnchanged(t *testing.T) { + cases := []struct { + name string + // setup prepares the session and returns the Options to run. + setup func(t *testing.T) (string, Options) + want string + }{ + { + name: "malformed header", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: fixture(t, "bad-header.cast")} + }, + want: "not an asciicast header", + }, + { + name: "malformed event", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: fixture(t, "bad-event.cast")} + }, + want: "malformed asciicast event", + }, + { + name: "decreasing v2 time", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: fixture(t, "decreasing.cast")} + }, + want: "must not decrease", + }, + { + name: "negative v3 interval", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: fixture(t, "negative-interval.cast")} + }, + want: "must not be negative", + }, + { + name: "no manifest", + setup: func(t *testing.T) (string, Options) { + dir := t.TempDir() + return dir, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)} + }, + want: "load manifest", + }, + { + name: "manifest with no usable t0", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, 0) + return dir, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)} + }, + want: "anchoring the terminal cast", + }, + { + name: "manifest with a negative t0", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, -1) + return dir, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)} + }, + want: "anchoring the terminal cast", + }, + { + name: "non-positive header timestamp", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: writeCast(t, `{"version":2,"timestamp":0}`, `[0,"o","x\r\n"]`)} + }, + want: "is not a recording instant", + }, + { + name: "negative header timestamp", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: writeCast(t, `{"version":2,"timestamp":-5}`, `[0,"o","x\r\n"]`)} + }, + want: "is not a recording instant", + }, + { + name: "implausible derived offset", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: writeCast(t, `{"version":2,"timestamp":9000000000000}`, `[0,"o","x\r\n"]`)} + }, + want: "derived cast offset", + }, + { + name: "no output events", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: writeCast(t, `{"version":2,"timestamp":1784300398}`, `[0,"i","l"]`, `[0.5,"r","80x24"]`)} + }, + want: "holds no output events", + }, + { + // Named apart from the no-output-events case above: this cast really is + // a terminal recording, it just displayed nothing legible. + name: "output events that all render empty", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: writeCast(t, + `{"version":2,"timestamp":1784300398}`, `[0,"o","\r\n"]`, `[0.5,"o","\r\n"]`)} + }, + want: "holds no importable output (2 record(s) rendered empty)", + }, + { + name: "neither -cast nor terminal.cast", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir} + }, + want: "and no -cast given", + }, + { + name: "-cast naming a missing file", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: filepath.Join(t.TempDir(), "absent.cast")} + }, + want: "cast file:", + }, + { + name: "-cast naming a directory", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + return dir, Options{SessionDir: dir, Cast: t.TempDir()} + }, + want: "it is not a regular file", + }, + { + name: "a symlink at terminal.cast", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + target := filepath.Join(t.TempDir(), "elsewhere.cast") + if err := os.WriteFile(target, []byte("{}\n"), 0o644); err != nil { + t.Fatal(err) + } + if err := os.Symlink(target, filepath.Join(dir, session.TerminalCastFile)); err != nil { + t.Fatal(err) + } + return dir, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)} + }, + want: "it is a symlink", + }, + { + name: "a symlink at interactions.jsonl", + setup: func(t *testing.T) (string, Options) { + dir := newSession(t, testT0) + target := filepath.Join(t.TempDir(), "elsewhere.jsonl") + if err := os.WriteFile(target, nil, 0o644); err != nil { + t.Fatal(err) + } + if err := os.Symlink(target, filepath.Join(dir, session.InteractionsFile)); err != nil { + t.Fatal(err) + } + return dir, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)} + }, + want: "it is a symlink", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + dir, opts := tc.setup(t) + opts.Log = &bytes.Buffer{} + before := snapshot(t, dir) + if _, err := Run(opts); err == nil { + t.Fatal("want a refusal, got nil") + } else if !strings.Contains(err.Error(), tc.want) { + t.Fatalf("error %q does not contain %q", err, tc.want) + } + assertSessionUnchanged(t, dir, before) + }) + } +} + +// writeCast writes a cast made of the given lines into a fresh temp directory +// and returns its path. +func writeCast(t *testing.T, lines ...string) string { + t.Helper() + path := filepath.Join(t.TempDir(), "session.cast") + if err := os.WriteFile(path, []byte(strings.Join(lines, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + return path +} + +func TestRefusesFIFOAtTerminalCast(t *testing.T) { + dir := newSession(t, testT0) + if err := syscall.Mkfifo(filepath.Join(dir, session.TerminalCastFile), 0o644); err != nil { + t.Skipf("FIFOs unavailable on this platform: %v", err) + } + before := snapshot(t, dir) + done := make(chan error, 1) + go func() { + _, err := Run(Options{SessionDir: dir, Cast: fixture(t, fixtureV2), Log: &bytes.Buffer{}}) + done <- err + }() + err := <-done + if err == nil || !strings.Contains(err.Error(), "not a regular file") { + t.Fatalf("error = %v, want a non-regular-file refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +func TestRefusesFIFOAsTheCastInput(t *testing.T) { + dir := newSession(t, testT0) + fifo := filepath.Join(t.TempDir(), "session.cast") + if err := syscall.Mkfifo(fifo, 0o644); err != nil { + t.Skipf("FIFOs unavailable on this platform: %v", err) + } + before := snapshot(t, dir) + done := make(chan error, 1) + go func() { + _, err := Run(Options{SessionDir: dir, Cast: fifo, Log: &bytes.Buffer{}}) + done <- err + }() + err := <-done + if err == nil || !strings.Contains(err.Error(), "not a regular file") { + t.Fatalf("error = %v, want a non-regular-file refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +func TestOffsetProvenance(t *testing.T) { + cases := []struct { + name string + header string + offset float64 + offsetSet bool + t0 int64 + wantMS int64 + wantProv string + wantErr string + }{ + { + name: "explicit -offset wins over the header", + header: `{"version":2,"timestamp":1784300398}`, offset: -12.4, offsetSet: true, t0: testT0, + wantMS: -12400, wantProv: "from -offset flag", + }, + { + name: "explicit -offset rounds to the millisecond", + header: `{"version":2}`, offset: 1.23456, offsetSet: true, t0: testT0, + wantMS: 1235, wantProv: "from -offset flag", + }, + { + name: "derived from the header timestamp", + header: `{"version":2,"timestamp":1784300398}`, t0: testT0, + wantMS: -2000, wantProv: "derived: cast header timestamp − manifest t0 (whole seconds, ±1s)", + }, + { + name: "no header timestamp defaults to zero", + header: `{"version":3}`, t0: testT0, + wantMS: 0, wantProv: "default 0: cast header carries no timestamp", + }, + { + name: "a null header timestamp counts as absent", + header: `{"version":3,"timestamp":null}`, t0: testT0, + wantMS: 0, wantProv: "default 0: cast header carries no timestamp", + }, + { + name: "an unusable t0 refuses even with an explicit offset", + header: `{"version":2,"timestamp":1784300398}`, offset: 1, offsetSet: true, t0: 0, + wantErr: "anchoring the terminal cast", + }, + { + name: "a non-positive header timestamp refuses", + header: `{"version":2,"timestamp":0}`, t0: testT0, + wantErr: "is not a recording instant", + }, + { + name: "an implausible derived offset refuses", + header: `{"version":2,"timestamp":9000000000000}`, t0: testT0, + wantErr: "derived cast offset", + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + hdr, err := scanCast(strings.NewReader(tc.header+"\n"), "session.cast", nil, nil) + if err != nil { + t.Fatalf("scanCast: %v", err) + } + man := session.Manifest{Session: "s", T0EpochMS: tc.t0} + opts := Options{Offset: tc.offset, OffsetSet: tc.offsetSet} + gotMS, gotProv, err := resolveOffset("session.cast", opts, man, hdr) + if tc.wantErr != "" { + if err == nil || !strings.Contains(err.Error(), tc.wantErr) { + t.Fatalf("error = %v, want one containing %q", err, tc.wantErr) + } + return + } + if err != nil { + t.Fatalf("resolveOffset: %v", err) + } + if gotMS != tc.wantMS { + t.Errorf("offsetMS = %d, want %d", gotMS, tc.wantMS) + } + // The three provenance strings are a documented contract, printed + // verbatim for the operator. + if gotProv != tc.wantProv { + t.Errorf("provenance = %q, want %q", gotProv, tc.wantProv) + } + }) + } +} + +func TestRunRefusesNonFiniteOffset(t *testing.T) { + dir := newSession(t, testT0) + before := snapshot(t, dir) + _, err := Run(Options{ + SessionDir: dir, Cast: fixture(t, fixtureV2), + Offset: math.Inf(1), OffsetSet: true, Log: &bytes.Buffer{}, + }) + if err == nil || !strings.Contains(err.Error(), "-offset must be a finite number") { + t.Fatalf("error = %v, want transcribe.CheckOffset's refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +// TestReimportIsIdempotent: the second run drops exactly what the first wrote +// and writes exactly the same records back. +func TestReimportIsIdempotent(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + path := filepath.Join(dir, session.InteractionsFile) + first, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + _, log := mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + second, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(first, second) { + t.Errorf("a re-import changed interactions.jsonl\nfirst: %s\nsecond: %s", first, second) + } + if !strings.Contains(log, "replaced 5 terminal_output record(s) from an earlier import") { + t.Errorf("printed output %q does not report the replaced records", log) + } +} + +// TestForeignRecordsPreserved: a -demo session's clicks and inputs survive an +// import byte-for-byte, in their original order, and so does a line this +// importer cannot decode at all. +func TestForeignRecordsPreserved(t *testing.T) { + dir := newSession(t, testT0) + foreign := []string{ + `{"t":1784300419200,"kind":"click","selector":"[data-testid=save-btn]","text":"Save","route":"#general"}`, + `{"t":1784300421000,"kind":"input","selector":"[data-testid=name]","value":"Alice"}`, + `{"kind":"terminal_output","t":1,"text":"from an earlier import"}`, + `not json at all`, + ``, + `{"t":1784300422000,"kind":"click","unknown_field":{"kept":true}}`, + } + path := filepath.Join(dir, session.InteractionsFile) + if err := os.WriteFile(path, []byte(strings.Join(foreign, "\n")+"\n"), 0o644); err != nil { + t.Fatal(err) + } + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + + got := readLines(t, path) + wantKept := []string{foreign[0], foreign[1], foreign[3], foreign[4], foreign[5]} + if len(got) != len(wantKept)+5 { + t.Fatalf("interactions.jsonl holds %d lines, want %d", len(got), len(wantKept)+5) + } + for i, want := range wantKept { + if got[i] != want { + t.Errorf("line %d = %q, want %q byte-for-byte", i+1, got[i], want) + } + } + for i, line := range got[len(wantKept):] { + if !strings.Contains(line, `"kind":"`+OutputKind+`"`) { + t.Errorf("appended line %d is not a terminal record: %q", i+1, line) + } + } +} + +func TestImportPreservesInteractionsFileMode(t *testing.T) { + dir := newSession(t, testT0) + path := filepath.Join(dir, session.InteractionsFile) + if err := os.WriteFile(path, []byte("{\"t\":1784300419200,\"kind\":\"click\"}\n"), 0o600); err != nil { + t.Fatal(err) + } + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + fi, err := os.Stat(path) + if err != nil { + t.Fatal(err) + } + if got := fi.Mode().Perm(); got != 0o600 { + t.Errorf("interactions.jsonl mode = %04o, want 0600 preserved", got) + } +} + +// TestCastArchivedVerbatim is where the byte-for-byte claim is made: the +// archived cast, not the decoded records. +func TestCastArchivedVerbatim(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + got, err := os.ReadFile(filepath.Join(dir, session.TerminalCastFile)) + if err != nil { + t.Fatal(err) + } + want, err := os.ReadFile(fixture(t, fixtureV2)) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(got, want) { + t.Errorf("terminal.cast is not byte-identical to the imported file") + } + fi, err := os.Stat(filepath.Join(dir, session.TerminalCastFile)) + if err != nil { + t.Fatal(err) + } + if fi.Mode().Perm()&0o400 == 0 { + t.Errorf("terminal.cast mode = %04o, want at least owner-readable", fi.Mode().Perm()) + } +} + +func TestArchivePreservesExistingMode(t *testing.T) { + dir := newSession(t, testT0) + archive := filepath.Join(dir, session.TerminalCastFile) + if err := os.WriteFile(archive, []byte("{\"version\":2}\n"), 0o600); err != nil { + t.Fatal(err) + } + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + fi, err := os.Stat(archive) + if err != nil { + t.Fatal(err) + } + if got := fi.Mode().Perm(); got != 0o600 { + t.Errorf("terminal.cast mode = %04o, want 0600 preserved", got) + } +} + +// TestInPlaceReimport: -cast omitted reuses the session's own terminal.cast, so +// a corrected -offset is a one-liner. +func TestInPlaceReimport(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + archive := filepath.Join(dir, session.TerminalCastFile) + before, err := os.ReadFile(archive) + if err != nil { + t.Fatal(err) + } + + n, log := mustRun(t, Options{SessionDir: dir, Offset: -12.4, OffsetSet: true}) + if n != 5 { + t.Fatalf("imported %d records, want 5", n) + } + if !strings.Contains(log, "offset: -12.40s (from -offset flag)") { + t.Errorf("printed output %q does not carry the explicit offset", log) + } + after, err := os.ReadFile(archive) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(before, after) { + t.Errorf("the in-place re-import rewrote terminal.cast") + } + recs := decodeRecords(t, filepath.Join(dir, session.InteractionsFile)) + if got := recs[0].T - testT0; got != -12400 { + t.Errorf("first record's session-relative t = %d ms, want -12400", got) + } +} + +// TestSameFileCastIsInPlace: -cast pointing at the session's own terminal.cast +// is the omitted case, so the archive is never copied onto itself. +func TestSameFileCastIsInPlace(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + archive := filepath.Join(dir, session.TerminalCastFile) + inPlace, log := mustRun(t, Options{SessionDir: dir, Cast: archive}) + if inPlace != 5 { + t.Fatalf("imported %d records, want 5", inPlace) + } + if !strings.Contains(log, "derived: cast header timestamp") { + t.Errorf("printed output %q does not re-derive the offset from the archived header", log) + } + entries, err := os.ReadDir(dir) + if err != nil { + t.Fatal(err) + } + for _, e := range entries { + if strings.HasPrefix(e.Name(), "."+session.TerminalCastFile) { + t.Errorf("a staging temp file was left behind: %s", e.Name()) + } + } +} + +// TestInputEventsDropped: keystrokes never reach a record, and the count says +// the recorder captured them. +func TestInputEventsDropped(t *testing.T) { + dir := newSession(t, testT0) + castPath := writeCast(t, + `{"version":2,"timestamp":1784300398}`, + `[0,"i","s"]`, + `[0.1,"i","e"]`, + `[0.2,"i","c"]`, + `[0.3,"i","r"]`, + `[0.4,"i","e"]`, + `[0.5,"i","t"]`, + `[0.6,"o","visible output\r\n"]`, + ) + n, log := mustRun(t, Options{SessionDir: dir, Cast: castPath}) + if n != 1 { + t.Fatalf("imported %d records, want 1", n) + } + recs := decodeRecords(t, filepath.Join(dir, session.InteractionsFile)) + for _, rec := range recs { + for _, key := range []string{"s", "e", "c", "r", "t"} { + if rec.Text == key { + t.Errorf("a keystroke reached a record: %q", rec.Text) + } + } + } + if got := recs[0].Text; got != "visible output\r\n" { + t.Errorf("record text = %q, want the output only", got) + } + if !strings.Contains(log, "dropped 6 input (i) event(s)") { + t.Errorf("printed output %q does not report the dropped keystrokes", log) + } +} + +// TestOversizedEventSplits is criterion 7 at the package boundary. +func TestOversizedEventSplits(t *testing.T) { + dir := newSession(t, testT0) + // 6 MiB raw, not the 12 MiB the coalescer unit uses: escaping inflates the + // records by about a third, and the assembled interactions.jsonl has its own + // 16 MiB cap, so a larger event would be refused for the file's size rather + // than split. 6 MiB is comfortably past the 4 MiB line limit, which is what + // the split needs. + unit := "abcé中\x1b[0;34m" + data := strings.Repeat(unit, 6<<20/len(unit)) + encoded, err := json.Marshal(data) + if err != nil { + t.Fatal(err) + } + castPath := writeCast(t, `{"version":2,"timestamp":1784300398}`, `[0,"o",`+string(encoded)+`]`) + + n, _ := mustRun(t, Options{SessionDir: dir, Cast: castPath}) + if n < 2 { + t.Fatalf("imported %d records, want more than one", n) + } + recs := decodeRecords(t, filepath.Join(dir, session.InteractionsFile)) + var joined strings.Builder + for i, rec := range recs { + entryLen, err := session.EncodedLen(timeline.EventEntry(rec, testT0)) + if err != nil { + t.Fatal(err) + } + if entryLen > session.MaxJSONLLine { + t.Errorf("record %d's timeline entry is %d bytes, over the %d-byte limit", i+1, entryLen, session.MaxJSONLLine) + } + joined.WriteString(rec.Text) + } + if joined.String() != data { + t.Errorf("the records do not reproduce the event's data (%d bytes read back, %d written)", joined.Len(), len(data)) + } + // The records are still mergeable, which is the whole point of the budget. + if _, _, err := timeline.Merge(dir); err != nil { + t.Errorf("merge refused the split records: %v", err) + } +} + +// TestImportedRecordsPassCheckInteraction runs every fixture's records through +// the guard merge applies, so import cannot persist a record merge refuses. +func TestImportedRecordsPassCheckInteraction(t *testing.T) { + for _, name := range []string{fixtureV2, fixtureV3, "v3-nots.cast"} { + t.Run(name, func(t *testing.T) { + dir := newSession(t, testT0) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, name)}) + for i, line := range readLines(t, filepath.Join(dir, session.InteractionsFile)) { + if err := timeline.CheckInteraction([]byte(line), testT0); err != nil { + t.Errorf("record %d %v", i+1, err) + } + } + }) + } +} + +func TestInteractionsFileSizeLimitRefuses(t *testing.T) { + dir := newSession(t, testT0) + // A pre-existing file just under the cap, so the import's own records are + // what push the assembly over it. + line := `{"t":1784300419200,"kind":"click","text":"` + strings.Repeat("x", 1000) + `"}` + var b strings.Builder + for b.Len() < session.MaxJSONLBytes-2000 { + b.WriteString(line) + b.WriteString("\n") + } + if err := os.WriteFile(filepath.Join(dir, session.InteractionsFile), []byte(b.String()), 0o644); err != nil { + t.Fatal(err) + } + big := strings.Repeat("y", 4000) + "\r\n" + encoded, err := json.Marshal(big) + if err != nil { + t.Fatal(err) + } + castPath := writeCast(t, `{"version":2,"timestamp":1784300398}`, `[0,"o",`+string(encoded)+`]`) + before := snapshot(t, dir) + _, err = Run(Options{SessionDir: dir, Cast: castPath, Log: &bytes.Buffer{}}) + if err == nil || !strings.Contains(err.Error(), "past its") { + t.Fatalf("error = %v, want a size refusal", err) + } + if !strings.Contains(err.Error(), session.InteractionsFile) { + t.Errorf("error %q does not name interactions.jsonl", err) + } + assertSessionUnchanged(t, dir, before) +} + +// TestMergedTimelineSizeLimitRefuses is the case only an offline importer can +// measure: interactions.jsonl itself stays under the cap while the timeline the +// import implies would not. +func TestMergedTimelineSizeLimitRefuses(t *testing.T) { + dir := newSession(t, testT0) + // Speech entries take up most of the merged budget. They live in + // transcript.jsonl, which does not count towards interactions.jsonl's own cap. + var utts []timeline.Utterance + text := strings.Repeat("z", 4000) + for i := 0; len(utts) < 4000; i++ { + utts = append(utts, timeline.Utterance{ + ID: fmt.Sprintf("utt-%03d", i+1), T0: float64(i), T1: float64(i) + 1, Speaker: "P1", Text: text, + }) + } + if err := session.WriteJSONL(filepath.Join(dir, session.TranscriptFile), utts); err != nil { + t.Skipf("transcript fixture over the writer's own cap: %v", err) + } + big := strings.Repeat("y", 900_000) + "\r\n" + encoded, err := json.Marshal(big) + if err != nil { + t.Fatal(err) + } + castPath := writeCast(t, `{"version":2,"timestamp":1784300398}`, `[0,"o",`+string(encoded)+`]`) + before := snapshot(t, dir) + _, err = Run(Options{SessionDir: dir, Cast: castPath, Log: &bytes.Buffer{}}) + if err == nil || !strings.Contains(err.Error(), "merged "+session.TimelineFile) { + t.Fatalf("error = %v, want a merged-timeline size refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +func TestRefusesOversizedExistingInteractions(t *testing.T) { + dir := newSession(t, testT0) + var b strings.Builder + line := `{"t":1784300419200,"kind":"click","text":"` + strings.Repeat("x", 1000) + `"}` + for b.Len() <= session.MaxJSONLBytes { + b.WriteString(line) + b.WriteString("\n") + } + if err := os.WriteFile(filepath.Join(dir, session.InteractionsFile), []byte(b.String()), 0o644); err != nil { + t.Fatal(err) + } + before := snapshot(t, dir) + _, err := Run(Options{SessionDir: dir, Cast: fixture(t, fixtureV2), Log: &bytes.Buffer{}}) + if err == nil || !strings.Contains(err.Error(), "refusing to read") { + t.Fatalf("error = %v, want a read refusal", err) + } + assertSessionUnchanged(t, dir, before) +} + +// --- Integration: import, merge, report --- + +// writeTranscript seeds a two-utterance transcript spanning the fixture's +// records, so the join can be observed. +func writeTranscript(t *testing.T, dir string) { + t.Helper() + utts := []timeline.Utterance{ + {ID: "utt-001", T0: -1.5, T1: -0.5, Speaker: "P1", Text: "Right, let me list the project files."}, + {ID: "utt-002", T0: 0.5, T1: 2, Speaker: "P1", Text: "The colours make that hard to read."}, + } + if err := session.WriteJSONL(filepath.Join(dir, session.TranscriptFile), utts); err != nil { + t.Fatalf("WriteJSONL: %v", err) + } +} + +// TestImportThenMergeInterleaves is criterion 1: one interleaved clock from the +// same t0, with no separate clock for the terminal stream. +func TestImportThenMergeInterleaves(t *testing.T) { + for _, name := range []string{fixtureV2, fixtureV3} { + t.Run(name, func(t *testing.T) { + dir := newSession(t, testT0) + writeTranscript(t, dir) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, name)}) + speech, events, err := timeline.Merge(dir) + if err != nil { + t.Fatalf("Merge: %v", err) + } + if speech != 2 || events != 5 { + t.Fatalf("merged %d speech and %d events, want 2 and 5", speech, events) + } + entries, err := timeline.ReadEntries(filepath.Join(dir, session.TimelineFile)) + if err != nil { + t.Fatalf("ReadEntries: %v", err) + } + type want struct { + t float64 + src string + } + wants := []want{ + {-2, "event"}, // the first prompt, two seconds before t0 + {-1.5, "speech"}, // Alice starts speaking + {-1.48, "event"}, // the echoed command + {-0.498, "event"}, + {0, "event"}, + {0.5, "speech"}, + {1, "event"}, + } + if len(entries) != len(wants) { + t.Fatalf("timeline holds %d entries, want %d", len(entries), len(wants)) + } + for i, w := range wants { + if entries[i].T != w.t || entries[i].Src != w.src { + t.Errorf("entry %d = (t %g, src %s), want (t %g, src %s)", i+1, entries[i].T, entries[i].Src, w.t, w.src) + } + } + }) + } +} + +// TestReportRendersTerminalEvents is criterion 4: the existing event rendering, +// unchanged. +func TestReportRendersTerminalEvents(t *testing.T) { + dir := newSession(t, testT0) + writeTranscript(t, dir) + mustRun(t, Options{SessionDir: dir, Cast: fixture(t, fixtureV2)}) + if _, _, err := timeline.Merge(dir); err != nil { + t.Fatalf("Merge: %v", err) + } + md, err := report.Render(dir, 2.5) + if err != nil { + t.Fatalf("Render: %v", err) + } + for _, want := range []string{ + "**[-00:02] P1:** ", + // report escapes the underscore for its Markdown sink; the record is + // rendered by the existing eventLine, unchanged. + "- [-00:02] terminal\\_output ", + "[00:00] terminal\\_output ", + "ls --color", + } { + if !strings.Contains(md, want) { + t.Errorf("report does not contain %q\n%s", want, md) + } + } + // SafeText strips the ESC byte, so no raw terminal control sequence reaches + // report.md — the printable CSI tail stays, which is why the how-to asks for + // colour to be disabled at record time. + if strings.ContainsRune(md, 0x1b) { + t.Error("report.md carries a raw ESC byte") + } + if !strings.Contains(md, "0;34m") { + t.Error("report.md does not show the printable CSI residue the docs describe") + } +} + +// TestEarlyStartedCastGoesNegative is criterion 6: an early-started recorder's +// events render exactly as an early-started audio recording's utterances do. +func TestEarlyStartedCastGoesNegative(t *testing.T) { + dir := newSession(t, testT0) + castPath := writeCast(t, + fmt.Sprintf(`{"version":2,"timestamp":%d}`, testT0/1000-30), + `[0,"o","early output\r\n"]`, + `[30,"o","on time\r\n"]`, + ) + mustRun(t, Options{SessionDir: dir, Cast: castPath}) + if _, _, err := timeline.Merge(dir); err != nil { + t.Fatalf("Merge: %v", err) + } + entries, err := timeline.ReadEntries(filepath.Join(dir, session.TimelineFile)) + if err != nil { + t.Fatal(err) + } + if len(entries) != 2 || entries[0].T != -30 || entries[1].T != 0 { + t.Fatalf("entry times = %v, want -30 and 0", entries) + } + md, err := report.Render(dir, 2.5) + if err != nil { + t.Fatalf("Render: %v", err) + } + if !strings.Contains(md, "[-00:30]") { + t.Errorf("report does not render the negative clock\n%s", md) + } +} + +// TestDropTallyIsBounded: a cast carrying a different event code on every line +// must not grow the tally, or the line it prints, in step with the file. +func TestDropTallyIsBounded(t *testing.T) { + dir := newSession(t, testT0) + lines := []string{`{"version":2,"timestamp":1784300398}`} + for i := 0; i < maxDropCodes*4; i++ { + lines = append(lines, fmt.Sprintf(`[%d,"c%d","x"]`, i, i)) + } + lines = append(lines, fmt.Sprintf(`[%d,"o","real output\r\n"]`, maxDropCodes*4)) + n, log := mustRun(t, Options{SessionDir: dir, Cast: writeCast(t, lines...)}) + if n != 1 { + t.Fatalf("imported %d records, want 1", n) + } + if !strings.Contains(log, "under further codes") { + t.Errorf("printed output %q does not report the codes past the bound", log) + } + if got := strings.Count(log, "unrecognised"); got > maxDropCodes { + t.Errorf("printed output names %d codes, over the %d bound", got, maxDropCodes) + } +} + +// TestInputCountSurvivesTheTallyBound is the privacy regression: the dropped- +// keystroke count is a disclosure, not a scoping note, so a cast that fills the +// tally with junk codes before its first `i` event must not swallow that count +// into the anonymous overflow and leave the operator unaware their recorder +// captured input. +func TestInputCountSurvivesTheTallyBound(t *testing.T) { + dir := newSession(t, testT0) + lines := []string{`{"version":2,"timestamp":1784300398}`} + for i := 0; i < maxDropCodes; i++ { + lines = append(lines, fmt.Sprintf(`[%d,"c%d","x"]`, i, i)) + } + lines = append(lines, + fmt.Sprintf(`[%d,"i","s"]`, maxDropCodes), + fmt.Sprintf(`[%d,"i","u"]`, maxDropCodes+1), + fmt.Sprintf(`[%d,"o","real output\r\n"]`, maxDropCodes+2), + ) + _, log := mustRun(t, Options{SessionDir: dir, Cast: writeCast(t, lines...)}) + if want := "dropped 2 input (i) event(s): keystrokes are never imported"; !strings.Contains(log, want) { + t.Errorf("want %q on stderr, got %q", want, log) + } +} + +// TestEmptyEventCodeIsNotOverflow keeps the overflow counter off a reserved map +// key: "" is a legitimate event code a cast can carry, so a sentinel key would +// report a real empty-coded event as overflow and overflow as a real event. +func TestEmptyEventCodeIsNotOverflow(t *testing.T) { + dir := newSession(t, testT0) + _, log := mustRun(t, Options{SessionDir: dir, Cast: writeCast(t, + `{"version":2,"timestamp":1784300398}`, + `[0,"","odd"]`, + `[1,"o","real output\r\n"]`, + )}) + if want := "dropped 1 other event(s): 1 unrecognised ()"; !strings.Contains(log, want) { + t.Errorf("want %q on stderr, got %q", want, log) + } + if strings.Contains(log, "under further codes") { + t.Errorf("an empty event code was reported as overflow: %q", log) + } +} + +// TestDropLineClipsALongCode keeps an attacker-authored code out of the +// operator's terminal at its own length. +func TestDropLineClipsALongCode(t *testing.T) { + dir := newSession(t, testT0) + long := strings.Repeat("z", 4096) + _, log := mustRun(t, Options{SessionDir: dir, Cast: writeCast(t, + `{"version":2,"timestamp":1784300398}`, + `[0,"`+long+`","ignored"]`, + `[1,"o","real output\r\n"]`, + )}) + if strings.Contains(log, strings.Repeat("z", 16)) { + t.Errorf("printed output carries the code at full length: %q", log) + } + if !strings.Contains(log, "…") { + t.Errorf("printed output %q does not mark the clip", log) + } +} diff --git a/internal/cast/coalesce.go b/internal/cast/coalesce.go new file mode 100644 index 0000000..5748b96 --- /dev/null +++ b/internal/cast/coalesce.go @@ -0,0 +1,240 @@ +package cast + +import ( + "fmt" + "strings" + "unicode/utf8" + + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" +) + +// The coalescing boundaries. A shell echoes a typed command back roughly one +// keystroke at a time, so one record per raw output event would fragment a +// single typed command across dozens of near-empty records. These are named +// constants rather than flags: an operator has no way to know what value to +// pass, and the values interact with report's -window, which is already a flag +// on the command that needs it. +const ( + // coalesceGapMS closes a record whose output never ends in a newline: a bare + // prompt, a read prompt, a progress line. 250 ms sits above a human's typical + // inter-keystroke interval (~100-200 ms), so a typed command still coalesces; + // three orders of magnitude above the gaps inside a program's output burst, + // so it never merges across a genuine pause; and an order of magnitude below + // report's 2.5 s join window, so coalescing alone can never pull content + // across a window boundary. + coalesceGapMS = 250 + + // maxCoalesceSpanMS caps how long one record may span, measured from its + // first event. A record states one time — its first event's — so unbounded + // coalescing would attribute minutes of output to a single early instant. One + // second keeps the attribution error well inside report's 2.5 s default + // window, so a record still joins to the utterance spoken over it. + maxCoalesceSpanMS = 1000 +) + +// castEntryIDMargin is demo's eventIDGrowthMargin twin, for the same reason: +// timeline.EventEntry stamps the placeholder id "ev-001", while the real +// ordinal depends on the record's position among every interaction in the +// session, so a measured entry is a lower bound and this margin covers the +// ordinal's growth. 32 spare bytes cover an "ev-%03d" ordinal up to 32 digits. +// (demo's constant is unexported, so the value is restated here with the +// citation rather than reached across the package boundary.) +const castEntryIDMargin = 32 + +// coalescer turns a stream of output events into interaction records: one +// record per line the terminal displayed, closed at the first of a newline, a +// coalesceGapMS inter-event gap, a maxCoalesceSpanMS span, or the JSONL line +// budget. It holds one pending record's text at a time. +// +// A record's t is always the time of the event that contributed its FIRST rune +// — never fabricated, never averaged — so a record cut by any of the four +// boundaries is honestly timed, and continuation records after a split carry +// the time of the event whose data they open with. +type coalescer struct { + t0 int64 // manifest t0, epoch milliseconds + offsetMS int64 // cast→session offset, milliseconds + + records []timeline.Interaction + dropped int // records whose text renders empty + + open bool + text strings.Builder + enc int // JSON-encoded length of the pending text + budget int // encoded-text budget for the pending record + firstMS int64 // cast-clock ms of the event that opened the record + lastMS int64 // cast-clock ms of the most recent event added +} + +// add appends one output event's runes to the pending record, closing and +// opening records at the boundaries above. +func (c *coalescer) add(ev castEvent) error { + ms := microsToMillis(ev.US) + if c.open && (ms-c.lastMS >= coalesceGapMS || ms-c.firstMS >= maxCoalesceSpanMS) { + if err := c.close(); err != nil { + return err + } + } + c.lastMS = ms + for _, r := range ev.Data { + if !c.open { + if err := c.begin(ms); err != nil { + return err + } + } + n := encodedRuneLen(r) + // A rune is appended only if it keeps the running total within budget; + // otherwise the record closes and the rune opens the next one, so a split + // never falls inside a rune and concatenating a split's records reproduces + // the event's decoded data exactly. The pending record must be non-empty + // to split, so a rune that cannot fit even an empty record's budget is + // still appended — and caught by close's measured check — rather than + // looping for ever on a record it can never open. + if c.enc > 0 && c.enc+n > c.budget { + if err := c.close(); err != nil { + return err + } + if err := c.begin(ms); err != nil { + return err + } + } + c.text.WriteRune(r) + c.enc += n + // The newline is included in the record, and the next rune starts a new + // one. This is the primary boundary and the one that makes the stream + // legible: one record per line the terminal displayed. Carriage returns + // are kept but are NOT a boundary — a progress bar emits many \r-separated + // frames for one displayed line, and one record per frame would flood the + // stream. + if r == '\n' { + if err := c.close(); err != nil { + return err + } + } + } + return nil +} + +// flush closes the pending record at end of stream. +func (c *coalescer) flush() error { + if !c.open { + return nil + } + return c.close() +} + +// begin opens a record at the cast-clock instant ms and computes its encoded +// text budget. +// +// The budget is computed per record, from that record's own time, rather than +// once per run: the session-relative t a timeline entry carries varies in +// encoded length across a session (a 15-byte worst case against a 1-byte best +// one), and a record's time is known the moment it opens, so measuring it is +// exact where a single run-wide envelope would have to guess. The probe carries +// a one-rune text because timeline.BuildEntries omits an empty text entirely, +// so an envelope measured without one under-counts by the whole `,"text":""` +// scaffolding. +func (c *coalescer) begin(ms int64) error { + probe := timeline.Interaction{T: c.recordTime(ms), Kind: OutputKind, Text: "x"} + n, err := session.EncodedLen(timeline.EventEntry(probe, c.t0)) + if err != nil { + return err + } + c.open = true + c.firstMS = ms + c.enc = 0 + c.text.Reset() + c.budget = session.MaxJSONLLine - (n - 1) - castEntryIDMargin + return nil +} + +// close finishes the pending record. +func (c *coalescer) close() error { + text := c.text.String() + c.open = false + c.enc = 0 + c.text.Reset() + // A record whose text renders empty — a blank line, a lone carriage return — + // is dropped and counted, so it does not become a timeline bullet showing + // only the word terminal_output. Presence is decided on the rendered form, + // the transcribe.mapSegments rule. The archived terminal.cast holds those + // bytes verbatim, which is what makes this a rendering decision rather than + // a loss of evidence. + if strings.TrimSpace(session.SafeText(text)) == "" { + c.dropped++ + return nil + } + rec := timeline.Interaction{T: c.recordTime(c.firstMS), Kind: OutputKind, Text: text} + // The per-rune budget above is an assumption about another package's + // encoder, so every finished record is additionally measured for real. The + // check is unreachable if the table is right, and it is the difference + // between a wrong table costing a refused run and a wrong table costing a + // session no command can read back. + n, err := session.EncodedLen(timeline.EventEntry(rec, c.t0)) + if err != nil { + return err + } + if n+castEntryIDMargin > session.MaxJSONLLine { + return fmt.Errorf("record %d encodes to a %d-byte timeline entry, over the %d-byte JSONL line limit; this is an importer bug — please report it with the cast that triggered it", + len(c.records)+1, n, session.MaxJSONLLine) + } + c.records = append(c.records, rec) + return nil +} + +// recordTime places a cast-clock instant on the session clock. An interaction's +// t is epoch milliseconds, so the arithmetic is integer throughout. +func (c *coalescer) recordTime(ms int64) int64 { + return c.t0 + c.offsetMS + ms +} + +// microsToMillis rounds a recording-clock instant from the microsecond grain +// scanCast carries to the millisecond an interaction records, half away from +// zero — math.Round's rule, in integers. This is the one place the rounding +// happens, which is what keeps a v2 and a v3 reading of the same recording on +// the same millisecond (see castTimeGrain). A recording clock never runs +// negative, since v2 refuses a decrease from zero and v3 a negative interval, +// but the negative case is handled rather than assumed. +func microsToMillis(us int64) int64 { + if us < 0 { + return -((-us + 500) / 1000) + } + return (us + 500) / 1000 +} + +// encodedRuneLen is the number of bytes r costs once encoding/json has written +// it into a JSON string with SetEscapeHTML(false) — the settings +// session.WriteJSONL and session.EncodedLen both use. Escaping, not raw length, +// is what consumes the line budget: an ESC byte costs six bytes encoded, and +// ANSI-coloured output is dense in them. +// +// The table is pinned against encoding/json by TestEncodedRuneLenMatchesJSON, +// so a stdlib change that invalidates it fails there rather than as a refused +// import. +func encodedRuneLen(r rune) int { + switch { + case r == '"' || r == '\\': + return 2 + case r == '\n' || r == '\r' || r == '\t': + return 2 + case r < 0x20: + // Every other C0 control, ESC (0x1b) included, becomes a six-byte + // u-escape. Backspace and form feed are the one place the table + // deliberately over-counts: encoding/json has written those two as a + // six-byte u-escape in some Go versions and as a two-byte short escape in + // others, and an over-count costs a few unused bytes of a 4 MiB budget + // while an under-count costs a false refusal. + return 6 + case r == 0x2028 || r == 0x2029: + // Escaped unconditionally, whatever SetEscapeHTML says. + return 6 + } + // Everything else is written literally, DEL and <, >, & included (the last + // three only because HTML escaping is off). + if n := utf8.RuneLen(r); n > 0 { + return n + } + // A rune no encoder can write is replaced by U+FFFD. Unreachable from a + // string encoding/json decoded, which is the only source here. + return utf8.RuneLen(utf8.RuneError) +} diff --git a/internal/cast/coalesce_test.go b/internal/cast/coalesce_test.go new file mode 100644 index 0000000..e571b06 --- /dev/null +++ b/internal/cast/coalesce_test.go @@ -0,0 +1,342 @@ +package cast + +import ( + "math" + "strings" + "testing" + "unicode/utf8" + + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" +) + +// testT0 is the anchor every hermetic case in this package uses. It matches the +// t0 the session-directory reference's own examples carry. +const testT0 = 1784300400000 + +// run feeds a sequence of output events through a coalescer and returns the +// records it produced. +func run(t *testing.T, offsetMS int64, evs ...castEvent) *coalescer { + t.Helper() + c := &coalescer{t0: testT0, offsetMS: offsetMS} + for _, ev := range evs { + if err := c.add(ev); err != nil { + t.Fatalf("add(%+v): %v", ev, err) + } + } + if err := c.flush(); err != nil { + t.Fatalf("flush: %v", err) + } + return c +} + +// out builds one output event at t seconds, on the microsecond grain scanCast +// delivers. +func out(t float64, data string) castEvent { + return castEvent{US: int64(math.Round(t * castTimeGrain)), Code: codeOutput, Data: data} +} + +// TestMicrosToMillis pins the one rounding step between the grain the clock is +// carried on and the millisecond a record records: half away from zero, which +// is math.Round's rule done in integers. +func TestMicrosToMillis(t *testing.T) { + cases := []struct { + us int64 + want int64 + }{ + {0, 0}, + {499, 0}, + {500, 1}, + {1499, 1}, + {1500, 2}, + {10500, 11}, // the half-millisecond tie the two formats used to split on + {-500, -1}, + {-499, 0}, + } + for _, tc := range cases { + if got := microsToMillis(tc.us); got != tc.want { + t.Errorf("microsToMillis(%d) = %d, want %d", tc.us, got, tc.want) + } + } +} + +func texts(recs []timeline.Interaction) []string { + out := make([]string, len(recs)) + for i, r := range recs { + out[i] = r.Text + } + return out +} + +func TestCoalescerBoundaries(t *testing.T) { + cases := []struct { + name string + events []castEvent + wantText []string + wantTimes []int64 // session-relative milliseconds + }{ + { + // The echo of a typed command carries no newline until Enter, so the + // whole command arrives as one record. + name: "keystroke echo coalesces into one record", + events: []castEvent{ + out(1.000, "l"), out(1.090, "s"), out(1.180, " "), + out(1.270, "-"), out(1.360, "a"), out(1.450, "\r\n"), + }, + wantText: []string{"ls -a\r\n"}, + wantTimes: []int64{1000}, + }, + { + name: "newline closes a record and the next rune opens one", + events: []castEvent{ + out(1.000, "first\nsecond\n"), + }, + wantText: []string{"first\n", "second\n"}, + wantTimes: []int64{1000, 1000}, + }, + { + name: "a trailing fragment is closed by flush", + events: []castEvent{ + out(1.000, "line\nprompt$ "), + }, + wantText: []string{"line\n", "prompt$ "}, + wantTimes: []int64{1000, 1000}, + }, + { + // A bare prompt never ends in a newline; the gap to the next output is + // what closes it. + name: "a 250 ms gap closes an unterminated record", + events: []castEvent{ + out(1.000, "$ "), out(1.250, "ls\r\n"), + }, + wantText: []string{"$ ", "ls\r\n"}, + wantTimes: []int64{1000, 1250}, + }, + { + name: "a gap just under 250 ms keeps one record", + events: []castEvent{ + out(1.000, "$ "), out(1.249, "ls\r\n"), + }, + wantText: []string{"$ ls\r\n"}, + wantTimes: []int64{1000}, + }, + { + // A record states one time, so the span cap bounds how stale that + // instant can be. + name: "the one-second span cap closes a long burst", + events: []castEvent{ + out(0.000, "a"), out(0.200, "b"), out(0.400, "c"), + out(0.600, "d"), out(0.800, "e"), out(1.000, "f"), + }, + wantText: []string{"abcde", "f"}, + wantTimes: []int64{0, 1000}, + }, + { + name: "a span just under one second keeps one record", + events: []castEvent{ + out(0.000, "a"), out(0.200, "b"), out(0.400, "c"), + out(0.600, "d"), out(0.800, "e"), out(0.999, "f"), + }, + wantText: []string{"abcdef"}, + wantTimes: []int64{0}, + }, + { + // Both boundaries fall on the same event: it closes once, not twice. + name: "gap and span together close one record", + events: []castEvent{ + out(0.000, "a"), out(1.500, "b"), + }, + wantText: []string{"a", "b"}, + wantTimes: []int64{0, 1500}, + }, + { + // A progress bar's frames are one record, because \r is not a boundary. + name: "carriage returns stay inside one record", + events: []castEvent{ + out(0.000, "10%\r"), out(0.100, "50%\r"), out(0.200, "100%\r\n"), + }, + wantText: []string{"10%\r50%\r100%\r\n"}, + wantTimes: []int64{0}, + }, + { + name: "an empty output event contributes no runes", + events: []castEvent{ + out(0.000, ""), out(0.100, "hi\n"), + }, + wantText: []string{"hi\n"}, + wantTimes: []int64{100}, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + c := run(t, 0, tc.events...) + if got := texts(c.records); !equalStrings(got, tc.wantText) { + t.Fatalf("records = %q, want %q", got, tc.wantText) + } + for i, want := range tc.wantTimes { + if got := c.records[i].T - testT0; got != want { + t.Errorf("record %d: session-relative t = %d ms, want %d", i+1, got, want) + } + } + for i, rec := range c.records { + if rec.Kind != OutputKind { + t.Errorf("record %d: kind = %q, want %q", i+1, rec.Kind, OutputKind) + } + if rec.Selector != "" || rec.Value != "" || rec.Route != "" { + t.Errorf("record %d: carries a field outside t/kind/text: %+v", i+1, rec) + } + } + }) + } +} + +func TestCoalescerDropsRecordsThatRenderEmpty(t *testing.T) { + c := run(t, 0, + out(0.000, "\r\n"), // a blank line + out(0.500, "\r"), // a lone carriage return + out(1.000, "​​\n"), // invisible-only Unicode + out(1.500, "real\n"), + ) + if got := texts(c.records); !equalStrings(got, []string{"real\n"}) { + t.Fatalf("records = %q, want one real record", got) + } + if c.dropped != 3 { + t.Errorf("dropped = %d, want 3", c.dropped) + } +} + +func TestCoalescerAppliesTheOffset(t *testing.T) { + c := run(t, -30000, out(0.000, "early\n")) + if len(c.records) != 1 { + t.Fatalf("got %d records, want 1", len(c.records)) + } + // t stays a positive epoch-millisecond value, so CheckInteraction's + // positivity rule is met while the merged session-relative time goes + // negative. + if want := int64(testT0 - 30000); c.records[0].T != want { + t.Errorf("t = %d, want %d", c.records[0].T, want) + } +} + +// TestCoalescerSplitsOversizedEvent is criterion 7's unit: one output event +// larger than the line limit becomes several records, each within the limit, +// with every rune preserved and no boundary inside a rune. +func TestCoalescerSplitsOversizedEvent(t *testing.T) { + // Mixed ASCII, multi-byte runes, and ESC bytes — the last cost six encoded + // bytes each, which is what makes the encoded budget rather than the raw + // length the binding limit. No newline, so only the budget can split it. + unit := "abcé中\x1b[0;34m" + data := strings.Repeat(unit, 12<<20/len(unit)) + c := run(t, 0, out(0.000, data)) + if len(c.records) < 2 { + t.Fatalf("got %d records, want more than one", len(c.records)) + } + var joined strings.Builder + for i, rec := range c.records { + if !utf8.ValidString(rec.Text) { + t.Errorf("record %d is not valid UTF-8; a split fell inside a rune", i+1) + } + n, err := session.EncodedLen(timeline.EventEntry(rec, testT0)) + if err != nil { + t.Fatalf("record %d: %v", i+1, err) + } + if n+castEntryIDMargin > session.MaxJSONLLine { + t.Errorf("record %d's timeline entry is %d bytes, over the %d-byte limit", i+1, n, session.MaxJSONLLine) + } + joined.WriteString(rec.Text) + } + if joined.String() != data { + t.Errorf("concatenating the records does not reproduce the event's data (%d bytes vs %d)", joined.Len(), len(data)) + } + // Every record but the last should be close to the budget, or the splitter is + // cutting far earlier than it needs to. + first := c.records[0] + if n, _ := session.EncodedLen(timeline.EventEntry(first, testT0)); n < session.MaxJSONLLine/2 { + t.Errorf("first record's entry is only %d bytes; the budget is not being used", n) + } +} + +// TestEncodedRuneLenMatchesJSON pins the per-rune table against the encoder +// session.WriteJSONL actually uses, so a stdlib change that invalidates it fails +// here rather than as a refused import. +// +// The table must never under-count: an under-count over-fills the budget and +// costs a false importer-bug refusal, while an over-count costs only a few +// unused bytes of a 4 MiB budget. Equality is therefore asserted for every rune +// except backspace and form feed, which encoding/json has written as a six-byte +// u-escape in some Go versions and as a two-byte short escape in others — the +// table takes the larger, on the safe side of that difference. +func TestEncodedRuneLenMatchesJSON(t *testing.T) { + overCounted := map[rune]bool{'\b': true, '\f': true} + var runes []rune + for r := rune(0); r < 0x80; r++ { + runes = append(runes, r) + } + runes = append(runes, + 0x00a3, // pound sign, 2 bytes + 0x00e9, // e-acute, 2 bytes + 0x4e2d, // CJK, 3 bytes + 0x1f600, // emoji, 4 bytes + 0x2028, 0x2029, // line and paragraph separator + utf8.RuneError, // U+FFFD, what the decoder substitutes + 0x00ad, 0x200b, // soft hyphen, zero-width space + ) + for _, r := range runes { + // Measure one rune's contribution as the difference between a two-rune + // and a one-rune payload, so the envelope around it cancels out. + base, err := session.EncodedLen(map[string]string{"t": "x"}) + if err != nil { + t.Fatal(err) + } + with, err := session.EncodedLen(map[string]string{"t": "x" + string(r)}) + if err != nil { + t.Fatal(err) + } + got, want := encodedRuneLen(r), with-base + if got < want { + t.Errorf("encodedRuneLen(%U) = %d, under encoding/json's %d; the budget would overfill", r, got, want) + } + if got != want && !overCounted[r] { + t.Errorf("encodedRuneLen(%U) = %d, encoding/json spends %d", r, got, want) + } + } +} + +func equalStrings(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} + +// TestCoalescerRefusesRecordOverTheLineLimit exercises the measured check that +// backs the per-rune table. It is unreachable if the table is right, so the +// budget is deliberately falsified here — which is exactly the failure the check +// exists to turn into a refusal rather than a session no command can read back. +func TestCoalescerRefusesRecordOverTheLineLimit(t *testing.T) { + c := &coalescer{t0: testT0} + if err := c.begin(0); err != nil { + t.Fatalf("begin: %v", err) + } + c.budget = math.MaxInt32 + text := strings.Repeat("x", session.MaxJSONLLine) + c.text.WriteString(text) + c.enc = len(text) + err := c.close() + if err == nil { + t.Fatal("want a refusal, got nil") + } + for _, want := range []string{"record 1 encodes to", "JSONL line limit", "this is an importer bug"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not contain %q", err, want) + } + } + if len(c.records) != 0 { + t.Errorf("a refused record was kept: %d records", len(c.records)) + } +} diff --git a/internal/cast/scan.go b/internal/cast/scan.go new file mode 100644 index 0000000..f952be1 --- /dev/null +++ b/internal/cast/scan.go @@ -0,0 +1,304 @@ +package cast + +import ( + "bufio" + "bytes" + "encoding/json" + "errors" + "fmt" + "io" + "math" + "unicode/utf8" + + "github.com/REPPL/Testimony/internal/session" +) + +// Asciicast event codes. Only output becomes records; the rest are dropped and +// counted (see Run). An unrecognised code is dropped rather than refused, so a +// future asciicast revision that adds one does not turn every cast it writes +// into an unimportable file; the printed count is what keeps that tolerance +// honest. +const ( + codeOutput = "o" + codeInput = "i" + codeResize = "r" + codeMarker = "m" + codeExit = "x" +) + +// codeName gives an event code its asciicast name, for the printed drop line. +func codeName(code string) string { + switch code { + case codeOutput: + return "output" + case codeInput: + return "input" + case codeResize: + return "resize" + case codeMarker: + return "marker" + case codeExit: + return "exit" + default: + return "unrecognised" + } +} + +// A cast is the same kind of input session.ReadJSONL bounds — line-oriented, +// operator-supplied, and possibly received rather than recorded here — so the +// scan is bounded the same way, at its own scale. +const ( + // maxCastLine bounds one line. A single o event can legitimately be one large + // write (a cat of a file), and JSON escaping inflates it — an ESC byte encodes + // to six bytes — so session.MaxJSONLLine's 4 MiB would refuse casts whose + // records this importer can split and persist perfectly well. A line at this + // bound still splits into records that fit. + maxCastLine = 16 << 20 // 16 MiB + + // maxCastBytes bounds the whole file. A cast large enough to matter cannot + // become records that fit interactions.jsonl's own 16 MiB cap anyway, so this + // exists only to bound the scan itself. + maxCastBytes = 64 << 20 // 64 MiB + + // maxCastSeconds bounds an event's absolute time on the recording clock, + // mirroring timeline's maxUtteranceSeconds and transcribe's maxOffsetSeconds, + // so a time this importer accepts is a time merge accepts. + maxCastSeconds = 1e9 + + // castTimeGrain is the integer grain the recording clock is carried on: + // microseconds. It is what makes the v2 and v3 readings of one recording + // agree bit-for-bit, which is the whole "the operator never needs to know + // which format their recorder wrote" guarantee. + // + // Carrying the clock in float64 seconds does not give that. v2 rounds a + // stated absolute time to milliseconds; v3 rounds a float64 running sum of + // intervals. At a half-millisecond tie the two land on different + // milliseconds — seven 0.0015 s intervals sum to 0.010499999999999999 and + // round to 10 ms, while v2's stated 0.0105 rounds to 11 ms — and one + // millisecond is enough to flip a 250 ms coalescing gap, so the same + // recording becomes one record in one format and two in the other. + // Accumulating on an exact integer grain removes the class: both formats + // reach the identical integer, and the rounding to milliseconds happens once, + // at the same place, from the same number. + // + // A microsecond is exact for every time either format writes — both cap a + // time at six decimal places — and 1e9 seconds on this grain is 1e15, three + // orders of magnitude inside int64. + castTimeGrain = 1e6 + + // maxCastMicros is maxCastSeconds on that grain. + maxCastMicros = int64(maxCastSeconds * castTimeGrain) +) + +// castHeader is the first line of a cast, reduced to the two fields this +// importer needs. Both are pointers so an absent field stays distinguishable +// from a genuine 0 — the timeline.rawInteraction.T and +// transcribe.offsetSidecar.OffsetSeconds precedent. Every other header field +// (width, height, term, env, theme, command, title, duration, idle_time_limit) +// is ignored: the importer needs the version and the anchor and nothing else, +// and unknown fields must not be rejected, since both formats are extensible. +type castHeader struct { + Version *int + Timestamp *int64 +} + +// castEvent is one event line. US is always absolute MICROSECONDS since +// recording start, so the v2/v3 difference is resolved inside scanCast and +// nothing downstream of it knows which format was read — the single seam behind +// the "identical timeline from either format" guarantee. It is an integer +// rather than a float64 of seconds because that guarantee is bit-for-bit: see +// castTimeGrain. +type castEvent struct { + Line int + US int64 + Code string + Data string +} + +// scanCast streams an asciicast: the header first, then one callback per event +// line. onHeader runs after the header is decoded and before the first event, +// so the caller can resolve the cast→session offset — which the header's +// timestamp anchors, and which every record's time needs — without buffering +// the events or reading the file twice; it may be nil. Both callbacks' errors +// abort the scan unchanged. +// +// Blank lines are skipped, matching every other line reader in the repository. +// Every refusal names the line it fired on and leaves the caller with nothing +// written, because the whole scan runs before any file in the session changes. +func scanCast(r io.Reader, name string, onHeader func(castHeader) error, fn func(castEvent) error) (castHeader, error) { + var hdr castHeader + sc := bufio.NewScanner(r) + sc.Buffer(make([]byte, 0, 64*1024), maxCastLine) + line := 0 + var total int64 + // v2 keeps the previous absolute time; v3 keeps the running sum of intervals. + // Both are microseconds (castTimeGrain), so the sum is exact and the two + // formats' readings of one recording cannot diverge: each event's time is + // rounded to the grain once, on the way in, and the rounding to the + // millisecond a record records happens once more, downstream, from the same + // integer whichever format was read. + var clock int64 + haveHeader := false + for sc.Scan() { + line++ + raw := sc.Bytes() + // Counted before the blank-line skip, and including the newline, exactly + // as session.ReadJSONL counts, so a file padded with blank lines past the + // cap is refused rather than scanned past forever. + total += int64(len(raw)) + 1 + if total > maxCastBytes { + return hdr, fmt.Errorf("%s: exceeds %d bytes across %d lines; refusing to read", name, maxCastBytes, line) + } + if len(bytes.TrimSpace(raw)) == 0 { + continue + } + if !haveHeader { + var err error + if hdr, err = parseHeader(raw, name, line); err != nil { + return hdr, err + } + haveHeader = true + if onHeader != nil { + if err := onHeader(hdr); err != nil { + return hdr, err + } + } + continue + } + ev, err := parseEvent(raw, name, line, *hdr.Version, &clock) + if err != nil { + return hdr, err + } + if fn != nil { + if err := fn(ev); err != nil { + return hdr, err + } + } + } + if err := sc.Err(); err != nil { + if errors.Is(err, bufio.ErrTooLong) { + // The scanner stopped on the line after the last one it delivered. + return hdr, fmt.Errorf("%s:%d: line exceeds %d bytes; refusing to read", name, line+1, maxCastLine) + } + return hdr, fmt.Errorf("%s: %w", name, err) + } + if !haveHeader { + return hdr, fmt.Errorf("%s:1: not an asciicast header (expected a JSON object with a \"version\" field)", name) + } + return hdr, nil +} + +// parseHeader decodes line 1. A cast with no header at all is caught here, too: +// an event line is a JSON array, not an object. +func parseHeader(raw []byte, name string, line int) (castHeader, error) { + var hdr castHeader + var obj map[string]json.RawMessage + // A JSON null decodes into a nil map without error, so it is refused + // explicitly rather than read as an object carrying no version. + if err := json.Unmarshal(raw, &obj); err != nil || obj == nil { + return hdr, fmt.Errorf("%s:%d: not an asciicast header (expected a JSON object with a \"version\" field)", name, line) + } + vraw, ok := obj["version"] + if !ok || isJSONNull(vraw) { + return hdr, fmt.Errorf("%s:%d: asciicast header carries no \"version\" field", name, line) + } + var version int + if err := json.Unmarshal(vraw, &version); err != nil || (version != 2 && version != 3) { + // Naming the file and the version found is the acceptance criterion's + // exact wording. The value is echoed from its JSON literal, so a + // non-integer version ("2", say) is named as written rather than as a + // decode failure — and it is neutralised and truncated first, because a + // cast is attacker-authorable and this message reaches a terminal. + return hdr, fmt.Errorf("%s: asciicast version %s is not supported (import reads version 2 and 3)", name, literal(vraw)) + } + hdr.Version = &version + if traw, ok := obj["timestamp"]; ok && !isJSONNull(traw) { + var ts int64 + if err := json.Unmarshal(traw, &ts); err != nil { + return hdr, fmt.Errorf("%s:%d: asciicast header timestamp %s is not an integer number of seconds; pass -offset SECONDS to anchor the cast explicitly", name, line, literal(traw)) + } + hdr.Timestamp = &ts + } + return hdr, nil +} + +// parseEvent decodes one [time, code, data] line and resolves its time onto the +// absolute recording clock, in microseconds. clock carries the previous event's +// absolute time (v2) or the running interval sum (v3) across calls. +func parseEvent(raw []byte, name string, line, version int, clock *int64) (castEvent, error) { + malformed := fmt.Errorf("%s:%d: malformed asciicast event (expected [time, code, data])", name, line) + var el []json.RawMessage + if err := json.Unmarshal(raw, &el); err != nil || len(el) != 3 { + return castEvent{}, malformed + } + var t float64 + var code, data string + // An out-of-range numeric literal (1e400) lands here too: encoding/json + // refuses it into a float64 rather than yielding +Inf. + if err := json.Unmarshal(el[0], &t); err != nil { + return castEvent{}, malformed + } + if err := json.Unmarshal(el[1], &code); err != nil { + return castEvent{}, malformed + } + if err := json.Unmarshal(el[2], &data); err != nil { + return castEvent{}, malformed + } + // Bound the value before it reaches the grain: int64(math.Round(x)) has no + // defined answer for a float past the integer range, so a 1e300 time (or + // interval) must be refused here rather than converted. The bound is the + // recording-clock bound itself, so nothing legitimate is refused earlier + // than it would have been after accumulating. + if math.Abs(t) > maxCastSeconds { + return castEvent{}, fmt.Errorf("%s:%d: event time %gs exceeds %g seconds; that is no recording clock", name, line, t, maxCastSeconds) + } + us := int64(math.Round(t * castTimeGrain)) + switch version { + case 2: + // v2 times are absolute seconds since recording start. + if us < *clock { + return castEvent{}, fmt.Errorf("%s:%d: event time %g precedes the previous event's %g; asciicast v2 times must not decrease", name, line, t, float64(*clock)/castTimeGrain) + } + *clock = us + default: + // v3 times are intervals since the previous event. + if us < 0 { + return castEvent{}, fmt.Errorf("%s:%d: event interval %g is negative; asciicast v3 intervals must not be negative", name, line, t) + } + *clock += us + } + if *clock > maxCastMicros || *clock < -maxCastMicros { + return castEvent{}, fmt.Errorf("%s:%d: event time %gs exceeds %g seconds; that is no recording clock", name, line, float64(*clock)/castTimeGrain, maxCastSeconds) + } + return castEvent{Line: line, US: *clock, Code: code, Data: data}, nil +} + +// isJSONNull reports whether a raw JSON value is the literal null, which this +// importer treats as an absent field rather than as a malformed one. +func isJSONNull(raw json.RawMessage) bool { + return string(bytes.TrimSpace(raw)) == "null" +} + +// literal renders a raw JSON value for an error message: neutralised +// (session.SafeText, since a cast is attacker-authorable and the message +// reaches a terminal) and clipped, so a header field holding a megabyte of text +// cannot become the error. +func literal(raw json.RawMessage) string { + return clip(session.SafeText(string(bytes.TrimSpace(raw))), 32) +} + +// clip shortens s to at most max runes, marking the cut. The cut falls on a +// rune boundary, so a clipped string never ends mid-rune. +func clip(s string, max int) string { + if utf8.RuneCountInString(s) <= max { + return s + } + n := 0 + for i := range s { + if n == max { + return s[:i] + "…" + } + n++ + } + return s +} diff --git a/internal/cast/scan_test.go b/internal/cast/scan_test.go new file mode 100644 index 0000000..51e7111 --- /dev/null +++ b/internal/cast/scan_test.go @@ -0,0 +1,342 @@ +package cast + +import ( + "errors" + "fmt" + "io" + "strings" + "testing" +) + +// collect runs scanCast over src and returns every event it delivered. +func collect(t *testing.T, src, name string) (castHeader, []castEvent) { + t.Helper() + var got []castEvent + hdr, err := scanCast(strings.NewReader(src), name, nil, func(ev castEvent) error { + got = append(got, ev) + return nil + }) + if err != nil { + t.Fatalf("scanCast: %v", err) + } + return hdr, got +} + +func TestScanCastV2AbsoluteTimes(t *testing.T) { + src := `{"version":2,"timestamp":1784300398} +[0,"o","a"] +[0.25,"o","b"] +[12.5,"o","c"] +` + hdr, got := collect(t, src, "v2") + if hdr.Version == nil || *hdr.Version != 2 { + t.Fatalf("version = %v, want 2", hdr.Version) + } + if hdr.Timestamp == nil || *hdr.Timestamp != 1784300398 { + t.Fatalf("timestamp = %v, want 1784300398", hdr.Timestamp) + } + // Microseconds, the grain the clock is carried on, compared exactly. + want := []int64{0, 250_000, 12_500_000} + if len(got) != len(want) { + t.Fatalf("got %d events, want %d", len(got), len(want)) + } + for i, ev := range got { + if ev.US != want[i] { + t.Errorf("event %d: us = %d, want %d", i+1, ev.US, want[i]) + } + if ev.Line != i+2 { + t.Errorf("event %d: line = %d, want %d", i+1, ev.Line, i+2) + } + } +} + +// TestScanCastV3RunningSum pins the one seam the "identical timeline from +// either format" guarantee rests on: a v3 cast's intervals reconstructed into +// the absolute times a v2 cast states outright. +func TestScanCastV3RunningSum(t *testing.T) { + cases := []struct { + name string + intervals []float64 + want []int64 // absolute microseconds + }{ + {"simple", []float64{0, 0.5, 0.5}, []int64{0, 500_000, 1_000_000}}, + {"zero intervals", []float64{0, 0, 0}, []int64{0, 0, 0}}, + {"first interval non-zero", []float64{1.25, 0.25}, []int64{1_250_000, 1_500_000}}, + {"millisecond tail", []float64{0.001, 0.001, 0.001, 0.001}, []int64{1000, 2000, 3000, 4000}}, + // The sum is exact on the grain, so a run of half-millisecond ties lands + // on the same integer a v2 cast states outright — the divergence class + // TestV2AndV3Agree pins end to end. + {"half-millisecond ties", []float64{0.0015, 0.0015, 0.0015, 0.0015, 0.0015, 0.0015, 0.0015}, + []int64{1500, 3000, 4500, 6000, 7500, 9000, 10500}}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + var b strings.Builder + b.WriteString("{\"version\":3}\n") + for _, iv := range tc.intervals { + fmt.Fprintf(&b, "[%g,\"o\",\"x\"]\n", iv) + } + _, got := collect(t, b.String(), "v3") + if len(got) != len(tc.want) { + t.Fatalf("got %d events, want %d", len(got), len(tc.want)) + } + for i, ev := range got { + // The sum is an integer, so this is an exact comparison rather than a + // tolerance — which is the point of the grain. + if ev.US != tc.want[i] { + t.Errorf("event %d: us = %d, want %d", i+1, ev.US, tc.want[i]) + } + } + }) + } +} + +// TestScanCastLongV3Tail checks that a long run of intervals does not drift +// past the millisecond a record records. +func TestScanCastLongV3Tail(t *testing.T) { + const n = 5000 + var b strings.Builder + b.WriteString("{\"version\":3}\n") + for i := 0; i < n; i++ { + b.WriteString("[0.001,\"o\",\"x\"]\n") + } + _, got := collect(t, b.String(), "v3") + if len(got) != n { + t.Fatalf("got %d events, want %d", len(got), n) + } + // Five thousand 1 ms intervals sum to exactly five seconds, with no drift to + // tolerate: the accumulator is an integer. + if last, want := got[n-1].US, int64(5_000_000); last != want { + t.Errorf("final absolute time = %d us, want %d", last, want) + } +} + +func TestScanCastDeliversEveryCode(t *testing.T) { + src := `{"version":2} +[0,"o","out"] +[0.1,"i","key"] +[0.2,"r","80x24"] +[0.3,"m","marker"] +[0.4,"x","0"] +[0.5,"z","future"] +` + _, got := collect(t, src, "codes") + want := []string{"o", "i", "r", "m", "x", "z"} + if len(got) != len(want) { + t.Fatalf("got %d events, want %d", len(got), len(want)) + } + for i, ev := range got { + if ev.Code != want[i] { + t.Errorf("event %d: code = %q, want %q", i+1, ev.Code, want[i]) + } + } +} + +func TestScanCastSkipsBlankLines(t *testing.T) { + src := "{\"version\":2}\n\n[0,\"o\",\"a\"]\n \n[0.5,\"o\",\"b\"]\n" + _, got := collect(t, src, "blanks") + if len(got) != 2 { + t.Fatalf("got %d events, want 2", len(got)) + } + // The line numbers still count the blank lines, so a refusal names the line + // the operator can find in the file. + if got[0].Line != 3 || got[1].Line != 5 { + t.Errorf("lines = %d, %d; want 3, 5", got[0].Line, got[1].Line) + } +} + +func TestScanCastRefusals(t *testing.T) { + cases := []struct { + name string + src string + want []string + }{ + { + "not an object", + "[0,\"o\",\"x\"]\n", + []string{"cast:1:", "not an asciicast header"}, + }, + { + "json null header", + "null\n", + []string{"cast:1:", "not an asciicast header"}, + }, + { + "not json at all", + "not json\n", + []string{"cast:1:", "not an asciicast header"}, + }, + { + "no version field", + "{\"timestamp\":1784300398}\n", + []string{"cast:1:", "carries no \"version\" field"}, + }, + { + "null version", + "{\"version\":null}\n", + []string{"cast:1:", "carries no \"version\" field"}, + }, + { + "version 1", + "{\"version\":1}\n", + []string{"cast:", "asciicast version 1 is not supported", "version 2 and 3"}, + }, + { + "version 4", + "{\"version\":4}\n", + []string{"cast:", "asciicast version 4 is not supported"}, + }, + { + "version as a string", + "{\"version\":\"2\"}\n", + []string{"cast:", "asciicast version \"2\" is not supported"}, + }, + { + "non-integer timestamp", + "{\"version\":2,\"timestamp\":\"noon\"}\n", + []string{"cast:1:", "is not an integer number of seconds", "-offset SECONDS"}, + }, + { + "event not an array", + "{\"version\":2}\n{\"t\":1}\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + "event with two elements", + "{\"version\":2}\n[0.5,\"o\"]\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + "event time not a number", + "{\"version\":2}\n[\"0.5\",\"o\",\"x\"]\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + "event code not a string", + "{\"version\":2}\n[0.5,3,\"x\"]\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + "event data not a string", + "{\"version\":2}\n[0.5,\"o\",7]\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + // encoding/json refuses an out-of-range literal into float64 rather + // than yielding +Inf, so it lands on the malformed path. + "event time out of float range", + "{\"version\":2}\n[1e400,\"o\",\"x\"]\n", + []string{"cast:2:", "malformed asciicast event"}, + }, + { + "v2 time decreasing", + "{\"version\":2}\n[1,\"o\",\"a\"]\n[0.5,\"o\",\"b\"]\n", + []string{"cast:3:", "precedes the previous event's 1", "must not decrease"}, + }, + { + "v3 interval negative", + "{\"version\":3}\n[0.1,\"o\",\"a\"]\n[-0.2,\"o\",\"b\"]\n", + []string{"cast:3:", "event interval -0.2 is negative", "must not be negative"}, + }, + { + "absolute time beyond the clock bound", + "{\"version\":2}\n[2e9,\"o\",\"a\"]\n", + []string{"cast:2:", "that is no recording clock"}, + }, + { + "v3 sum beyond the clock bound", + "{\"version\":3}\n[1e9,\"o\",\"a\"]\n[1e9,\"o\",\"b\"]\n", + []string{"cast:3:", "that is no recording clock"}, + }, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + _, err := scanCast(strings.NewReader(tc.src), "cast", nil, func(castEvent) error { return nil }) + if err == nil { + t.Fatal("want a refusal, got nil") + } + for _, want := range tc.want { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not contain %q", err, want) + } + } + }) + } +} + +func TestScanCastRefusesEmptyInput(t *testing.T) { + _, err := scanCast(strings.NewReader(""), "cast", nil, nil) + if err == nil || !strings.Contains(err.Error(), "not an asciicast header") { + t.Fatalf("error = %v, want a missing-header refusal", err) + } +} + +func TestScanCastRefusesOverlongLine(t *testing.T) { + src := "{\"version\":2}\n[0,\"o\",\"" + strings.Repeat("x", maxCastLine) + "\"]\n" + _, err := scanCast(strings.NewReader(src), "cast", nil, func(castEvent) error { return nil }) + if err == nil { + t.Fatal("want a refusal, got nil") + } + for _, want := range []string{"cast:2:", "line exceeds", "refusing to read"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not contain %q", err, want) + } + } +} + +// repeater emits chunk for ever, so the whole-file bound can be exercised +// without allocating 64 MiB of fixture. +type repeater struct { + chunk []byte + off int +} + +func (r *repeater) Read(p []byte) (int, error) { + n := copy(p, r.chunk[r.off:]) + r.off = (r.off + n) % len(r.chunk) + return n, nil +} + +func TestScanCastRefusesOverlongFile(t *testing.T) { + line := "[0,\"o\",\"" + strings.Repeat("x", 64<<10) + "\"]\n" + src := io.MultiReader(strings.NewReader("{\"version\":2}\n"), &repeater{chunk: []byte(line)}) + _, err := scanCast(src, "cast", nil, func(castEvent) error { return nil }) + if err == nil { + t.Fatal("want a refusal, got nil") + } + for _, want := range []string{"cast:", "exceeds", "across", "refusing to read"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("error %q does not contain %q", err, want) + } + } +} + +func TestScanCastPropagatesCallbackErrors(t *testing.T) { + sentinel := errors.New("stop") + _, err := scanCast(strings.NewReader("{\"version\":2}\n[0,\"o\",\"x\"]\n"), "cast", + func(castHeader) error { return sentinel }, nil) + if !errors.Is(err, sentinel) { + t.Fatalf("header callback error = %v, want %v", err, sentinel) + } + _, err = scanCast(strings.NewReader("{\"version\":2}\n[0,\"o\",\"x\"]\n"), "cast", + nil, func(castEvent) error { return sentinel }) + if !errors.Is(err, sentinel) { + t.Fatalf("event callback error = %v, want %v", err, sentinel) + } +} + +func TestLiteralTruncatesAndNeutralises(t *testing.T) { + long := "\"" + strings.Repeat("é", 100) + "\"" + got := literal([]byte(long)) + if !strings.HasSuffix(got, "…") { + t.Errorf("literal over the cap = %q, want a truncation", got) + } + if n := len([]rune(got)); n != 33 { + t.Errorf("literal truncated to %d runes, want 33 (32 plus the ellipsis)", n) + } + // A cast is attacker-authorable and this text reaches a terminal, so the + // control bytes an ANSI sequence rides on are stripped first. + if got := literal([]byte("\"a\x1b[31mb\"")); strings.ContainsRune(got, 0x1b) { + t.Errorf("literal kept an ESC byte: %q", got) + } +} diff --git a/internal/cast/testdata/bad-event.cast b/internal/cast/testdata/bad-event.cast new file mode 100644 index 0000000..85d40e8 --- /dev/null +++ b/internal/cast/testdata/bad-event.cast @@ -0,0 +1,3 @@ +{"version":2,"timestamp":1784300398} +[0.0,"o","ok\r\n"] +[0.5,"o"] diff --git a/internal/cast/testdata/bad-header.cast b/internal/cast/testdata/bad-header.cast new file mode 100644 index 0000000..19cda91 --- /dev/null +++ b/internal/cast/testdata/bad-header.cast @@ -0,0 +1 @@ +[0.0,"o","x"] diff --git a/internal/cast/testdata/bad-version.cast b/internal/cast/testdata/bad-version.cast new file mode 100644 index 0000000..2c7dd54 --- /dev/null +++ b/internal/cast/testdata/bad-version.cast @@ -0,0 +1,2 @@ +{"version":4,"timestamp":1784300398} +[0.0,"o","x"] diff --git a/internal/cast/testdata/decreasing.cast b/internal/cast/testdata/decreasing.cast new file mode 100644 index 0000000..8074784 --- /dev/null +++ b/internal/cast/testdata/decreasing.cast @@ -0,0 +1,3 @@ +{"version":2,"timestamp":1784300398} +[1.0,"o","a\r\n"] +[0.5,"o","b\r\n"] diff --git a/internal/cast/testdata/golden.interactions.jsonl b/internal/cast/testdata/golden.interactions.jsonl new file mode 100644 index 0000000..355841c --- /dev/null +++ b/internal/cast/testdata/golden.interactions.jsonl @@ -0,0 +1,5 @@ +{"t":1784300398000,"kind":"terminal_output","text":"\u001b[0;32malice@example.test\u001b[0m:~/project$ "} +{"t":1784300398520,"kind":"terminal_output","text":"ls --color\r\n"} +{"t":1784300399502,"kind":"terminal_output","text":"\u001b[0;34mdocs\u001b[0m \u001b[0;34minternal\u001b[0m README.md\r\n"} +{"t":1784300400000,"kind":"terminal_output","text":"building 0%\rbuilding 50%\rbuilding 100%\r\n"} +{"t":1784300401000,"kind":"terminal_output","text":"\u001b[0;32malice@example.test\u001b[0m:~/project$ "} diff --git a/internal/cast/testdata/negative-interval.cast b/internal/cast/testdata/negative-interval.cast new file mode 100644 index 0000000..07c1df1 --- /dev/null +++ b/internal/cast/testdata/negative-interval.cast @@ -0,0 +1,3 @@ +{"version":3,"timestamp":1784300398} +[0.1,"o","a\r\n"] +[-0.2,"o","b\r\n"] diff --git a/internal/cast/testdata/v2-ties.cast b/internal/cast/testdata/v2-ties.cast new file mode 100644 index 0000000..35a7cb1 --- /dev/null +++ b/internal/cast/testdata/v2-ties.cast @@ -0,0 +1,9 @@ +{"version":2,"width":80,"height":24,"timestamp":1784300398} +[0.0015,"o","a"] +[0.003,"o","b"] +[0.0045,"o","c"] +[0.006,"o","d"] +[0.0075,"o","e"] +[0.009,"o","f"] +[0.0105,"o","g"] +[0.26,"o","h\r\n"] diff --git a/internal/cast/testdata/v2.cast b/internal/cast/testdata/v2.cast new file mode 100644 index 0000000..539b400 --- /dev/null +++ b/internal/cast/testdata/v2.cast @@ -0,0 +1,25 @@ +{"version":2,"width":120,"height":40,"timestamp":1784300398,"env":{"SHELL":"/bin/zsh","TERM":"xterm-256color"}} +[0.0,"o","\u001b[0;32malice@example.test\u001b[0m:~/project$ "] +[0.512,"i","l"] +[0.52,"o","l"] +[0.6,"i","s"] +[0.61,"o","s"] +[0.7,"o"," "] +[0.79,"o","-"] +[0.88,"o","-"] +[0.97,"o","c"] +[1.06,"o","o"] +[1.15,"o","l"] +[1.24,"o","o"] +[1.33,"o","r"] +[1.45,"o","\r\n"] +[1.502,"o","\u001b[0;34mdocs\u001b[0m "] +[1.51,"o","\u001b[0;34minternal\u001b[0m README.md\r\n"] +[1.7,"r","120x40"] +[2.0,"o","building 0%\r"] +[2.15,"o","building 50%\r"] +[2.3,"o","building 100%\r\n"] +[2.7,"m","build finished"] +[2.8,"o","\r\n"] +[3.0,"o","\u001b[0;32malice@example.test\u001b[0m:~/project$ "] +[3.4,"x","0"] diff --git a/internal/cast/testdata/v3-nots.cast b/internal/cast/testdata/v3-nots.cast new file mode 100644 index 0000000..4a76590 --- /dev/null +++ b/internal/cast/testdata/v3-nots.cast @@ -0,0 +1,4 @@ +{"version":3,"term":{"cols":80,"rows":24}} +[0.0,"o","alice@example.test:~$ "] +[0.4,"o","echo hi\r\n"] +[0.1,"o","hi\r\n"] diff --git a/internal/cast/testdata/v3-ties.cast b/internal/cast/testdata/v3-ties.cast new file mode 100644 index 0000000..17e9eb9 --- /dev/null +++ b/internal/cast/testdata/v3-ties.cast @@ -0,0 +1,9 @@ +{"version":3,"term":{"cols":80,"rows":24},"timestamp":1784300398} +[0.0015,"o","a"] +[0.0015,"o","b"] +[0.0015,"o","c"] +[0.0015,"o","d"] +[0.0015,"o","e"] +[0.0015,"o","f"] +[0.0015,"o","g"] +[0.2495,"o","h\r\n"] diff --git a/internal/cast/testdata/v3.cast b/internal/cast/testdata/v3.cast new file mode 100644 index 0000000..e36639b --- /dev/null +++ b/internal/cast/testdata/v3.cast @@ -0,0 +1,25 @@ +{"version":3,"term":{"cols":120,"rows":40,"type":"xterm-256color"},"timestamp":1784300398} +[0.0,"o","\u001b[0;32malice@example.test\u001b[0m:~/project$ "] +[0.512,"i","l"] +[0.008,"o","l"] +[0.08,"i","s"] +[0.01,"o","s"] +[0.09,"o"," "] +[0.09,"o","-"] +[0.09,"o","-"] +[0.09,"o","c"] +[0.09,"o","o"] +[0.09,"o","l"] +[0.09,"o","o"] +[0.09,"o","r"] +[0.12,"o","\r\n"] +[0.052,"o","\u001b[0;34mdocs\u001b[0m "] +[0.008,"o","\u001b[0;34minternal\u001b[0m README.md\r\n"] +[0.19,"r","120x40"] +[0.3,"o","building 0%\r"] +[0.15,"o","building 50%\r"] +[0.15,"o","building 100%\r\n"] +[0.4,"m","build finished"] +[0.1,"o","\r\n"] +[0.2,"o","\u001b[0;32malice@example.test\u001b[0m:~/project$ "] +[0.4,"x","0"] diff --git a/internal/cast/write.go b/internal/cast/write.go new file mode 100644 index 0000000..3744f0a --- /dev/null +++ b/internal/cast/write.go @@ -0,0 +1,315 @@ +package cast + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "io" + "os" + "path/filepath" + "strconv" + "syscall" + + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" +) + +// rewriteInteractions replaces interactions.jsonl whole: prior records from +// this importer are dropped, every other line is kept byte-for-byte in file +// order, and the new records are appended. It returns how many prior records +// were replaced. +// +// The rewrite cannot go through session.WriteJSONL: that encodes values, and +// would silently rewrite a foreign record's field order, number formatting, or +// an unknown field it cannot model. The checks WriteJSONL would have applied +// are therefore applied here instead, before anything is written — plus the +// merged-timeline total, which demo's capture endpoint can only estimate and an +// offline importer can measure. +// +// castName is the cast's display name, for the size refusal; t0 anchors the +// merged-timeline measurement. +func rewriteInteractions(dir, castName string, t0 int64, records []timeline.Interaction) (replaced int, err error) { + path := filepath.Join(dir, session.InteractionsFile) + prior, err := readInteractionLines(path) + if err != nil { + return 0, err + } + + var buf bytes.Buffer + for _, l := range prior.kept { + buf.Write(l) + // A final line with no terminating newline is the one byte this rewrite + // adds: without it the first imported record would be appended onto that + // line and neither would survive a read back. + buf.WriteByte('\n') + } + for i, rec := range records { + line, err := encodeRecord(rec) + if err != nil { + return 0, fmt.Errorf("record %d: %w", i+1, err) + } + if len(line)+1 > session.MaxJSONLLine { + return 0, fmt.Errorf("record %d encodes to %d bytes, over the %d-byte JSONL line limit", i+1, len(line)+1, session.MaxJSONLLine) + } + buf.Write(line) + buf.WriteByte('\n') + } + if buf.Len() > session.MaxJSONLBytes { + return 0, fmt.Errorf("importing %s would take %s past its %d-byte limit (%d bytes across %d lines); record shorter terminal sessions, or start a fresh session", + castName, session.InteractionsFile, session.MaxJSONLBytes, buf.Len(), len(prior.kept)+len(records)) + } + if err := checkMergedTimelineFits(dir, t0, prior.sized, records); err != nil { + return 0, err + } + + // Temp file plus rename, symlink refused up front, an existing file's mode + // preserved exactly: a failure at any point leaves the prior + // interactions.jsonl untouched. + if err := session.WriteFileAtomicNoFollow(path, buf.Bytes(), 0o644); err != nil { + return 0, err + } + return prior.replaced, nil +} + +// priorInteractions is what a rewrite keeps of the file it replaces. +type priorInteractions struct { + kept [][]byte // lines preserved verbatim, in file order + replaced int // records from an earlier import, dropped + sized []timeline.Interaction // kept lines that decode, for the merged-timeline measurement +} + +// readInteractionLines reads the existing interactions.jsonl and classifies its +// lines. A missing file is zero lines, not an error. +// +// A line whose kind equals OutputKind is a record from an earlier import and is +// dropped — that marker, and nothing else, is how a re-run identifies its own +// prior output. Every other line — a demo click, an input, a line this importer +// cannot decode at all, a blank line — is kept exactly as it was read, which is +// what makes a re-import byte-identical and leaves a -demo session's capture +// untouched. +func readInteractionLines(path string) (priorInteractions, error) { + var prior priorInteractions + // The no-follow guard, so a FIFO or symlink planted at the name in a received + // session is refused rather than followed or blocked on. + f, err := session.OpenFileNoFollowRead(path) + if err != nil { + if errors.Is(err, os.ErrNotExist) { + return prior, nil + } + return prior, err + } + defer f.Close() + + // Read one byte past the cap so an over-large file is refused as too big + // rather than silently truncated and then rewritten short — the same pair of + // bounds session.ReadJSONL enforces, read whole because the replacement is + // assembled in memory anyway. + b, err := io.ReadAll(io.LimitReader(f, session.MaxJSONLBytes+1)) + if err != nil { + return prior, err + } + if len(b) > session.MaxJSONLBytes { + return prior, fmt.Errorf("%s: exceeds %d bytes; refusing to read", path, session.MaxJSONLBytes) + } + line := 0 + for len(b) > 0 { + line++ + raw := b + if i := bytes.IndexByte(b, '\n'); i >= 0 { + raw, b = b[:i], b[i+1:] + } else { + b = nil + } + if len(raw)+1 > session.MaxJSONLLine { + return priorInteractions{}, fmt.Errorf("%s:%d: line exceeds %d bytes; refusing to read", path, line, session.MaxJSONLLine) + } + var probe struct { + Kind string `json:"kind"` + } + if json.Unmarshal(raw, &probe) == nil && probe.Kind == OutputKind { + prior.replaced++ + continue + } + prior.kept = append(prior.kept, raw) + // An interaction line that does not decode is not sized: merge will refuse + // that session for its own, pre-existing reason, and import must not be + // blamed for it. + var rec timeline.Interaction + if json.Unmarshal(raw, &rec) == nil { + prior.sized = append(prior.sized, rec) + } + } + return prior, nil +} + +// castIDGrowth is demo.idGrowth's twin: how many bytes longer nth — the real, +// 1-based ordinal merge's "ev-%03d" gives an interaction at this position among +// every interaction in the session — runs than the "ev-001" placeholder +// timeline.EventEntry sizes an entry with. It is 0 until the 1000th +// interaction, which is why charging a flat margin per record instead would +// waste a real fraction of the file's capacity on growth that has not happened. +// (demo's is unexported, so the arithmetic is restated here with the citation +// rather than reached across the package boundary.) +func castIDGrowth(nth int64) int64 { + if nth < 1000 { + return 0 + } + return int64(len(strconv.FormatInt(nth, 10))) - 3 +} + +// checkMergedTimelineFits measures the timeline.jsonl this import implies — +// every decodable interaction plus, when transcript.jsonl is present, every +// utterance — against the same total-size cap session.WriteJSONL applies when +// merge writes it. +// +// Each entry is charged its id growth, because timeline.EventEntry sizes every +// entry with the placeholder id "ev-001" while merge assigns "ev-%03d" by the +// interaction's position among every interaction in the file: past the 1000th, +// the real id is longer than the measured one. Without the charge a session +// sitting just under the 16 MiB cap with a few thousand interactions could pass +// this pre-flight and still be refused by merge — the exact "import succeeds, +// merge is permanently unable to read it back" state the pre-flight exists to +// prevent. The ordinal is the position across the prior records and then the new +// ones, which is the order they are written in and so the order merge numbers +// them in. +// +// Like demo's running estimate, the guarantee is "as of import time": a +// transcribe run afterwards adds speech entries this pass cannot see. +func checkMergedTimelineFits(dir string, t0 int64, prior, records []timeline.Interaction) error { + total := 0 + nth := int64(0) + for _, set := range [][]timeline.Interaction{prior, records} { + for _, rec := range set { + nth++ + n, err := session.EncodedLen(timeline.EventEntry(rec, t0)) + if err != nil { + return err + } + total += n + int(castIDGrowth(nth)) + } + } + // An unreadable or malformed transcript is not sized, for readInteractionLines' + // reason: merge refuses that session on its own account, and this pre-flight + // must not turn a pre-existing fault into an import refusal. + if utts, err := session.ReadJSONL[timeline.Utterance](filepath.Join(dir, session.TranscriptFile)); err == nil { + for _, u := range utts { + n, err := session.EncodedLen(timeline.SpeechEntry(u)) + if err != nil { + return err + } + total += n + } + } + if total > session.MaxJSONLBytes { + return fmt.Errorf("the imported records would take the merged %s past its %d-byte limit; record shorter terminal sessions, or start a fresh session", + session.TimelineFile, session.MaxJSONLBytes) + } + return nil +} + +// encodeRecord renders one record as the JSONL line session.WriteJSONL would +// write for it, without the terminating newline. HTML escaping is off, matching +// session.EncodedLen's encoder, so a record's measured size and its written +// bytes cannot disagree. +func encodeRecord(rec timeline.Interaction) ([]byte, error) { + var buf bytes.Buffer + enc := json.NewEncoder(&buf) + enc.SetEscapeHTML(false) + if err := enc.Encode(rec); err != nil { + return nil, err + } + return bytes.TrimSuffix(buf.Bytes(), []byte("\n")), nil +} + +// stageCast streams the operator's cast into a temp file beside terminal.cast +// and returns its path. import copies the file rather than requiring it to be +// in place already, mirroring transcribe -audio: the hand-off pattern's whole +// point is that the operator hands over a file and the session becomes +// self-contained. +// +// src is the descriptor the scan already read, rewound here rather than +// re-opened by path: the archive must hold the bytes the records were derived +// from, and between two opens of a path the operator's file can be replaced or +// rewritten, which would leave the session asserting a byte-for-byte archive of +// something else. name is that file's display name, for the size refusal. +// +// The prior-mode rule is transcribe.atomicConvert's: an existing +// terminal.cast's own mode is reapplied, and a new file takes 0o644 &^ umask, +// so a privacy-conscious operator's umask is honoured. The caller removes the +// temp file on every failure path. +func stageCast(dir, name string, src *os.File) (string, error) { + target := filepath.Join(dir, session.TerminalCastFile) + // checkPlainOutput's rule: a symlink planted at the archive name would + // redirect the copy outside the session, and a FIFO would block the rename's + // successor for ever. Refused before the temp file is created. + priorPerm, havePrior := os.FileMode(0), false + if fi, err := os.Lstat(target); err == nil { + if fi.Mode()&os.ModeSymlink != 0 { + return "", fmt.Errorf("refusing to write %s: it is a symlink", target) + } + if !fi.Mode().IsRegular() { + return "", fmt.Errorf("refusing to write %s: it is not a regular file", target) + } + priorPerm, havePrior = fi.Mode().Perm(), true + } else if !errors.Is(err, os.ErrNotExist) { + return "", err + } + + if _, err := src.Seek(0, io.SeekStart); err != nil { + return "", fmt.Errorf("archive terminal cast: %w", err) + } + + tmp, err := os.CreateTemp(dir, "."+session.TerminalCastFile+".tmp-*") + if err != nil { + return "", fmt.Errorf("archive terminal cast: %w", err) + } + tmpPath := tmp.Name() + fail := func(err error) (string, error) { + tmp.Close() + os.Remove(tmpPath) + return "", err + } + // One byte past the cap, so a file that grew past the bound between the scan + // and the copy is refused rather than archived truncated — the archive is the + // one place this design makes a byte-for-byte claim. + n, err := io.Copy(tmp, io.LimitReader(src, maxCastBytes+1)) + if err != nil { + return fail(fmt.Errorf("archive terminal cast: %w", err)) + } + if n > maxCastBytes { + return fail(fmt.Errorf("%s: exceeds %d bytes; refusing to read", name, maxCastBytes)) + } + perm := priorPerm + if !havePrior { + // os.CreateTemp reserves the name at 0600; restore the mode a plain create + // would have given the file, so the archive matches every sibling artefact + // rather than staying private by accident of the temp file's mode. The + // brief probe is safe here — import creates no other file concurrently. + um := syscall.Umask(0) + syscall.Umask(um) + perm = 0o644 &^ os.FileMode(um) + } + if err := tmp.Chmod(perm); err != nil { + return fail(fmt.Errorf("archive terminal cast: %w", err)) + } + // Close before rename, and surface the Close error: a filesystem that defers + // write-back errors to close would otherwise rename a corrupt copy into place + // (session.WriteFileAtomicNoFollow's identical stance). + if err := tmp.Close(); err != nil { + os.Remove(tmpPath) + return "", fmt.Errorf("archive terminal cast: %w", err) + } + return tmpPath, nil +} + +// commitCast renames a staged copy over terminal.cast. It runs last, after the +// records are written, so the only residual state a failure can leave is +// records with no archival copy. +func commitCast(tmpPath string) error { + target := filepath.Join(filepath.Dir(tmpPath), session.TerminalCastFile) + if err := os.Rename(tmpPath, target); err != nil { + return fmt.Errorf("archive terminal cast: %w", err) + } + return nil +} diff --git a/internal/cast/write_test.go b/internal/cast/write_test.go new file mode 100644 index 0000000..883dd19 --- /dev/null +++ b/internal/cast/write_test.go @@ -0,0 +1,115 @@ +package cast + +import ( + "io" + "os" + "path/filepath" + "strings" + "testing" + + "github.com/REPPL/Testimony/internal/session" + "github.com/REPPL/Testimony/internal/timeline" +) + +func TestEncodeRecordMatchesWriteJSONL(t *testing.T) { + rec := timeline.Interaction{T: testT0, Kind: OutputKind, Text: "a & c\r\n"} + line, err := encodeRecord(rec) + if err != nil { + t.Fatal(err) + } + // HTML escaping is off, matching session.WriteJSONL, so a literal <, >, or & + // stays one byte rather than inflating to a six-byte escape. + if !strings.Contains(string(line), "a & c") { + t.Errorf("encodeRecord escaped HTML: %s", line) + } + if strings.HasSuffix(string(line), "\n") { + t.Errorf("encodeRecord kept the terminating newline: %q", line) + } + n, err := session.EncodedLen(rec) + if err != nil { + t.Fatal(err) + } + if len(line)+1 != n { + t.Errorf("encodeRecord wrote %d bytes, session.EncodedLen measures %d", len(line)+1, n) + } +} + +// TestRewriteTerminatesAnUnterminatedFinalLine is the one byte this rewrite +// changes: without the newline the first imported record would be appended onto +// a malformed final line and neither would survive a read back. +func TestRewriteTerminatesAnUnterminatedFinalLine(t *testing.T) { + dir := newSession(t, testT0) + path := filepath.Join(dir, session.InteractionsFile) + foreign := `{"t":1784300419200,"kind":"click"}` + if err := os.WriteFile(path, []byte(foreign), 0o644); err != nil { + t.Fatal(err) + } + rec := timeline.Interaction{T: testT0, Kind: OutputKind, Text: "out\r\n"} + if _, err := rewriteInteractions(dir, "session.cast", testT0, []timeline.Interaction{rec}); err != nil { + t.Fatalf("rewriteInteractions: %v", err) + } + lines := readLines(t, path) + if len(lines) != 2 || lines[0] != foreign { + t.Fatalf("lines = %q, want the foreign line kept and the record appended", lines) + } +} + +func TestRewriteRefusesOverlongExistingLine(t *testing.T) { + dir := newSession(t, testT0) + path := filepath.Join(dir, session.InteractionsFile) + long := `{"t":1784300419200,"kind":"click","text":"` + strings.Repeat("x", session.MaxJSONLLine) + `"}` + if err := os.WriteFile(path, []byte(long+"\n"), 0o644); err != nil { + t.Fatal(err) + } + rec := timeline.Interaction{T: testT0, Kind: OutputKind, Text: "out\r\n"} + _, err := rewriteInteractions(dir, "session.cast", testT0, []timeline.Interaction{rec}) + if err == nil || !strings.Contains(err.Error(), "line exceeds") { + t.Fatalf("error = %v, want an over-long-line refusal", err) + } + // The refusal is before the write, so the file is as it was. + b, err := os.ReadFile(path) + if err != nil { + t.Fatal(err) + } + if string(b) != long+"\n" { + t.Error("the refused rewrite changed interactions.jsonl") + } +} + +func TestCommitCastRenamesIntoPlace(t *testing.T) { + dir := newSession(t, testT0) + src := filepath.Join(t.TempDir(), "session.cast") + if err := os.WriteFile(src, []byte("{\"version\":2}\n"), 0o644); err != nil { + t.Fatal(err) + } + f, err := os.Open(src) + if err != nil { + t.Fatal(err) + } + defer f.Close() + // Read the descriptor to end of file first, as Run's scan leaves it, so the + // rewind stageCast performs is what the copy depends on. + if _, err := io.Copy(io.Discard, f); err != nil { + t.Fatal(err) + } + tmpPath, err := stageCast(dir, src, f) + if err != nil { + t.Fatalf("stageCast: %v", err) + } + if filepath.Dir(tmpPath) != dir { + t.Errorf("staged outside the session directory: %s", tmpPath) + } + if _, err := os.Stat(filepath.Join(dir, session.TerminalCastFile)); !os.IsNotExist(err) { + t.Error("stageCast committed the archive itself") + } + if err := commitCast(tmpPath); err != nil { + t.Fatalf("commitCast: %v", err) + } + b, err := os.ReadFile(filepath.Join(dir, session.TerminalCastFile)) + if err != nil { + t.Fatal(err) + } + if string(b) != "{\"version\":2}\n" { + t.Errorf("terminal.cast = %q", b) + } +} diff --git a/internal/cli/cli.go b/internal/cli/cli.go index 461cb12..48dc716 100644 --- a/internal/cli/cli.go +++ b/internal/cli/cli.go @@ -12,6 +12,7 @@ import ( "time" "github.com/REPPL/Testimony/internal/analyze" + "github.com/REPPL/Testimony/internal/cast" "github.com/REPPL/Testimony/internal/demo" "github.com/REPPL/Testimony/internal/drafttests" "github.com/REPPL/Testimony/internal/record" @@ -34,6 +35,8 @@ Usage: testimony transcribe [-session DIR] [-audio FILE] transcribe a voice recording into transcript.jsonl (reuses the session's audio.wav when -audio is omitted) [-engine auto|whisperx|whispercpp] [-model large-v3-turbo] [-language en] [-offset SECONDS] [-device auto|cpu|cuda] [-compute_type auto|int8|float16|…] [-vad auto|silero|pyannote] (whisperx only) + testimony import [-session DIR] [-cast FILE] import an asciinema recording's output into interactions.jsonl (reuses the session's terminal.cast when -cast is omitted) + [-offset SECONDS] testimony merge [-session DIR] merge transcript + interactions into timeline.jsonl testimony report [-session DIR] [-window 2.5] render timeline.jsonl as a Markdown report testimony analyze [-session DIR] [-out FILE] emit the analysis request (rubric + timeline) on stdout or to FILE @@ -49,9 +52,10 @@ Usage: testimony help A session directory is described in docs/reference/session-directory.md. -Omitting -session on transcribe, merge, report, analyze, draft-tests, or review -uses the current directory when it holds a Testimony session manifest.json (one -with a session field), and names the inferred session on stderr. +Omitting -session on transcribe, import, merge, report, analyze, draft-tests, or +review uses the current directory when it holds a Testimony session +manifest.json (one with a session field), and names the inferred session on +stderr. ` // Run executes the CLI and returns a process exit code. @@ -318,6 +322,73 @@ func Run(args []string) int { fmt.Printf("transcribed %d utterances → %s\n", n, filepath.Join(sess, session.TranscriptFile)) return 0 + case "import": + fs := flag.NewFlagSet("import", flag.ExitOnError) + dir := fs.String("session", "", "session directory") + castFile := fs.String("cast", "", "asciicast file (v2 or v3); omit to reuse the session's terminal.cast") + offset := fs.Float64("offset", 0, "cast→session clock offset in seconds (default: derived from the cast header's timestamp)") + fs.Parse(rest) + if err := rejectArgs(fs); err != nil { + return usageErr(err) + } + castSet, offsetSet := false, false + fs.Visit(func(f *flag.Flag) { + switch f.Name { + case "cast": + castSet = true + case "offset": + offsetSet = true + } + }) + // An explicitly-empty -cast is a wrong invocation (an unset shell variable + // spliced into the flag, say), not "omit -cast" — transcribe's -audio + // precedent, and for the same reason: left unchecked it silently selects the + // in-place branch, re-importing the session's own terminal.cast instead of + // the file the caller named, at exit 0. + // + // -cast carries no extension check, unlike -audio's closed .m4a/.mov/.wav + // set. That set exists because ffmpeg accepts only those containers; here the + // cast header's version field is the authority on whether a file is + // importable, and a name rule would refuse a legitimately-named cast + // (`asciinema rec` writes whatever name the operator gives it, and a + // redirected recording may carry no extension at all). + if castSet && *castFile == "" { + return usageErr(fmt.Errorf("import: -cast must not be empty")) + } + // The same bound transcribe's -offset obeys, from the same function, refused + // at exit 2 before anything reads the cast: a non-finite offset has no + // millisecond value to round to, and a finite but absurd one would write + // records merge refuses one command later, naming interactions.jsonl rather + // than the flag. + if offsetSet { + if err := transcribe.CheckOffset(*offset); err != nil { + return usageErr(fmt.Errorf("import: %w", err)) + } + } + // Resolved last of the invocation checks, as on every other pipeline + // command: import is one of them, so -session obeys the one shared rule + // rather than a required-flag refusal of its own, and a run refused for + // another flag announces no session it never used. + sess, err := resolveSession(fs, *dir) + if err != nil { + return usageErr(err) + } + // The offset provenance line, the dropped-event counts, and the replaced-record + // count go to stderr, beside resolveSession's own inference line, so stdout + // carries just the one summary line a script reads. + n, err := cast.Run(cast.Options{ + SessionDir: sess, + Cast: *castFile, + Offset: *offset, + OffsetSet: offsetSet, + Log: os.Stderr, + }) + if err != nil { + return fail(err) + } + fmt.Printf("imported %d records → %s\n", n, filepath.Join(sess, session.InteractionsFile)) + return 0 + case "analyze": fs := flag.NewFlagSet("analyze", flag.ExitOnError) dir := fs.String("session", "", "session directory") diff --git a/internal/cli/cli_test.go b/internal/cli/cli_test.go index 9d6b1a0..4c50d7d 100644 --- a/internal/cli/cli_test.go +++ b/internal/cli/cli_test.go @@ -1,6 +1,7 @@ package cli import ( + "fmt" "io" "os" "path/filepath" @@ -172,6 +173,7 @@ func TestStrayPositionalIsAUsageError(t *testing.T) { {"merge", "-session", dir, "junk"}, {"report", "-session", dir, "junk", "-window", "NaN"}, {"transcribe", "-session", dir, "junk", "-offset", "99"}, + {"import", "-session", dir, "junk", "-offset", "99"}, {"analyze", "-session", dir, "junk", "-out", "x", "-ingest", "-"}, {"review", "-session", dir, "junk", "-finding", "F-001", "-verdict", "confirmed"}, {"draft-tests", "-session", dir, "junk", "-render"}, @@ -252,6 +254,9 @@ func TestInvalidFlagValuesExitTwo(t *testing.T) { {[]string{"transcribe", "-session", dir, "-engine", ""}, `transcribe: -engine must not be empty`}, {[]string{"transcribe", "-session", dir, "-device", ""}, `transcribe: -device must not be empty`}, {[]string{"transcribe", "-session", dir, "-vad", ""}, `transcribe: -vad must not be empty`}, + {[]string{"import", "-session", dir, "-cast", ""}, `import: -cast must not be empty`}, + {[]string{"import", "-session", dir, "-offset", "NaN"}, `import: -offset must be a finite number of seconds, got NaN`}, + {[]string{"import", "-session", dir, "-offset", "1e10"}, `import: -offset 1e+10 exceeds 1e+09 seconds in magnitude`}, {[]string{"review", "-session", dir, "-finding", "", "-verdict", ""}, `review: -finding must not be empty`}, {[]string{"review", "-session", dir, "-finding", "F-001", "-verdict", ""}, `review: -verdict must not be empty`}, {[]string{"review", "-session", dir, "-finding", "F-001", "-verdict", "duplicate-of-F-001"}, `review: -finding cannot be a duplicate of itself`}, @@ -299,7 +304,7 @@ func TestInvalidFlagValuesExitTwo(t *testing.T) { func TestUsageListsEveryFlagAndCommand(t *testing.T) { for _, want := range []string{"-commit HASH", "testimony help", "testimony draft-tests", "-window 10", "-kind findings|tests", "-decision edited -edit FILE", - "transcribe, merge, report, analyze, draft-tests, or review"} { + "transcribe, import, merge, report, analyze, draft-tests, or"} { if !strings.Contains(usage, want) { t.Errorf("usage text does not mention %q", want) } @@ -318,7 +323,7 @@ func TestMissingSessionIsAUsageError(t *testing.T) { // manifest.json, which is no longer merely incidental now that its // absence is what sends these invocations down the refusal path. chdir(t, t.TempDir()) - for _, cmd := range []string{"merge", "report", "transcribe", "analyze", "draft-tests", "review"} { + for _, cmd := range []string{"merge", "report", "transcribe", "import", "analyze", "draft-tests", "review"} { var code int stderr := captureStderr(t, func() { code = Run([]string{cmd}) }) if code != 2 { @@ -481,7 +486,7 @@ func TestNoSessionAndNoManifestIsAUsageError(t *testing.T) { func TestEmptySessionIsAUsageErrorNotInference(t *testing.T) { dir := miniSession(t) chdir(t, dir) - for _, cmd := range []string{"merge", "report", "transcribe", "analyze", "review"} { + for _, cmd := range []string{"merge", "report", "transcribe", "import", "analyze", "review"} { var code int stderr := captureStderr(t, func() { code = Run([]string{cmd, "-session", ""}) }) if code != 2 { @@ -694,6 +699,7 @@ func TestRefusedInvocationAnnouncesNoSession(t *testing.T) { {"report", "-window", "NaN"}, {"transcribe", "-engine", "bogus"}, {"transcribe", "-offset", "NaN"}, + {"import", "-offset", "NaN"}, {"analyze", "-out", "req.md", "-ingest", "-"}, {"review", "-finding", "F-001"}, } @@ -1031,3 +1037,206 @@ func TestDraftTestsInfersSession(t *testing.T) { t.Errorf("the inference notice reached stdout: %q", req) } } + +// castTestT0 anchors the import tests' session. It matches the t0 the +// session-directory reference's own examples carry, and the fixture cast below +// declares a header timestamp two seconds earlier, so the imported records land +// on a session-relative clock that starts negative and crosses zero. +const ( + castTestT0 = 1784300400000 + castHeaderUnix = 1784300398 +) + +// castSession writes a session with a usable t0 (which import requires on every +// path, since the records it writes are epoch-millisecond-timed) and a small +// asciicast v2 file beside it, and returns both paths. The cast carries a +// keystroke-echoed command, an input event import must drop, and a resize event +// it must count rather than normalise. +func castSession(t *testing.T) (dir, castPath string) { + t.Helper() + dir = t.TempDir() + if err := session.SaveManifest(dir, session.Manifest{ + Session: "s", App: "a shell", Participant: "P1", T0EpochMS: castTestT0, + }); err != nil { + t.Fatalf("SaveManifest: %v", err) + } + lines := []string{ + fmt.Sprintf(`{"version":2,"width":80,"height":24,"timestamp":%d}`, castHeaderUnix), + `[0,"o","alice@example.test:~/project$ "]`, + `[0.52,"i","l"]`, + `[0.53,"o","l"]`, + `[0.62,"o","s"]`, + `[0.75,"o","\r\n"]`, + `[0.8,"r","120x40"]`, + `[0.81,"o","docs internal README.md\r\n"]`, + } + castPath = filepath.Join(t.TempDir(), "session.cast") + if err := os.WriteFile(castPath, []byte(strings.Join(lines, "\n")+"\n"), 0o644); err != nil { + t.Fatalf("write cast: %v", err) + } + return dir, castPath +} + +// TestImportWritesTerminalRecords is the well-formed run: exit 0, the summary +// line on stdout, the records in interactions.jsonl, and the cast archived in +// the session so a later bare re-import has something to read. +func TestImportWritesTerminalRecords(t *testing.T) { + dir, castPath := castSession(t) + var code int + var stderr string + stdout := captureStdout(t, func() { + stderr = captureStderr(t, func() { + code = Run([]string{"import", "-session", dir, "-cast", castPath}) + }) + }) + if code != 0 { + t.Fatalf("import: exit %d, want 0 (stderr %q)", code, stderr) + } + want := "imported 3 records → " + filepath.Join(dir, session.InteractionsFile) + if !strings.Contains(stdout, want) { + t.Errorf("want %q on stdout, got %q", want, stdout) + } + b, err := os.ReadFile(filepath.Join(dir, session.InteractionsFile)) + if err != nil { + t.Fatalf("read interactions: %v", err) + } + for _, want := range []string{`"kind":"terminal_output"`, `"text":"ls\r\n"`, `"t":1784300398000`} { + if !strings.Contains(string(b), want) { + t.Errorf("interactions.jsonl does not contain %q: %s", want, b) + } + } + // A keystroke must never reach the derived text, whatever the recorder did. + if strings.Contains(string(b), `"text":"l"`) { + t.Errorf("an input event reached interactions.jsonl: %s", b) + } + archived, err := os.ReadFile(filepath.Join(dir, session.TerminalCastFile)) + if err != nil { + t.Fatalf("read archived cast: %v", err) + } + original, err := os.ReadFile(castPath) + if err != nil { + t.Fatal(err) + } + if string(archived) != string(original) { + t.Errorf("terminal.cast is not byte-identical to the imported file") + } +} + +// TestImportDiagnosticsStayOffStdout pins the stream split: the offset +// provenance line and the dropped-event counts are diagnostics that belong +// beside the session-inference line on stderr, so stdout carries only the one +// summary line a script reads. +func TestImportDiagnosticsStayOffStdout(t *testing.T) { + dir, castPath := castSession(t) + var code int + var stderr string + stdout := captureStdout(t, func() { + stderr = captureStderr(t, func() { + code = Run([]string{"import", "-session", dir, "-cast", castPath}) + }) + }) + if code != 0 { + t.Fatalf("import: exit %d, want 0 (stderr %q)", code, stderr) + } + for _, want := range []string{ + "offset: -2.00s (derived: cast header timestamp − manifest t0 (whole seconds, ±1s))", + "dropped 1 input (i) event(s): keystrokes are never imported", + "dropped 1 other event(s): 1 resize (r)", + } { + if !strings.Contains(stderr, want) { + t.Errorf("want %q on stderr, got %q", want, stderr) + } + } + if strings.Contains(stdout, "offset:") || strings.Contains(stdout, "dropped") { + t.Errorf("import's diagnostics reached stdout: %q", stdout) + } + if lines := strings.Count(strings.TrimSpace(stdout), "\n"); lines != 0 { + t.Errorf("stdout carries %d extra line(s) beyond the summary: %q", lines, stdout) + } +} + +// TestImportInfersSessionAndReImportsInPlace is the pair the how-to's +// offset-correction recipe rests on: import resolves -session by the same +// shared rule as every other pipeline command, and with -cast omitted it +// re-reads the session's own terminal.cast, so correcting a wrong offset is one +// command with no file to find again. +func TestImportInfersSessionAndReImportsInPlace(t *testing.T) { + dir, castPath := castSession(t) + captureStdout(t, func() { + captureStderr(t, func() { + if code := Run([]string{"import", "-session", dir, "-cast", castPath}); code != 0 { + t.Fatalf("first import: exit %d, want 0", code) + } + }) + }) + chdir(t, dir) + + var code int + var stderr string + stdout := captureStdout(t, func() { + stderr = captureStderr(t, func() { code = Run([]string{"import", "-offset", "-12.4"}) }) + }) + if code != 0 { + t.Fatalf("bare import from inside a session: exit %d, want 0 (stderr %q)", code, stderr) + } + for _, want := range []string{ + "import: using session . (inferred from the current directory)", + "offset: -12.40s (from -offset flag)", + "replaced 3 terminal_output record(s) from an earlier import", + } { + if !strings.Contains(stderr, want) { + t.Errorf("want %q on stderr, got %q", want, stderr) + } + } + if want := "imported 3 records → " + session.InteractionsFile; !strings.Contains(stdout, want) { + t.Errorf("want %q on stdout, got %q", want, stdout) + } + // The explicit offset replaced the derived one rather than being applied on + // top of it: the first record sits 12.4 s before t0, not 14.4 s. + b, err := os.ReadFile(session.InteractionsFile) + if err != nil { + t.Fatalf("read interactions: %v", err) + } + if want := `"t":1784300387600`; !strings.Contains(string(b), want) { + t.Errorf("interactions.jsonl does not carry the corrected time %s: %s", want, b) + } + if strings.Count(string(b), `"kind":"terminal_output"`) != 3 { + t.Errorf("the re-import did not replace the first import's records: %s", b) + } +} + +// TestImportRefusesUnreadableCastAtRuntime keeps the exit-status contract: +// a well-formed invocation whose cast cannot be read is a runtime failure (1), +// not a usage error, so a script can tell a mistyped flag from a missing file. +func TestImportRefusesUnreadableCastAtRuntime(t *testing.T) { + dir, _ := castSession(t) + var code int + stderr := captureStderr(t, func() { + code = Run([]string{"import", "-session", dir, "-cast", filepath.Join(t.TempDir(), "absent.cast")}) + }) + if code != 1 { + t.Errorf("import with an absent -cast: exit %d, want 1 (runtime error)", code) + } + if want := "testimony: cast file:"; !strings.Contains(stderr, want) { + t.Errorf("want %q on stderr, got %q", want, stderr) + } +} + +// TestUsageListsImport pins the surface change the intent's last criterion +// asks for: the new verb is visible in the usage text, and record's own flag +// set is untouched by it. +func TestUsageListsImport(t *testing.T) { + for _, want := range []string{ + "testimony import [-session DIR] [-cast FILE]", + "[-offset SECONDS]", + "import an asciinema recording's output into interactions.jsonl", + "Omitting -session on transcribe, import, merge, report, analyze, draft-tests, or", + } { + if !strings.Contains(usage, want) { + t.Errorf("usage text does not mention %q", want) + } + } + if strings.Contains(usage, "-terminal") { + t.Error("usage text offers a -terminal flag; terminal capture is a hand-off, not a record mode") + } +} diff --git a/internal/session/session.go b/internal/session/session.go index 321f830..f69af32 100644 --- a/internal/session/session.go +++ b/internal/session/session.go @@ -8,6 +8,7 @@ // audio.offset.json audio→session offset for an external recording (local only) // screen.mp4 screen recording (local only; -video capture) // events.rrweb.jsonl raw rrweb events (archival; web sessions only) +// terminal.cast raw asciicast (archival; written by import; local only) // interactions.jsonl normalised interaction events (epoch ms) // transcript.jsonl word-aligned utterances (session-relative seconds) // timeline.jsonl merged, session-relative timeline @@ -52,6 +53,7 @@ const ( AudioOffsetFile = "audio.offset.json" ScreenFile = "screen.mp4" RawEventsFile = "events.rrweb.jsonl" + TerminalCastFile = "terminal.cast" InteractionsFile = "interactions.jsonl" TranscriptFile = "transcript.jsonl" TimelineFile = "timeline.jsonl" diff --git a/internal/transcribe/transcribe.go b/internal/transcribe/transcribe.go index 73b5f3a..464fd10 100644 --- a/internal/transcribe/transcribe.go +++ b/internal/transcribe/transcribe.go @@ -384,12 +384,18 @@ func checkSegmentTime(v float64) bool { // a sidecar readOffsetSidecar itself refuses on the next bare run. No // genuine offset is refused: a value past ±1e9 seconds already fails every // downstream reader. +// +// It is the one home for the rule, so `import` validates its own -offset here +// too (cast.Run and the CLI both call it): the cast→session offset obeys the +// same finiteness and magnitude bound for the same reason, and two copies of a +// numeric bound are two places for it to drift. The message therefore names a +// recording→session offset rather than an audio-specific one. func CheckOffset(v float64) error { if math.IsNaN(v) || math.IsInf(v, 0) { return fmt.Errorf("-offset must be a finite number of seconds, got %v", v) } if math.Abs(v) > maxOffsetSeconds { - return fmt.Errorf("-offset %g exceeds %g seconds in magnitude; no audio→session offset is that large", v, maxOffsetSeconds) + return fmt.Errorf("-offset %g exceeds %g seconds in magnitude; no recording→session offset is that large", v, maxOffsetSeconds) } return nil }