Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .abcd/development/brief/04-surfaces/08-import.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# `testimony import`

Normalises an operator-recorded terminal session — an asciinema recording, in
either asciicast format — into a session's `interactions.jsonl` on the shared
session clock, and keeps the raw `.cast` in the session as an archival
`terminal.cast`. It is `transcribe -audio`'s peer for the terminal: an artefact
the CLI never produced, an anchor read out of that artefact's own metadata, an
explicit `-offset` that always wins, a mandatory printed provenance line, an
idempotent re-run, and an all-or-nothing write. Nothing here calls a model,
spawns a process, allocates a pty, or touches the network.

`record` is deliberately untouched: no `-terminal` flag, no wrapped recorder,
no second way for a session to end. The operator runs their own recorder in the
window where the work happens, and hands the file over afterwards — the pattern
`demo` already uses for QuickTime and external audio.

## Flags

| Flag | Default | Meaning |
|---|---|---|
| `-session` | (required) | session directory |
| `-cast` | (optional) | asciicast file to import; omit to re-import the session's own `terminal.cast` |
| `-offset` | derived | cast→session clock offset in seconds |

## Behaviour

- Accepts **asciicast v2 and v3**, told apart by the header's `version` field:
v2 event times are absolute seconds since recording start, v3 event times are
intervals since the previous event, reconstructed by a running sum. Any other
version is refused by name. The difference is confined to one accumulator, and
that accumulator runs on an exact integer grain — microseconds, finer than
either format writes — so the same recording in either format yields
byte-identical records. A `float64`-seconds clock does not give that: v2 rounds
a stated time while v3 rounds a running sum, and at a half-millisecond tie the
two land on different milliseconds, which is enough to flip a coalescing cut.
- Resolves the cast→session offset in `transcribe`'s order: an explicit
`-offset` wins; otherwise the offset is derived from the header's `timestamp`
minus the manifest's `t0_epoch_ms`, in exact integer arithmetic; otherwise 0.
The offset and its provenance are always printed, and the derived provenance
carries the `(whole seconds, ±1s)` caveat, because the header field is an
integer in both formats and the report's default join window is 2.5 s. No
sidecar is persisted: the archived cast keeps its own header, so a re-import
re-derives the identical offset from the identical bytes.
- Requires a usable `t0` on every path, unlike `transcribe`: the records are
epoch-millisecond-timed and `-offset` is defined against the session clock,
and `merge` already refuses a session with interactions and no anchor.
- Keeps only `o` (output) events. `i` (input), `r` (resize), `m` (marker), `x`
(exit), and any unrecognised code are dropped and counted by code, with the
counts printed. The tally is bounded against a cast carrying a different code
on every line, but the input count is exempt from that bound: it is a privacy
disclosure rather than a scoping note, and must not be suppressible. Dropping input is a privacy requirement, not a
simplification: a cast recorded with input capture still cannot put
keystrokes into the derived text. An unrecognised code is dropped rather than
refused, so a future asciicast revision does not make its casts unimportable.
- Coalesces adjacent output into one record per line the terminal displayed,
closed at the first of a newline, a 250 ms inter-event gap, a 1 s span cap
measured from the record's first event, or the encoded JSONL line budget. A
record's time is the instant its first rune arrived. Carriage returns are kept
and are not a boundary, so a redrawn progress line is one record rather than
one per frame. A record whose text renders empty is dropped and counted.
- Splits a single oversized output event across consecutive records at rune
boundaries, budgeted against the **encoded** length of the timeline entry
`merge` will wrap the record in — escaping is what consumes the budget, an
escape byte costing six bytes — and every finished record is then measured for
real, so a wrong assumption about the encoder costs a refusal rather than a
session no command can read back.
- Writes `interactions.jsonl` whole and atomically: records from an earlier
import (identified by `kind: "terminal_output"`, and nothing else) are
dropped, every other line is kept byte-for-byte in file order, and the new
records are appended. So a re-import is byte-identical, a `-demo` session's
clicks survive untouched, and an import that would yield zero records refuses
rather than erase — naming which of the two cases it hit, no output events at
all or output that all rendered empty. Before writing, the assembly is checked
against the line and file limits `session.ReadJSONL` enforces, and the merged
timeline the import implies is measured against the file limit too — the case
only an offline importer can compute rather than estimate — with each entry
charged the id growth `merge` adds past the thousandth interaction, so a
session this pre-flight passes is one `merge` can still read back.
- Archives the cast in two phases: the copy is staged into a temp file beside
`terminal.cast`, the records are written, and the staged copy is renamed into
place last. A failure anywhere before that rename leaves the session exactly
as it was; the one residual state is records with no archival copy, which is
the less misleading of the two. With `-cast` omitted there is no copy phase.
- Keeps ANSI escape sequences raw in the record, because the record is
evidence and a hand-written escape-sequence parser would corrupt it rather
than merely litter it. `report`'s existing sink strips the escape byte, so no
terminal control sequence reaches `report.md`; the printable residue is
answered by recording guidance (`NO_COLOR=1`), not by code.
- Requires no change to `merge`, `report`, `analyze`, or `review`: the records
carry only `t`, `kind`, and `text`, `kind` is an open set, and the timeline
learns no new `src` value. Output schema:
[`../05-internals/02-schemas.md`](../05-internals/02-schemas.md).
14 changes: 11 additions & 3 deletions .abcd/development/brief/05-internals/01-packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ Everything lives under `internal/`; `cmd/testimony/main.go` is a thin
entrypoint that calls `cli.Run` and exits with its return code.

- **`internal/cli`** — the command-line interface: usage text, one
`flag.FlagSet` per subcommand (`demo`, `record`, `transcribe`, `merge`,
`report`, `analyze`, `draft-tests`, `review`, `version`, `help`), and dispatch
into the other packages. Holds the `Version` variable stamped by the release
`flag.FlagSet` per subcommand (`demo`, `record`, `transcribe`, `import`,
`merge`, `report`, `analyze`, `draft-tests`, `review`, `version`, `help`), and
dispatch into the other packages. Holds the `Version` variable stamped by the release
process. Errors print as `testimony: <err>` and map to exit codes (1
failure, 2 usage).
- **`internal/demo`** — the instrumented demo app: an embedded single-page
Expand Down Expand Up @@ -40,6 +40,14 @@ entrypoint that calls `cli.Run` and exits with its return code.
engines' JSON output files into engine-neutral segments, and the mapping of
segments to the `Utterance` schema. Fixture-tested against golden JSONL
files in `testdata/`.
- **`internal/cast`** — the terminal import path: a streaming asciicast reader
that resolves v2's absolute times and v3's interval sums into one absolute
recording clock, the offset resolution from the cast header's own timestamp,
the coalescer that turns adjacent output events into one record per displayed
line, and the all-or-nothing rewrite of `interactions.jsonl` that replaces
only the records this importer wrote. Named for the artefact it parses because
`import` is a Go keyword. Fixture-tested against a v2/v3 pair describing one
recording, plus a golden JSONL file, in `testdata/`.
- **`internal/report`** — Markdown rendering of a merged timeline: the
event↔utterance attachment pass (the same window test as
`timeline.EventsNear`, inlined and keyed by position so the join does not
Expand Down
18 changes: 18 additions & 0 deletions .abcd/development/brief/05-internals/02-schemas.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ sessions/<timestamp>/
audio.offset.json # audio→session offset for an external recording (written by transcribe; local only)
screen.mp4 # screen capture (written by record -video; local only)
events.rrweb.jsonl # raw rrweb events (archival; web sessions only)
terminal.cast # raw asciicast (archival; written by import; local only)
interactions.jsonl # normalised interaction events (epoch ms)
transcript.jsonl # word-aligned utterances (session-relative seconds)
timeline.jsonl # merged, session-relative timeline
Expand Down Expand Up @@ -61,6 +62,23 @@ schema changes update code, sample, and tests together
| `value` | string | optional input value |
| `route` | string | optional |

`kind` is an open set with one reserved value: `terminal_output`, written only
by [`import`](../04-surfaces/08-import.md), which identifies its own earlier
records by that value alone. Such a record carries `t`, `kind`, and `text` only;
one record is one line the terminal displayed, its `text` keeping carriage
returns and ANSI escape sequences verbatim.

## `terminal.cast` — one asciicast, as recorded

Not a session schema this project defines: the file is an
[asciicast](https://docs.asciinema.org/manual/asciicast/v2/) v2 or v3 recording,
copied into the session byte-for-byte by `import` and read back by a later
`import` that omits `-cast`. Archival only — nothing downstream reads it — and
local only, since it holds every event the recorder captured, keystrokes
included. `import` reads only the header's `version` and `timestamp`, ignoring
every other header field, and bounds a read at 16 MiB per line and 64 MiB per
file.

## `timeline.jsonl` — one `Entry` per line

| Field | Type | Notes |
Expand Down
2 changes: 1 addition & 1 deletion .abcd/development/brief/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ note is preserved as drafted in
- [`02-dependencies.md`](02-constraints/02-dependencies.md) — zero Go dependencies; external capability means a subprocess.
- [`03-invariants.md`](02-constraints/03-invariants.md) — the one-wall-clock rule, the privacy boundary, schema discipline.
- [`04-ethics.md`](02-constraints/04-ethics.md) — validity limits and the privacy/ethics posture.
- [`04-surfaces/`](04-surfaces/) — one file per command: [`demo`](04-surfaces/01-demo.md), [`transcribe`](04-surfaces/02-transcribe.md), [`merge`](04-surfaces/03-merge.md), [`report`](04-surfaces/04-report.md), [`record`](04-surfaces/05-record.md), [`analyze`](04-surfaces/06-analyze.md), [`review`](04-surfaces/07-review.md).
- [`04-surfaces/`](04-surfaces/) — one file per command: [`demo`](04-surfaces/01-demo.md), [`transcribe`](04-surfaces/02-transcribe.md), [`merge`](04-surfaces/03-merge.md), [`report`](04-surfaces/04-report.md), [`record`](04-surfaces/05-record.md), [`analyze`](04-surfaces/06-analyze.md), [`review`](04-surfaces/07-review.md), [`import`](04-surfaces/08-import.md).
- [`05-internals/`](05-internals/)
- [`01-packages.md`](05-internals/01-packages.md) — the package map under `internal/`.
- [`02-schemas.md`](05-internals/02-schemas.md) — the JSON schemas of the session artefacts.
Expand Down
Loading