testimony <command> [flags]
Running testimony with no command, or with an unknown command, prints the usage text and exits with status 2.
| Status | Meaning |
|---|---|
| 0 | success |
| 1 | runtime error — the message is printed to stderr as testimony: <error> |
| 2 | usage error — no command, an unknown command, a stray positional argument (no command takes one), an unparseable or invalid flag value, a missing required flag, or a flag combination that is not allowed |
The seven pipeline commands — transcribe, import, merge, report, analyze, draft-tests, and review — take their session directory from -session DIR. When -session is omitted and the current directory itself holds a Testimony session manifest.json, that directory is the session: the command operates on it exactly as -session . does, and prints one line to stderr naming what it inferred (merge: using session . (inferred from the current directory)) before it starts work, so the implicit choice is visible in the output of the run. The line goes to stderr, never stdout, so analyze's emitted request stays a clean pipe. An explicit -session always wins, is used verbatim, and prints no such line — the current directory is not consulted at all.
The marker is a session manifest, not merely the file name. manifest.json is one of the most common file names in software, so the file must be a regular file (a directory or a symlink at that name is not a marker) and, when it parses, must carry the session field every testimony session has (see manifest.json). A manifest.json that belongs to something else leaves the command refusing rather than writing into a directory that is not a session:
report: -session is required (the current directory holds a manifest.json, but it is not a session manifest: no "session" field)
A session manifest.json that fails to parse still counts as the marker, so a corrupt session reports its own parse error rather than a claim that there is no session here.
Inference covers the exact current directory only: no parent directory is searched, so a directory inside a session (or above one) is not a session. An explicitly empty -session (an unset shell variable spliced into the flag) is a usage error rather than an omission. With neither an explicit -session nor a session manifest in the current directory, the command exits 2 naming both:
report: -session is required (no -session flag, and the current directory holds no regular manifest.json file)
Resolution is the last of a command's invocation checks, so a run refused for any other flag reports only that flag and announces no session.
record and demo are unaffected: they create sessions rather than operate on an existing one, and their -out root is a different flag with a different meaning — see where a new session lands.
The two capture commands — record and demo — create a session under the -out root. That root defaults to ~/Testimony/sessions, a fixed location resolved against the home directory at the moment of the run, so every session lands in the same place whatever directory the command was run from. The root, and any missing parent of it, is created on demand.
-out DIR overrides it per invocation and is used verbatim, relative or absolute: -out sessions keeps a project-local sessions/ beside the working directory. It is the only override — there is no environment variable and no configuration file.
When -out is omitted and the home directory cannot be resolved, the command refuses at status 2 rather than fall back to a relative root, since a session written beside whichever directory the operator was standing in is exactly what the fixed default exists to prevent:
record: -out is required (the default root ~/Testimony/sessions cannot be resolved: $HOME is not defined); pass -out DIR
Both commands print the session directory they created, as the real path, in their start-up output.
Serves the instrumented demo app and captures a session.
testimony demo [-addr :8737] [-out ~/Testimony/sessions]
| Flag | Default | Meaning |
|---|---|---|
-addr |
:8737 |
listen address (a bare :port binds loopback 127.0.0.1 only) |
-out |
~/Testimony/sessions |
root directory for new session folders, created on demand (see where a new session lands) |
Behaviour: creates a new session directory named after the current time (YYYY-MM-DD_HHMMSS) under the -out root, writes manifest.json (app testimony demo, participant P1, one seeded default task, t0_epoch_ms set to now), serves the demo page at /, and appends captured events via two endpoints:
POST /api/interactions— one JSON object per request, appended as one line ofinteractions.jsonl.POST /api/events— a JSON array per request, each element appended as one line ofevents.rrweb.jsonl.
Both accept POST only (405 otherwise) and require a loopback remote peer, a loopback Host, and — when an Origin header is present — a loopback origin, and Content-Type: application/json (403 for any of the first three, 415 for the last). This guards the unauthenticated write endpoints against cross-site and DNS-rebinding forgery, and against a non-browser client on the same network simply forging a loopback Host. They return 204 on success and 400 on malformed bodies; /api/interactions also refuses with 400 any record merge would refuse — a body that is not a JSON object, or one missing the required t (a positive epoch-millisecond time on a plausible session clock) or kind. POST /api/interactions limits the body to 4 MiB — the readable JSONL line limit, since one request becomes one line — and also refuses with 413 a record that fits that limit alone but whose timeline entry would not once merge wraps it (a src/id/payload envelope on the session-relative clock, see timeline.jsonl), one that would push interactions.jsonl past the session's 16 MiB total-size limit, or one whose own timeline entry would push the session's merged timeline.jsonl past that same limit even while interactions.jsonl itself stays under it (see session-directory.md); POST /api/events limits the batch body to 8 MiB, refusing with 413 a batch over that cap or a batch element that would itself exceed the 4 MiB line limit — events.rrweb.jsonl is archival and carries no total-size cap. A body that passes every check but fails to append (a filesystem error) answers 500 rather than a false 204. Both paths are exact: any other /api/… path answers 404, and every other path serves the demo page to GET and HEAD, answering 405 (Allow: GET, HEAD) to any other method — so a capture post that misses an endpoint by a character is refused rather than answered with the page. Every refused capture write is logged to stderr, the operator's only signal, since the page posts via sendBeacon, which surfaces no status. The command blocks until interrupted (Ctrl+C).
The loopback remote-peer requirement means an explicit non-loopback -addr host (e.g. 0.0.0.0:8737) serves the page to other devices but refuses their capture posts — even one that sends a loopback Host — so only the machine running record/demo itself can ever write evidence; the command warns at startup that such a bind serves the page only.
Transcribes a voice recording into transcript.jsonl using a local ASR engine.
testimony transcribe [-session DIR] [-audio FILE]
[-engine auto|whisperx|whispercpp] [-model large-v3-turbo]
[-language en] [-offset SECONDS]
[-device auto|cpu|cuda] [-compute_type auto|int8|float16|…]
[-vad auto|silero|pyannote]
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-audio |
(optional) | voice recording (.m4a, .mov, or .wav) to convert into the session's audio.wav. Omit to reuse an audio.wav already in the session (as a testimony record session has); required only when the session has none |
-engine |
auto |
ASR engine: auto, whisperx, or whispercpp. auto prefers whisperx on PATH, then whisper-cli |
-model |
large-v3-turbo |
Whisper model name, or (whispercpp) a ggml model file path. A whispercpp model name resolves to ggml-<name>.bin searched in ~/.cache/whisper.cpp, ~/.cache/whisper, ~/.local/share/whisper.cpp, and ~/models |
-language |
en |
spoken language code |
-device |
auto |
(whisperx) inference device: auto, cpu, or cuda. auto picks cuda only when an NVIDIA GPU is present, and never on macOS |
-compute_type |
auto |
(whisperx) compute type: auto, int8, float16, … . auto follows the device: float16 on CUDA, int8 on CPU |
-vad |
auto |
(whisperx) VAD method: auto, silero, or pyannote. auto picks silero; pyannote fails under newer torch versions |
-offset |
derived | audio-to-session clock offset in seconds. When not given: with -audio naming a file other than the session's own audio.wav, derived from the recording's creation time minus the manifest's t0_epoch_ms, or 0 when derivation is impossible; without it (including -audio audio.wav), read back from audio.offset.json when the session has one, else 0. A non-finite value, or one beyond ±10⁹ seconds (the bound every derived or persisted offset already meets), is a usage error |
Behaviour: reads manifest.json (required). With -audio, requires ffmpeg on PATH and converts the recording to 16 kHz mono audio.wav in the session directory; without it (or when -audio points at the session's own audio.wav), it uses the existing audio.wav in place and skips the conversion. It then runs the engine, applies the offset, and writes transcript.jsonl. While the engine is still running, a status line ("whisperx: still running (Ns elapsed)" or the whisper-cli equivalent) prints every 5 seconds, since a CPU-only run can otherwise sit silent for minutes with no other output. Always prints the offset it used and its provenance — one of:
from -offset flag— the explicit flag, which always wins;derived: audio creation_time − manifest t0— derived for an external recording;default 0: audio creation time unavailable— an external recording whose creation time could not be read, or for whichffprobewas not found on PATH;persisted: audio.wav converted from an external recording (+3.20s)— read back fromaudio.offset.json, the printed value being the persisted offset;default 0: session audio.wav captured at t0— a session whoseaudio.wavwas captured here and has no sidecar.
It then prints transcribed N utterances → <path>. With an -audio that names a file other than the session's own audio.wav, the offset in force is written to audio.offset.json; without it (including -audio audio.wav), the sidecar is rewritten only when an explicit -offset is given and the session already has one. A later bare run reuses the persisted value.
Imports an operator-recorded terminal session — an asciinema recording — into the session's interactions.jsonl on the shared session clock, and keeps the raw cast in the session as terminal.cast. It is transcribe -audio's peer for the terminal: the CLI never runs the recorder, so the operator records their own shell and hands the file over afterwards. Nothing here spawns a process, allocates a pty, or touches the network, and record is unaffected — there is no -terminal flag and no change to how a session ends.
testimony import [-session DIR] [-cast FILE] [-offset SECONDS]
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-cast |
(optional) | asciicast file to import. Omit to re-import the session's own terminal.cast, which is what makes correcting an offset a one-line command. A -cast resolving to that same file is the omitted case, so the archive is never copied onto itself. Unlike -audio, no extension is required: the cast's own header decides whether a file is importable |
-offset |
derived | cast-to-session clock offset in seconds — the value added to every cast-clock time to place it on the session clock. A non-finite value, or one beyond ±10⁹ seconds, is a usage error |
What is read. Both asciicast formats are accepted and told apart by the header's version field: v2 event times are absolute seconds since recording start, v3 event times are intervals since the previous event, reconstructed by a running sum. Both are reconstructed onto one exact integer clock — microseconds, which is finer than either format writes — so the same recording in either format yields byte-identical records, and the operator never has to state which their recorder wrote. Any other version is refused by name. The header's version and timestamp are the only fields read; every other one is ignored, since both formats are extensible. A cast is read within two bounds: 16 MiB for a single line and 64 MiB for the whole file.
Which events are kept. Only o (output) becomes records. i (input), r (resize), m (marker), x (exit), and any code this importer does not recognise are dropped and counted by code, with the counts printed. Dropping input is unconditional: a cast recorded with input capture still cannot put keystrokes into the derived text. An unrecognised code is dropped rather than refused, so a later asciicast revision does not make its casts unimportable.
How output becomes records. A shell echoes a typed command back roughly one keystroke at a time, so adjacent output accumulates into one record per line the terminal displayed. A record closes at the first of: a newline (included in the record); an inter-event gap of 250 ms or more, which closes output that never ends in a newline, such as a bare prompt; a span of 1 second measured from the record's first event, so a record's stated time is never more than a second stale against report's 2.5-second join window; or the JSONL line budget. A record's t is the instant its first rune arrived. Carriage returns are kept and are not a boundary, so a progress line redrawing over itself is one record holding its frames rather than one record per frame. A record whose text would render empty — a blank line, a lone carriage return — is dropped and counted.
A single output event too large for the line limit is split across consecutive records at rune boundaries, so concatenating their text reproduces the event exactly and nothing is truncated. The budget is measured against the encoded length of the timeline entry merge will wrap the record in, because escaping is what consumes it: an escape byte costs six bytes encoded, and coloured output is dense in them.
Records carry t, kind, and text and nothing else, with kind always terminal_output — a reserved kind (see interactions.jsonl). ANSI escape sequences are kept verbatim, because the record is evidence; report strips the escape byte at its own boundary, so recording with colour disabled (NO_COLOR=1) is the remedy for the printable residue. See record a terminal session.
The archival copy. The cast is copied into the session as terminal.cast, staged into a temp file beside it before the records are written and renamed into place afterwards, so a failure anywhere before that rename leaves the session exactly as it was. An existing terminal.cast's own file mode is preserved; a new one is created honouring the umask. With -cast omitted there is no copy at all.
Re-running. interactions.jsonl is rewritten whole and atomically: records from an earlier import (identified by their terminal_output kind, and nothing else) are dropped, every other line is kept byte-for-byte in its original order, and the new records are appended. So importing the same cast twice yields a byte-identical file, a record -demo session's clicks and inputs survive untouched, and a line this importer cannot decode at all is preserved as it stands. A session holds one terminal recording: importing a different cast replaces the first one's records and its terminal.cast.
Printed output. The offset in force and its provenance are printed on every run, so the default is never a silent assumption — one of:
from -offset flag— the explicit flag, which always wins;derived: cast header timestamp − manifest t0 (whole seconds, ±1s)— taken from the cast header's own timestamp. The caveat is part of the line because the field is an integer in both formats, so the reconstructed clock can sit up to a second adrift oft0's millisecond precision — material against the 2.5-second default join window, which is why the spoken "session start" marker stays the cross-check;default 0: cast header carries no timestamp— the cast carries no anchor, so the recording is taken to start att0.
The dropped-event counts, the count of records that render empty, and the number of earlier records replaced follow it. Dropped input events are counted on a line of their own, because that count is how you learn a recorder captured keystrokes; the other codes share a line, ordered by code. All of these go to stderr, beside the session-inference line; stdout carries only imported N records → <path>, so a script reads one line.
Failure modes. Exit 2 (a wrong invocation, refused before any work): a missing -session with no session manifest in the current directory, an explicitly empty -session or -cast, a stray positional argument, or an unusable -offset. Exit 1 (a runtime failure): no manifest.json, or one with no usable t0_epoch_ms — required on every path, including with an explicit -offset, since the records are epoch-millisecond-timed; neither -cast nor a terminal.cast to fall back on; a -cast naming a missing or non-regular file; an unsupported version, a malformed header or event, or a line or file past its bound, each naming the line; a header timestamp at or before the epoch, or a derived offset beyond ±10⁹ seconds, both asking for an explicit -offset; a cast holding no importable output — either no output events at all, or output that all rendered empty, each named as such — which refuses rather than erase an earlier import's records; and any size limit the assembled interactions.jsonl — or the merged timeline.jsonl it implies — would cross. Every exit-1 path above fires before any file in the session changes.
Merges the transcript and interaction stream into timeline.jsonl.
testimony merge [-session DIR]
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
Terminal records written by import merge exactly like any other interaction: they carry no new source type, flag, or schema field.
Behaviour: reads manifest.json (required), transcript.jsonl, and interactions.jsonl; converts interaction epoch-millisecond times to session-relative seconds via t0_epoch_ms; writes the time-sorted timeline.jsonl; prints merged N utterances + M events → <path>. A missing transcript.jsonl or interactions.jsonl counts as zero records rather than an error, so a default audio-only record session (which never writes interactions.jsonl) still merges to a speech-only timeline. If the two sources together yield zero entries — missing, empty, or both — and a timeline.jsonl from an earlier merge already exists and is non-empty, merge refuses rather than truncate it to zero entries; a session with no timeline yet, or one already empty, still merges to an empty one. When interactions are present, t0_epoch_ms is required: without it their epoch-millisecond times cannot be placed on the session clock, so merge fails rather than write a corrupt timeline.
Renders timeline.jsonl as a Markdown report.
testimony report [-session DIR] [-window 2.5]
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-window |
2.5 |
utterance-to-event join window, in seconds |
Behaviour: reads manifest.json (required) and timeline.jsonl, plus findings.jsonl when present (the Findings section; without the file it is a short notice pointing at analyze and review; if the file exists but cannot be read, the section reports that instead, and the command still exits 0). A timeline entry whose src is neither speech nor event is refused rather than silently omitted from the rendered record. Attaches each event to the first utterance whose span, widened by the window on both sides, contains it; events matched by no utterance appear as standalone lines. Writes report.md into the session directory and prints wrote <path>.
Managed capture: creates the session directory and manifest, starts the recorders, and runs until interrupted.
testimony record [-out ~/Testimony/sessions] [-app NAME] [-participant P1] [-commit HASH]
[-task ...] [-video|-no-video] [-demo [-addr :8737]]
| Flag | Default | Meaning |
|---|---|---|
-out |
~/Testimony/sessions |
root directory for new session folders, created on demand (see where a new session lands) |
-app |
(empty) | application under test; with -demo, defaults to the demo app |
-participant |
P1 |
participant pseudonym |
-commit |
(empty) | build/commit hash under test |
-task |
(none) | a task the participant will attempt; repeat the flag for several tasks. With -demo and no -task, the demo app's default task is seeded, matching a standalone demo session |
-video |
off | also capture the screen to screen.mp4 (needs Screen Recording permission) |
-no-video |
— | explicitly disable screen capture; this is the default, and it wins when both -video and -no-video are given |
-demo |
off | also serve the instrumented demo app into the same session directory |
-addr |
:8737 |
demo server listen address (with -demo) |
Behaviour: creates a new session directory named after the current time (YYYY-MM-DD_HHMMSS) under the -out root and writes manifest.json (app, participant, tasks, commit, t0_epoch_ms set to now) through the same code path as demo, refusing the write if long -task/-app text would push it past its own 1 MiB size limit (see session-directory.md). On macOS it captures the default microphone to audio.wav (16 kHz mono PCM, the canonical ASR input — no re-conversion needed downstream) and, with -video, the screen to screen.mp4; both recorders need ffmpeg on PATH. Audio-only is the default; -video opts in. With -demo it also serves the demo app so one command captures voice and clicks into the same directory.
The command blocks until interrupted (Ctrl+C). On SIGINT/SIGTERM — and on SIGHUP, so closing the terminal window mid-session finalises exactly like Ctrl+C — it sends each recorder an interrupt so it finalises its container, waits up to five seconds, and hard-kills only on timeout. It then validates each recorder's artefact — audio.wav, and screen.mp4 with -video — and prints the exact next commands with the real session directory: with a usable audio.wav in place it offers transcribe → merge → report without -audio, because the recording is already present.
Any of the following leaves the command exiting with status 1, each diagnosed differently, and each appending the recorder's own output for diagnosis:
- A recorder that leaves no usable artefact at all — most often because its macOS permission was never granted, so it blocked on the prompt and captured nothing until it was stopped — names the missing file and points at the exact System Settings pane (Privacy & Security → Microphone, or → Screen Recording).
- A recorder that instead exits on its own before it is asked to stop is diagnosed by when: at start-up with ffmpeg's own permissions-failure signature, the same System Settings pane is named; an early exit without that signature, or an exit after the session was already underway, is reported without a pane pointer, since the cause is not provably a permission.
- A recorder that is still running when asked to stop but does not finalise its container within five seconds is force-killed and flagged as likely truncated or unplayable even when it left data — a microphone recording usually survives a kill intact and is still offered to
transcribe, but the command still exits 1 to flag the risk.
When there is no usable audio.wav, the next-command block omits the bare transcribe line and instead keeps merge and report (interactions may still be captured) and explains how to get audio: on a platform with microphone capture, re-run record after granting the permission; either way, transcribe an external recording with -audio FILE.
On platforms other than macOS, audio and screen capture are unavailable, and the status output states what was skipped. Without -demo, the command exits 0 immediately: it still writes a valid manifest and session directory. With -demo it still serves the demo app and captures clicks — the demo server is not macOS-specific — so the command blocks until interrupted exactly as it does on macOS; the next-command block omits the bare transcribe line because there is no audio.
The first-pass analysis layer. analyze never calls a model, holds no keys, and adds no network dependency: it emits a self-contained analysis request that any assistant (or a human) runs, then ingests and validates the JSON answer into findings.jsonl.
testimony analyze [-session DIR] [-out FILE] # emit the request
testimony analyze [-session DIR] -ingest FILE [-backend local|cloud] [-model NAME] # validate the answer → findings.jsonl
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-out |
(stdout) | emit mode: write the request to FILE instead of stdout |
-ingest |
(off) | ingest mode: validate the answer JSON at FILE (or - for stdin) into findings.jsonl |
-backend |
(unrecorded) | ingest mode: record which backend answered the request — local or cloud |
-model |
(not recorded) | ingest mode: record the model that answered the request; free text, at most 200 characters, and refused when it renders as nothing (whitespace, invisible characters, or backticks alone) |
analyze runs in exactly one mode: emit (no -ingest) or ingest (-ingest). Combining -out and -ingest is an error. -backend and -model belong to ingest alone — emit mutates nothing in the session directory, so there is nothing for them to be recorded against — and passing either in emit mode is a usage error rather than a silently ignored flag. -model without -backend is a usage error too: the backend is the field that carries the claim, so a model name on its own records nothing about where the request ran. unrecorded is not accepted from -backend; it is what the absence of the flag records. Emit reads manifest.json and timeline.jsonl; ingest reads timeline.jsonl only. Both hint to run merge first when the timeline is missing, and both refuse a timeline whose entries carry a src other than speech or event, or a duplicated entry id — findings cite evidence by id, so a reused one cannot be resolved unambiguously (a merge-produced timeline never carries either defect: merge refuses a transcript whose utterance ids repeat or collide with the ev-NNN event ids it synthesises).
Emit behaviour: writes a single self-contained prompt — the rubric version header (testimony-analysis/v1), the second-coder stance, two-pass instructions (segment coding, then session synthesis), the rubric body (five type definitions, the 1..4 severity scale, the evidence hard-constraints), the session context (app, participant, tasks), the timeline lines inline, and the required output shape with a worked example. Nothing in the session directory is mutated. The timeline is emitted whole (v1 does not chunk by task boundary; the manifest carries no task timestamps). With -out FILE the prompt goes to a file and the command prints wrote <path>; otherwise it prints to stdout.
Ingest behaviour: reads the answer from FILE (or stdin when -), accepting a top-level object with a findings array (optionally a rubric, which must be a known version) or a bare array. Ingest is the sole validation boundary and never trusts the model. Each finding is decoded with unknown fields disallowed, then checked against every schema rule (see session directory reference): id format and uniqueness, t within the session, the type, severity, and mode enums, non-empty evidence of at most 64 ids with every id real and at least one spoken utt-* anchor, a quote that is a verbatim substring of one cited evidence utterance, and any ui selector/route matching a real event. Validation is transactional — all errors are reported at once and nothing is written on any failure. On success every finding is forced to status: unverified, findings.jsonl is written, and the command prints validated N findings → <path>, followed by the birth state and the provenance the run recorded (the exact forms are under Provenance below). An answer with no findings (a bare [], {"findings":[]}, or a truncated file) is refused rather than written, so it cannot erase a prior findings.jsonl; an answer whose findings would together push findings.jsonl past the session's 16 MiB total-size limit is refused the same way (see session-directory.md). Ingest refuses to overwrite a findings.jsonl that already holds verdict records — counting any kind:"verdict" line, even one whose value is outside the closed enum.
Provenance. Every ingest writes one provenance record as the first line of findings.jsonl, in the same write as the findings, so a findings file always states what produced it. The record carries the rubric version, the backend, the model when one was given, and the date. It is the operator's declaration: analyze never calls a model and has no way to observe where the emitted request ran, so it records what you tell it.
With neither flag the record states "backend":"unrecorded" and the run says so on stderr before it reads the answer, so the choice is visible in the output of the run that made it (stated as an intention, because a run that then fails validation writes nothing):
analyze: no -backend given; the provenance will record "backend not recorded"
The success line names what was recorded:
validated 5 findings → sessions/x/findings.jsonl (all unverified; local backend, model llama3.1:70b)
validated 5 findings → sessions/x/findings.jsonl (all unverified; cloud backend, model not recorded)
validated 5 findings → sessions/x/findings.jsonl (all unverified; backend not recorded, model not recorded)
A re-ingest replaces the provenance record together with the findings it accompanies, so a declaration can never outlive the findings it describes. The verdict guard is unchanged and outranks it: once a findings.jsonl holds verdicts, no re-ingest may rewrite it — including one whose only purpose is to correct the provenance.
For the fully local route end to end, see Analyse a session locally.
The regression-test drafting layer. Like analyze, draft-tests never calls a model, holds no keys, and adds no network dependency: it emits a self-contained drafting request that any assistant (or a human) runs, then ingests and validates the JSON answer into tests.jsonl. A third mode renders the accepted drafts as Markdown test cases. The step sits downstream of verification, so only a finding a person already confirmed can be drafted from.
testimony draft-tests [-session DIR] [-window 10] [-out FILE] # emit the request
testimony draft-tests [-session DIR] -ingest FILE # validate the answer → tests.jsonl
testimony draft-tests [-session DIR] -render [-out FILE] # render the accepted drafts
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-window |
10 |
emit mode: the event-window half-width in seconds around each finding's cited evidence |
-out |
(stdout) | emit or render mode: write the document to FILE instead of stdout |
-ingest |
(off) | ingest mode: validate the answer JSON at FILE (or - for stdin) into tests.jsonl |
-render |
(off) | render mode: write Markdown test cases for the accepted drafts |
draft-tests runs in exactly one mode: emit (neither -ingest nor -render), ingest (-ingest), or render (-render). -ingest combines with neither -out nor -render; -out pairs with emit or render. -window belongs to emit alone — ingest validates against the findings and render reads only what is on disk — so passing it with -ingest or -render is a usage error rather than a silently ignored flag. A non-finite -window is a usage error too, as it is for report; a negative one is legitimate and narrows the window.
Emit reads manifest.json, findings.jsonl, and timeline.jsonl; ingest reads manifest.json and findings.jsonl only, because drafts are validated against the findings rather than re-derived from the timeline; render reads manifest.json, findings.jsonl, and tests.jsonl. Emit hints to run merge first when the timeline is missing; every mode hints to run analyze -ingest first when there is no findings.jsonl, and ingest, render, and review -kind tests hint to run draft-tests -ingest first when there is no tests.jsonl.
Eligibility. A finding may be drafted from when its effective status is confirmed and its mode is not B. Effective status is the same computation review and report use, so a later verdict overriding an earlier one is honoured: a finding confirmed and then rejected is not eligible, and one rejected and then confirmed is. unverified, rejected, and duplicate findings are never eligible — including a duplicate whose target is confirmed, since the canonical finding carries the evidence. The finding's type is not filtered: the request carries it so the model can calibrate, and a draft with nothing to regress against is what the reject decision is for.
Emit behaviour: writes a single self-contained prompt — the rubric version header (testimony-testdraft/v1), the proposal stance, the per-field instructions, the rubric body (the field definitions, how to read each finding record, and the hard constraints ingest enforces), the session context (session, app, participant, tasks), then, per eligible finding in id order, a prose header naming its id, type, severity, clock and the date of the verdict that confirmed it, its own record verbatim in a ```jsonl fence, and its event window in a second fence, followed by the required output shape with a worked example. Nothing in the session directory is mutated. With -out FILE the prompt goes to a file and the command prints `wrote `; otherwise it prints to stdout.
The event window is the only material the steps may be reconstructed from. For each finding it is every timeline entry whose time falls between the earliest cited evidence entry's start minus -window and the latest cited entry's end plus -window, in time order, speech and events together — the utterances carry the expected behaviour and the events carry the steps. The default half-width is 10 seconds rather than report's 2.5, because a reproduction needs the lead-up and the aftermath and not only the moment: on the bundled sample, 2.5 seconds excludes the one utterance in which the participant states what they expected. A finding whose evidence resolves to no entry falls back to the window around its own t. The steps are stated to end at the last cited evidence event at or before the finding's t; a cited event after it belongs in observed.
Ingest behaviour: reads the answer from FILE (or stdin when -), accepting a top-level object with a tests array (optionally a rubric, which must be a known version) or a bare array. Ingest is the sole validation boundary and never trusts the model. Each draft is decoded with unknown fields disallowed, then checked against every schema rule (see session directory reference): id format and uniqueness, a finding that is currently confirmed and not mode B, a session equal to the manifest's, a non-empty title of at most 200 characters, non-empty steps of at most 32 non-empty entries, non-empty expected and observed, a rationale_quote equal to the source finding's quote byte for byte, and a severity equal to the source finding's. Validation is transactional — all errors are reported at once and nothing is written on any failure. On success every draft is forced to status: proposed, tests.jsonl is written, and the command prints validated N test drafts → <path> (all proposed). An answer with no drafts (a bare [], {"tests":[]}, or a truncated file) is refused rather than written, so it cannot erase a prior tests.jsonl; so is an answer whose drafts would together push tests.jsonl past the session's 16 MiB total-size limit. Ingest refuses to overwrite a tests.jsonl that already holds decision records — counting any kind:"decision" line, even one whose value is outside the closed enum.
Render behaviour: writes one Markdown test-case block per draft whose effective status is accepted or edited, in id order, with the latest edited decision's fields applied over the draft. proposed and rejected drafts are omitted: a proposal is not a test, and a rejected draft stays in tests.jsonl for the record rather than for the plan. Each block names the source finding and session, the decision and its date, the numbered steps, the expected and observed behaviour, and the participant's quote as the rationale. With -out FILE the plan goes to a file and the command prints wrote <path>; otherwise it prints to stdout, which is the default because Testimony never writes into the application's repository — where a docs-as-code test plan lives is the operator's choice.
Loud staging. Two states are refused at exit 1 — a well-formed invocation whose work cannot be done — with the counts by status and nothing written:
testimony: no confirmed findings to draft tests from (5 findings: 0 confirmed, 2 unverified, 1 duplicate, 2 rejected); confirm one with `testimony review -session ~/Testimony/sessions/x` first
testimony: no accepted test drafts to render (3 drafts: 0 accepted, 0 edited, 2 proposed, 1 rejected); accept one with `testimony review -session ~/Testimony/sessions/x -kind tests` first
The first applies to emit and to ingest, and on ingest it fires before a byte of the answer is read: with no eligible finding, every draft in the answer would fail the same rule. The second keeps -out FILE from truncating an existing test plan into an empty document.
The one human-decision verb, across both record families. With -kind findings (the default) it records a verdict on each candidate finding; with -kind tests it records an accept / edit / reject decision on each drafted regression test. Either way the decision is appended — to findings.jsonl or to tests.jsonl — without ever rewriting the machine record in place, so the record's birth state and the full decision history are retained as the precision measure.
testimony review [-session DIR] [-kind findings|tests]
testimony review [-session DIR] -finding F-NNN -verdict confirmed|rejected|duplicate-of-F-NNN
testimony review [-session DIR] -kind tests -test T-NNN -decision accepted|rejected
testimony review [-session DIR] -kind tests -test T-NNN -decision edited -edit FILE
| Flag | Default | Meaning |
|---|---|---|
-session |
(inferred) | session directory; when omitted, the current directory if it holds a Testimony session manifest.json (see session directory inference) |
-kind |
findings |
which record family to review: findings or tests |
-finding |
(interactive) | non-interactive: the finding to judge (F-NNN), -kind findings only |
-verdict |
(interactive) | non-interactive: confirmed, rejected, or duplicate-of-F-NNN, -kind findings only |
-test |
(interactive) | non-interactive: the test draft to decide (T-NNN), -kind tests only |
-decision |
(interactive) | non-interactive: accepted, edited, or rejected, -kind tests only |
-edit |
(off) | with -decision edited: the replacement fields as a JSON object at FILE (or - for stdin) |
A flag belonging to the other record family is a usage error, not a silently ignored value: -finding or -verdict with -kind tests, and -test, -decision or -edit with -kind findings, each exit 2. So do an unknown -kind, a -test that is not T-NNN, a -decision outside the enum, -decision edited without -edit, and -edit alongside any other decision.
Behaviour: loads findings and existing verdicts (hinting to run analyze -ingest first when there is no findings.jsonl) and computes each finding's effective status (every finding starts unverified; the last verdict for a finding wins).
Interactive (review -session DIR): walks the unverified findings in id order, printing each finding's id, type, severity, clock, quote, and anchor, then prompting [c]onfirm [r]eject [d]uplicate-of [s]kip [q]uit; d asks for the canonical F-NNN. Each decision appends a verdict record stamped with today's date. Interactive mode is gated on stdin being a character device — true for an interactive terminal, but also for /dev/null, so this is not simply "not a terminal". When stdin is a pipe or a redirected regular file, review prints a one-line notice and exits 0 instead of walking, so CI never blocks; redirected from /dev/null (< /dev/null) it still enters the walk and immediately reaches end of input, since redirection makes stdin the character device itself rather than a pipe reading from it.
Non-interactive (-finding F-003 -verdict confirmed, or -verdict duplicate-of-F-002): validates that the finding exists, the verdict parses, and any duplicate target exists and differs; appends one verdict record and prints a one-line confirmation. A verdict may be appended even when one already exists (append-only correction; the latest wins), unless appending it would push findings.jsonl past the session's 16 MiB total-size limit, which both interactive and non-interactive review refuse (see session-directory.md). The stored verdict enum is exactly confirmed | rejected | duplicate; duplicate-of-F-NNN is stored as verdict: "duplicate" with of: "F-NNN".
Behaviour: loads the test drafts and existing decisions (hinting to run draft-tests -ingest first when there is no tests.jsonl) and computes each draft's effective status — every draft starts proposed, decision records apply in file order, and the last one for a draft wins. A decision naming an unknown draft, or carrying a value outside the closed enum, is ignored rather than applied, so a draft never vanishes from both the walk and the plan.
Interactive (review -session DIR -kind tests): walks the proposed drafts in id order, printing each draft's id, its source finding with that finding's type, severity and clock, the title, the numbered steps, the expected and observed behaviour, and the participant's quote, then prompting [a]ccept [e]dit [r]eject [s]kip [q]uit. e asks for each editable field in turn showing the current value; a blank answer keeps it, and steps are read one per line until a blank line. A pass through the prompts that changes nothing prints no changes; recorded as accepted. and records accepted, because an edited decision with an empty edit records a change that did not happen. The character-device gate is the same as the findings walk's.
Non-interactive (-kind tests -test T-001 -decision accepted): validates that the draft exists and the decision parses, appends one decision record, and prints recorded: T-001 accepted (<date>). -decision edited requires -edit FILE (or - for stdin), a JSON object holding the replacement fields — a subset of title, steps, expected, and observed with at least one member, each held to the draft's own rule for that field:
{"title":"Saving a display name gives no confirmation","steps":["Open #general.","Click Save."]}The edit object is closed: one naming id, finding, session, severity, or rationale_quote is an error rather than a silently dropped key, so no edit can re-point a draft at different evidence. The only way to change the link is to reject the draft and ingest a new one. A decision may be appended even when one already exists (append-only correction; the latest wins), and the accepted and edited drafts are what draft-tests -render puts in the test plan.
Prints testimony <version> — the version stamped at release, or dev.
Prints the usage text (also -h or --help).