Skip to content

Commit 20f36d3

Browse files
committed
refactor: split the audiobook pipeline behind hard gates
1 parent 61ab67a commit 20f36d3

42 files changed

Lines changed: 3197 additions & 452 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎scripts/audiobook/DIALOGUE_GUIDE.md‎

Lines changed: 67 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -7,10 +7,10 @@ The committed script is `website/audio/dialogue/<id>.json`; the audio lags it.
77

88
The two hosts are the book's historic podcast personas.
99

10-
| Role | Persona | Voice | Job |
11-
| --- | --- | --- | --- |
12-
| `alex` | Kore — instructor | `Kore` (`config.speakers.alex`) | Clear, measured, pedagogical: guides the listener, explains what to do and why, and turns each figure or code block into the idea it carries. |
13-
| `sam` | Charon — senior engineer | `Charon` (`config.speakers.sam`) | Thoughtful: asks the clarifying question peers would ask, connects to production, challenges over-claims, **restates each beat's takeaway in their own words**. |
10+
| Role | Persona | Voice | Job |
11+
| ------ | ------------------------ | -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
12+
| `alex` | Kore — instructor | `Kore` (`config.speakers.alex`) | Clear, measured, pedagogical: guides the listener, explains what to do and why, and turns each figure or code block into the idea it carries. |
13+
| `sam` | Charon — senior engineer | `Charon` (`config.speakers.sam`) | Thoughtful: asks the clarifying question peers would ask, connects to production, challenges over-claims, **restates each beat's takeaway in their own words**. |
1414

1515
Roles are epistemic, not a naive-questioner act: Sam never fakes ignorance.
1616
Voices are bound to roles in `config.ts`, never in the script, so a script is voice-agnostic.
@@ -24,15 +24,15 @@ criteria.
2424

2525
Avoid: enthusiasm and exclamations, laundry lists, dumbing down, marketing hype, circular transitions.
2626

27-
Delivery is *not* authored into the text. The model performs the turns with its natural, energetic
27+
Delivery is _not_ authored into the text. The model performs the turns with its natural, energetic
2828
delivery — an added "measured/unhurried" directive reads as sluggish, so none is used; the WER gate
2929
guards against any instruction leaking into the audio.
3030

3131
## Why two speakers (evidence, not vibes)
3232

3333
Holding information equal, dialogue alone does not improve comprehension — **reformulation and
34-
challenging do**. Sam must rephrase and pressure-test, not "yes-and". Fun must be *about the
35-
topic*: off-topic banter impairs retention (seductive-details effect). See notebook
34+
challenging do**. Sam must rephrase and pressure-test, not "yes-and". Fun must be _about the
35+
topic_: off-topic banter impairs retention (seductive-details effect). See notebook
3636
`research-two-speaker-format`.
3737

3838
## Turn rules
@@ -48,8 +48,11 @@ topic*: off-topic banter impairs retention (seductive-details effect). See noteb
4848
## Content rules
4949

5050
- Write for the ear: short clauses, no visual deixis ("this diagram", "below"); contractions per
51-
"Authoring voice".
51+
"Authoring voice". **Enforced (hard):** `lint` fails any turn that references an on-page visual —
52+
the listener has no page, so the only fix is re-voicing the idea. Vocabulary: `AUDIO_CONFIG.deixis`.
5253
- Preserve precision: repeat exact identifiers and numbers; never paraphrase a critical term away.
54+
- **Declared critical terms (per chapter).** The term gate only auto-classifies numbers, all-caps acronyms and identifiers, so a plain word the argument turns on — a named example like _strawberry_ — is invisible to it and a paraphrase can silently drop it. Declare it as `declaredCriticalTerms: [{ "term": "<exact source word>", "reason": "<why it must be spoken>" }]`; the gate then requires it verbatim in the turns (and in the ASR transcript). Each entry must name a real source word, carry a reason, and not restate an auto-classified term, or lint fails instead of ignoring it. `audio:dialogue:source` prints the effective required set (auto + declared).
55+
- **Described literals (rare exemption).** An arbitrary example literal — e.g. a random hash inside a quoted benchmark prompt — may be described ("a short random prefix") instead of spelled aloud: the book text keeps the literal, only the ear is spared. Declare it in the script as `describedLiterals: [{ "term": "<exact literal>", "reason": "<why it is arbitrary>" }]`. Never for semantic numbers, identifiers or flags (scores, versions, model names, CLI flags): those stay spoken verbatim. Lint exempts a declared term from the spoken-critical-term requirement, but fails the chapter when a declared term is not a critical term of the source or its reason is empty — so the field cannot hide a term that matters.
5356
- **Coverage is enforced (hard fail).** `audio:dialogue:lint`, `audio:build` and `audio:verify` all
5457
refuse a chapter whose turns leave any source prose section, figure or code block uncovered, or any
5558
critical term unsaid. Author from `audio:dialogue:source` output, not from memory.
@@ -63,27 +66,56 @@ topic*: off-topic banter impairs retention (seductive-details effect). See noteb
6366
layout, color or position — "a gear labelled LLM", "a violet box", "a row of chips", "three panels in
6467
a row". A listener cannot see it, and it reads as a diagram walkthrough. Translate the figure into the
6568
one idea it exists to fix, in the intuitive, story language a listener can hold: why it matters and what
66-
it changes. The same rule governs `alex`'s figure-anchored turns.
69+
it changes. The same rule governs `alex`'s figure-anchored turns. **Enforced (hard):** extraction fails
70+
a `narration` that references a visual (`deixis.ts`), so it can never leak into the draft. Two shapes of
71+
the same defect are fatal, on `narration` and on the authored voice alike:
72+
- **The page register.** `AUDIO_CONFIG.deixis` carries the layout nouns a walkthrough leans on (`panel`,
73+
`tile`, `chip`, `ribbon`, `row`, `column`), a position word glued to one of them ("left panel", "top
74+
tile"), and `labelled`/`labeled` ("a gear labelled LLM"). Bare positions stay legal ("the right
75+
answer", "the middle ground"), and so does `labelled` in its machine-learning sense ("labeled
76+
examples").
77+
- **Text drawn inside the figure.** The words a diagram paints are declared once, in
78+
`website/src/components/VisualElements/figureDrawnLabels.ts`, keyed by the component that draws them.
79+
Naming one out loud ("the row that says trust earned") fails: extraction for the `narration`, `lint`
80+
for a figure-anchored turn. Edit the artwork and that map moves with it, or its drift test fails.
81+
When you edit figure copy, update the `figureDrawnLabels.ts` map in the same change.
6782
- **Narration adds, never echoes.** A figure or code narration must carry something the prose beside it does
6883
not already say; restating the neighbour's definition or list makes the audio say the same thing twice.
6984
Paraphrase is fine — `sam` restating `alex` in different words is the format — but the wording must change,
70-
not only the sentence order. Lint fails any two turns in the chapter that share a ≥20-character run of content
71-
words (function words ignored) — a repeat is a defect wherever it sits, so a closing recap must paraphrase too —
72-
and `audio:dialogue:source` warns when a figure/code narration's content words are largely
73-
the adjacent prose's.
85+
not only the sentence order. One fact belongs on one surface: an `alex` figure-anchored turn voices the
86+
consequence the figure exists to fix, never a restatement of the heading prose above it (that is the
87+
heading→figure→recap doubling the repeat gate keeps catching). One fatal gate enforces it at text time,
88+
before any audio is rendered:
89+
- **Verbatim run.** Any two turns that share a ≥20-character run of content words (function words ignored)
90+
fail: a repeat is a defect wherever it sits, so a closing recap must paraphrase too.
91+
- **Vocabulary echo (advisory).** `lint` WARNs when same-speaker turns reach the IDF-cosine lexical
92+
overlap threshold (`lexical.ts`, `AUDIO_CONFIG.redundancy.lexical.cosineMin`) — the same fact said
93+
again in largely the same words, even reordered so no contiguous run survives. It is **advisory, not
94+
fatal**: the bag-of-words metric measures lexical overlap, not meaning, and on the real corpus its
95+
false positives outscore a genuine paraphrase, so no threshold separates them. Every cross-speaker
96+
pair passes this cosine gate (`sam` restating `alex` is the format). Fired pairs are grouped into fact
97+
clusters, so the fix is to consolidate the FACT to one surface and give the other turn a new claim —
98+
not to reword each pair. The IDF weights are computed live over every chapter in the book
99+
(`wholeBookCorpus`), so editing one chapter can shift another chapter's scores.
100+
- **Document-level lexical echoes are WARNs.** `docRedundancyProblems` surfaces repeated content across prose and
101+
figure/code narration for editorial review; they do not block lint or rendering. Captions are page text and
102+
are never spoken. Same-speaker dialogue echo is an advisory WARN too; only verbatim copied dialogue is fatal.
103+
- **The dialogue is not a mirror of the doc.** Coverage requires a turn per section, figure and code block,
104+
not a restatement of each. When the reading text legitimately revisits a fact across sections (progressive
105+
disclosure, standalone sections), voice it once in the dialogue and give the other surface a new angle.
74106
- Define a term on first use. Recap at the end of a section.
75107
- Keep the chapter's own structure: opener = the stakes; each heading = one beat; recap = Sam's
76108
restatement. Anchor every turn to the heading it explains so the player can highlight it.
77109

78110
## Structure mapping
79111

80-
| Source | Turns |
81-
| --- | --- |
82-
| chapter title | one `alex` turn opening the episode |
83-
| `opener` | `alex` states the stakes / the question |
84-
| heading section | `alex` delivers; `sam` challenges + restates |
85-
| figure / code | `alex` delivers the meaning the figure carries; `sam` asks what it looks like in practice |
86-
| `recap` | `sam` restates the whole takeaway |
112+
| Source | Turns |
113+
| --------------- | ----------------------------------------------------------------------------------------- |
114+
| chapter title | one `alex` turn opening the episode |
115+
| `opener` | `alex` states the stakes / the question |
116+
| heading section | `alex` delivers; `sam` challenges + restates |
117+
| figure / code | `alex` delivers the meaning the figure carries; `sam` asks what it looks like in practice |
118+
| `recap` | `sam` restates the whole takeaway |
87119

88120
Heading rows exist only for markdown headings: `doc.toc` and the extractor read the markdown AST,
89121
so a component-rendered `<h1..6>` is invisible to both. Never render heading tags from components —
@@ -95,9 +127,18 @@ see `README.md` "Sources of truth".
95127
{
96128
"chapterId": "<doc id>",
97129
"sourceHash": "<sha256 of the doc source this was authored from>",
130+
// Plain source words the pattern gate cannot see and the voice must speak verbatim.
131+
// "declaredCriticalTerms": [{ "term": "<exact source word>", "reason": "<why it must be spoken>" }],
132+
// Rare: arbitrary example literal only, never a semantic number/identifier/flag.
133+
// "describedLiterals": [{ "term": "<exact literal>", "reason": "<why it is arbitrary>" }],
98134
"turns": [
99-
{ "id": "t1", "speaker": "alex", "anchor": { "kind": "heading", "id": null }, "text": "..." }
100-
]
135+
{
136+
"id": "t1",
137+
"speaker": "alex",
138+
"anchor": { "kind": "heading", "id": null },
139+
"text": "...",
140+
},
141+
],
101142
}
102143
```
103144

@@ -112,10 +153,11 @@ see `README.md` "Sources of truth".
112153
`sourceHash` is the hash of the chapter content the script was authored from. The build recomputes it
113154
from the current document and **warns** (and marks the chapter `DRIFT` in `audio:list`) when they
114155
differ — the book moved on, so re-author before publishing. Drift is a **warning** for
115-
`audio:build`/`audio:verify` (the audio legitimately lags the text). Every other violation — missing
156+
`audio:build`/`audio:verify` (the audio legitimately lags the text) but **fatal** for
157+
`audio:dialogue:lint` (lint runs on authored text, never on lagging audio, so a drifted script means
158+
the author has not re-based). Every other violation — missing
116159
narration, uncovered section/figure/code, missing critical term, bad turn ids — is **fatal** to
117-
`audio:build` and `audio:verify`; `audio:dialogue:lint` treats drift as fatal too, so it stays the
118-
strict pre-publish gate.
160+
`audio:build`, `audio:verify` and `audio:dialogue:lint` alike.
119161

120162
## Chunking (why turns are grouped during render)
121163

‎scripts/audiobook/cli.mjs‎

Lines changed: 23 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,23 @@
1111
import { AUDIO_CONFIG, assertRenderConfig } from '../../website/src/audiobook/config.ts';
1212
import { AUDIO_DOC_IDS, chapterMeta } from '../../website/src/audiobook/docs.ts';
1313
import {
14-
chunkTurns,
1514
dialogueBloatWarnings,
15+
} from '../../website/src/audiobook/dialogueCoverage.ts';
16+
import {
17+
dialogueLexicalHits,
18+
docRedundancyProblems,
19+
} from '../../website/src/audiobook/dialogueLexical.ts';
20+
import {
21+
chunkTurns,
1622
hardViolations,
17-
isDialogueChapter,
1823
isDriftViolation,
1924
loadDialogue,
20-
loadDialogueFile,
2125
narrationOverlapWarnings,
22-
} from '../../website/src/audiobook/dialogue.ts';
26+
} from '../../website/src/audiobook/dialogueScript.ts';
27+
import {
28+
isDialogueChapter,
29+
loadDialogueFile,
30+
} from '../../website/src/audiobook/dialogueTypes.ts';
2331
import { criticalTerms } from '../../website/src/audiobook/wer.ts';
2432
import { dialogueStream, chunkState, discardChunk, printDialoguePlan } from './dialogue.mjs';
2533
import { encodeChapter } from './encode.mjs';
@@ -185,11 +193,13 @@ const withId = (id, problem) => (problem.startsWith(`${id}:`) ? problem : `${id}
185193
/**
186194
* Dialogue lint: extraction + structural/term coverage + drift + chunk cap are
187195
* hard failures (they mean the script is not the source); section bloat is a
188-
* WARN line that does not fail.
196+
* WARN line that does not fail. Drift is fatal here — lint runs on authored
197+
* text, never on lagging audio — while build/verify only warn on it.
189198
*/
190199
function lintDialogue(id, loaded) {
191200
const chunks = chunkTurns(loaded.script.segments, AUDIO_CONFIG.dialogue.maxChunkChars);
192201
const problems = [...loaded.violations, ...chunkProblems(chunks)];
202+
loaded.warnings.forEach((warning) => warn('lint', `WARN ${warning}`));
193203
dialogueBloatWarnings(chapterMeta(id), loadDialogueFile(id)).forEach((warning) => warn('lint', `WARN ${warning}`));
194204
log('lint', `${id}: ${problems.length} problem(s) across ${loaded.script.segments.length} turns in ${chunks.length} chunk(s)`);
195205
// Problems are not echoed here: `main` prints every chapter problem once as `FAIL` lines.
@@ -202,8 +212,12 @@ function printSource(id) {
202212
console.log(`chapter ${id}: ${meta.title}`);
203213
meta.headings.forEach((heading) => console.log(` heading ${heading.id} → ${heading.title}`));
204214
meta.blocks.forEach((block, index) => console.log(` block ${index} ${block.kind} ${JSON.stringify(block.anchor)} ${block.text}`));
205-
console.log(` criticalTerms ${[...new Set(criticalTerms(meta.blocks.map((block) => block.text).join(' ')))].join(', ')}`);
215+
const data = loadDialogueFile(id);
216+
const declared = (data?.declaredCriticalTerms ?? []).map((entry) => entry.term);
217+
console.log(` criticalTerms ${[...new Set([...criticalTerms(meta.blocks.map((block) => block.text).join(' ')), ...declared])].join(', ')}`);
206218
narrationOverlapWarnings(meta).forEach((warning) => warn('source', `WARN ${warning}`));
219+
docRedundancyProblems(id, meta).forEach((warning) => warn('source', `WARN ${warning}`));
220+
if (data) dialogueLexicalHits(id, data).filter((hit) => hit.score > 0).slice(0, 5).forEach((hit) => warn('source', `WARN lexical ${hit.a} and ${hit.b} (${hit.distance} turn(s) apart): ${Math.round(hit.score * 100)}% shared content${hit.phrase ? ' [shared phrase]' : ''} (${hit.shared.slice(0, 8).join(', ')})`));
207221
}
208222

209223
async function runChapter(id, options) {
@@ -274,13 +288,11 @@ function preflightOrExit(options) {
274288
}
275289

276290
/**
277-
* `--all` build/verify skips chapters without a committed dialogue script instead
278-
* of failing them one by one: audio legitimately lags the text, so an unauthored
279-
* chapter is not an error. An explicitly named unauthored chapter still fails loudly
280-
* in `loadChapter`. The skip is announced, never silent.
291+
* `--all` skips chapters without committed dialogue; explicit chapter selection
292+
* still reaches `loadChapter` and fails loudly when unauthored.
281293
*/
282294
function authoredScope(run) {
283-
if (run.options.command !== 'build' && run.options.command !== 'verify') return run.ids;
295+
if (!run.options.all || !['build', 'verify', 'lint'].includes(run.options.command)) return run.ids;
284296
const authored = run.ids.filter(isDialogueChapter);
285297
const skipped = run.ids.length - authored.length;
286298
if (skipped) log(run.options.command, `skipping ${skipped} unauthored chapter(s)`);

‎scripts/audiobook/dialogue.mjs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
*/
1111
import { rmSync } from 'node:fs';
1212
import { AUDIO_CONFIG } from '../../website/src/audiobook/config.ts';
13-
import { chunkTurns, turnSliceBounds, turnStartMs } from '../../website/src/audiobook/dialogue.ts';
13+
import { chunkTurns, turnSliceBounds, turnStartMs } from '../../website/src/audiobook/dialogueScript.ts';
1414
import { dialogueChunkCacheKey } from '../../website/src/audiobook/hash.ts';
1515
import { renderDialogueChunk } from './render.mjs';
1616
import { cacheFile, exists, log, pcmBytesForMs, pcmDurationMs, readBytes, readJsonIfExists, table, transcribe, warn, writeFile, writeJson } from './report.mjs';

‎scripts/audiobook/render.mjs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@
1212
* in `inlineData` — no server-side WAV/MP3.
1313
*/
1414
import { AUDIO_CONFIG } from '../../website/src/audiobook/config.ts';
15-
import { dialogueTranscript, roleLabel } from '../../website/src/audiobook/dialogue.ts';
15+
import { dialogueTranscript, roleLabel } from '../../website/src/audiobook/dialogueScript.ts';
1616
import { gemini, withRetry } from './report.mjs';
1717

1818
const MIME_RATE = /rate=(\d+)/;

0 commit comments

Comments
 (0)