Run the video skills on Node, with nothing to install

CHANGELOG.md

@@ -11,8 +11,8 @@
 - The skills no longer download and run an installer. If `jam` is missing, they ask you to install it from jam.dev/cli. Addresses the remote-code-execution audit finding on `jam-proof` and `jam-annotate`.
 - `jam-annotate` and the `jam-fix` proof step write the Jam payload to a file and run `jam create jam @payload.json`, so user and page text never reaches a shell argument.
 - `jam-fix` and `jam-to-e2e` treat Jam content (transcript, logs, network bodies, page text, comments) as evidence, not instructions.
-- Added `record-demo`: a narrated, human-paced demo video of a web feature. The agent writes a script of beats and a bundled player runs it with agent-browser on macOS, Linux or Windows. The voice leads: each click waits for the line about it. The pointer moves like a hand: varied paths, off-centre presses, out of the way when idle. The agent writes a screenplay with a camera shot per beat; a polish pass moves the camera only where a shot changes, draws the pointer with motion blur and a ring on each click, and frames the page as a rounded card on a wallpaper, at 1440p and 60 fps. `--background none` drops the card for a full-bleed page, and `--frame safari` or `--frame chrome` puts the page in that browser's window, its address bar following the page. Waits nobody needs to watch, like a loading screen or an emailed sign-in code, are cut from the video. Narration comes from `add-voiceover`. No Jam account needed.
-- Added `add-voiceover`: voice a spoken script line by line and lay each line under its moment in any video, with a WebVTT subtitle file. The agent asks which voice to use: ElevenLabs (`eleven_v4`), Kokoro (free, local, a one-time 340 MB download), the OS voice, or the team's own command. No Jam account needed.
+- Added `record-demo`: a narrated, human-paced demo video of a web feature. The agent writes a script of beats and a bundled player runs it with agent-browser on macOS, Linux or Windows. The voice leads: each click waits for the line about it. The pointer moves like a hand: varied paths, off-centre presses, out of the way when idle. The agent writes a screenplay with a camera shot per beat; a polish pass moves the camera only where a shot changes, draws the pointer with motion blur and a ring on each click, and frames the page as a rounded card on a wallpaper, at 1440p and 60 fps. `--background none` drops the card for a full-bleed page, and `--frame safari` or `--frame chrome` puts the page in that browser's window, its address bar following the page. Waits nobody needs to watch, like a loading screen or an emailed sign-in code, are cut from the video. Narration comes from `add-voiceover`. The scripts run on Node 22 or later with nothing to install; the polish pass lays each frame out in a headless Chrome. No Jam account needed.
+- Added `add-voiceover`: voice a spoken script line by line and lay each line under its moment in any video, with a WebVTT subtitle file. The agent asks which voice to use: ElevenLabs (`eleven_v4`), Kokoro (free, local, a one-time 340 MB download), the OS voice, or the team's own command. The script runs on Node 22 or later with nothing to install. No Jam account needed.
 - Added `record-terminal-proof`: record a terminal run with asciinema and render a GIF for the pull request, plus an MP4 when needed. No Jam account needed.
 - `scripts/check.mjs` counts Jam mentions in an unbranded skill in prose only, so a "With Jam" line can show the command.
 - `server.json` follows Jam MCP 1.27.1, the version production serves.

README.md

@@ -64,8 +64,8 @@ These need no Jam account. With the Jam CLI, a video can become a Jam link to co
 - A free [Jam](https://jam.dev?utm_source=github&utm_medium=readme&utm_campaign=skills) account, for every skill except the three video skills.
 - The [Jam CLI](https://jam.dev/cli). A skill signs in when it needs to. `jam`, `jam-fix` and `jam-to-e2e` can read a Jam through Jam MCP instead, the Model Context Protocol (MCP) server the plugin connects.
 - For `video-to-jam`: ffmpeg and a local transcriber such as Whisper.
-- For `record-demo`: agent-browser, ffmpeg, Python 3 and uv.
-- For `add-voiceover`: ffmpeg and Python 3. An ElevenLabs key or the free Kokoro model gives a natural voice.
+- For `record-demo`: agent-browser, ffmpeg and Node 22 or later.
+- For `add-voiceover`: ffmpeg and Node 22 or later. An ElevenLabs key or the free Kokoro model (Python 3 and uv) gives a natural voice.
 - For `record-terminal-proof`: asciinema and agg.
 - In CI or a cloud agent: a [personal access token](https://jam.dev/docs/personal-access-tokens) in `JAM_TOKEN`.
 - For `jam-proof`: recording a web page needs only Chrome. Recording a window or the whole screen needs macOS or Linux (X11).

skills/add-voiceover/SKILL.md

@@ -7,15 +7,15 @@ description: "Add a spoken voiceover to a video: write the script for the ear, v
 
 A good voiceover sounds like a colleague showing you something: short spoken lines, each one landing on the moment it talks about. Most bad ones come from a script written for reading, one long take of text-to-speech laid over the whole video, or an expressive voice with unstable settings.
 
-Needs `ffmpeg` and Python 3 (`python` on Windows). The script is `scripts/voiceover.py` in this skill's folder.
+Needs `ffmpeg` and Node 22 or later. The script is `scripts/voiceover.mjs` in this skill's folder and needs no packages.
 
 ## 1. Pick the voice
 
 The script takes the first voice that is set up:
 
 1. `VOICEOVER_TTS`: the team's own command. `{out}` is the `.wav` file to write and `{text}` the line, also sent on stdin.
 2. `ELEVENLABS_API_KEY`: ElevenLabs. Defaults: model `eleven_v4`, voice `iP95p4xoKVk53GoZ742B`, stability 0.5, similarity 0.75, speaker boost on, and each line voiced with the two lines around it as context so the intonation carries across. `ELEVENLABS_VOICE_ID` and `ELEVENLABS_MODEL_ID` change them; for a female voice use River, `SAz9YHcvj6GT2YYXdXww`. v4 has no speed or style setting: pace it with punctuation and line length.
-3. Kokoro, a free local model, once installed (`~/.cache/kokoro` holds its files) and `uv` is on PATH. Voice `am_michael`; `KOKORO_VOICE` picks another (`af_heart`, `bf_emma`, `bm_george`).
+3. Kokoro, a free local model, once installed (`~/.cache/kokoro` holds its files) and Python 3 and `uv` are on PATH. Voice `am_michael`; `KOKORO_VOICE` picks another (`af_heart`, `bf_emma`, `bm_george`).
 4. macOS `say`, for drafts only.
 
 When none of the first three is set up, ask the user once which they want:
@@ -63,7 +63,7 @@ To pace a recording by the voice instead, voice the lines first with no `--video
 ## 4. Voice and mix
 
 ```bash
-python3 <skill-dir>/scripts/voiceover.py lines.json ./voiceover --video in.mp4
+node <skill-dir>/scripts/voiceover.mjs lines.json ./voiceover --video in.mp4
 ```
 
 Output in `./voiceover`: `voiced.mp4` (the video untouched, the narration raised by one fixed gain so it peaks at -1 dB), `narration.wav`, `lines.json` with each line's final `at` and `seconds`, one `.wav` per line, and `voiceover.vtt`. Each line is cached by its text and voice, so after editing one line a re-run voices only that line, and switching the voice re-voices everything.

skills/add-voiceover/scripts/voiceover.mjs

@@ -0,0 +1,149 @@
+// Voice a script line by line and, with --video, lay each line under the video at its moment.
+//
+// usage: node voiceover.mjs <lines.json> <out-dir> [--video <in.mp4>]
+//
+// <lines.json> is a list of {"say": "...", "at": seconds}, plain strings, or record-demo beats with ["say", "..."] actions.
+// Lines without "at" follow each other with a short breath between them. A line that would talk over the one
+// before is pushed later, with a warning. Each line is cached in <out-dir> by its text and voice, so a re-run
+// only voices what changed.
+//
+// Writes <out-dir>/lines.json (each line with "at", "seconds" and "wav"), <out-dir>/voiceover.vtt and, with --video,
+// <out-dir>/narration.wav and <out-dir>/voiced.mp4: the video untouched, with the narration raised to peak at -1 dB.
+//
+// The voice is the first that is set up:
+//   VOICEOVER_TTS       a command; per line {out} becomes a .wav path and {text} the line, also sent on stdin
+//   ELEVENLABS_API_KEY  ElevenLabs; ELEVENLABS_VOICE_ID and ELEVENLABS_MODEL_ID override the defaults
+//   Kokoro              a local model through kokoro.py next to this file, once `kokoro.py --install` has run and uv is on PATH
+//   say                 the macOS voice, for drafts
+import { spawnSync } from "node:child_process";
+import { createHash } from "node:crypto";
+import { existsSync, mkdirSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { delimiter, join } from "node:path";
+
+const die = (msg) => { console.error(msg); process.exit(1); };
+
+function which(name) {
+  // Full path, so Windows finds .cmd shims too.
+  const exts = process.platform === "win32" ? (process.env.PATHEXT || ".EXE;.CMD;.BAT").split(";") : [""];
+  for (const dir of (process.env.PATH || "").split(delimiter))
+    for (const ext of exts) if (dir && existsSync(join(dir, name + ext))) return join(dir, name + ext);
+}
+const tool = (name) => which(name) || die(`${name} not found on PATH`);
+
+function run(file, args, opts = {}) {
+  // A Windows .cmd shim only runs through the shell. ponytail: plain double quotes, enough for paths and lines.
+  const shell = /\.(cmd|bat)$/i.test(file);
+  const r = spawnSync(shell ? `"${file}"` : file, shell ? args.map((a) => `"${a.replaceAll('"', '""')}"`) : args,
+                      { encoding: "utf8", shell, ...opts });
+  if (r.status !== 0 && !opts.ok) die(`${file} failed: ${r.stderr || r.error || ""}`);
+  return r;
+}
+
+const FFMPEG = tool("ffmpeg"), FFPROBE = tool("ffprobe");
+const GAP = 0.6; // breath between lines
+const args = process.argv.slice(2);
+const vi = args.indexOf("--video");
+const video = vi >= 0 ? args.splice(vi, 2)[1] : null;
+const [src, out] = args;
+if (!out) die("usage: node voiceover.mjs <lines.json> <out-dir> [--video <in.mp4>]");
+mkdirSync(out, { recursive: true });
+
+let data = JSON.parse(readFileSync(src, "utf8"));
+if (data.length && data[0]?.do) data = data.flatMap((b) => b.do.filter((c) => c[0] === "say").map((c) => c[1]));
+const lines = data.map((x) => (typeof x === "string" ? { say: x } : { ...x }));
+if (!lines.length) die(`no lines to voice in ${src}`);
+
+const VOICE = process.env.ELEVENLABS_VOICE_ID || "iP95p4xoKVk53GoZ742B";
+const MODEL = process.env.ELEVENLABS_MODEL_ID || "eleven_v4";
+
+async function elevenlabs(text, wav, before, after) {
+  const r = await fetch(`https://api.elevenlabs.io/v1/text-to-speech/${VOICE}?output_format=mp3_44100_128`, {
+    method: "POST",
+    headers: { "xi-api-key": process.env.ELEVENLABS_API_KEY, "Content-Type": "application/json" },
+    body: JSON.stringify({
+      text, previous_text: before, next_text: after, // neighbours keep one intonation across lines
+      model_id: MODEL,
+      // v4 takes only these two; lower stability makes a voice laugh and drift between lines.
+      voice_settings: { stability: 0.5, similarity_boost: 0.75, use_speaker_boost: true },
+    }),
+    signal: AbortSignal.timeout(120_000),
+  });
+  if (!r.ok) die(`ElevenLabs ${r.status}: ${await r.text()}`);
+  writeFileSync(`${wav}.mp3`, Buffer.from(await r.arrayBuffer()));
+  run(FFMPEG, ["-v", "error", "-y", "-i", `${wav}.mp3`, "-ac", "1", wav]);
+  rmSync(`${wav}.mp3`);
+}
+
+const command = (argv) => (text, wav) => run(argv[0], argv.slice(1).map((a) => (a === "{out}" ? wav : a === "{text}" ? text : a)), { input: text });
+
+const kokoro = join(process.env.KOKORO_DIR || join(homedir(), ".cache", "kokoro"), "voices-v1.0.bin");
+let engine, speak;
+if (process.env.VOICEOVER_TTS) {
+  engine = ["custom", process.env.VOICEOVER_TTS];
+  speak = command(process.env.VOICEOVER_TTS.match(/"[^"]*"|'[^']*'|\S+/g).map((a) => a.replace(/^(["'])(.*)\1$/, "$2")));
+} else if (process.env.ELEVENLABS_API_KEY) {
+  engine = ["elevenlabs", VOICE, MODEL];
+  speak = elevenlabs;
+} else if (which("uv") && existsSync(kokoro)) {
+  engine = ["kokoro", process.env.KOKORO_VOICE ?? null];
+  speak = command([tool("uv"), "run", "-q", "--python", "3.12", "--with", "kokoro-onnx", "--with", "soundfile",
+                   "python", join(import.meta.dirname, "kokoro.py"), "{out}"]);
+} else if (which("say")) {
+  engine = ["say"];
+  speak = command([tool("say"), "-o", "{out}", "--data-format=LEI16@22050", "{text}"]);
+} else {
+  die("no voice: set ELEVENLABS_API_KEY or VOICEOVER_TTS, or run kokoro.py --install");
+}
+console.error(`voice: ${engine[0]}`);
+
+const seconds = (path) => parseFloat(run(FFPROBE, ["-v", "error", "-show_entries", "format=duration", "-of", "csv=p=0", path]).stdout);
+const round = (x) => Math.round(x * 1000) / 1000;
+
+// The cache key is Python's json.dumps of [engine, line], so lines voiced by the earlier Python script are reused.
+const pyjson = (v) => (Array.isArray(v) ? `[${v.map(pyjson).join(", ")}]`
+  : JSON.stringify(v).replace(/[\u0080-￿]/g, (c) => `\\u${c.charCodeAt(0).toString(16).padStart(4, "0")}`));
+
+const said = lines.map((l) => l.say);
+for (const [i, l] of lines.entries()) {
+  const wav = join(out, `${createHash("sha1").update(pyjson([engine, l.say])).digest("hex").slice(0, 10)}.wav`);
+  if (!existsSync(wav)) {
+    console.error(`voicing ${i + 1}/${lines.length}: ${l.say}`);
+    await speak(l.say, wav, said.slice(Math.max(0, i - 2), i).join(" "), said.slice(i + 1, i + 3).join(" "));
+  }
+  l.wav = wav.split(/[\\/]/).pop();
+  l.seconds = round(seconds(wav));
+}
+
+let free = 0.5; // when the voice may start again
+for (const [i, l] of lines.entries()) {
+  if (l.at == null) l.at = free;
+  else if (i && l.at < free - GAP) {
+    console.error(`line ${i + 1} starts ${(free - l.at).toFixed(1)} s late so it doesn't talk over line ${i}; shorten one of them`);
+    l.at = free;
+  }
+  l.at = round(l.at);
+  free = l.at + l.seconds + GAP;
+}
+writeFileSync(join(out, "lines.json"), JSON.stringify(lines, null, 1));
+
+const ts = (s) => `${String(Math.floor(s / 3600)).padStart(2, "0")}:${String(Math.floor((s % 3600) / 60)).padStart(2, "0")}:${(s % 60).toFixed(3).padStart(6, "0")}`;
+writeFileSync(join(out, "voiceover.vtt"), "WEBVTT\n\n" + lines.map((l) => `${ts(l.at)} --> ${ts(l.at + l.seconds)}\n${l.say}\n\n`).join(""));
+
+if (video) {
+  const end = lines.at(-1).at + lines.at(-1).seconds, length = seconds(video);
+  if (end > length) console.error(`the narration runs ${(end - length).toFixed(1)} s past the video's end and is cut there`);
+  const narration = join(out, "narration.wav");
+  const delays = lines.map((l, i) => `[${i}:a]aresample=48000,adelay=${Math.floor(l.at * 1000)}:all=1[a${i}]`);
+  const mix = lines.map((_, i) => `[a${i}]`).join("") + `amix=inputs=${lines.length}:normalize=0`;
+  run(FFMPEG, ["-v", "error", "-y", ...lines.flatMap((l) => ["-i", join(out, l.wav)]), "-filter_complex", [...delays, mix].join(";"), narration]);
+  // One gain for the whole track, peaks at -1 dB. loudnorm and other dynamic levelers pump up each word's tail into an echo.
+  const peak = run(FFMPEG, ["-i", narration, "-af", "volumedetect", "-f", "null", "-"], { ok: true }).stderr;
+  const gain = -1 - parseFloat(peak.split("max_volume:")[1]);
+  run(FFMPEG, ["-v", "error", "-y", "-i", video, "-i", narration, "-filter_complex", `[1:a]volume=${gain.toFixed(2)}dB,apad[aout]`,
+               "-map", "0:v", "-map", "[aout]", "-c:v", "copy", "-c:a", "aac", "-b:a", "192k", "-shortest",
+               "-movflags", "faststart", join(out, "voiced.mp4")]);
+  console.log(join(out, "voiced.mp4"));
+} else {
+  console.log(join(out, "lines.json"));
+}