Skip to content

Record narrated demo videos, voiceovers and terminal proof - #26

Merged
baransu merged 19 commits into
masterfrom
tomasz/wave-1-record-skills
Nov 2, 2026
Merged

baransu merged 19 commits into
masterfrom
tomasz/wave-1-record-skills

Conversation

@baransu

@baransu baransu commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Making a good demo video of a feature meant recording the screen by hand, editing out the waits and recording a voiceover separately. A proof that a terminal command ran had no good format for a pull request at all. These three skills let the agent make both, with no Jam account needed.

  • record-demo: the agent writes a screenplay of beats, then plays it in agent-browser at human pace. The pointer moves like a hand: it turns into a pointing hand over links and an I-beam over text fields, and presses off the label so the viewer can read what was clicked. The player warns about any pause over 3 s, and the polish pass also writes poster.png, a still for places a video won't play. The camera moves only where the screenplay changes the shot. The page sits on a rounded card over a wallpaper, rendered at 1440p and 60 fps. Waits nobody needs to watch, like a loading screen or an emailed sign-in code, are cut from the take.
  • add-voiceover: writes the script for the ear, voices it line by line and lays each line on the moment it describes, with a WebVTT file. ElevenLabs eleven_v4 with a conversational voice by default, or Kokoro (free, local), the OS voice or the team's own command.
  • record-terminal-proof: records a terminal run with asciinema and renders a GIF that plays inline on the pull request.

Also: a "Make a video" group in the README, rows for the three skills in the jam router, and scripts/check.mjs counts Jam mentions in prose only, so a "With Jam" line can show the command.

The scripts run on Node 22 or later with nothing to install. The polish pass lays out each frame in a headless Chrome through agent-browser. Kokoro, the free local voice, needs Python and uv for its small runner, because the model only ships for Python.

Tested end to end on TodoMVC and on a full sign-up flow with an emailed sign-in code: a 68 s narrated video with three cuts.

This video was made by these skills:

jam-skills-demo.mp4

Council review

Rounds: 5

Delegate Round 1 Round 2 Round 3 Round 4 Round 5
Claude Opus 7/10 8/10 8.5/10 8/10 9/10
GPT-6 Astra 7/10 7/10 9/10 8/10 9/10

Created with Claude Opus 5.5 in Claude Code.

@baransu
baransu marked this pull request as ready for review November 2, 2026 09:18
@baransu baransu self-assigned this Nov 2, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

- record-demo: a screenplay of beats played in agent-browser at human pace, polished to a 1440p, 60 fps video with a directed camera, a hand-like pointer and a wallpaper card. Waits like a loading screen or an emailed sign-in code are cut from the take.
- add-voiceover: voices a spoken script line by line (ElevenLabs eleven_v4 with Chris by default, Kokoro, the OS voice or a custom command) and lays each line under its moment, with a WebVTT file.
- record-terminal-proof: records a terminal run with asciinema and renders a GIF for the pull request.
- README gets a "Make a video" group, the jam router names the three skills, and check.mjs counts Jam mentions in prose only.
ffmpeg's concat read each JPEG at 25 fps and rounded frame times to 40 ms,
so takes stuttered. Frames now get a 1 ms timebase and land on an even 30 fps
grid; Chrome screencasts at JPEG quality 80 to keep up at 3x.

Also: scrollto eases over up to 4 s, address bar and tab titles shift with
cuts, a page-leaving cut waits for the line and measures the shot first.
record.py, polish.py and voiceover.py needed Python 3, uv, Pillow and
websockets. The ports use Node 22's WebSocket and fetch; polish lays out
each frame in a headless Chrome through agent-browser instead of drawing
with Pillow. Kokoro keeps its Python runner, since the model is Python-only.
…oster

The pointer turns into a hand over links and an I-beam over text fields,
cross-fading at the target's edge, and presses off the label so the viewer
can read what was clicked. The player warns about more than 3 s with
nothing moving or said, the payoff holds 2 s, and polish.mjs writes
poster.png from the beat marked poster. SKILL.md adds a fresh-eyes review
of the beat frames before sharing.
A live screencast drops frames unevenly under load, so a scroll stuttered.
scrollto now moves the page to each 60th-of-a-second position itself, takes
the frame Chrome paints once the scroll lands, and stamps it on the 60 fps
grid; the time the steps took is cut. Live parts stay on an even 30 fps.
polish.mjs keeps only take frames that change, so a 60 fps take costs no
more disk.
…, quoted TTS options

- voiceover.mjs mixes the narration over a video's own audio instead of
  dropping it.
- The ElevenLabs cache key includes the neighbouring lines it hears, so
  editing a line re-voices its neighbours.
- VOICEOVER_TTS keeps quoted option values like --voice="Jane Doe" whole.
- record.mjs prints each $ command before it runs it, and SKILL.md says to
  play only a screenplay you wrote or read, and why the player drives
  agent-browser rather than the project's browser driver.
@baransu
baransu force-pushed the tomasz/wave-1-record-skills branch from 2061f07 to 10e3fac Compare November 2, 2026 11:12
devin-ai-integration[bot]

This comment was marked as resolved.

…ing doc

- A video whose own sound ends early keeps its full length: the mix runs
  to the longer input and -shortest ends it with the picture.
- A scrollto with nowhere to go waits out its steps, so its cut never runs
  backward, and a scroll is clamped at the top of the page.
- SKILL.md says how to quote VOICEOVER_TTS arguments.
devin-ai-integration[bot]

This comment was marked as resolved.

- scrollto scrolls the panel that holds the target when the page itself
  doesn't scroll, and takes a screenshot when a step gets no frame.
- polish.mjs quotes the agent-browser path on Windows; a shot centred where
  the wide view is centred now zooms.
- voiceover.mjs caches a line only once it is fully written, and replaces
  {out} and {text} anywhere in an argument.
- record.mjs warns when a line is missing from --voice.
- SKILL.md: AGENT_BROWSER_PROFILE keeps a sign-in across takes, scrollto
  ends a beat, re-record to change a shot, Jam upload carries the poster.
- record-terminal-proof records with --return; CONTRIBUTING names the
  record-demo exception to rule 6.
devin-ai-integration[bot]

This comment was marked as resolved.

…e cases

- scrollto scrolls the nearest panel that hides the target, else the
  window, so a panel on a long page scrolls too.
- A scroll in a framed beat measures the shot before it moves.
- record.mjs sends eval code on one line (cmd.exe ends at a newline) and
  stops on a scroll that throws.
- polish.mjs cleans up even when closing its session fails.
- voiceover.mjs keeps $ characters in a line literal.
- SKILL.md shows AGENT_BROWSER_PROFILE inline, not exported.
devin-ai-integration[bot]

This comment was marked as resolved.

A short scrollto waits out its steps with stepped capture still on, so a
late repaint, like an image loading in, was dropped. The newest frame
painted after the last step now becomes the scroll's final frame.
One frame can't hold content that moves mid-beat. The frame keeps the
shot as it was before the scroll, which is right when scrollto ends the
beat as SKILL.md says; the player now warns when it doesn't.
…llback

- voiceover.mjs measures the narration mixed with a video's own sound and
  applies one more fixed gain when the two together would pass -1 dB.
- record.mjs keeps each occurrence's length when the same words are voiced
  twice, saves its timing files before the final encode, and on a page
  that doesn't scroll falls back to the nearest scrolling panel.
- SKILL.md: move the poster in timeline.json for a polish-only rerun.
devin-ai-integration[bot]

This comment was marked as resolved.

- voiceover.mjs measures the narration-plus-sound mix with astats on float
  samples; volumedetect clamps at 0 dB, so a louder mix still clipped.
- record.mjs keeps voiced line lengths in a prototype-free object, so a
  line like "constructor" can't break the lookup.
- record.mjs closes its session however it ends, retries its first
  command once when the last session is still closing, and warns only when
  a framed beat acts after scrolling (wait and say are fine).
- polish.mjs removes a cut-short polished.mp4 when a frame fails, and
  reports a dead ffmpeg instead of hanging on a pipe.
- SKILL.md text matches the framing, mix and caching behaviour.
- record-terminal-proof runs the script with bash, so it needs no chmod.
devin-ai-integration[bot]

This comment was marked as resolved.

- polish.mjs watches ffmpeg's exit from the moment it starts and races
  every backpressured write against it, so an encoder that dies is reported
  instead of hanging; a failing frame stops ffmpeg before removing the
  cut-short file.
- record.mjs and polish.mjs close their browser sessions on any exit,
  Ctrl-C and SIGTERM included.
However polish stops, a timeout or Ctrl-C included, it removes the
cut-short polished.mp4 and its frames, and it notices an ffmpeg killed
by a signal straight away. SKILL.md says to give it a long timeout.
devin-ai-integration[bot]

This comment was marked as resolved.

A failed rerun no longer deletes the last good polished.mp4, and a run
cut short on Windows, where an open file can't be deleted, leaves only a
file named as unfinished.
devin-ai-integration[bot]

This comment was marked as resolved.

poster.png is written as poster.part.png and renamed together with the
video, so a rerun whose encode fails keeps the old pair.
devin-ai-integration[bot]

This comment was marked as resolved.

… skills

README lists agent-browser 0.38.1+, no npm packages, add-voiceover as the
narration source, the macOS voice fallback, asciinema 3 and WSL on
Windows. CHANGELOG adds the stepped 60 fps scroll, panel scrolling,
AGENT_BROWSER_PROFILE and keeping a video's own sound.
@baransu
baransu merged commit 4cc9604 into master Nov 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant