Turn a voice recording + an avatar image into a transparent-background video of the avatar pulsing with the voice — glow, expanding rings on transients, a spectrum-deformed outline and a waveform scope ring. Drop the file into DaVinci Resolve (or Premiere / FCP) on a track above your screencast, scale it into a corner, done. No keying, no green screen.
- Node 20+
- ffmpeg on your PATH —
brew install ffmpeg/apt install ffmpeg --codec hevcadditionally needshevc_videotoolbox, which is macOS-only. On Linux use--codec proresor--codec png; the renderer checks up front and tells you which codecs your ffmpeg actually supports instead of failing with a wall of output.
npm install
node src/make-avatar.js # placeholder avatar (replace assets/avatar.png)
node src/render.js --audio assets/voice.wav --avatar assets/avatar.pngProduces:
| file | what |
|---|---|
out/avatar_alpha.mov |
ProRes 4444 w/ alpha + the source audio. This is the file you import. |
out/preview_opaque.mp4 |
The same thing flattened onto a solid background, so you can watch it in any player. It is opaque — importing this is what puts a grey box on your footage. |
node src/serve.js # serves the page AND persists your settings
open http://localhost:8777/preview.htmlChrome pauses the render loop for background tabs, so bring the window to the front before pressing play — otherwise it looks frozen.
The stage has Behind and Size toggles: preview the overlay over transparency, black, white or a footage-like gradient, and at full / half / corner size. A corner avatar is far smaller than 512px, and an effect that reads well full-size can turn to mush down there — so check it at the size it will actually ship at. Both choices persist.
Your avatar image is remembered across reloads in localStorage (downscaled to 512px, which
is more than the visual uses), so you only pick it once — the render uses the remembered
image too, not the default. There's a forget link next to it. Audio is not remembered:
recordings are far too large for the ~5 MB storage budget.
Press 🎬 Render the final file and the page uploads your audio and avatar to the local server, which shells out to the renderer and streams progress back; you get a download link when it finishes. No terminal, and no need to know where your files live — the browser hands over the bytes, not a path.
Press 💾 Save settings for the agent and the current values land in settings.json,
which render.js picks up automatically (explicit CLI flags still win). Open the page with
query params to preconfigure it: preview.html?hueA=200&glow=140.
Sliders for hue, bounce, glow, blob amount and sensitivity, driven by a file or your
microphone. src/visual.js is shared by the preview and the renderer, so what you tune
is exactly what renders.
--style pulse | constellation | waterfall | packets | handshake
Left to right, all shown in the modem palette:
- pulse — glow, rings on transients, spectrum-deformed outline, waveform ring. The default.
- constellation — a QAM constellation diagram, which is how a modem actually encodes bits onto a carrier: each symbol is a point in amplitude/phase space, and noise smears the cluster. Loud clean speech snaps the lattice tight; silence lets it drift.
- waterfall — a radial spectrogram. Each frame pushes the current spectrum outward, so the last ~1.5 s of your voice trails away from the avatar. The one that most obviously reads as that is my sound.
- packets — transients emit framed bursts of dashes travelling outward, each burst with its own bit pattern, like data on the wire.
- handshake — a carrier arc that sweeps while searching and locks when you speak, combined with symbols and packets.
--preset modem # modem.dev teal #44BDA3 + cream #F8F8ED
--colorA '#44BDA3' --colorB '#F8F8ED'
--hueA 190 --hueB 285 # or drive it by hue
--audio path audio (or video) file to react to default assets/voice.wav
--avatar path image, centre-cropped into the circle default assets/avatar.png
--outdir path output directory default out
--fps n frame rate — match your timeline default 30
--size n square output in px default 512
--codec c prores | hevc | png default prores
--gain n multiply reactivity default 1
--range n dB range: lower = more per-syllable default 13
--hueA/--hueB n gradient hues 0-360 default 190 / 285
--bounce n avatar scale with loudness, 0-30 default 9
--glow n glow intensity, 0-200 default 100
--blob n spectrum deformation, 0-30 default 9
--no-preview skip the flattened mp4
Every one of these matches a slider in `preview.html`, which prints the exact command
for your current settings with a copy button — tune visually, paste, render.
The overlay carries the source audio (PCM, bit-identical to your input) so you can line it
up against the screencast by waveform, or let Resolve auto-sync. Mute or unlink that track
once it's positioned — otherwise you'll hear it twice. --no-audio renders it silent.
Almost always the NLE ignoring the alpha channel rather than anything wrong with the file. In rough order of likelihood:
- Composite mode. In Resolve, select the clip → Inspector → Settings → Composite, and make sure the mode is one that respects alpha (Normal). If it is ignoring alpha you get the clip's opaque base layer, which is a solid rectangle.
- You imported
preview_opaque.mp4. That is the flattened check file, opaque by design. The overlay isavatar_alpha.mov. - The codec.
--codec hevcrelies on Apple HEVC-with-alpha, which some tools read and some silently ignore.--codec prores(ProRes 4444) is the bulletproof option if you hit trouble — bigger, but no app disagrees about it.
To check the file itself rather than the NLE:
ffmpeg -i out/avatar_alpha.mov -vf "select=eq(n\,50)" -frames:v 1 -update 1 /tmp/f.pngOpen /tmp/f.png — the area outside the effects should be transparent, not black.
A soft circular haze instead of a hard edge is a different problem (glow spread, below), and a bright white halo is an alpha premultiply mismatch (further below).
Everything outside the effects is fully transparent, and that is worth guarding. A wide, slowly-fading glow puts low alpha across most of the frame, which composites as a flat grey film — the overlay stops reading as a floating avatar and starts looking like a translucent box sitting on your footage. Two things prevent it:
- the glow hugs the avatar and falls off steeply, rather than spreading to the frame edge
- alpha below 4/255 is snapped to zero on output; invisible on its own, but across half a frame it is exactly that grey film
On a loud frame two thirds of the output is now fully transparent, against 29% before.
If you want a bigger halo, raise --glow — that scales intensity without spreading haze
back across the frame.
--alpha premultiplied # default
--alpha straight
Canvas2D produces straight alpha, but Resolve (and most NLEs) composite as premultiplied by default. Handing straight data to a premultiplied compositor adds the full-brightness colour instead of scaling it, so soft glows blow out to a solid white blob around the avatar. Yapatar therefore premultiplies on the way out by default, which is what your NLE expects.
If you already have a render that shows a white halo, you don't have to re-render: in
Resolve, right-click the clip -> Clip Attributes -> Alpha Mode -> Straight. Use
--alpha straight if you specifically want that workflow.
Measured on 18.9 s @ 512×512 / 30 fps:
--codec |
size | notes |
|---|---|---|
prores |
286 MB | ProRes 4444, 16-bit alpha. Safest, works everywhere. |
hevc |
14 MB | Apple HEVC w/ alpha. ~20× smaller, 8-bit alpha, 4:2:0 colour. macOS NLEs read it fine. |
png |
112 MB | Lossless RGBA in a .mov. Slow to decode. |
ProRes is roughly 15 MB/s of runtime, so a 30-minute take is ~27 GB — use --codec hevc
for anything long, or render at the size the overlay actually appears at.
Measured on this machine, 512x512 @ 30 fps, a real 30-minute (1815 s) file:
| render time | output size | peak RAM | |
|---|---|---|---|
--codec hevc |
91 s | 1.3 GB | 1.2 GB |
--codec prores |
~100 s | ~27 GB | 1.2 GB |
Render time is fine either way — about 20x faster than realtime. Size is the thing that
bites: ProRes 4444 is a fixed ~15 MB per second of runtime regardless of content, so a
half-hour take is 27 GB. Use --codec hevc for anything long; it's the same picture with
8-bit alpha instead of 16-bit, which no one will see in a corner overlay.
RAM is ~1.2 GB at 30 minutes because the whole decoded PCM plus per-frame features are held at once. That's comfortable, but it grows linearly — a multi-hour file would need chunking.
- Record voice + screen as usual.
- Extract or point at the voice track:
--audio take01.wav(a video file works too — ffmpeg pulls the audio out). - Render at your timeline's fps:
--fps 30. - In Resolve: import
avatar_alpha.mov, drop it on video track 2 above the screencast. Alpha is honoured automatically — no need to set a composite mode. Scale/position into a corner. The clip is silent; keep using your original audio track. - Because the render is frame-locked to the audio file, it stays in sync as long as the overlay starts at the same timecode as that audio.
src/serve.js also exposes the render pipeline over HTTP:
POST /upload?name=x.wav |
raw body, returns {id} |
POST /render |
{audioId, avatarId, style, preset, codec, ...} -> {jobId} |
GET /job/:id |
{state, pct, output, error} |
GET/POST /settings |
the tuned values |
It spawns subprocesses, so it is deliberately locked down: bound to 127.0.0.1 only,
requests carrying a foreign Origin are rejected, and nothing reaches a shell — render.js
is spawned with an argv array, style/preset/codec are checked against allowlists, and
every numeric field is coerced and clamped. Uploads land in .uploads/ and rendered jobs in
out/web/<jobId>/; both are gitignored and safe to delete.
This project doubles as a plugin. skills/yapatar/SKILL.md teaches an agent the
whole loop: start the server, open a preconfigured preview, wait for you to tune and save,
then render with your values. It's symlinked into ~/.claude/skills/, so it's live now —
just ask for a voice-reactive avatar overlay.
To share it, .claude-plugin/plugin.json makes the repo installable as a plugin.
npm test # unit + end-to-end (ffmpeg-dependent tests skip if it is missing)
npm run serve # preview at http://localhost:8777/preview.html
npm run render # render with the bundled sample assetsSee CONTRIBUTING.md for how to add an effect — it is one function plus a line in a style table, and the suite picks it up automatically.
MIT.
src/audio.js— ffmpeg decodes to mono f32 PCM; per video frame a 2048-sample Hann window gives RMS loudness and a 64-bin log-spaced spectrum (60 Hz–8 kHz). Loudness is auto-calibrated per clip: the 97th percentile becomes "full pulse", 13 dB below it becomes rest. Without that, ordinary speech pins the visual at maximum and nothing appears to react. Then fast-attack / slow-release smoothing, so it moves like speech.src/visual.js— pure Canvas2D, no DOM, so the identical code runs in the browser preview and headless in Node via@napi-rs/canvas. Deterministic: driven by frame index, never by wall-clock, so renders are reproducible.src/render.js— pipes raw RGBA frames straight into ffmpeg's stdin. ~10× faster than realtime (18.9 s of audio renders in 2 s).
The layout keeps a deliberate margin: glow and expanding rings are sized so they never touch the frame edge, which would show as a hard clipped square once composited.