feat(engine): sub-frame multi-sample motion blur for rotation, scale and opacity

[miga-heygen]

Summary

The motion blur catalog component (#4008) integrates an element's translation over the shutter window: it reads the element's position at the start and end of the window and stacks N equal-weight copies along that path. That matches a 180 degree shutter reference within a pixel for elements that move across the frame.

It cannot blur change that is not a position change inside the same frame:

• rotation during the shutter window (a spinning logo has crisp edges),
• scale changes (a zoom pop has no radial smear),
• opacity ramps and colour changes,
• anything driven by a filter, clip-path, or child animation rather than a transform on the blurred element.

After Effects handles all of these with one mechanism: the layer is re-rendered at several sub-frame times inside the shutter window and the results are averaged.

Proposal

Add an engine-level accumulation mode to frame capture: for each output frame, seek the timeline to K sub-frame positions spread over the shutter window (default 180 degrees, phase -90 degrees, K configurable), capture each, and average the captures into the output frame.

• Opt-in per composition (root attribute, for example data-motion-blur-samples="8" and data-shutter-angle="180"), default off so render cost is unchanged for existing projects.
• Reuse the existing seek plumbing: the timeline is already sought per frame; this is K seeks per frame followed by a blend.
• Blend in linear light or at least document that the blend is in sRGB, so results match the component's behaviour.
• Preview/Studio can show the effect at a lower K or disable it, the way HDR preview already degrades.

Acceptance

• A spinning element renders with radial smear proportional to angular velocity; a scaling element renders with a radial zoom smear; an opacity ramp renders as the integrated opacity over the window.
• A translation-only composition rendered with the engine mode matches the catalog component output within the tolerance measured in fix(registry): make motion-blur match After Effects shutter-based blur #4008.
• Render time scales linearly with K and is unchanged when the mode is off.

Context

The catalog component stays as the cheap default for moving elements. The engine mode is what makes AE-style blur complete for rotation, scale, and fades, which are the cases people currently call "impossible on the web".

[miga-heygen]

Implementation notes from Vance Ingalls (2026-09-16), posted verbatim except for two local-path lines. Line numbers are against origin/main edf6e4b (v0.8.41); re-grep symbols before trusting them.

Implementation notes — issue #4010, engine sub-frame multi-sample motion blur

Handoff notes for whoever implements this.

**All line numbers are against heygen-com/hyperframes at origin/main = edf6e4b37 (v0.8.41),
All line numbers are against heygen-com/hyperframes at origin/main = edf6e4b37 (v0.8.41). Line numbers drift; re-grep the
symbol before trusting a number.
872 commits behind origin/main.

---

What the issue asks for

For each output frame, seek the timeline to K sub-frame times spread across the shutter window,
capture each, and average the captures into one output frame. Opt-in, default off.

Read the issue itself (gh issue view 4010 --repo heygen-com/hyperframes) and PR #4008
(gh pr view 4008). #4008's body is the single most useful document for this task: it already
solved the sub-frame sampling problem in-page, documents the shutter-window maths, and built a
960fps ground-truth reference harness you should reuse.

---

Where the code lives

The capture primitive — this is your insertion point.
packages/engine/src/services/frameCapture.ts (4435 lines).

• captureFrame(session, frameIndex, time) — line 3727. Note it already takes time as an
  argument separate from frameIndex. That is the seam: sub-frame accumulation calls this (or
  its buffer sibling) K times with K different time values and averages the buffers.
• captureFrameToBuffer — line 3764
• captureFrameToBufferPipelined — line 3812
• captureFramesBatchPipelined — line 3976
• BeforeCaptureHook = (page: Page, time: number) => Promise<void> — line 75. Fires after seek,
  before screenshot.

packages/producer/src/services/frameCapture.ts is only a re-export barrel (21 lines). Don't edit it.

The orchestration / frame loop.

• packages/producer/src/services/render/stages/captureStage.ts (440 lines)
• packages/producer/src/services/render/stages/captureStreamingStage.ts (960 lines)

Both need the change. The streaming stage is the one that runs for long or high-frame-count
compositions, and it is where the parallel-worker and batching logic lives.

The runtime seek.
packages/core/src/runtime/init.ts:388:

seek: (timeSeconds: number, options?: { keepPlaying?: boolean }) => void;

Float seconds. Sub-frame seeking requires no new runtime API. The page-side protocol is
window.__hf.seek; the readiness expression is at frameCapture.ts:1582.

---

Blocker you will hit — read this before writing code

quantizeTimeToFrame at packages/core/src/inline-scripts/parityContract.ts:37:

const frameIndex = Math.floor(safeTime * safeFps + 1e-9);
return frameIndex / safeFps;

It floors time onto the output frame grid. Called at frameCapture.ts:3173 and :4327
(quantizeTimeToFrame(frameIdx / fps, fps)). If you pass a fractional sub-frame time through the
existing path it will be snapped down to the containing frame, every sub-sample will land on the
same instant, and your averaged frame will be byte-identical to the unblurred one. This will look
like "the feature does nothing" rather than like an error.

Two ways out, both acceptable:

1. Quantize on the K×fps grid during the accumulation pass — quantizeTimeToFrame(t, fps * K).
   Keeps sub-sample times exact and deterministic, which the pipeline depends on.
2. Use snapTimeToFrameBoundary (same file, line 45). Its contract is explicitly "snap decimal
   noise near a frame boundary without moving genuinely fractional timing" — i.e. it preserves
   fractional times, which is what you want.

Prefer (1). Determinism is a hard requirement here; there is a whole parityContract module
because of it.

---

Static-frame dedup — compose with it, do not bypass it

Documented at frameCapture.ts:88-111. Default-on, opt out with HF_STATIC_DEDUP=false. It
identifies frames byte-identical to their predecessor and reuses lastFrameBuffer (screenshot
path) or lastEncodeResult (drawElement path) instead of re-seeking and re-screenshotting.

This is the difference between the feature being usable and being unusable. Real compositions are
mostly at rest. Measured on the registry motion-blur example: its rows are stationary for 82%,
91% and 95% of the clip respectively. A render I ran logged
[static-dedup] reused 27/30 frame(s) (90%), est. ~1215ms saved.

So: pay K× only on frames that are not static. If a frame is static, one capture is the correct
answer and K captures of the same instant average to the same buffer anyway.

Known interaction: dedup is disabled when a before-capture hook is active. I observed
static-frame dedup: disabled (before-capture hook / video injection active) in a render that
injected video. If accumulation is implemented via a before-capture hook, it will silently disable
dedup and you lose the optimisation above. Don't implement it as a before-capture hook.

Also note BeforeCaptureHook fires per capture, so K captures = K hook invocations. Video
injection running K× per frame is probably wrong. Decide deliberately.

---

Event suppression on sub-frame seeks

#4008 hit this hard and the write-up is worth reading in full. Summary of its finding: seeking GSAP
from inside its own render re-enters the timeline mid-iteration, reads stale tween state on the
first sample entering a tween, and suppresses the composition's own tween callbacks on forward
seeks. It fixed this with tl.time(t, true) (events suppressed) plus a microtask deferral, and
proved it with a randomised stress test — 3,000 arbitrary seeks went from 13,779 stale samples and
1,436 wrong restored states, to zero of each.

The engine path seeks via window.__hf.seek rather than from inside a tween callback, so the
re-entrancy may not apply. Verify rather than assume. What definitely must hold: a user's
onUpdate/onComplete callbacks must not fire K times per output frame. If __hf.seek does not
already suppress them, add an option that does.

Reuse #4008's stress test. Do not write a new one from scratch.

---

What does and does not follow a sub-frame seek

Verified by reading the adapters:

• GSAP — yes.
• CSS @keyframes — yes. packages/core/src/runtime/adapters/css.ts:56 sets
  animation.currentTime = timeMs.
• Web Animations (WAAPI) — yes. packages/core/src/runtime/adapters/waapi.ts:166, same
  mechanism.

Both take float milliseconds, so sub-frame times work.

• <video> content — no. Video goes through a separate frame-extraction path, not the
  timeline seek. A video's own motion will not blur. Document this as a known limitation; don't
  let it be discovered later as a bug.

This matters because the issue claims "one mechanism" handles "anything driven by a filter,
clip-path, or child animation". That is true for GSAP/CSS/WAAPI-driven change. It is not true for
video.

---

API surface

The issue proposes root attributes data-motion-blur-samples="8" and data-shutter-angle="180".

Gap to close: composition-level opt-in is all-or-nothing. Every animated element in the
composition gets blurred, including captions, UI chrome and logo lockups you want crisp. Both
existing precedents are narrower — #4008 attaches per-selector, and the AE path marks per-element
with data-ae. Add a per-element opt-out (e.g. data-motion-blur="off") or an allowlist.

Match #4008's naming and clamp: it uses samplesPerFrame, default 16, max 64.

Shutter window maths — copy #4008's, verbatim, so the two implementations cannot drift:

shutterTime  = shutterAngle / 360 / fps
windowStart  = t + shutterPhase / 360 / fps
sample k at   windowStart + (k + 0.5) / N * shutterTime

Defaults shutterAngle: 180, shutterPhase: -90, samplesPerFrame: 16 — which integrate
[t − ¼/fps, t + ¼/fps], i.e. AE's defaults.

---

Blending colour space — the issue contradicts itself, resolve it this way

The issue says "blend in linear light or at least document that the blend is in sRGB", and sets an
acceptance criterion of matching #4008 within its measured tolerance. Those conflict. #4008
deliberately averages in sRGB (color-interpolation-filters="sRGB") to match AE's default
non-linearised 8-bpc working space. Linear-light averaging diverges from sRGB precisely on
high-contrast edges — which is exactly where blur gets judged.

Recommendation: sRGB by default (matches AE and #4008), linear as an explicit opt-in. Say so in
the PR description so the divergence is a decision rather than an accident.

---

Acceptance criteria — corrections to the issue's list

1. Do not anchor to fix(registry): make motion-blur match After Effects shutter-based blur #4008. The issue says a translation-only composition should match the
   catalog component. That caps the engine mode at the accuracy of an approximation. The engine
   re-renders rather than offsetting copies, so it should be more accurate. Test against the
   960fps shutter-integral reference fix(registry): make motion-blur match After Effects shutter-based blur #4008 already built (four 240fps passes with the timeline
   shifted by ±1/1920s and ±3/1920s, interleaved, then average the 16 sub-frames inside the 180°
   window). fix(registry): make motion-blur match After Effects shutter-based blur #4008 scored 72.4 dB mean PSNR / 0.9998 SSIM against it; beat that.

2. Add a high-velocity case. "A spinning element" is low displacement. Required K scales with
   per-frame displacement: K ≈ displacement ÷ acceptable step. The registry motion-blur example's
   fastest row peaks near 1020 px/frame at 30fps; K=16 leaves 64px gaps and you get visible
   discrete copies, not a smear. Measured directly: at 720° shutter, 16 samples produced hard-edged
   repeated copies and 64 resolved them into a continuous arc.

3. Add an at-rest case. A stationary element must be pixel-sharp. fix(registry): make motion-blur match After Effects shutter-based blur #4008 removes its filter
   entirely at rest for this reason and measured hold frames as sharp.

4. Strengthen the cost criterion. "Render time unchanged when off" is too weak. It should be
   "scales with K only on non-static frames" — see the dedup section.

---

De-risking evidence already gathered

I prototyped the issue's mechanism from outside the engine, to confirm it produces the claimed
result before anyone writes engine code. It does.

Method: render the composition at 8× frame rate, then average sub-frames back down with ffmpeg.

hyperframes render --ae off --fps 240 --low-memory-mode --output super240.mp4
# 180 degree shutter: average the middle 4 of every group of 8
ffmpeg -i super240.mp4 -vf "tmix=frames=4,select='not(mod(n-5,8))',setpts=N/30/TB" -r 30 sub180.mp4
# 360 degree shutter: average all 8
ffmpeg -i super240.mp4 -vf "tmix=frames=8,select='not(mod(n-7,8))',setpts=N/30/TB" -r 30 sub360.mp4

Result: all six transform phases blurred — x translation, y translation, z depth, rotateX,
rotateY and rotateZ. The three rotations and the depth move are what #4008 cannot do at all and
what AE's Pixel Motion Blur only manages weakly. Shutter scaling checked out too: measured smear
ratio 360°/180° = 2.16× against a predicted 2.00×.

Artifacts, if still present: /tmp/hf-4010-text/ (sub180.mp4, sub360.mp4, phases-4010.png,
index.html). /tmp/hf-4008-text/ holds the #4008-component render of the same composition for
comparison. Both are under /tmp and will not survive a reboot — re-create from the commands above
if they are gone.

Two caveats on the prototype:

• The visible stepping in those files is the prototype's fault, not the mechanism's. fps is capped
  at 240 (packages/core/src/core.types.ts:91 and :111), so a single pass yields 8 sub-frames per
  output frame and only 4 inside a 180° window. The engine implementation has no such cap — it
  seeks arbitrary sub-frame times, so K is free. Expect materially better output than the prototype.
• Rendering at 240fps wanted ~12 GB of temp frame storage and failed on a full disk.
  --low-memory-mode streams frames and works. Irrelevant to the engine path, which never
  materialises a high-frame-rate intermediate.

---

Context you should know but not act on

There is an unmerged local branch implementing After Effects motion blur by exporting a marked
element to a PNG sequence, applying AE's Pixel Motion Blur, and compositing a transparent WebM back
in. It is at 73ae89b98 in the worktree named above, with a completed review.

Measurements from it, relevant because they bound what #4010 should be compared against: AE Pixel
Motion Blur delivers roughly 76% of the theoretically correct smear at 68 px/frame, 75% at 136,
58% at 291, and zero at 583 and 1020 px/frame — optical flow stops tracking. It also costs
72–110s per marked element, requires macOS with AE installed and running, and rewrites the author's
source composition file in place.

If #4010 lands, it supersedes that branch for motion blur specifically: no external dependency, no
velocity ceiling, no source mutation. Do not delete or modify that branch as part of this work —
flag the overlap and let a human decide.