feat(engine): adaptive per-frame sample count for motion blur (floor 16, ceiling 64)

[miguel-heygen]

Summary

data-motion-blur-samples / CaptureOptions.motionBlur.samplesPerFrame (packages/engine/src/services/motionBlur.ts:23,53,55,70-77) is a fixed per-render sample count, default 16, ceiling 64 (MAX_SAMPLES_PER_FRAME). Measuring the shipped feature (v0.8.45) against the accuracy bar set in #4010 (72.4 dB mean PSNR vs. the 960fps shutter integral) shows the default does not universally clear it: it depends on the content's edge hardness and displacement.

Measurements

Harness: an independent 128-sample reference per frame, built by hand (raw sub-frame window.__hf.seek(t, {subframe, suppressEvents}) calls + screenshots, averaged with a standalone accumulator — not the production accumulation code). Validated before trusting it: the production K=16 output and a hand-rolled K=16 reference built the same way are byte-identical, confirming the harness measures the real pipeline rather than a stand-in.

Composition: a single 20px-wide opaque bar on a 1536x200 transparent canvas, 30fps, default shutter (180°, phase -90°).

1. PSNR, moderate translation (66.7 px/frame — the same displacement basis used for the original 72.4 dB number in fix(registry): make motion-blur match After Effects shutter-based blur #4008): default K=16 → 30.1 dB mean PSNR, well under the bar. K=64 → 80.0 dB, clears it comfortably.
2. High-velocity (1020 px/frame, matching the registry motion-blur example's fastest row): K=16 → 15 true zero-coverage gaps across the swept region (the smear visibly breaks into separate copies). K=64 → 0 gaps (fully continuous, resolved into a smear).
3. At-rest: blur on vs. off is byte-identical — stays pixel-sharp.

Cases 2 and 3 match the acceptance criteria from #4010 as written. Case 1 does not.

Caveat: this is one hard-edge synthetic frame, not the original multi-frame text composition that produced 72.4 dB. A sharp, narrow, fully-opaque edge is a harder case for discretization at low K than typical content (antialiased text, softer boundaries) — this is not "the feature is broken," it's evidence that a fixed default sample count is content-dependent and can miss the bar on legitimately common content (thin high-contrast elements: text strokes, iconography, UI chrome).

Proposal

Adaptive per-frame sample count, derived from the motion magnitude measured across the shutter window (e.g. displacement of the animated properties between the window's sub-frame endpoints), rather than a fixed samplesPerFrame. Floor at the current default (16), ceiling at the current max (64, MAX_SAMPLES_PER_FRAME), so:

• Static and slow-moving content pays what it already pays today, at the floor — case 3 above must stay free. Cost never drops below today's fixed 16; it only rises on the frames that need it.
• Fast or hard-edged content scales up toward 64 automatically, closing the gap in case 1 without every render paying the 64-sample cost.

Open question for whoever picks this up: what "motion across the shutter window" should be measured from — screen-space displacement of the blurred element's bounding box between the window's endpoint samples is the cheapest signal and is already implicitly available since the endpoints are captured either way; whether that alone is a good enough proxy for rotation/scale-driven blur (as opposed to pure translation) needs checking against a case like case 1 but with rotation instead of translation.

[miguel-heygen]

Plan (P2, per Miguel)

Gate 1 (unchanged). computeStaticFrameSet / session.staticFrames (frameCapture.ts:2739, motionBlurWindowIsStatic in motionBlur.ts:125) still decides "no active tween in this frame's span → 1 capture, reuse the neighbour." Adaptive sampling never runs for these frames; still content keeps paying nothing extra.

Gate 2 (new): non-spatial change does not need high K. A fade or colour change reads as a large pixel diff with no actual motion to resolve, and would otherwise buy 64 samples it doesn't need. Extend the existing GSAP walk in computeStaticFrameSet to record, per active interval, whether the tween's vars touch a spatial property (x, y, rotation, scale, skew, top, left, width, height, transform) as opposed to only non-spatial ones (opacity, color, backgroundColor, filter, ...). For CSS/WAAPI-driven animation, the same classification reads element.getAnimations()[].effect.getKeyframes() property keys (mirrors packages/core/src/runtime/adapters/css.ts and waapi.ts). A frame whose only active animation(s) are non-spatial skips the probe step below and uses the floor (16) directly — identical cost to today.

Probes (new), measurement-only. For a frame that clears both gates, capture two extra samples at the true shutter-window edges (not the existing (k+0.5)/K midpoint placement motionBlur.ts:70-88) purely to measure motion magnitude. Decode both with the accumulator's existing decodePng and diff them. These two buffers are never passed to MotionBlurAccumulator (motionBlur.ts:182-243) — real accumulation keeps its current (k+0.5)/K placement at whichever K gets picked, so the 80 dB measured at K=64 and the 30→80 dB spread already reported need no re-validation.

K selection (new). A monotone step function, diff magnitude → K ∈ [16, 64], thresholds in one exported table in motionBlur.ts next to MAX_SAMPLES_PER_FRAME, each with a comment citing the composition/velocity it was calibrated against — calibrated from a small sweep (not guessed), extending the harness used for the numbers above.

Cost, per frame:

• static (gate 1): 1 capture, unchanged.
• moving, non-spatial only (gate 2): 1×floor(16) captures, unchanged from today's default.
• moving, spatial: 2 probes + K accumulation captures, K ∈ [16, 64] — this is the only case that changes.

Building against this shape now.