Skip to content

perf(player): scope preview readiness to first frame - #4239

Open
miguel-heygen wants to merge 6 commits into
mainfrom
hfoss36/first-frame-gate
Open

miguel-heygen wants to merge 6 commits into
mainfrom
hfoss36/first-frame-gate

Conversation

@miguel-heygen

@miguel-heygen miguel-heygen commented Sep 21, 2026

Copy link
Copy Markdown
Collaborator

What changed

Preview readiness now waits for media and images active at t=0, plus fonts. The full scan remains the default for callers that need every asset. Runtime and player callers share one first-frame scope constant. Runtime visibility and readiness now use the same timing-window helper.

What I measured

  • Launch composition: before 1.04s, after 1.04s to the first visible crop, with no black intervals.
  • 20-clip composition with the first clip at t=0 and later clips scheduled from t=1 onward: before 1.04s, after 1.04s, with no black intervals.
  • The current capture crop sees the editor canvas before it can isolate preview pixels, so these equal readings are a regression check, not a claimed speedup.
  • Core readiness and timeline tests: 182 core tests and 167 player tests passed. Core and player typechecks passed.

Before

Before

After

After

Export and producer path

The producer and engine paths do not import the composition-readiness module. This was checked with rg -n 'composition-readiness|compositionReadiness' packages/producer packages/engine, which returned no matches. Producer parity waits on __playerReady and __renderReady; engine capture readiness separately checks seek availability and positive duration.

The explicit { scope: "all" } public-path test waits for a late asset, pinning the full-wait behavior.

For preview, readiness now favors showing the first frame over buffering later clips. A later clip may still stall when playback reaches it on a slow connection.

What I did not exercise

  • End-to-end export rendering was not run in this change.
  • The capture crop did not isolate preview pixels from the editor canvas.

Before

After

@miguel-heygen

Copy link
Copy Markdown
Collaborator Author

Launch composition capture at 12s. The before and after frames use the same capture path; the current crop includes the editor canvas, so these are visual artifacts only and not a claimed speedup.

before

after

@miguel-heygen
miguel-heygen force-pushed the hfoss36/first-frame-gate branch from 4cf88d1 to a3c50d0 Compare September 21, 2026 04:08

@terencecho terencecho left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVING at a3c50d0b3e2138a593b3cfb25e3a89ef322246ee — the preview-vs-export split is fenced by both API default (scope: "all") and import-graph isolation (producer/engine don't touch this module), and the explicit full-scan public-path test pins the invariant.

Export-safety invariant — verified through three independent gates

Gate 1 — API-default fail-open: scope: "all" is the default on every public entry (scanPendingCompositionAssets, mediaReadinessInput, settleCompositionReadiness). Any caller that doesn't explicitly opt in keeps the old wait-for-every-asset behavior. The new settleFirstFrameCompositionReadiness is a thin wrapper that has to be called by name to trigger the reduced-scope path.

Gate 2 — Import-graph isolation: packages/producer and packages/engine don't import compositionReadiness at all (verified by GH code search on composition-readiness / compositionReadiness scoped to each path — 0 hits each). Even if the default flipped tomorrow, export wouldn't reach through this module.

Gate 3 — Consumer census: Only 6 files touch this module, and I read each one:

  • packages/core/src/compositionReadiness.ts — the module itself
  • packages/core/src/compositionReadiness.test.ts — tests (covered below)
  • packages/core/src/runtime/init.ts:2896converted to first-frame — the sandbox runtime posts to the "hf-preview" protocol channel, so this is preview-only
  • packages/core/src/runtime/transportPark.test.ts — mock updated to preserve importOriginal + mock both variants
  • packages/player/src/hyperframes-player.tsconverted to first-frame in _settleAssetsReady (:1060) and _warnStuckAssets (:1083)
  • packages/studio/src/player/store/readinessSlice.ts:38NOT converted, still uses default scope: "all" (see non-blocking note below)

Grep for settleFirstFrameCompositionReadiness across the head confirms only two call sites (init.ts:2896, hyperframes-player.ts:1060) — no leaking use in export/producer/engine.

First-frame scope predicate is correct at t=0

isActiveAtFirstFrame (compositionReadiness.ts:214-237) walks ancestors and returns false only if some ANCESTOR carries timing (data-start or data-track-index) AND that ancestor isn't visible at t=0. Two important properties fall out of this:

  • Untimed media stays in scope — because isTimedElement is false all the way up the chain, so no ancestor "votes to skip". .test.ts:32 pins this. Correct fail-safe: no timing metadata → wait for it.
  • Nested composition timing decisions align with runtime visibility owner — the walk uses the same isRuntimeElementVisibleAt predicate the runtime uses to draw or hide the element, with matching exportRenderSeek: false for the first-frame scan (interactive, not export). .test.ts:40-58 pins this by wiring both sides against the same doc + resolver and asserting they agree.

Fonts are NOT gated by scope — they always count as pending if doc.fonts?.status === "loading". That's the right call: font swap is visible to the user regardless of which clip is active at t=0, and blocking font readiness prevents FOIT/FOUT flashes on the first frame.

Test coverage is non-vacuous, not framework text

The load-bearing test is .test.ts:153-173 — "keeps the explicit full-scan path waiting for a later video". It:

  • Sets image complete = true (first-frame ready)
  • Sets video readyState = 0 (later video NOT ready)
  • Calls settleCompositionReadiness(..., { scope: "all", timeoutMs: 1 })
  • Asserts { timedOut: true } — the promise never resolved because it was still waiting on the video

That test WOULD fail if scope: "all" silently became "first-frame". Pinned.

The mirror test at :113-151 — "does not wait for a later video through the public first-frame path" — pins the OTHER direction: settleFirstFrameCompositionReadiness resolves once the first image + fonts settle, without waiting on the far video. Uses flushMicrotasks() and asserts result stays undefined until both the image AND fonts resolve, then settles as { timedOut: false }.

.test.ts:67-83 and :85-105 pin the same behavior at the mediaReadinessInput layer.

Refactor observations (non-blocking, all good)

  • isRuntimeElementVisibleAt extracted from the inline isTimedElementVisibleAt in init.ts into runtime/timeline.ts. Same logic (start/duration resolution, composition-id live-duration handoff, frame-snap on exportRenderSeek), now shared between the sandbox runtime's visibility check and the first-frame scope predicate. Both call sites in init.ts:2481-2500 pass through matching parameters — no drift.
  • AUTHORED_DURATION_ATTR and AUTHORED_END_ATTR moved from a private init.ts const to exported constants in authoredTiming.ts so both init.ts and timeline.ts reference the same names. Prevents string-literal drift between the two visibility-check sites.
  • transportPark.test.ts mock update uses importOriginal to preserve the module's other exports (constants, types) while mocking both settle* entry points — the right pattern for partial mocks.

Non-blocking follow-up: Studio's readinessSlice.ts stayed on scope: "all"

packages/studio/src/player/store/readinessSlice.ts:38 calls settleCompositionReadiness(doc, ...) with no scope — defaults to "all". This is the Studio-editor preview store, gating a timelineReady flag consumed by 18 files (autoplay, scrub, seek, timeline overlay tests, etc). Two ways to read this:

  • Intentional: the Studio editor's timelineReady needs to gate scrub/autoplay/seek across the whole composition, so waiting for every asset avoids stutter when the user scrubs to a later clip. Preview display (which this PR speeds up) is a different concern from preview interactivity.
  • Oversight: just missed as a caller during the sweep.

If the former, worth an inline comment on why Studio's readiness stays on the full scan while the player web-component uses first-frame. If the latter, converting readinessSlice.ts to settleFirstFrameCompositionReadiness would let Studio's initial timelineReady=true land earlier for the same UX win. Either way, doesn't block this PR — the export-safety invariant is intact and this is a UX question inside the preview world.

Stamp mechanics

  • Head a3c50d0b3e2138a593b3cfb25e3a89ef322246ee, authored by miguel-heygen (trusted stamper).
  • CI green: 54 SUCCESS, 8 SKIPPED, 0 FAILURE at head.
  • reviewDecision: REVIEW_REQUIRED, mergeStateStatus: BLOCKED — my approve clears the gate.
  • hf-oss require_last_push_approval=true → this approval is at head, valid until a new push.
  • Rebased-onto-main clean; diff scope is 7 files (all within packages/core/ + packages/player/), no producer/engine/studio-source drift beyond what's diff-visible.

— Review by tai (pr-review)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants