Skip to content

fix(core): preserve native audio fallback - #3048

Draft
cwhy wants to merge 1 commit into
heygen-com:mainfrom
cwhy:cy-cd/core-mixed-webaudio-ownership
Draft

fix(core): preserve native audio fallback#3048
cwhy wants to merge 1 commit into
heygen-com:mainfrom
cwhy:cy-cd/core-mixed-webaudio-ownership

Conversation

@cwhy

@cwhy cwhy commented Aug 5, 2026

Copy link
Copy Markdown

Summary

  • use the global output mute only for parent-proxy ownership
  • keep WebAudio muting scoped to each element actually owned by WebAudio
  • cover mixed WebAudio/native fallback plus parent force-mute behavior in the runtime integration test

Root cause

The runtime supports mixed ownership on a per-element basis:

  • a track that decodes successfully may be played by WebAudio
  • another track in the same composition may fail WebAudio fetch/decode and continue through its native HTMLMediaElement
  • video soundtracks may likewise remain native while separate audio tracks use WebAudio

Before this change, one active WebAudio source made webAudio.isActive() true and passed outputMuted: true to syncRuntimeMedia. Because outputMuted means “force-mute every media element,” one WebAudio-owned track globally muted unrelated native fallback tracks. In the reported mixed-CORS case, background music entered WebAudio while narration could not; the narration continued advancing natively but was silent.

Ownership invariant

The intended rule is:

muteThisElement =
  parentOwnsAllAudio ||
  userMuted ||
  webAudio.ownsElement(thisElement);

syncRuntimeMedia already implements that split:

  • outputMuted || userMuted force-mutes all elements
  • isWebAudioOwned(el) mutes only the element duplicated through WebAudio
  • an unowned element remains audible through the native fallback

This PR therefore passes only state.mediaOutputMuted as the global output mute and retains isWebAudioOwned: (el) => webAudio.ownsElement(el) for per-element ownership.

Why the removed clauses are not lost safeguards

nativeMediaSyncDisabled

The call to syncRuntimeMedia is already inside if (!state.nativeMediaSyncDisabled). Rechecking the same flag inside the outputMuted expression was redundant. When native media sync is disabled, this call is skipped entirely.

webAudioMediaDisabled

Disabling WebAudio prevents new WebAudio scheduling and stops active sources. With no active source, webAudio.ownsElement(el) is false and native media is the intended output. This flag should choose the transport, not globally silence native media.

webAudio.isActive()

This reports whether any WebAudio source is active; it does not report whether WebAudio owns a particular element. Using it as a global mute conflated composition-level activity with per-element ownership and defeated the existing ownsElement(el) contract.

Safety behavior that remains unchanged

  • Parent-proxy ownership still sets state.mediaOutputMuted, immediately mutes the WebAudio master and every DOM media element, and is reasserted on subsequent ticks.
  • The user's mute preference remains independently enforced through state.bridgeMuted / userMuted.
  • A WebAudio source mutes its corresponding DOM element when scheduled, and ownsElement(el) continues to prevent double audio.
  • Stopping WebAudio restores the element's prior mute state.
  • Native autoplay rejection still reports media-autoplay-blocked, allowing the player to promote audio ownership to parent proxies.

Regression coverage

The new runtime test creates two simultaneous tracks while WebAudio is globally active:

  1. a WebAudio-owned track remains muted in the DOM
  2. an unowned native fallback remains unmuted
  3. after parent ownership sends set-media-output-muted: true, both tracks are muted

Existing media and transport tests additionally cover owned/unowned separation, global parent/user mute, exact-element ownership, and ownership cleanup.

Verification

  • core typecheck
  • 1,498/1,498 core tests
  • formatter and linter
  • exact mixed-CORS Chrome canary: native narration remained unmuted and advanced while background music remained WebAudio-owned at its authored gain
  • parent force-mute behavior verified after mixed ownership was established

Draft status and remaining validation

This PR intentionally remains a draft while stronger end-to-end coverage is added or reviewed. Useful follow-ups before marking it ready:

  • real asynchronous mixed decode success/failure rather than mocked ownership
  • WebAudio music plus a native video soundtrack
  • native NotAllowedError promotion to parent-proxy ownership
  • enabling/disabling WebAudio and native synchronization during playback

Those gaps do not change the ownership reasoning above, but documenting and exercising them reduces regression risk before merge.

@cwhy
cwhy marked this pull request as draft August 5, 2026 09:17
@cwhy
cwhy marked this pull request as ready for review August 5, 2026 09:36
@cwhy
cwhy marked this pull request as draft August 5, 2026 09:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant