Skip to content

Adds E-AC-3 / Dolby Atmos (E-AC-3 JOC) support across the MP4 extractor and muxer.#3317

Open
rexjin-meta wants to merge 4 commits into
androidx:mainfrom
rexjin-meta:muxer-atmos-v2
Open

Adds E-AC-3 / Dolby Atmos (E-AC-3 JOC) support across the MP4 extractor and muxer.#3317
rexjin-meta wants to merge 4 commits into
androidx:mainfrom
rexjin-meta:muxer-atmos-v2

Conversation

@rexjin-meta

@rexjin-meta rexjin-meta commented Jul 2, 2026

Copy link
Copy Markdown

Adds E-AC-3 / Dolby Atmos (E-AC-3 JOC) support across the MP4 extractor and muxer.

  • New FormatSpecificTransmuxingData (Metadata.Entry, media3-container) carries a raw box payload for transmuxing.
  • Extractor attaches the dec3 payload to Format.metadata (not initializationData), so the E-AC-3 decode path is unchanged.
  • Muxer writes the dec3 box / ec-3 fourcc from that metadata entry; AUDIO_E_AC3 / AUDIO_E_AC3_JOC added to Mp4Muxer + FragmentedMp4Muxer supported types, so Transformer can transmux Atmos.
  • Tests: BoxesTest + end-to-end transmux (Mp4MuxerEndToEndTest, FragmentedMp4MuxerEndToEndTest); extractor golden dumps regenerated.

@snow2405 snow2405 self-assigned this Jul 3, 2026
@FongMi

FongMi commented Jul 3, 2026

Copy link
Copy Markdown

Findings

  1. P1 Mp4Muxer.java:417 / FragmentedMp4Muxer.java:184
    Adding AUDIO_E_AC3 / AUDIO_E_AC3_JOC to the supported list causes the default Transformer to directly transmux E-AC-3. However, dec3Box() requires the raw dec3 payload in Format.initializationData, while the existing MP4 extractor parses dec3 into Format fields without preserving initialization data. For common E-AC-3/Atmos MP4 or TS inputs, this may change behavior from transcoding to AAC into direct copy, then fail later while muxing moov because csd-0 is missing. I’d recommend either preserving raw dec3 in BoxParser in the same PR, or not adding these MIME types to the factory supported list yet.

  2. P3 Boxes.java:1658
    dec3Box() only checks that initializationData is non-empty, but does not check that csd-0 itself has bytes. An empty payload would produce an empty dec3 box, which is not a valid EC3SpecificBox. This should mirror esdsBox() / hvcCBox() with checkArgument(csd0.length > 0, ...), plus a test for an empty byte array.

@rexjin-meta

rexjin-meta commented Jul 3, 2026

Copy link
Copy Markdown
Author

Hey @FongMi and @snow2405 , Thanks for the reviews — addressed both comments:

  • P1 (Transformer regression): Dropped AUDIO_E_AC3 / AUDIO_E_AC3_JOC from SUPPORTED_AUDIO_SAMPLE_MIME_TYPES in both Mp4Muxer and FragmentedMp4Muxer, so the default Transformer keeps transcoding E-AC-3 to AAC — no behavior change. The muxer retains the dec3/ec-3 write capability for direct callers that supply csd-0. Added a comment on dec3Box() explaining the deferral.

  • P3 (empty csd-0): Added checkArgument(csd0.length > 0, "csd-0 is empty for dec3 box."), mirroring esdsBox() / hvcCBox(), plus a test for the empty-byte[] case.

Also added a unit test asserting the ec-3 fourcc on the audio sample entry.

This PR will laser focus on enabling muxer to write atmos.
Follow-up (separate PR): preserve the raw dec3 payload in BoxParser, then add E-AC-3 to the supported list and add an end-to-end mux test (with the audio_sample_entry_box_eac3 golden dump). This staging keeps the current PR free of the extractor change and any Transformer impact.

@rexjin-meta

Copy link
Copy Markdown
Author

cc: @andrewlewis This PR is the new attempt for my previously closed PR: #3312 (comment)

@rexjin-meta

Copy link
Copy Markdown
Author

Updated the PR to fix the extractor and muxer together, as suggested.

Extractor: BoxParser now preserves the raw dec3 (EC3SpecificBox) payload in Format.initializationData when parsing the MP4 dec3 box — mirroring the existing av1C/dOps handling (box body without header). This is the piece that was missing.

Muxer: implements the dec3 box + ec-3 fourcc and adds AUDIO_E_AC3 / AUDIO_E_AC3_JOC to the supported audio MIME types. With the extractor change in place, the default Transformer now correctly stream-copies E-AC-3 / Atmos (instead of transcoding), and it no longer risks the missing-csd-0 failure — the two changes are consistent across extractor, muxer, and transformer.

Tests: added an end-to-end Mp4MuxerEndToEndParameterizedTest pass for sample_eac3.mp4 and sample_eac3joc.mp4; regenerated the extractor golden dumps (they now include the dec3 initializationData); BoxesTest covers dec3 wrapping, the ec-3 fourcc, and the missing/empty csd-0 guards.

@rexjin-meta rexjin-meta changed the title Add muxer support for E-AC-3 and Dolby Atmos (E-AC-3 JOC) Add muxer/demuxer support for E-AC-3 and Dolby Atmos (E-AC-3 JOC) Jul 6, 2026
@rexjin-meta

rexjin-meta commented Jul 8, 2026

Copy link
Copy Markdown
Author

Reworked per your suggestion. Added FormatSpecificTransmuxingData (a Metadata.Entry in media3-container, boxType + raw data). The extractor now attaches the dec3 payload to Format.metadata instead of initializationData, so the E-AC-3 decode path is unchanged; the muxer reads it back from Format.metadata to write the dec3 box.

Tests: BoxesTest covers dec3 wrapping, the ec-3 fourcc, and missing/empty-payload guards; Mp4MuxerEndToEndTest + FragmentedMp4MuxerEndToEndTest do a full Mp4Extractor → mux → re-extract round trip for mple_eac3(.mp4|joc.mp4). Extractor golden dumps regenerated.

Made the entry general (boxType + data) rather than E-AC-3-specific — open to either.

@rexjin-meta rexjin-meta changed the title Add muxer/demuxer support for E-AC-3 and Dolby Atmos (E-AC-3 JOC) Adds E-AC-3 / Dolby Atmos (E-AC-3 JOC) support across the MP4 extractor and muxer. Jul 8, 2026
rexjin-meta and others added 3 commits July 9, 2026 14:30
Introduce FormatSpecificTransmuxingData, a typed Metadata.Entry that
carries a raw MP4 box payload (the box body, without header) so it can be
reproduced when transmuxing without re-encoding.

Extractor: when parsing the MP4 dec3 (EC3SpecificBox), attach its payload
to Format.metadata as a FormatSpecificTransmuxingData("dec3", ...) entry
instead of Format.initializationData. This keeps E-AC-3's decode path
unchanged (no csd-0 is injected) while preserving the bytes for muxing.

Muxer: implement the dec3 box and ec-3 fourcc in Boxes.java, reading the
payload from the FormatSpecificTransmuxingData metadata entry, and add
AUDIO_E_AC3 / AUDIO_E_AC3_JOC to the supported audio MIME types in
Mp4Muxer and FragmentedMp4Muxer. The default Transformer can now
stream-copy (transmux) E-AC-3 / Dolby Atmos audio.

Tests:
- BoxesTest: dec3 box wrapping for AUDIO_E_AC3 and AUDIO_E_AC3_JOC, the
  ec-3 fourcc, and guards for missing / empty dec3 payload.
- Mp4MuxerEndToEndTest and FragmentedMp4MuxerEndToEndTest: Mp4Extractor-
  driven transmux of sample_eac3.mp4 and sample_eac3joc.mp4 (reads via
  Mp4Extractor so Format.metadata is preserved).
- Extractor golden dumps regenerated to carry the dec3 metadata entry.
@snow2405

snow2405 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Hey @rexjin-meta
I'm going to send this for internal review now. You may see some more commits being added as I make changes in response to review feedback. Please refrain from pushing any more substantive changes as it will complicate the internal review - thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants