diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..b113183 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,126 @@ +# CLAUDE.md + +Guidance for AI agents (Claude, Copilot, etc.) and new contributors working in +this repository. Keep this file up to date when project structure, conventions, +or workflows change. + +## What this project is + +`react-native-compressor` is a React Native library that compresses **video**, +**image**, and **audio** files (and provides background upload/download helpers) +with results comparable to WhatsApp-style compression. It ships native code for +both **iOS** (Swift/Obj-C) and **Android** (Kotlin), exposed to JavaScript through +a TurboModule-capable spec, and supports both the old and new React Native +architectures plus an Expo config plugin. + +- Package name: `react-native-compressor` +- Package manager: **Yarn 4** (`packageManager: yarn@4.14.1`); Yarn workspaces with `examples/*`. +- Upstream: https://github.com/numandev1/react-native-compressor — this is a fork + (`XChikuX/react-native-compressor`) that triages and fixes upstream issues. + +## Repository layout + +``` +src/ JavaScript/TypeScript public API (the npm entry point) + index.tsx Re-exports the public surface + Main.tsx Aggregates the default export object + Spec/NativeCompressor.ts TurboModule spec (single source of truth for native methods) + Video/ Image/ Audio/ Per-domain JS wrappers (compress(), options, events) + utils/ Uploader/Downloader/helpers (uuid, path normalization) + expo-plugin/ Expo config plugin +ios/ Native iOS implementation (Swift + Obj-C bridge) + Video/VideoMain.swift Video compression entry (auto/manual helpers) + Video/NextLevelSessionExporter.swift AVAssetReader/Writer export engine + Image/ Audio/ Utils/ Image, audio, upload/download, thumbnails + Compressor.mm / Compressor.h Obj-C bridge to the Swift module +android/ Native Android implementation (Kotlin) + src/main/java/com/reactnativecompressor/ + Video/ Video compression (MediaCodec transcode pipeline) + VideoCompressor/compressor/Compressor.kt Core encode/decode loop + VideoCompressor/utils/CompressorUtils.kt Format/codec helpers + Image/ Audio/ Utils/ Image, audio, upload/download, helpers + src/oldarch / src/newarch Architecture-specific TurboModule specs +__tests__/ Jest unit tests for the JS wrapper (native is mocked) +harness/ react-native-harness on-device smoke test definitions +examples/bare Bare React Native example app (build + harness target) +examples/expo Expo example app +TRIAGE.md Running triage of upstream issues and fixes in this fork +``` + +## Public API surface + +The default export aggregates these modules/functions (see +`__tests__/compressor.test.ts` for the authoritative list): +`Audio`, `Image`, `Video`, `UploadType`, `UploaderHttpMethod`, `backgroundUpload`, +`cancelUpload`, `clearCache`, `createVideoThumbnail`, `download`, +`generateFilePath`, `getDetails`, `getFileSize`, `getImageMetaData`, +`getRealPath`, `getVideoMetaData`, `uuidv4`. + +Video compression supports `compressionMethod: 'auto' | 'manual'`, `maxSize`, +`bitrate`, `progressDivider`, `minimumFileSizeForCompress`, and `stripAudio`. + +## Build, test, and validate + +Run JS-level checks from the repo root: + +| Command | Purpose | +| --- | --- | +| `yarn install` | Install dependencies (Yarn 4) | +| `yarn jest` / `yarn test` | Run the JS wrapper unit tests | +| `yarn typecheck` | `tsc --noEmit` | +| `yarn lint` | ESLint over `**/*.{js,ts,tsx}` | +| `yarn test:pr` | `test --runInBand && typecheck && lint` (run before opening a PR) | +| `yarn build:android` | Assemble the bare example (`arm64-v8a`) | +| `yarn build:ios` | Build the bare example for the iOS simulator | +| `yarn test:harness:android` / `yarn test:harness:ios` | On-device/simulator smoke tests | + +**Important:** The Jest tests mock the native module, so they validate only the +JS contract. Real media decoding/encoding **cannot** be verified by unit tests — +it must be smoke-tested in the example app on a simulator or device. When you +change native Swift/Kotlin code, state clearly that it was not runtime-verified +in CI and, where possible, validate via the example app or harness. + +## Native video pipeline notes (high-signal, easy to get wrong) + +### iOS (`ios/Video/`) +- `VideoMain.swift` builds the `videoOutputConfiguration` / `compressionDict` + and drives `NextLevelSessionExporter`. +- `NextLevelSessionExporter.setupVideoOutput` only creates the video writer input + when `writer.canApply(outputSettings:forMediaType:) == true`; otherwise it logs + `"Unsupported output configuration"` and writes **audio only**, yet still ends + as `.completed`. That means a bad `videoOutputConfiguration` can silently yield + an **audio-only** MP4 reported as success. +- Do **not** add undocumented H.264 (`avc1`) compression properties such as + `AVVideoExpectedSourceFrameRateKey` or `AVVideoAverageNonDroppableFrameRateKey`: + `canApply(...)` accepts them but the iOS encoder drops the video track + (regression in #392, fixed for #400). After export, verify the output asset + actually contains a video track before resolving success. + +### Android (`android/.../Video/VideoCompressor/`) +- `Compressor.kt` runs an `MediaExtractor` → decoder (Surface) → encoder + (`video/avc`) → `MP4Builder` transcode loop. +- The decoder is created from the **input** track's MIME. Some containers (notably + iPhone `.MOV`) report `video/dolby-vision`, which fails with `NAME_NOT_FOUND` + on devices lacking a Dolby Vision decoder. `CompressorUtils.ensureDecodableVideoFormat` + remaps such inputs to their backward-compatible HEVC/AVC base layer (profiles 8/4 + → HEVC, profile 9 → AVC) or throws a clear error for non-compatible profiles + (5/7). See #398. +- The encoder is intentionally `c2.android.avc.encoder` (when QTI codecs exist) or + `MediaCodec.createEncoderByType("video/avc")`; QTI AVC encoders can produce MP4s + that do not play on Mac/iPhone, so avoid switching this without testing. + +## Conventions + +- Keep changes surgical and aligned with surrounding style. Native helper objects + (e.g. `CompressorUtils`) use member imports and unqualified calls in + `Compressor.kt` — match that. +- When fixing an upstream issue, record it in `TRIAGE.md` (triage row + the + "Minor fixes made in this branch" list) referencing the issue number. +- Prefer graceful, descriptive failures over cryptic native crashes for + unsupported media (clear error messages that tell the user what happened). + +## Merging back upstream + +This fork accumulates many incremental commits. When contributing back to +`numandev1/react-native-compressor`, use a **squash merge** so the history lands +as a single, well-described commit rather than the full incremental series. diff --git a/TRIAGE.md b/TRIAGE.md index b45157c..8a8b013 100644 --- a/TRIAGE.md +++ b/TRIAGE.md @@ -13,6 +13,8 @@ Legend: | Issue | Triage | Notes | | --- | --- | --- | +| #400 | real, fixed here | iOS regression from #392: H.264 `videoOutputConfiguration` added `AVVideoExpectedSourceFrameRateKey` / `AVVideoAverageNonDroppableFrameRateKey`, which `canApply(...)` accepts but the iOS encoder silently drops the video track for, yielding an audio-only MP4 reported as success. This branch removes those keys and verifies the exported file actually contains a video track. | +| #398 | real, fixed here | Android could not compress Dolby Vision `.MOV` inputs (iPhone HDR): `MediaCodec.createDecoderByType("video/dolby-vision")` fails with `NAME_NOT_FOUND` on devices without a Dolby Vision decoder. This branch remaps the input to its backward-compatible HEVC/AVC base layer when possible, and otherwise fails with a clear, actionable error. | | #390 | not a bug | Reports `start` / `end` time behavior for video compression, but the current public video API does not expose trim parameters. | | #387 | needs info | Gradle binary store corruption looks environment-specific; report does not isolate a library code change. | | #384 | needs info | Performance question, not a reproducible defect report. | @@ -86,7 +88,9 @@ These should be closed upstream unless a current repro still exists on the lates - Android: clamp metadata parsing and reject invalid transcode output - Android: adaptive video compression profile for high-resolution inputs - Android: fast-start compressed MP4 outputs and skip unsupported copied audio sample metadata +- Android: decode the backward-compatible base layer of Dolby Vision `.MOV` inputs, or fail with a clear error (#398) - Android/iOS: clamp image and thumbnail JPEG quality values - Android/iOS: harden thumbnail frame extraction for difficult source videos - iOS: guard missing video tracks and use the same adaptive sizing/bitrate strategy +- iOS: drop unsupported H.264 frame-rate keys and verify the output has a video track to prevent silent audio-only results (#400) - iOS: return background-upload response bodies consistently diff --git a/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/compressor/Compressor.kt b/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/compressor/Compressor.kt index ecd5fa3..8b6827c 100644 --- a/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/compressor/Compressor.kt +++ b/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/compressor/Compressor.kt @@ -11,6 +11,7 @@ import android.net.Uri import android.os.Build import android.util.Log import com.reactnativecompressor.Video.VideoCompressor.CompressionProgressListener +import com.reactnativecompressor.Video.VideoCompressor.utils.CompressorUtils.ensureDecodableVideoFormat import com.reactnativecompressor.Video.VideoCompressor.utils.CompressorUtils.findTrack import com.reactnativecompressor.Video.VideoCompressor.utils.CompressorUtils.hasQTI import com.reactnativecompressor.Video.VideoCompressor.utils.CompressorUtils.prepareVideoHeight @@ -790,6 +791,10 @@ object Compressor { ): MediaCodec { val originalMime = inputFormat.getString(MediaFormat.KEY_MIME)!! + // Some inputs (e.g. iPhone .MOV files) report a "video/dolby-vision" MIME + // type that many devices cannot decode. Remap to a decodable base-layer + // codec, or fail with a clear error, before creating the decoder (#398). + ensureDecodableVideoFormat(inputFormat) // Dolby Vision (video/dolby-vision) has no standalone decoder on most Android // devices and throws NAME_NOT_FOUND. Profiles 8.1/8.4 carry an HEVC base layer // that the standard HEVC decoder can render, so we remap them to HEVC. Profile 5 diff --git a/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/utils/CompressorUtils.kt b/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/utils/CompressorUtils.kt index 801d4a6..b24b9cb 100644 --- a/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/utils/CompressorUtils.kt +++ b/android/src/main/java/com/reactnativecompressor/Video/VideoCompressor/utils/CompressorUtils.kt @@ -207,4 +207,76 @@ object CompressorUtils { } return false } + + // MIME type reported by MediaExtractor for Dolby Vision tracks (e.g. iPhone .MOV files) + private const val MIMETYPE_VIDEO_DOLBY_VISION = "video/dolby-vision" + + /** + * Check whether the device exposes a decoder for the given MIME type. + */ + private fun hasDecoderForMime(mime: String): Boolean { + val codecList = MediaCodecList(MediaCodecList.REGULAR_CODECS) + for (codec in codecList.codecInfos) { + if (codec.isEncoder) continue + for (type in codec.supportedTypes) { + if (type.equals(mime, ignoreCase = true)) return true + } + } + return false + } + + /** + * Resolve the backward-compatible base-layer MIME type for a Dolby Vision track. + * + * Dolby Vision profiles 8.x (DvheSt) and 4 (DvheDtr) carry an HEVC base layer, and + * profile 9 (DvavSe) carries an AVC base layer; these can be decoded by the standard + * HEVC/AVC decoders. Profiles 5 (DvheStn) and 7 (DvheDtb) have no usable single base + * layer, so they return null. When the profile is unknown we assume HEVC, which covers + * the common consumer case (e.g. iPhone records Dolby Vision profile 8). + */ + private fun dolbyVisionBaseLayerMime(inputFormat: MediaFormat): String? { + if (!inputFormat.containsKey(MediaFormat.KEY_PROFILE)) { + return MediaFormat.MIMETYPE_VIDEO_HEVC + } + return when (inputFormat.getInteger(MediaFormat.KEY_PROFILE)) { + MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvavSe -> MediaFormat.MIMETYPE_VIDEO_AVC + MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheSt, + MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheDtr -> MediaFormat.MIMETYPE_VIDEO_HEVC + else -> null + } + } + + /** + * Ensure the input video format can be decoded on this device. + * + * Some containers (notably iPhone `.MOV` files) expose the video track as + * `video/dolby-vision`. Many Android devices have no Dolby Vision decoder, so + * `MediaCodec.createDecoderByType("video/dolby-vision")` fails with NAME_NOT_FOUND. + * When the dedicated decoder is missing but the stream carries a backward-compatible + * base layer, this rewrites the format MIME to the base-layer codec so the standard + * HEVC/AVC decoder can decode it. If no compatible decoder exists, it throws a clear + * error instead of letting the cryptic native failure surface (see issue #398). + */ + fun ensureDecodableVideoFormat(inputFormat: MediaFormat) { + val mime = inputFormat.getString(MediaFormat.KEY_MIME) ?: return + if (hasDecoderForMime(mime)) return + + if (mime.equals(MIMETYPE_VIDEO_DOLBY_VISION, ignoreCase = true)) { + val fallbackMime = dolbyVisionBaseLayerMime(inputFormat) + if (fallbackMime != null && hasDecoderForMime(fallbackMime)) { + Log.w( + "Compressor", + "No Dolby Vision decoder on this device; decoding the $fallbackMime base layer instead." + ) + inputFormat.setString(MediaFormat.KEY_MIME, fallbackMime) + return + } + throw IllegalStateException( + "This video uses Dolby Vision, which is not supported by this device's decoders " + + "and has no backward-compatible base layer to fall back to." + ) + } + + throw IllegalStateException("No decoder available for video format: $mime") + } } diff --git a/ios/Video/VideoMain.swift b/ios/Video/VideoMain.swift index 3e29497..7d702f7 100644 --- a/ios/Video/VideoMain.swift +++ b/ios/Video/VideoMain.swift @@ -320,11 +320,15 @@ class VideoCompressor { exporter.outputURL = tmpURL exporter.outputFileType = AVFileType.mp4 + // NOTE: Do not add AVVideoExpectedSourceFrameRateKey or + // AVVideoAverageNonDroppableFrameRateKey here. They are not documented + // H.264 (avc1) compression properties on iOS and, while AVFoundation's + // `canApply(...)` check still returns true, the iOS encoder silently + // drops the video track, producing an audio-only MP4 that still reports + // success. See issue #400. let compressionDict: [String: Any] = [ AVVideoAverageBitRateKey: bitRate, AVVideoProfileLevelKey: AVVideoProfileLevelH264HighAutoLevel, - AVVideoExpectedSourceFrameRateKey: frameRate, - AVVideoAverageNonDroppableFrameRateKey: frameRate, ] exporter.optimizeForNetworkUse = true; exporter.videoOutputConfiguration = [ @@ -366,7 +370,16 @@ class VideoCompressor { switch result { case .success: if let outputURL = exporter.outputURL { - onCompletion(outputURL) + // Guard against the iOS encoder silently dropping the video track + // and producing an audio-only file that still reports success. + // See issue #400. + let outputAsset = AVAsset(url: outputURL) + if outputAsset.tracks(withMediaType: AVMediaType.video).isEmpty { + try? FileManager.default.removeItem(at: outputURL) + onFailure(CompressionError(message: "Compression produced a file with no video track")) + } else { + onCompletion(outputURL) + } } else { onFailure(CompressionError(message: "Compression succeeded but output URL is unavailable")) }