Skip to content

Latest commit

 

History

5 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

OpenTranscode

Open-source batch video transcoder for Linux. Encodes folders of video files to AV1 / VP9 / HEVC / Theora with configurable audio codecs, resolution scaling, and source-file management. Built on ffmpeg (default) with an optional av1an chunk-parallel path for users with a working VapourSynth setup. NVIDIA GPU (NVENC) encoding is used automatically when available.

The application is a single self-contained file — open-transcode.py. Everything (codec profiles, encoder worker, environment probe, UI, rebuild-from-git builder) is inlined in it; run it with nothing but Python + PySide6 on the box. The source file is the source of truth; this README describes it.

  • Default encoder: ffmpeg + libsvtav1 (reliable across distros)
  • GPU encoder: NVENC (hevc_nvenc / av1_nvenc) auto-detected with a live encode test — used by the Auto engine when it actually works
  • Optional encoder: av1an chunk-parallel (opt-in via UI toggle)
  • Self-sufficient rebuild: the always-enabled REBUILD FROM GIT button installs its own build dependencies (distro-aware), then builds VapourSynth + BestSource + av1an into ~/.local / ~/.cargo
  • Codecs: AV1 (SVT-AV1), VP9, x265 (HEVC), Theora (OGV) video; Opus, Vorbis, FLAC, IAMF audio
  • Containers: MKV, WebM, MP4, OGV (Ogg)
  • Title scrubbing: output names drop the source release's stale codec/container tags (Movie.x264.1080p.x265-GRP → Movie.1080p. WEBRip.GRP_archived.mkv) — a title that still says x264/webm on an AV1/Opus/MKV output misdescribes the archive
  • Resolution: Original or scaled (16:9, 21:9, 32:9 presets from 480p to 4K)
  • Skip-existing: Probes output with ffprobe; skips files whose codec matches
  • Audio normalization: Per-file loudness analysis with volume gain
  • Subtitle mux: Optional English subtitle passthrough
  • Source management: Optional verified-source deletion after encode

Requirements

  • Linux (POSIX)
  • Python ≥ 3.12
  • ffmpeg (with libsvtav1, libvpx, libx265, libopus, libvorbis, flac; libtheora only for the OGV output target)
  • ffprobe
  • PySide6 (for the GUI)
  • Optional: NVIDIA GPU + driver for NVENC hardware encoding
  • Optional: av1an + VapourSynth (only if using the av1an toggle)

Quick start

python open-transcode.py             # launches the GUI — that's it

CLI entry points (--version, --dry-run, --verify-only, flag overrides) are library access on the inlined main() / build_parser() — the script itself opens the GUI directly and ignores arguments.

Usage

Encode engine (GPU vs CPU)

The ENGINE selector (UI) picks the video encoder:

  • Auto (default) — uses the NVENC hardware encoder for the selected codec family when the environment probe's live encode test proved it works (x265 → hevc_nvenc, AV1 → av1_nvenc on RTX 40+); otherwise the software encoders. VP9 has no NVENC encoder and always uses the CPU.
  • GPU — force NVENC; falls back to CPU with a log line when unavailable. GPU encodes run via single-pass ffmpeg (-c:v hevc_nvenc -rc vbr -cq N), with CPU decode. av1an chunk-parallel is not used — one NVENC process outruns chunk-parallel CPU workers.
  • CPU — force the software encoders (SVT-AV1 / VP9 / x265).
  • Hybrid (v4.7.0) — run the GPU lane AND a CPU lane concurrently: the queue is scanned up front and split by file size (LPT, balanced with a GPU:CPU speed ratio) so both lanes finish at about the same time. The CPU lane's thread budget is reduced by a small reserve for the GPU lane's decode/mux, and with --use-av1an the CPU lane uses chunk-parallel — so NVENC + chunk workers + software encoders can all be busy at once. Lanes split FILES, never one file across encoders (mixing hevc_nvenc and libx265 chunks inside a single file would make scene-by-scene quality visibly inconsistent, and av1an cannot drive NVENC anyway). Needs 2+ encodable files and a functional GPU encoder; otherwise it falls back to a single CPU queue. GPU-lane files get NVENC quality, CPU-lane files get software-encoder quality.

The GPU gate is a real encode smoke test, not just an encoder-list grep: a ffmpeg build can list hevc_nvenc while the installed NVIDIA driver is too old for the NVENC API version it was compiled against (e.g. "Driver does not support the required nvenc API version. Required: 13.1 Found: 13.0") — in that case the probe reports the exact driver fix and encoding stays on the CPU. The startup log prints the GPU verdict.

When NVENC is "present but NOT usable" (driver / ffmpeg API mismatch)

ffmpeg is compiled against a specific NVENC API version and the NVIDIA driver must be at least that new. On legacy driver branches (580 is the last branch supporting Pascal cards such as the GTX 10xx) upgrading the driver is not an option — so invert the fix: build ffmpeg against the NVENC API your driver does provide and install it to ~/.local/bin (which PATHs before the distro binary; OpenTranscode picks it up automatically on the next launch). Use the included helper:

# "Found: 13.0" in the error → pass 13.0
./scripts/build-ffmpeg-nvenc-matched.sh 13.0

The script copies your distro ffmpeg's feature set (same version, same libraries — a drop-in), builds against the matching nv-codec-headers, and installs only into $HOME/.local. Nothing system-wide is touched.

Default (ffmpeg-only, recommended)

opentranscode
  • Encodes with ffmpeg -c:v libsvtav1 (or libvpx-vp9 / libx265 based on selection)
  • Single-pass per file
  • Reliable across distros; no VapourSynth dependency

Skip-existing (default ON)

Files whose output already exists with a matching video+audio codec are skipped. Detection uses ffprobe — verifies codec_name for both video and audio streams, plus resolution when scaling is requested.

# flag handling lives in the inlined build_parser()/main()
# (library access); the GUI reflects the same options

Output filenames (title scrubbing)

Output names are built from the scrubbed source title plus this app's own markers: <clean-title>[_<w>x<h>]_archived.<ext>. The scrub (opentranscode/title_clean.py) removes:

  • Codec tags — video (x264, h.265, hevc, xvid, divx, theora, av1, …), audio (ac3, dts, aac, flac, …) and container (avi, webm, mkv, …) tokens, matched on token boundaries so names like MP4Box or Aviator pass through
  • This app's earlier suffixes — _archived and _<w>x<h>, so re-encoding never stacks markers
  • Separator residue — doubled dots/dashes left by removals

A stem that scrubs to nothing (x264.mkv) keeps its original name. The resolution suffix is added only when a target resolution is selected. Verbosity (--verbose) logs every title rewrite.

Verbose logging

Default log output is minimal — two lines per file (start + finish):

Found 180 file(s) to process.
[1/180] filename.mkv
[1/180] filename.mkv — OK: 1.6MB -> 1.3MB (81%)
[2/180] already_done.mkv — SKIP (already av1/opus)
[3/180] next.mkv
[3/180] next.mkv — OK: 2.4MB -> 1.8MB (75%)
QUEUE COMPLETE. Success: 178, Failed: 0, Skipped: 2.

For diagnostics (CMD lines, live tail of ffmpeg/av1an stderr, disk-space warnings, heartbeats):

Verbose detail is also available via the inlined CLI parser.

Massive-file support

For 30GB+ BluRay rips:

  • 24-hour per-file timeout (configurable via --timeout SECONDS)
  • Inline scaling on the ffmpeg path — the ffmpeg (and GPU) path scales with -vf directly in the encode command; no intermediate file, no extra encode pass, no pre-scale timeout. The av1an path still writes a CRF-16 (visually lossless) intermediate for VapourSynth compatibility, but under the full per-file timeout instead of a flat 30-minute cap.
  • 1KB absolute integrity minimum (no false "output too small" failures on high-bitrate sources — duration check is the real gate)
  • Disk-space warnings — severe warnings (free space below the source size on the output or temp partition) are user-facing even in quiet mode

The per-file timeout is configurable (24h default).

GPU capability profiles

The GPU dropdown (next to ENGINE) lists combined card generations — same silicon, same encoding behaviour — rather than individual SKUs: NVIDIA Kepler through Ada/Blackwell, Intel Arc + integrated (QSV), and AMD GCN/RDNA (VAAPI), including data-center and crypto-era oddballs (Tesla P40/T4, CMP 30–90HX, and the GA100-based CMP 170HX + V100/A100/ H100 boards, which ship without NVENC and route to the CPU path). Auto-detect matches your card and the live encode probe decides what works; forcing a profile extends the REBUILD FROM GIT dependency tree with that GPU's packages.

Rebuilding the encode stack from git

The REBUILD FROM GIT button is available at all times — even on a bare system with no build tools installed:

  1. Dependency tree — detects missing tools/libraries and installs them via the distro package manager (pacman/apt-get/dnf/zypper; one privilege prompt). Includes zimg, which VapourSynth requires.
  2. VapourSynth from git → ~/.local (self-contained: module, libs, headers in the python user site-packages).
  3. BestSource plugin from git → compiled against that fresh VS, so av1an gets a fast, frame-accurate chunk method (no more slow select).
  4. av1an from git → ~/.cargo/bin.

Restart the app afterwards; the probe picks up the new stack automatically. ffmpeg + IAMF builds libiamf + ffmpeg into ~/.local/bin for the IAMF audio codec.

av1an chunk-parallel (opt-in)

For users with a working VapourSynth + source plugin (lsmash, ffms2, bestsource) setup who want scene-detection-based chunk-parallel encoding:

  • UI: Check the "av1an (chunk-parallel)" checkbox

Known chunking failure modes (all handled with per-file retries + a diagnosis block, and safe to hit):

  • ffmpeg ≥ 7 removed -vsync — av1an's segment/hybrid chunk extraction calls ffmpeg -vsync and dies with "Unrecognized option 'vsync'" on modern ffmpeg. Without VS source plugins the app forces --chunk-method select (works on any ffmpeg — verified — but is slow, so the default ffmpeg-only path is usually faster for long files). Install bestsource/ffms2/lsmash to give av1an a fast plugin chunk method.
  • FRAME MISMATCH / "encoder crashed: exit status: 0" — chunk extraction drift on sources with sparse keyframes; the app retries the file with select automatically.
  • y4m pipe breaks ("Failed to read y4m frame delimiter") — same select retry.

When av1an fails per-file, the code automatically falls back to ffmpeg for that file. When av1an fails systematically (VSScript API mismatch, missing encoder), the queue aborts with an actionable diagnostic.

CLI reference (library access — the script opens the GUI directly)

opentranscode [--version] [--dry-run] [--verify-only PATH] [--force]
              [--engine {auto,gpu,cpu,hybrid}]
              [--chunk-method METHOD] [--max-workers N] [--threads-per-worker N]
              [--use-av1an] [--verbose] [--skip-existing | --force-reencode]
              [--timeout SECONDS]
Flag Default Description
--version — Print version and exit
--dry-run — Probe environment + smoke test, no encode
--verify-only PATH — Re-verify an existing output file
--force off Skip ffprobe pre-validation
--engine auto auto = NVENC GPU encoder when the live encode test passes, else CPU; gpu = force NVENC; cpu = force software encoders
--chunk-method METHOD auto Force av1an chunk method (select, hybrid, ffms2, lsmash, bestsource, dgdecnv)
--max-workers N auto Cap chunk-parallel worker count
--threads-per-worker N auto Per-encoder thread cap
--use-av1an off Use av1an chunk-parallel (UI toggle also available)
--verbose off Full tech-detail log output
--skip-existing on Skip files whose output has matching codec
--force-reencode off Re-encode everything
--timeout SECONDS 86400 Per-file encode timeout (24h default)

Architecture

open-transcode.py        # THE application — single self-contained file
                         #   (title scrubbing, codec profiles, Theora q-map,
                         #    encoder worker, env probe, UI, rebuild builder)
pyproject.toml           # Project metadata + tool config (nothing to install)
pytest.ini               # Test suite configuration
tests/                   # 300+ tests; conftest.py loads the single file
scripts/                 # Helper: matched-NVENC ffmpeg build script

Encode pipeline

  1. Environment probe — detects distro, ffmpeg/ffprobe/av1an paths, encoder library availability, VapourSynth + source plugins, CPU topology
  2. Pre-flight validation — ffprobe scans all input files; reports valid vs invalid counts before encoding starts
  3. Per-file pipeline:
    • _validate_file — ffprobe pre-check (skip if invalid)
    • _check_disk_space — warn (verbose) if free space < source size
    • _output_already_encoded — skip if output exists with matching codec
    • _prepare_input — pre-scale to a CRF-16 intermediate (av1an path only) or symlink to temp; the ffmpeg/GPU path scales inline
    • _encode_one — dispatch to ffmpeg (default) or av1an (opt-in)
    • _run_with_stop_check — subprocess with STOP-button interrupt support
    • _verify_and_finalize — duration check (≥95%), subtitle mux, source deletion
  4. Final cleanup — sweep per-worker temp dir, delete verified sources

Thread safety

  • Each EncoderWorker runs in its own QThread
  • Per-worker temp subdirectory (~/.cache/OpenTranscode/tmp/worker-<pid>/) created with mode 0700 (SEI CERT FIO09-C)
  • Process-group signaling (start_new_session=True + os.killpg) reaches av1an's child encoders (SvtAv1EncApp / vpxenc / x265)
  • Drainer threads read stdout/stderr continuously to prevent pipe-buffer deadlock (same pattern as subprocess.run._communicate)

Coding standards

The codebase follows:

  • PEP 868 — parameterized type hints (dict[str, object], not Dict[str, object])
  • SEI CERT — MSC04-C (single source of truth for diagnostics), FIO09-C (secure temp directory), ERR01-C (narrow exception scope), STR09-C (no substring matches for encoder names)
  • POSIX — start_new_session=True for process-group signaling, signal.SIGTERM → SIGKILL escalation, os.killpg for child cleanup
  • MISRA (where applicable to Python) — single exit point per function where practical, no early returns from try blocks without cleanup
  • Unix philosophy — decision forks step down (guard, guard, main path) and each module does one thing (title_clean.py scrubs titles, codec_profiles.py holds the profile tables); preference for table-driven dispatch over nested if ladders and comprehensions over accumulation loops where side effects allow
  • Comment voice — comments state decisions, not history; wording like "restored" or "brought back" does not appear in code or docs

Testing

python -m pytest tests/ -q          # 315 tests, ~50s
python -m pytest tests/ -v          # verbose
python -m pytest tests/ -k "skip_existing"  # subset

Test categories:

  • Smoke tests — av1an VSScript compatibility probe
  • Encoder pipeline — _validate_file, _prepare_input, _verify_and_finalize
  • Real-encode e2e — generates real test videos with ffmpeg, runs the full pipeline
  • Chunk-method retry — y4m pipe break recovery
  • Stop button — SIGTERM/SIGKILL on process group
  • Concurrent workers — per-PID temp directory isolation
  • Skip-existing — codec matching, ffprobe failure, resolution mismatch
  • Theora/OGV — inverted quality-scale wrapper, container pairing rules
  • Title scrubbing — codec-token removal, residue cleanup, passthrough
  • Massive files — timeout flag, 1KB integrity threshold, disk-space checks
  • Package structure — public API surface, submodule imports, CLI parser

License

AGPL-3.0-or-later. See LICENSE.

Third-party tools invoked (not bundled): ffmpeg, ffprobe, av1an, VapourSynth, SvtAv1EncApp, vpxenc, x265, mkvmerge. Licenses flow through from upstream.

Project

Releases

Packages

Contributors

Languages