feat(cosyvoice3): add native reference-voice synthesis

[ErliCai]

Background

Refs #1263.

Add native TensorRT reference-voice synthesis for Fun-CosyVoice3-0.5B-2512. This advances the issue's initial component-packaging proposal: the standard build entry point now builds the complete bundle, and callers supply reference audio per request.

Exit Criteria

• Resolve the checkpoint to the CosyVoice3 family and build all six learned components through the standard build API.
• Generate 24 kHz mono audio through the native CLI and public Task interfaces, with request-time reference conditioning.
• Keep model implementation, dependencies, and correctness tests family-owned.
• Supported scope: offline, single-GPU, batch-one FP32 synthesis. Streaming, quantization, control/phoneme tags, and formal perceptual-quality qualification are not claimed.

Implementation

• Add family-owned checkpoint mapping, CAMPPlus, speech tokenizer, LLM, conditioning, Flow, HiFT, bundle construction, and native orchestration.
• Add an optional model-agnostic IReferenceAudioGeneration interface and CLI --reference-audio / --reference-text inputs. Existing IAudioGeneration virtual methods remain unchanged.
• Append BOOL tensor support for attention masks without renumbering existing dtype values.
• Execute learned runtime stages in TensorRT; native DSP handles reference preprocessing. No Python or ONNX Runtime learned fallback is used by the native runtime.
• Remove the obsolete fixed-voice packager and schema-1 loader. Native bundles require per-request reference audio; old development bundles must be rebuilt.
• No sibling-family edits, root dependency changes, private recordings, weights, generated plans, local notes, or standalone diagnostic/validation tools are included.

Change categories

• Model or runtime behavior
• Public API
• ABI
• Bundle or artifact format
• Dependencies
• Documentation only
• CI or developer tooling

Validation

Commands and Results

Latest cleanup: 0d65f7c16cc449740e5553c7877b51b7913611f6. Paths below use $BUILD_DIR for the local native build directory.

• cmake --build "$BUILD_DIR" --target trtmc trtmc_backend_trt test_cosyvoice3_runtime test_cli -j 2: passed. Native sources were synchronized to the feature worktree used by the existing build directory.
• ctest --test-dir "$BUILD_DIR" -R cosyvoice3 --output-on-failure: 1 passed; request-time frontend/orchestration, repeat-request state, error handling, and engine lifetime checks.
• ctest --test-dir "$BUILD_DIR" -R cli --output-on-failure: 1 passed.
• python -m pytest families/cosyvoice3/tests/test_bundle.py -k fixed_voice -q: 1 passed, 1 deselected.
• ruff check families/cosyvoice3, changed C++ formatting, git diff --check github/main...HEAD, python tools/legal_headers.py --check, python -m tools.model_ci validate, and python tools/test_impact.py --validate: passed; legal headers reported zero findings.

Earlier evidence, not a fresh full-suite result for this head:

• At edaabeca, python -m pytest families/cosyvoice3/tests -k 'not test_cosyvoice3_build_to_reference_audio' -q -ra --e2e-model=cosyvoice3 with checkpoint, native, official-reference, and GPU inputs configured: 182 passed, 1 deselected.
• Before PR cleanup, python -m pytest families/cosyvoice3/tests/test_e2e.py --e2e-model=cosyvoice3: 2 passed, including a fresh default-profile checkpoint build and native reference-audio synthesis. This establishes execution, not perceptual quality.

Hardware, Environment, and Revisions

• Base: 31701eb142b1cd6e47dc04ff80a3c1ecb13051b3; head: 0d65f7c16cc449740e5553c7877b51b7913611f6.
• WSL Ubuntu 24.04, Python 3.12, NVIDIA RTX 4070 Laptop GPU (8 GB), CUDA toolkit 13.2, TensorRT 11.1.0.106, FP32.
• Checkpoint: FunAudioLLM/Fun-CosyVoice3-0.5B-2512 at 29e01c4e8d000f4bcd70751be16fa94bf3d85a18.
• Official reference source: FunAudioLLM/CosyVoice at 074ca6dc9e80a2f424f1f74b48bdd7d3fea531cc; Transformers 5.2.0 and x-transformers 2.11.24 used for reference validation.

Not Run / Remaining Gaps

• Full HiFT waveform parity, ASR/content accuracy, listening acceptance, and voice-similarity qualification are not claimed.
• Other hardware, precisions, larger profiles, and shared-contract consumers beyond the recorded checks are not newly qualified.
• Public PR CI and protected premerge results are pending/not run. This is a draft, not a merge-ready or fully qualified support claim.

Contributor Self-Review

• I have completed a self-review of this change.

Notes For Future Readers

Review the public reference-audio contract and CLI first, then family build/runtime and tests. The shared edits express reference-audio inputs and BOOL tensors; model semantics remain in the family. All introduced commits carry author-matching DCO sign-offs.

Checkpoint provenance and license: pinned model card. Reference equations follow the pinned Apache-2.0 upstream source. Model weights and audio are not redistributed in this PR.

Risk level

• Low
• Medium
• High

New multi-engine model integration and additive public/native contracts warrant maintainer review and exact-head CI. Local synthesis and component checks do not establish general audio-quality acceptance.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}