Skip to content

fix(whisper): match reference mel preprocessing - #1282

Draft
WenJing95 wants to merge 1 commit into
NVIDIA:mainfrom
WenJing95:fix-whisper-mel-boundaries
Draft

fix(whisper): match reference mel preprocessing#1282
WenJing95 wants to merge 1 commit into
NVIDIA:mainfrom
WenJing95:fix-whisper-mel-boundaries

Conversation

@WenJing95

Copy link
Copy Markdown
Contributor

Background

Nonzero audio edges produced different Whisper mel features because STFT used zero padding. The discarded final frame could also change the log floor.

Exit Criteria

Match reference features at chunk boundaries.

Implementation

Reflect both ends of the waveform and remove the last frame before normalization.

Change categories

  • Model or runtime behavior

Validation

Commands and Results

Native reference goldens passed at 80/128 mels; source-quality: 164 passed.

g++ -std=c++17 -O2 -Wall -Wextra -Wpedantic -I. families/whisper/tests/cpp/test_whisper_mel_spectrogram.cpp families/whisper/runtime/whisper_mel_spectrogram.cpp -o /tmp/trtmc-four-whisper-test
/tmp/trtmc-four-whisper-test
python -m tools.community_ci source-quality --base 714f1fc0d567213a7b79a488e9dcfdf405279c6a

Hardware, Environment, and Revisions

8a95b0e: Ubuntu CPU, GCC 13.3; Transformers 5.2.0 reference.

Not Run / Remaining Gaps

TensorRT inference and full CMake suite: TensorRT environment unavailable.

Contributor Self-Review

  • I have completed a self-review of this change.

Notes For Future Readers

Reflect after padding/truncating to 30 seconds so short clips keep their zero-padded tail.

Risk level

  • Low

Changes are confined to Whisper's center padding and log normalization.

Signed-off-by: wj <126436871+WenJing95@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant