Skip to content

fix: pad mel with spec_min instead of 0.0 in acoustic collater#313

Open
KakaruHayate wants to merge 2 commits into
openvpi:mainfrom
KakaruHayate:fix/mel-pad-value
Open

fix: pad mel with spec_min instead of 0.0 in acoustic collater#313
KakaruHayate wants to merge 2 commits into
openvpi:mainfrom
KakaruHayate:fix/mel-pad-value

Conversation

@KakaruHayate

Copy link
Copy Markdown

Problem

The acoustic collater pads mel with raw log-mel 0.0, which is not a neutral value: norm_spec maps it to +1.0 — the top of the normalized range, i.e. maximum loudness. Every batch fills the padded tail of shorter samples with full-loudness garbage.

Two consequences:

  1. Backbone receptive-field leakage. The diffusion backbones (WaveNet / LYNXNet / LYNXNet2) receive no padding mask. LYNXNet2's receptive field is ~181 frames (kernel_size=31 × 6 layers), so the fake full-loudness signal leaks into the trailing valid frames of every non-longest sample. The loss mask (mel2ph > 0) only hides padding frames themselves — the contaminated valid frames near the boundary are fully counted. This is a systematic bias on utterance tails, exactly where breathy endings and vibrato decay live.

  2. Aux decoder trained on garbage. The aux decoder (shallow diffusion) loss is not masked at all: with zero-padding it was actively trained to predict maximum loudness on padding frames from near-zero condition.

Measured on an untrained LYNXNet2 (1024ch × 6L): valid-frame output shift up to ~1.3% within the receptive field of the padding boundary, zero beyond it.

Fix

Pad with spec_min (-12 by default) instead. It maps to -1.0 (silence) after normalization and sits next to the mel extractor's true silence floor log(1e-5) = -11.51, so padded regions now look like ordinary trailing silence — consistent with what the model sees at inference.

One-line behavioral change + comment; falls back to -12.0 if spec_min is absent from config.

Compatibility

  • Checkpoint-compatible: no parameter/shape changes.
  • Training data distribution changes slightly (padded tails now silence instead of max loudness). Models trained before/after differ only in padded-region behavior; fine-tuning across this change is safe.
  • Inference is unaffected (no padding at inference).

KakaruHayate and others added 2 commits July 15, 2026 14:42
Raw log-mel 0.0 is not a neutral padding value: norm_spec maps it to
+1.0 — the very top of the normalized range, i.e. maximum loudness.
Every batch therefore filled the padded tail of shorter samples with
full-loudness garbage. Two consequences:

1. The diffusion backbones (WaveNet / LYNXNet / LYNXNet2) receive no
   padding mask, so their receptive field (~181 frames for LYNXNet2
   with kernel_size=31 x 6 layers) leaks the fake signal into the
   trailing valid frames. The loss mask (mel2ph > 0) hides this from
   the loss on padding frames, but the contaminated valid frames near
   the boundary are fully counted — a systematic bias on utterance
   tails, exactly where breathy endings and vibrato decay live.

2. The aux decoder loss is not masked at all: with zero-padding the
   aux decoder was actively trained to predict maximum loudness on
   padding frames from near-zero condition.

Padding with spec_min (-12 by default) maps to -1.0 (silence) and sits
next to the mel extractor's true silence floor log(1e-5) = -11.51, so
padded regions now look like ordinary trailing silence — consistent
with what the model sees at inference time.

Note: this changes the training data distribution slightly; models
trained before/after this fix are checkpoint-compatible but their
padded-region behavior differs.
Replace zero-padding with spec_min for mel padding to avoid full-loudness garbage in shorter samples.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant