Skip to content

perf(whisper): remove redundant timestamp normalization - #2

Open
unohee wants to merge 1 commit into
feat/kt-641-exact-ceilingfrom
feat/kt-647-timestamp-reduction
Open

perf(whisper): remove redundant timestamp normalization#2
unohee wants to merge 1 commit into
feat/kt-641-exact-ceilingfrom
feat/kt-647-timestamp-reduction

Conversation

@unohee

@unohee unohee commented Jul 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • remove the full-vocabulary normalization that algebraically cancels when comparing aggregate timestamp logits against the best text logit
  • preserve all timestamp masks, legacy lower-bound semantics, token selection, word timestamps, and full JSON output

Exact-output benchmarks

Production MLX 0.31.2, whisper-large-v3-turbo, fixed seed, word timestamps enabled:

  • 4-minute remote-commit run: 5.8807s
  • 4-minute full JSON SHA-256: 4d966dbd85fe2a2405098dcf8f51d1daadea075a0a9f61c7fb6210697b7fd6ae
  • 71-minute baseline: 112.4515s
  • 71-minute candidate monkeypatch: 111.2430s
  • reduction: 1.2085s / 1.07%
  • 71-minute remote-commit run: 110.4300s
  • 1,735 segments / 7,085 words / 25,984 chars
  • 71-minute full JSON SHA-256: ab0a926a8e18d32458329a5129dc8c3da6f28c2e5edc605a2d938de9c7113d52

Review

OpenSwarm Codex was invoked against the installed package source, but it could not produce a verdict because the connector-created branch was not a local Git worktree (No working-tree changes to review). Exact 4-minute and 71-minute remote-commit golden runs are retained as the deterministic gate.

Tracking


Note

Low Risk
Single localized algebraic equivalence in timestamp logit filtering; behavior is gated by exact-output golden hashes in the PR description.

Overview
ApplyTimestampRules no longer runs a full-vocabulary logsumexp and per-token log-prob subtraction on every decode step when deciding whether aggregate timestamp mass beats the best text token.

That decision now compares logsumexp over timestamp logits directly to the max text-token logit, relying on the fact that the shared softmax normalization constant cancels in the inequality. All other timestamp masking (pairs, initial timestamp cap, legacy lower bound, etc.) is unchanged.

This is a hot-path perf tweak inside the autoregressive loop; PR benchmarks report ~1% faster long runs with identical full JSON hashes.

Reviewed by Cursor Bugbot for commit 99e22af. Bugbot is set up for automated code reviews on this repo. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant