feat(nabrah): add Nabrah STT plugin for LiveKit Agents with installation and usage instructions - #6873
feat(nabrah): add Nabrah STT plugin for LiveKit Agents with installation and usage instructions#6873MagdiWaleed wants to merge 3 commits into
Conversation
…ion and usage instructions
|
|
- add py.typed marker so mypy can check the package - give _recognize_impl the base-class signature - narrow self._stt and use LanguageCode for SpeechData - guard NotGivenOr word timestamps before segment bounds - register the plugin in the uv workspace sources and the livekit-agents extras, align version with the repo baseline - flesh out package metadata, run ruff format
…word tracking in SpeechStream
0964d9a to
d0f1f3c
Compare
| self._utt_flushed_clean = _normalize_whitespace( | ||
| self._utt_flushed_clean + " " + self._utt_clean, | ||
| ) |
There was a problem hiding this comment.
🟡 Already-finalized speech can be repeated in a later transcript within the same turn
Text already sent as final is recorded with an extra space inserted before punctuation (_utt_flushed_clean + " " + self._utt_clean at livekit-plugins/livekit-plugins-nabrah/livekit/plugins/nabrah/stt.py:422-424), so the bookkeeping no longer matches what the recognizer sends and the same speech is emitted again.
Impact: A user's earlier sentence can be duplicated in the next transcript of the same turn, which is then fed to the LLM.
Prefix-matching mismatch between `_flush_eos` accumulation and the recognizer's cumulative text
_process_message computes the not-yet-finalized part of the current utterance by stripping the already-flushed prefix: clean_now[len(self._utt_flushed_clean):] if self._utt_flushed_clean and clean_now.startswith(self._utt_flushed_clean) (livekit-plugins/livekit-plugins-nabrah/livekit/plugins/nabrah/stt.py:523-527). That prefix is rebuilt in _flush_eos by joining pieces with a literal space and normalizing whitespace, unlike _current_text which deliberately omits the space when the next piece starts with punctuation (livekit-plugins/livekit-plugins-nabrah/livekit/plugins/nabrah/stt.py:367-370).
Concrete sequence inside one recognizer utterance (no is_final in between, which the module docstring says is routine):
- message:
"مرحبا <eot>"→_utt_clean = "مرحبا"; EOT confirm fires → flush;_utt_flushed_clean = "مرحبا". - message:
"مرحبا. كيف"→clean_now.startswith("مرحبا")→_utt_clean = ". كيف". - second EOT → flush →
_utt_flushed_clean = normalize("مرحبا" + " " + ". كيف") = "مرحبا . كيف". - next message
"مرحبا. كيف حالك"does not start with"مرحبا . كيف", so_utt_cleanfalls back to the wholeclean_now, and the following FINAL_TRANSCRIPT repeats text already emitted as final.
The same unconditional " " join is used when rolling an utterance into the turn (livekit-plugins/livekit-plugins-nabrah/livekit/plugins/nabrah/stt.py:500-502), which additionally inserts a stray space before punctuation in emitted transcripts.
Prompt for agents
In livekit-plugins/livekit-plugins-nabrah/livekit/plugins/nabrah/stt.py, `_flush_eos` rebuilds the already-emitted prefix of the current utterance as `_normalize_whitespace(self._utt_flushed_clean + " " + self._utt_clean)`, and `_process_message` later strips that prefix from the recognizer's cumulative normalized text via `clean_now.startswith(self._utt_flushed_clean)`. Because the recognizer's own text has no space before punctuation ("مرحبا. كيف") while the reconstruction always inserts one ("مرحبا . كيف"), the prefix check can fail after a mid-utterance flush whose following diff starts with punctuation; `_utt_clean` then falls back to the entire utterance and already-finalized text is emitted again in the next FINAL_TRANSCRIPT. Consider tracking the flushed prefix by character offset into the normalized cumulative utterance text (e.g. store `len(clean_now)` consumed at flush time) instead of re-concatenating strings, or reuse the same punctuation-aware separator logic used in `_current_text` for both this join and the `_turn_text` join in the `not continues` branch.
Was this helpful? React with 👍 or 👎 to provide feedback.
No description provided.