Add Gandr TTS plugin - #2270
Conversation
Adds @livekit/agents-plugin-gandr, a TTS provider plugin for Gandr, mirroring the shape of the python plugin (livekit/agents PR #6814). - TTS extends the agents tts.TTS base, ChunkedStream extends tts.ChunkedStream - Gandr's /v1/audio/speech OpenAI-compatible endpoint, response_format wav/pcm (mp3 returns a deliberate 400, the door has no mp3 encoder) - 11 OpenAI voice aliases map onto stock gandr-* voices; gandr-* ids pass through - speed clamped 0.6-1.5; free key 100,000 tokens at gandr.ai - unit tests + a live-gated harness
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| } catch (error) { | ||
| throw new APIConnectionError({ | ||
| message: `Gandr request failed: ${String(error)}`, | ||
| }); | ||
| } |
There was a problem hiding this comment.
🔴 Cancelled speech is reported as a connection failure and the request is retried
An interrupted speech request is turned into a generic connection failure (new APIConnectionError at plugins/gandr/src/tts.ts:153-155) instead of being recognised as a cancellation, so cancelling playback produces error events and repeated pointless requests.
Impact: Every user interruption surfaces a spurious TTS error and burns extra retry attempts against the provider.
Abort handling masked by the inner catch around fetch
When this.abortSignal fires, fetch rejects with a DOMException named AbortError. The inner try/catch at plugins/gandr/src/tts.ts:152-156 immediately re-wraps any fetch rejection into APIConnectionError, so the outer handler's error.name === 'AbortError' check at plugins/gandr/src/tts.ts:203-205 never matches. APIConnectionError is retryable by default (agents/src/_exceptions.ts), so the base ChunkedStream retry loop (agents/src/tts/tts.ts:661-711) re-invokes run() with an already-aborted signal, failing instantly each time, and finally emits a non-recoverable tts_error and throws. The reference plugins (e.g. plugins/openai/src/tts.ts:150-154) let the AbortError reach the outer check untouched.
| } catch (error) { | |
| throw new APIConnectionError({ | |
| message: `Gandr request failed: ${String(error)}`, | |
| }); | |
| } | |
| } catch (error) { | |
| if (error instanceof Error && error.name === 'AbortError') { | |
| throw error; | |
| } | |
| throw new APIConnectionError({ | |
| message: `Gandr request failed: ${String(error)}`, | |
| }); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
| const { done, value } = await reader.read(); | ||
| if (done) break; | ||
|
|
||
| const frames = audioByteStream.write(value.buffer); |
There was a problem hiding this comment.
🔴 Spoken audio can come out as noise because unrelated memory is fed into the player
Each received piece of audio is handed to the player as its whole underlying memory block (audioByteStream.write(value.buffer) at plugins/gandr/src/tts.ts:194) rather than just the received bytes, so unrelated leftover bytes get played as audio.
Impact: Generated speech can be garbled with bursts of noise and wrong timing whenever the network delivers data in pooled buffers.
Uint8Array view vs. underlying ArrayBuffer
reader.read() yields a Uint8Array that, under Node's undici fetch, is frequently a view into a larger pooled socket buffer (non-zero byteOffset, buffer.byteLength much larger than value.byteLength). AudioByteStream.write (agents/src/audio.ts:52-56) already handles ArrayBufferView correctly by honouring byteOffset/byteLength, but passing .buffer discards that information and injects the entire pool contents (including other, unrelated data) into the PCM stream. All other plugins pass the view directly, e.g. plugins/rime/src/tts.ts:343 uses audioByteStream.write(value).
| const frames = audioByteStream.write(value.buffer); | |
| const frames = audioByteStream.write(value); |
Was this helpful? React with 👍 or 👎 to provide feedback.
|
|
||
| sendLastFrame(true); |
There was a problem hiding this comment.
🟡 The tail of every spoken response is silently dropped
The leftover audio still held in the buffer is never released before finishing (no flush before sendLastFrame(true) at plugins/gandr/src/tts.ts:201), so up to the last 100 ms of each utterance is discarded.
Impact: Sentences can end abruptly with the final syllable clipped off.
Missing AudioByteStream.flush()
AudioByteStream only emits complete 100 ms frames from write() (agents/src/audio.ts:52-72); any remainder stays in its internal buffer until flush() is called. Since the response byte length is almost never an exact multiple of 4800 bytes (24 kHz mono, 100 ms), the trailing partial frame is lost. Comparable plugins flush before marking the final frame, e.g. plugins/rime/src/tts.ts:349-355.
| sendLastFrame(true); | |
| for (const frame of audioByteStream.flush()) { | |
| if (frame.samplesPerChannel === 0) continue; | |
| sendLastFrame(false); | |
| lastFrame = frame; | |
| } | |
| sendLastFrame(true); |
Was this helpful? React with 👍 or 👎 to provide feedback.
| # @livekit/agents-plugin-gandr | ||
|
|
||
| ## 1.6.2 | ||
|
|
||
| ### Patch Changes | ||
|
|
||
| - Initial release. Gandr TTS for LiveKit Node Agents: speaks the OpenAI-compatible | ||
| `POST /v1/audio/speech` endpoint at `https://tts.gandr.ai/v1`, maps | ||
| `voice`/`response_format`/`speed`, and raises the framework `APIError` | ||
| subclasses on failure. |
There was a problem hiding this comment.
🟡 Release notes file was written by hand instead of leaving it to the release tooling
A hand-authored release-notes file with a pinned version is added (plugins/gandr/CHANGELOG.md:1-10), which the project's contribution rules explicitly forbid because a bot generates it.
Impact: Release automation may produce conflicting or duplicated release notes for the new package.
Rule reference
CONTRIBUTING.md states: "There's no need to mess around with CHANGELOG.md or package manifests — we have a bot handle that for us. A maintainer will add the necessary notes before merging." Additionally, CLAUDE.md requires pnpm changeset to be run before PRing; no changeset file for @livekit/agents-plugin-gandr exists under .changeset/.
Prompt for agents
CONTRIBUTING.md forbids hand-editing CHANGELOG.md / version fields in package manifests; the release bot owns those. Remove plugins/gandr/CHANGELOG.md (and let the version in plugins/gandr/package.json be handled by the release tooling), and instead add a changeset via `pnpm changeset` under .changeset/ describing the new @livekit/agents-plugin-gandr package, as required by CLAUDE.md.
Was this helpful? React with 👍 or 👎 to provide feedback.
| export default defineConfig({ | ||
| ...defaults, | ||
| }); No newline at end of file |
There was a problem hiding this comment.
🟡 New files are not formatted with the project formatter, which will fail CI
Several new files are committed unformatted, e.g. missing a final newline (plugins/gandr/tsup.config.ts:7), so the repository's mandatory formatting check fails.
Impact: Continuous integration rejects the pull request until the formatter is run.
Affected files and rule
CONTRIBUTING.md requires running pnpm -w format:write and pnpm -w lint:fix before committing, and CLAUDE.md lists "Prettier formatting passes" as a CI requirement. plugins/gandr/tsup.config.ts, plugins/gandr/tsconfig.json, plugins/gandr/api-extractor.json and plugins/gandr/src/tts.ts all end without a trailing newline; in addition the multi-line arrays in plugins/gandr/tsconfig.json:3-5,15-17 and the expanded fetch(...) argument layout in plugins/gandr/src/tts.ts:134-151 do not match Prettier's output (100-char width, last-arg hugging).
Prompt for agents
Run `pnpm -w format:write` (Prettier) across the newly added plugins/gandr files. At minimum: add trailing newlines to plugins/gandr/tsup.config.ts, plugins/gandr/tsconfig.json, plugins/gandr/api-extractor.json and plugins/gandr/src/tts.ts, collapse the tsconfig arrays onto single lines, and let Prettier reformat the fetch() call in plugins/gandr/src/tts.ts so `format:check` passes in CI.
Was this helpful? React with 👍 or 👎 to provide feedback.
Numbers and order IDs are not read correctly in the target language; the reddit lesson proved they render in English. Removed the claim from the Accuracy section. WER stays, it is verified.
| } finally { | ||
| this.queue.close(); | ||
| } |
There was a problem hiding this comment.
🟡 Network connection is left open when speech is cancelled
The open download of the audio response is abandoned without being released or cancelled (loop exits at plugins/gandr/src/tts.ts:190 with no reader.releaseLock()/cancel() in the cleanup at plugins/gandr/src/tts.ts:213-215), so cancelled requests can keep holding a connection.
Impact: Frequent interruptions can accumulate unreleased connections and memory over a long-running session.
Comparison with the equivalent plugin code
When this.abortSignal.aborted becomes true the while loop at plugins/gandr/src/tts.ts:190-199 exits, but the finally block only closes the queue. The reader keeps its lock on response.body and the body is never cancelled. plugins/baseten/src/tts.ts:200-203 calls reader.releaseLock() in its finally. Also note that the loop only observes the abort between reads: because reader.read() is awaited without racing an abort promise (unlike plugins/baseten/src/tts.ts:175-180), a stalled body keeps the task pending until the fetch signal aborts the stream.
| } finally { | |
| this.queue.close(); | |
| } | |
| } finally { | |
| try { | |
| await reader?.cancel(); | |
| } catch { | |
| // stream already errored or closed | |
| } | |
| this.queue.close(); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
| // SPDX-FileCopyrightText: 2026 Gandr | ||
| // | ||
| // SPDX-License-Identifier: Apache-2.0 |
There was a problem hiding this comment.
🟡 New source files use a non-standard copyright header
The new files declare a third-party copyright line (// SPDX-FileCopyrightText: 2026 Gandr at plugins/gandr/src/tts.ts:1) instead of the header the repository mandates for every new file.
Impact: The added files do not carry the project's required license header, which the repository's compliance rules demand.
Rule reference and affected files
CLAUDE.md ("Code Conventions") requires exactly:
// SPDX-FileCopyrightText: 2026 LiveKit, Inc.
//
// SPDX-License-Identifier: Apache-2.0
and CONTRIBUTING.md says to copy the first three lines from any other TypeScript file in the repo. Affected: plugins/gandr/src/tts.ts:1-3, plugins/gandr/src/index.ts:1-3, plugins/gandr/src/models.ts:1-3, plugins/gandr/src/tts.test.ts:1-3, plugins/gandr/README.md:1-5.
| // SPDX-FileCopyrightText: 2026 Gandr | |
| // | |
| // SPDX-License-Identifier: Apache-2.0 | |
| // SPDX-FileCopyrightText: 2026 LiveKit, Inc. | |
| // | |
| // SPDX-License-Identifier: Apache-2.0 |
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Adds @livekit/agents-plugin-gandr, a TTS provider plugin for Gandr, mirroring the shape of the python plugin (livekit/agents PR #6814).
Happy to adjust anything to match house style.