Skip to content

feat(voice): speak assistant text while it streams - #135

Draft
johnmatthewtennant wants to merge 6 commits into
jtennant/stream-pocket-tts-audiofrom
jtennant/speak-streaming-assistant-text
Draft

feat(voice): speak assistant text while it streams#135
johnmatthewtennant wants to merge 6 commits into
jtennant/stream-pocket-tts-audiofrom
jtennant/speak-streaming-assistant-text

Conversation

@johnmatthewtennant

@johnmatthewtennant johnmatthewtennant commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements the conversation side of the duplex stream: streaming LLM response -> TTS.

As ACP text deltas arrive, Berd appends the raw text to the Pocket utterance stream instead of waiting for the assistant message to complete. The backend owns sentence and token-limit segmentation. Tool boundaries flush speakable text without ending the utterance; response completion finishes it.

This is distinct from PR #133, which streams Pocket PCM synthesis chunks to playback and overlaps later synthesis with already-playing audio.

Reviewer-reproducible examples

  • Ask for a long multi-paragraph story. The transcript should grow while the model responds, and speech should begin after the first complete sentence, before the full response is visible.
  • During that response, watch the text block status. It should show one Speaking state for the utterance, not a separate Speaking block for every synthesized sentence or token chunk.
  • Listen across sentence boundaries. Later sentences should already be synthesizing while earlier audio plays, avoiding the long gaps caused by serial synthesize-then-play jobs.
  • Interrupt once in the middle of the response. Playback and pending synthesis should stop, with one Interrupted status and one delivery notice for the utterance.

@johnmatthewtennant
johnmatthewtennant marked this pull request as ready for review August 19, 2026 22:12
@johnmatthewtennant
johnmatthewtennant marked this pull request as draft August 20, 2026 01:43
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/speak-streaming-assistant-text branch 3 times, most recently from b58163e to ac4ef5a Compare August 20, 2026 02:40
@johnmatthewtennant
johnmatthewtennant force-pushed the jtennant/speak-streaming-assistant-text branch from ac4ef5a to 46aabf3 Compare August 20, 2026 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant