Skip to content

Release 0.63.3: Moonshine streaming external decoder hook - #522

Merged
michalharakal merged 5 commits into
developfrom
release/0.63.3
Oct 28, 2026
Merged

michalharakal merged 5 commits into
developfrom
release/0.63.3

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Against engine 0.62.0, transformers-only and additive. The Android runtime (runtime-iree-android) gains an external-decoder hook for streaming Moonshine, next to 0.63.2's external encoder.

Added

  • External decoder hook (moonshine_stream_jni.c, libskainet_moonshine_stream.so both ABIs).
    • A prefill path ext:<library.so>|<model dir> replaces the prefill and step graphs with a dlopened library exporting moonshine_ext_decoder().
    • ABI 1: create, cross, reset, step, destroy.
    • The streaming loop, greedy choice, token budgets, repeat guard and text handling stay in the runtime. Partial transcripts keep their locked prefix.
    • With both sides external, no params archive is needed.
    • Without the prefix nothing changes: the IREE step moved into one helper, behaviour identical.
  • moonshine_stream_check: MOONSHINE_DEC replaces the prefill path.
  • KDoc on IreeMoonshineStream for the external decoder form.

Checked on an arm32 device, 444 German commands:

  • External encoder + decoder on an accelerator outside IREE: 13.5 % WER, transcript 238 ms after the end of the audio (p50), against 1080 ms with the Vulkan decoder; no GPU use.
  • Regression: the external-encoder + Vulkan-decoder path gives 444/444 transcripts identical to 0.63.2.

Release steps:

  • Commits: version bump to 0.63.3 (gradle.properties, README, docs BOM snippets) and the CHANGELOG entry.
  • After merge: tag 0.63.3 on develop.

A prefill path of the form ext:<library.so>|<model dir> replaces the prefill and step graphs with a library
that exports moonshine_ext_decoder() (ABI 1: create, cross, reset, step, destroy). The streaming loop, greedy
choice, budgets and guards stay in the runtime; the library decodes one token per step from the embedded previous
token. Partials keep their locked prefix: cross() refreshes the memory without touching the token state.
With an external encoder too, no parameter archive is needed. The IREE step moves into iree_step(); its
behaviour is unchanged. moonshine_stream_check takes MOONSHINE_DEC like MOONSHINE_ENC.
…both ABIs)

Built with native/build-moonshine-stream.sh <abi> --vulkan from the previous commit.
@michalharakal
michalharakal merged commit 19c5fbc into develop Oct 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant