diff --git a/CHANGELOG.md b/CHANGELOG.md index 64cf1f19..a0e363e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.63.2] — 2026-10-27 + +Against engine **0.62.0**, transformers-only and additive. The Android runtime (`runtime-iree-android`) gains an +external-encoder hook for streaming Moonshine; it still depends on nothing but the Kotlin standard library. + +### Added + +- **Moonshine streaming: external encoder hook** (`moonshine_stream_jni.c`, `libskainet_moonshine_stream.so` both + ABIs). An encoder path of the form `ext:|` replaces the frontend, encoder and adapter graphs + with a native library loaded by `dlopen` that exports `moonshine_ext_encoder()` (ABI 1: one whole window of audio + in, the window's 64 memory rows out). An accelerator outside IREE can then run the encoder side while the decoder + stays on the IREE device (Vulkan or CPU). A bare library name resolves among the app's own native libraries. Without + the prefix nothing changes; `IreeMoonshineStream` passes the path through, so the Kotlin API is unchanged. + Checked on an arm32 Mali device over 444 German commands with an external encoder: 13.1 % WER against 13.9 % for + the all-Vulkan path from the same library. +- `native/moonshine_stream_check.c`: `MOONSHINE_ENC` replaces the encoder path (e.g. an `ext:` encoder). + ## [0.63.1] — 2026-10-22 Against engine **0.62.0**, transformers-only and additive. The Android runtime (`runtime-iree-android`) gains the diff --git a/README.md b/README.md index 4ee9dccd..93a07212 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,7 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.63.1**, against **SKaiNET 0.62.0**: the streaming Moonshine runtime gains an incremental frontend (an optional `frontend_step.vmfb` takes most of the frontend out of the end-of-utterance flush, transcripts unchanged) and `IreeModuleRunner` runs one compiled IREE module on Android. 0.63.0 brought expert heads and routing for Laya — a decision head fine-tuned on top of the frozen encoder (`expert.safetensors` + `expert.json`) loads on its own with `LayaExpertLoader`, any number of heads share one encoder pass per question (`LayaDecisionModel.logitsFrom`), `LayaRouterSpec` + `LayaMixtureAgent` route a request by asking the base head what kind of text it is, and `LayaMixtureDecisionAdapter` puts the mixture behind the typed decision contract; verified against the published layaMOE evaluation. The ModernBERT-large `laya-typed-decisions` checkpoint loads (`position_embedding_type: absolute`). The whisper StableHLO export ships as `WhisperExportCli`. Engine 0.62.0 brings the JVM ↔ Kotlin/Wasm parity: every kernel on the wasm tier, resident activations, fused attention / norms / softmax, browser IO parity. The Laya Q8_0 lane on the desktop JVM with the native provider needs engine 0.62.1 (SKaiNET #1436). +The current release is **0.63.2**, against **SKaiNET 0.62.0**: the streaming Moonshine runtime can hand the encoder side to an external native library (`ext:|` as the encoder path), so an accelerator outside IREE runs it while the decoder stays on the IREE device. 0.63.1 gave the streaming Moonshine runtime an incremental frontend (an optional `frontend_step.vmfb` takes most of the frontend out of the end-of-utterance flush, transcripts unchanged) and `IreeModuleRunner` runs one compiled IREE module on Android. 0.63.0 brought expert heads and routing for Laya — a decision head fine-tuned on top of the frozen encoder (`expert.safetensors` + `expert.json`) loads on its own with `LayaExpertLoader`, any number of heads share one encoder pass per question (`LayaDecisionModel.logitsFrom`), `LayaRouterSpec` + `LayaMixtureAgent` route a request by asking the base head what kind of text it is, and `LayaMixtureDecisionAdapter` puts the mixture behind the typed decision contract; verified against the published layaMOE evaluation. The ModernBERT-large `laya-typed-decisions` checkpoint loads (`position_embedding_type: absolute`). The whisper StableHLO export ships as `WhisperExportCli`. Engine 0.62.0 brings the JVM ↔ Kotlin/Wasm parity: every kernel on the wasm tier, resident activations, fused attention / norms / softmax, browser IO parity. The Laya Q8_0 lane on the desktop JVM with the native provider needs engine 0.62.1 (SKaiNET #1436). It builds on **0.62.2**, against **SKaiNET 0.61.0**: a transformers-only fix, the Android runtime's streaming Moonshine libraries are rebuilt with the utterance-start fix (#500; 0.62.0 and 0.62.1 packed the old build, which garbles transcripts on device). It builds on **0.62.1**, a transformers-only fix on 0.62.0, whose JVM `LayaModelLoader.load(directory, …)` had become an extension function that existing callers did not import; it is a member again (`@JvmOverloads`, so Java callers keep the 0.61 signature). @@ -303,7 +303,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc index c314212b..ed7b0588 100644 --- a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc +++ b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc @@ -32,7 +32,7 @@ From your own build, resolve the published module and run its main class: ---- val exportTool by configurations.creating dependencies { - exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) exportTool("sk.ainet.transformers:skainet-transformers-inference-moonshine") } tasks.register("exportMoonshine") { diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc index 498e693d..e1196a0f 100644 --- a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -24,7 +24,7 @@ a DSL decoder is future work. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") } ---- diff --git a/docs/modules/ROOT/pages/reference/typed-decisions.adoc b/docs/modules/ROOT/pages/reference/typed-decisions.adoc index 3e4afa9b..4f449f3f 100644 --- a/docs/modules/ROOT/pages/reference/typed-decisions.adoc +++ b/docs/modules/ROOT/pages/reference/typed-decisions.adoc @@ -12,7 +12,7 @@ below put that on the JVM path with the same contract for every model. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-decisions-api") // the contract, no engine dependency implementation("sk.ainet.transformers:skainet-transformers-decisions-runtime") // validation + acceptance policy implementation("sk.ainet.transformers:skainet-transformers-inference-laya") // Laya as a DecisionAdapter diff --git a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc index 47176d07..1a374ae6 100644 --- a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc @@ -55,7 +55,7 @@ dependencies { // self-registers it on ART at process start — nothing to call. runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu") - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-core") implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-inference-llama") diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index 9be01ecf..7446e8bc 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.63.1 + 0.63.2 pom import diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc index 12956621..befc2b16 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc @@ -34,7 +34,7 @@ encoder output to the advertised dimensionality. The runtime applies it automati [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-providers") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index a85db76f..5d9d6efd 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.63.2")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index cd66fca3..660fb55a 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.63.1 +VERSION_NAME=0.63.2 POM_DESCRIPTION=SKaiNET-transformers diff --git a/llm-runtime/iree-android/native/moonshine_stream_check.c b/llm-runtime/iree-android/native/moonshine_stream_check.c index 3d39e5c9..ef5ccee5 100644 --- a/llm-runtime/iree-android/native/moonshine_stream_check.c +++ b/llm-runtime/iree-android/native/moonshine_stream_check.c @@ -11,6 +11,7 @@ * frontend_step.vmfb when the library should use the incremental frontend * pcm headerless little-endian s16 mono 16 kHz; a directory = every *.pcm in it, name order * --fast feed without pacing (timings then no longer resemble a live stream) + * MOONSHINE_ENC (env) replaces the encoder path, e.g. "ext:|" for an external encoder * * Build (same image as the library): * docker run --rm -v "$PWD/native":/src -w /src --entrypoint sh skainet/iree-android:3.11.0 -c \ @@ -77,6 +78,7 @@ int main(int argc, char** argv) { const char* names[] = {"frontend.vmfb", "encoder.vmfb", "adapter.vmfb", "prefill.vmfb", "step.vmfb", "params.irpa", "vocab.bin", "dec_embed.bin"}; static char p[8][1024]; for (int i = 0; i < 8; ++i) snprintf(p[i], sizeof p[i], "%s/%s", argv[3], names[i]); + if (getenv("MOONSHINE_ENC")) snprintf(p[1], sizeof p[1], "%s", getenv("MOONSHINE_ENC")); double t0 = now_ms(); jlong h = create(env, NULL, (jstring)argv[2], (jstring)p[0], (jstring)p[1], (jstring)p[2], (jstring)p[3], (jstring)p[4], (jstring)p[5], (jstring)p[6], (jstring)p[7]); if (!h) { fprintf(stderr, "nativeCreate failed\n"); return 4; } diff --git a/llm-runtime/iree-android/native/moonshine_stream_jni.c b/llm-runtime/iree-android/native/moonshine_stream_jni.c index 7ba4c862..1e4c6709 100644 --- a/llm-runtime/iree-android/native/moonshine_stream_jni.c +++ b/llm-runtime/iree-android/native/moonshine_stream_jni.c @@ -24,9 +24,16 @@ * one exact full re-decode so the transcript handed to NLU is bit-faithful. * * One engine = one utterance at a time; calls are single-threaded (host serializes). + * + * External encoder: an `enc` path of the form "ext:|" replaces the frontend, encoder and + * adapter graphs (fe and adp are then ignored, the incremental frontend is off). The library exports + * `const MoonshineExtEncoder* moonshine_ext_encoder(void)` (struct below): one whole window of audio in, the + * window's 64 memory rows (adapter applied) out. It lets an accelerator outside IREE run the encoder side while + * the decoder stays on the IREE device. * Per-hop stage timings go to stderr/logcat as "moonshine-timing" lines. */ #include +#include #include #include #include @@ -129,12 +136,23 @@ typedef struct { int n; char** t; } Vocab; +/* External encoder ABI, version 1. window(): pcm[FE_IN] holds `lc` frames of left context then the window, + * zero past the fed audio; out[CHUNK * DIM] receives the memory rows of positions start .. start+CHUNK-1; + * ms[3] (may be NULL) the frontend / encoder / adapter times. Returns 0 on success. */ +typedef struct { + int abi; + void* (*create)(const char* dir); + int (*window)(void* h, const float* pcm, int lc, int start, float* out, double* ms); + void (*destroy)(void* h); +} MoonshineExtEncoder; + typedef struct { iree_runtime_instance_t* inst; iree_hal_device_t* dev; iree_hal_allocator_t* alloc; iree_runtime_session_t* fe, *enc, *adp, *pre, *step; iree_runtime_session_t* fes; /* incremental frontend, NULL = whole-window frontend */ + void* ext_lib; const MoonshineExtEncoder* ext; void* ext_h; /* external encoder, replaces fe/enc/adp */ float (*feat)[ENC_DIM]; /* [MAXF] frontend features of this utterance (fes only) */ int fe_done; /* features [0, fe_done) are exact: backed by audio, never recomputed */ double fe_feed_ms; int fe_calls; /* incremental frontend cost while the audio streams in */ @@ -392,6 +410,21 @@ static void fe_advance(Engine* e, int produced) { } } +/* finalize: [nmem .. newFinal); at flush the lookahead holds nothing back */ +static void finalize_rows(Engine* e, int start, int produced, int flush, int peek, float (*mem64)[DIM]) { + int limit = flush ? start + CHUNK : start + CHUNK - LOOKAHEAD; + int new_final = limit < produced ? limit : produced; + if (new_final > MAXMEM) new_final = MAXMEM; + /* The real window 0 recomputes and OVERWRITES rows an early peek finalized with a + * zero-padded tail; regular windows only append (their overlap rows were finalized with a + * fuller attention band by the previous window and must not be downgraded). */ + int first = (!peek && start == 0 && e->peeked) ? 0 : e->nmem; + for (int f = first; f < new_final; ++f) + memcpy(e->mem[f], mem64[f - start], DIM * sizeof(float)); + if (new_final > e->nmem) e->nmem = new_final; + if (!peek && start == 0) e->peeked = 0; +} + /* Process one encoder window starting at feature frame `start`; finalize rows into e->mem. */ static int process_window(Engine* e, int start, int produced, int flush, int peek) { double t0 = now_ms(); @@ -406,6 +439,22 @@ static int process_window(Engine* e, int start, int produced, int flush, int pee const int lc = start == 0 ? 0 : FE_LC; static float feats[FE_OUT][ENC_DIM]; const float* encsrc; + if (e->ext) { + static float fein[FE_IN]; + static float mem64[CHUNK][DIM]; + const int s0 = (start - lc) * SPF; + for (int i = 0; i < FE_IN; ++i) { + int si = s0 + i; + fein[i] = (si >= 0 && si < e->npcm) ? e->pcm[si] : 0.0f; + } + double ms[3] = {0, 0, 0}; + if (e->ext->window(e->ext_h, fein, lc, start, &mem64[0][0], ms)) return -1; + finalize_rows(e, start, produced, flush, peek, mem64); + e->windows++; + note_evt(e, EVT_WINDOW, t0 - e->t_first_feed, ms[0], ms[1], ms[2]); + TLOG("win %d ext fe %.0fms enc %.0fms adp %.1fms nmem %d\n", e->win, ms[0], ms[1], ms[2], e->nmem); + return 0; + } if (e->fes) { /* features up to the fed audio exist already; only the part of the window past them (a peek's or * the flush's zero-padded tail) is computed here. Frames before `produced` come out exact either @@ -494,18 +543,7 @@ static int process_window(Engine* e, int start, int produced, int flush, int pee if (pull(e, memo, mem64, sizeof(mem64))) { iree_hal_buffer_view_release(memo); return -1; } iree_hal_buffer_view_release(memo); - /* finalize: [nmem .. newFinal); at flush the lookahead holds nothing back */ - int limit = flush ? start + CHUNK : start + CHUNK - LOOKAHEAD; - int new_final = limit < produced ? limit : produced; - if (new_final > MAXMEM) new_final = MAXMEM; - /* The real window 0 recomputes and OVERWRITES rows an early peek finalized with a - * zero-padded tail; regular windows only append (their overlap rows were finalized with a - * fuller attention band by the previous window and must not be downgraded). */ - int first = (!peek && start == 0 && e->peeked) ? 0 : e->nmem; - for (int f = first; f < new_final; ++f) - memcpy(e->mem[f], mem64[f - start], DIM * sizeof(float)); - if (new_final > e->nmem) e->nmem = new_final; - if (!peek && start == 0) e->peeked = 0; + finalize_rows(e, start, produced, flush, peek, mem64); double t3 = now_ms(); e->windows++; note_evt(e, EVT_WINDOW, t0 - e->t_first_feed, t1 - t0, t2 - t1, t3 - t2); @@ -682,6 +720,11 @@ static void warmup(Engine* e) { double t0 = now_ms(); static float za[FE_IN]; memset(za, 0, sizeof(za)); + if (e->ext) { + static float zm[CHUNK * DIM]; + e->ext->window(e->ext_h, za, 0, 0, zm, NULL); + goto decoder; + } iree_hal_buffer_view_t* v = mkview(e, (iree_hal_dim_t[]){1, FE_IN}, 2, IREE_HAL_ELEMENT_TYPE_FLOAT_32, za, sizeof(za)); iree_hal_buffer_view_t* o = v ? run1(e, e->fe, FN_FE, v) : NULL; @@ -717,6 +760,7 @@ static void warmup(Engine* e) { if (posbv) iree_hal_buffer_view_release(posbv); iree_hal_buffer_view_release(enco); } +decoder: /* decoder: fake 64 zero memory frames, prefill + one step, then reset */ memset(e->mem, 0, sizeof(e->mem)); e->nmem = CHUNK; @@ -738,6 +782,26 @@ static void reset_utterance(Engine* e) { e->ntoks = 0; e->pos = 0; e->text[0] = 0; } +/* "|": dlopen the external encoder and create it over the model dir. */ +static void load_ext(Engine* e, const char* spec) { + char lib[1024]; + const char* bar = strchr(spec, '|'); + if (!bar || (size_t)(bar - spec) >= sizeof(lib)) { TLOG("create: ext(%s) -> FAIL (want lib|dir)\n", spec); return; } + memcpy(lib, spec, bar - spec); lib[bar - spec] = 0; + e->ext_lib = dlopen(lib, RTLD_NOW | RTLD_LOCAL); + const MoonshineExtEncoder* (*get)(void) = e->ext_lib ? (const MoonshineExtEncoder* (*)(void))dlsym(e->ext_lib, "moonshine_ext_encoder") : NULL; + const MoonshineExtEncoder* x = get ? get() : NULL; + if (x && x->abi == 1) e->ext_h = x->create(bar + 1); + if (e->ext_h) e->ext = x; + TLOG("create: ext(%s) -> %s%s\n", spec, e->ext ? "ok" : "FAIL ", e->ext ? "" : (e->ext_lib ? "(symbol/abi/create)" : dlerror())); +} + +static void release_ext(Engine* e) { + if (e->ext && e->ext_h) e->ext->destroy(e->ext_h); + if (e->ext_lib) dlclose(e->ext_lib); + e->ext = NULL; e->ext_h = NULL; e->ext_lib = NULL; +} + /* ---- JNI -------------------------------------------------------------------------------- */ #define JNIFN(name) Java_sk_ainet_transformers_iree_android_IreeMoonshineStream_##name @@ -764,10 +828,14 @@ JNIEXPORT jlong JNICALL JNIFN(nativeCreate)(JNIEnv* env, jobject thiz, iree_make_cstring_view(dev), &e->dev))) { e->params = load_params(e, par); TLOG("create: params(%s) -> %s\n", par, e->params ? "ok" : "FAIL"); - e->fe = load_session(e, fe, 0); - TLOG("create: fe(%s) -> %s\n", fe, e->fe ? "ok" : "FAIL"); - if (e->fe && e->params) { - e->alloc = iree_runtime_session_device_allocator(e->fe); + const int ext = !strncmp(enc, "ext:", 4); + if (ext) load_ext(e, enc + 4); + else { + e->fe = load_session(e, fe, 0); + TLOG("create: fe(%s) -> %s\n", fe, e->fe ? "ok" : "FAIL"); + } + if (e->fe) e->alloc = iree_runtime_session_device_allocator(e->fe); + if (!ext && e->fe && e->params) { /* optional incremental frontend, by convention next to frontend.vmfb */ char fesp[1024]; const char* slash = strrchr(fe, '/'); @@ -784,6 +852,8 @@ JNIEXPORT jlong JNICALL JNIFN(nativeCreate)(JNIEnv* env, jobject thiz, TLOG("create: enc(%s) -> %s\n", enc, e->enc ? "ok" : "FAIL"); e->adp = load_session(e, adp, 0); TLOG("create: adp(%s) -> %s\n", adp, e->adp ? "ok" : "FAIL"); + } + if ((ext ? (void*)e->ext : (void*)e->fe) && e->params) { e->pre = load_session(e, pre, 1); TLOG("create: pre(%s) -> %s\n", pre, e->pre ? "ok" : "FAIL"); e->step = load_session(e, stp, 1); @@ -803,7 +873,8 @@ JNIEXPORT jlong JNICALL JNIFN(nativeCreate)(JNIEnv* env, jobject thiz, } e->pcm = malloc(MAXPCM * sizeof(float)); rope_init(e); - if (e->enc && e->adp && e->pre && e->step && e->vocab.n > 0 && e->embed && e->pcm) { + if (!e->alloc && e->pre) e->alloc = iree_runtime_session_device_allocator(e->pre); + if ((e->ext || (e->enc && e->adp)) && e->pre && e->step && e->vocab.n > 0 && e->embed && e->pcm) { warmup(e); ret = (jlong)(intptr_t)e; } @@ -814,7 +885,7 @@ JNIEXPORT jlong JNICALL JNIFN(nativeCreate)(JNIEnv* env, jobject thiz, (*env)->ReleaseStringUTFChars(env, jpre, pre); (*env)->ReleaseStringUTFChars(env, jstep, stp); (*env)->ReleaseStringUTFChars(env, jparams, par); (*env)->ReleaseStringUTFChars(env, jvocab, voc); (*env)->ReleaseStringUTFChars(env, jembed, emb); - if (!ret && e) { free(e->embed); free(e->pcm); free(e->feat); free(e); } + if (!ret && e) { release_ext(e); free(e->embed); free(e->pcm); free(e->feat); free(e); } return ret; } @@ -996,6 +1067,7 @@ JNIEXPORT void JNICALL JNIFN(nativeReset)(JNIEnv* env, jobject thiz, jlong handl JNIEXPORT void JNICALL JNIFN(nativeDestroy)(JNIEnv* env, jobject thiz, jlong handle) { Engine* e = (Engine*)(intptr_t)handle; if (!e) return; reset_utterance(e); + release_ext(e); if (e->fe) iree_runtime_session_release(e->fe); if (e->fes) iree_runtime_session_release(e->fes); if (e->enc) iree_runtime_session_release(e->enc); diff --git a/llm-runtime/iree-android/src/main/jniLibs/arm64-v8a/libskainet_moonshine_stream.so b/llm-runtime/iree-android/src/main/jniLibs/arm64-v8a/libskainet_moonshine_stream.so index 4690023f..0f819b13 100755 Binary files a/llm-runtime/iree-android/src/main/jniLibs/arm64-v8a/libskainet_moonshine_stream.so and b/llm-runtime/iree-android/src/main/jniLibs/arm64-v8a/libskainet_moonshine_stream.so differ diff --git a/llm-runtime/iree-android/src/main/jniLibs/armeabi-v7a/libskainet_moonshine_stream.so b/llm-runtime/iree-android/src/main/jniLibs/armeabi-v7a/libskainet_moonshine_stream.so index 1922ffa3..8c214130 100755 Binary files a/llm-runtime/iree-android/src/main/jniLibs/armeabi-v7a/libskainet_moonshine_stream.so and b/llm-runtime/iree-android/src/main/jniLibs/armeabi-v7a/libskainet_moonshine_stream.so differ diff --git a/llm-runtime/iree-android/src/main/kotlin/sk/ainet/transformers/iree/android/IreeMoonshineStream.kt b/llm-runtime/iree-android/src/main/kotlin/sk/ainet/transformers/iree/android/IreeMoonshineStream.kt index 22ab5ad3..df0206ed 100644 --- a/llm-runtime/iree-android/src/main/kotlin/sk/ainet/transformers/iree/android/IreeMoonshineStream.kt +++ b/llm-runtime/iree-android/src/main/kotlin/sk/ainet/transformers/iree/android/IreeMoonshineStream.kt @@ -16,6 +16,11 @@ package sk.ainet.transformers.iree.android * only runs the encoder and adapter, which takes most of the frontend out of [finish]. The features are bit-identical * either way, so the transcript does not change; without the file the runtime behaves as before. See [OPTIONAL_FILE_NAMES]. * + * External encoder: an [Files.encoderVmfb] of the form `ext:|` replaces frontend, encoder and + * adapter with a native library exporting `moonshine_ext_encoder()` (see moonshine_stream_jni.c for the ABI), loaded + * with dlopen — a bare library name resolves among the app's own native libraries. The decoder stays on [device]; + * frontend and adapter paths are then ignored. + * * One instance = one utterance at a time; callers serialize access. Per-hop stage timings are logged * under the `moonshine-timing` tag. *