diff --git a/CHANGELOG.md b/CHANGELOG.md index efa69174..44c19e61 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.64.2] — 2026-11-08 + Against engine **0.63.0**, transformers-only and additive. ### Added @@ -24,6 +26,9 @@ Against engine **0.63.0**, transformers-only and additive. checkpoint. LFM2.5-350M with Q8_0 projections and a Q4_0 head: all 93 matmuls fold; against the FP32 model the logits cosine is 0.997–0.999 with the same top-1 over a 10-token sequence. Compiled with Torq 2.3.0 it is one NPU dispatch and runs on an SL2610 at 533 ms per step, logits cosine 0.996 to FP32 with the same top-1. +- `Lfm2DecodeStepExportCli --parameters external-bf16`: every weight as a model-scope parameter, written as BF16 + safetensors, with the parameter globals typed bf16 and converted to f32 where they load, for a GPU that serves + weights from an archive. Writes the same port table, export manifest and embedding rows as the inline mode. - `ShortConvLowering.MULTIPLY_SUM`: the single-token short convolution as `sum(window * taps)` instead of a grouped `conv1d`, numerically equal, for compilers without grouped convolutions (the decode step's default). - `DecodeStepExport.exportPorts(inputs, dtype) { ports, ctx -> results }` (`llm-core`, jvm): the decode-step export diff --git a/README.md b/README.md index 0f78f172..08b19293 100644 --- a/README.md +++ b/README.md @@ -115,7 +115,7 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.64.1**, against **SKaiNET 0.63.0**: a packaging fix: Maven Central refused 0.64.0 (the `skainet-transformers-inference-lfm2` POM lacked dependency versions), so 0.64.0 was never published and its content ships here. `DecodeStepExport` (`llm-core`) traces one single-token step of any KV-cache decoder with a static past and keeps Q8_0 / Q4_0 weights packed for the engine's block-quant matmul fusion, so FunctionGemma 270M's decode step compiles to one dispatch on the Torq NPU of an SL2610 (162 ms per step, same top-1 as the f32 reference); the with-past types (`RopeCosSin`, `ChunkContext`, `KvStepOutput`) are shared by Gemma and the generic KV decoder in `sk.ainet.lang.nn.dsl.decoder`. LFM2 arrives as `llm-inference:lfm2` (dense hybrid short-conv + GQA model, verified checkpoint loading, fixed-shape StableHLO export) and the `llm-runtime:klfm2` chat frontend. 0.63.3 let the streaming Moonshine runtime hand the decoder to an external native library too (`ext:|` as the prefill path), so with 0.63.2's external encoder the whole model can run on an accelerator outside IREE while the runtime keeps the streaming loop, greedy decoding and its guards. 0.63.2 handed the encoder side to an external library (`ext:` as the encoder path) while the decoder stays on the IREE device. 0.63.1 gave the streaming Moonshine runtime an incremental frontend (an optional `frontend_step.vmfb` takes most of the frontend out of the end-of-utterance flush, transcripts unchanged) and `IreeModuleRunner` runs one compiled IREE module on Android. 0.63.0 brought expert heads and routing for Laya — a decision head fine-tuned on top of the frozen encoder (`expert.safetensors` + `expert.json`) loads on its own with `LayaExpertLoader`, any number of heads share one encoder pass per question (`LayaDecisionModel.logitsFrom`), `LayaRouterSpec` + `LayaMixtureAgent` route a request by asking the base head what kind of text it is, and `LayaMixtureDecisionAdapter` puts the mixture behind the typed decision contract; verified against the published layaMOE evaluation. The ModernBERT-large `laya-typed-decisions` checkpoint loads (`position_embedding_type: absolute`). The whisper StableHLO export ships as `WhisperExportCli`. Engine 0.62.0 brings the JVM ↔ Kotlin/Wasm parity: every kernel on the wasm tier, resident activations, fused attention / norms / softmax, browser IO parity. The Laya Q8_0 lane on the desktop JVM with the native provider needs engine 0.62.1 (SKaiNET #1436). +The current release is **0.64.2**, against **SKaiNET 0.63.0**: LFM2 gets a decode step for an NPU, one fixed-capacity single-token graph with Q8_0 / Q4_0 weights kept packed; `Lfm2DecodeStepExportCli` writes it from a verified checkpoint, and LFM2.5-350M compiled with Torq 2.3.0 is one NPU dispatch on an SL2610 at 533 ms per step, with the same top-1 as FP32. 0.64.1 was a packaging fix: Maven Central refused 0.64.0 (the `skainet-transformers-inference-lfm2` POM lacked dependency versions), so 0.64.0 was never published and its content ships here. `DecodeStepExport` (`llm-core`) traces one single-token step of any KV-cache decoder with a static past and keeps Q8_0 / Q4_0 weights packed for the engine's block-quant matmul fusion, so FunctionGemma 270M's decode step compiles to one dispatch on the Torq NPU of an SL2610 (162 ms per step, same top-1 as the f32 reference); the with-past types (`RopeCosSin`, `ChunkContext`, `KvStepOutput`) are shared by Gemma and the generic KV decoder in `sk.ainet.lang.nn.dsl.decoder`. LFM2 arrives as `llm-inference:lfm2` (dense hybrid short-conv + GQA model, verified checkpoint loading, fixed-shape StableHLO export) and the `llm-runtime:klfm2` chat frontend. 0.63.3 let the streaming Moonshine runtime hand the decoder to an external native library too (`ext:|` as the prefill path), so with 0.63.2's external encoder the whole model can run on an accelerator outside IREE while the runtime keeps the streaming loop, greedy decoding and its guards. 0.63.2 handed the encoder side to an external library (`ext:` as the encoder path) while the decoder stays on the IREE device. 0.63.1 gave the streaming Moonshine runtime an incremental frontend (an optional `frontend_step.vmfb` takes most of the frontend out of the end-of-utterance flush, transcripts unchanged) and `IreeModuleRunner` runs one compiled IREE module on Android. 0.63.0 brought expert heads and routing for Laya — a decision head fine-tuned on top of the frozen encoder (`expert.safetensors` + `expert.json`) loads on its own with `LayaExpertLoader`, any number of heads share one encoder pass per question (`LayaDecisionModel.logitsFrom`), `LayaRouterSpec` + `LayaMixtureAgent` route a request by asking the base head what kind of text it is, and `LayaMixtureDecisionAdapter` puts the mixture behind the typed decision contract; verified against the published layaMOE evaluation. The ModernBERT-large `laya-typed-decisions` checkpoint loads (`position_embedding_type: absolute`). The whisper StableHLO export ships as `WhisperExportCli`. Engine 0.62.0 brings the JVM ↔ Kotlin/Wasm parity: every kernel on the wasm tier, resident activations, fused attention / norms / softmax, browser IO parity. The Laya Q8_0 lane on the desktop JVM with the native provider needs engine 0.62.1 (SKaiNET #1436). It builds on **0.62.2**, against **SKaiNET 0.61.0**: a transformers-only fix, the Android runtime's streaming Moonshine libraries are rebuilt with the utterance-start fix (#500; 0.62.0 and 0.62.1 packed the old build, which garbles transcripts on device). It builds on **0.62.1**, a transformers-only fix on 0.62.0, whose JVM `LayaModelLoader.load(directory, …)` had become an extension function that existing callers did not import; it is a member again (`@JvmOverloads`, so Java callers keep the 0.61 signature). @@ -305,7 +305,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc index fca223fb..fb137a85 100644 --- a/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc +++ b/docs/modules/ROOT/pages/how-to/export-moonshine-v2-streaming.adoc @@ -32,7 +32,7 @@ From your own build, resolve the published module and run its main class: ---- val exportTool by configurations.creating dependencies { - exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + exportTool(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) exportTool("sk.ainet.transformers:skainet-transformers-inference-moonshine") } tasks.register("exportMoonshine") { diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc index e79ed59a..d4829ab9 100644 --- a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -24,7 +24,7 @@ a DSL decoder is future work. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") } ---- diff --git a/docs/modules/ROOT/pages/reference/typed-decisions.adoc b/docs/modules/ROOT/pages/reference/typed-decisions.adoc index 00f51045..76bd5c6e 100644 --- a/docs/modules/ROOT/pages/reference/typed-decisions.adoc +++ b/docs/modules/ROOT/pages/reference/typed-decisions.adoc @@ -12,7 +12,7 @@ below put that on the JVM path with the same contract for every model. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-decisions-api") // the contract, no engine dependency implementation("sk.ainet.transformers:skainet-transformers-decisions-runtime") // validation + acceptance policy implementation("sk.ainet.transformers:skainet-transformers-inference-laya") // Laya as a DecisionAdapter diff --git a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc index e8d514a0..d657799f 100644 --- a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc @@ -55,7 +55,7 @@ dependencies { // self-registers it on ART at process start — nothing to call. runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu") - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-core") implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-inference-llama") diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index fd7eb364..cf13ddd3 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.64.1 + 0.64.2 pom import diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc index 24fc4a73..e9cfe7f9 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc @@ -34,7 +34,7 @@ encoder output to the advertised dimensionality. The runtime applies it automati [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-providers") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index f4164a59..be74795e 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.64.2")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index 119b53aa..0d602d90 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.64.1 +VERSION_NAME=0.64.2 POM_DESCRIPTION=SKaiNET-transformers