Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 38 additions & 4 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,50 @@

## [Unreleased]

### Fixed

- **Block-quant export reads loader-backed weights** (`skainet-compile-hlo`): `BlockQuantMatmulConverter` read Q8_0 / Q4_0 blocks only from heap `packedData`, so a weight from the GGUF loader (`BufferPackedTensorData`, whose `packedData` throws) failed the export. It now reads the ggml blocks through the weight's packed view, which works for both.
- **StableHLO export maps `Int4` to `i4` and fails on a dtype without an element type** (`skainet-compile-hlo`): `TypeMapper.mapDType` turned every name it did not know, `Int4` and `Ternary` among them, into `f32`, so a typed `Int4` or `Ternary` tensor exported as a module whose types disagreed with its data. `Int4`/`I4` now map to `i4`; an unset dtype (`""`, `unknown`, what the DSL records when none is given) stays `f32`; any other unknown name throws `IllegalArgumentException` naming the dtype. Quantized weights are unaffected: they reach the emitter as dense tensors with a storage encoding.
## [0.63.0] - 2026-11-06

Headline: **block-quantized weights reach an NPU through StableHLO, and the engine gains mixed precision, explicit
re-encoding, `convTranspose2d` and a working ONNX import path** (see *Added*). With `PackedConstantHandling.KEEP_PACKED`
and `BlockQuantMatmulFusionPass`, a Q8_0 / Q4_0 `Linear` exports as the dequantize-then-`batch_matmul` chain that
Torq 2.3.0 compiles to its `mixed_quant_matmul` kernel: the full FunctionGemma 270M decode step (127 projections) runs
on an SL2610 NPU in one dispatch at 162 ms per step, picking the same next token as the f32 reference. A layer can run
in another dtype through `PrecisionBoundary` (`dense(64, dtype = BF16) { }`), storages re-encode with `repackTo` /
`requantizeTo`, and ONNX graphs with `ConvTranspose`, `LeakyRelu`, `Silu`, `Clip`, `BatchNormalization` and `Pad`
import as executable ops. The generic `dense<T>()` / `stage<T>()` overloads, which could only ever throw, are now
deprecated at `ERROR` level.

### Added

- **Block-quantized weights through StableHLO export** (`skainet-compile-dag`, `skainet-compile-opt`, `skainet-compile-hlo`): `PackedConstantHandling.KEEP_PACKED` keeps a packed weight as its storage instead of failing or dequantizing; `BlockQuantMatmulFusionPass` folds a Q8_0 / Q4_0 weight used as `x · Wᵀ` into a `block_quant_matmul` node; `BlockQuantMatmulConverter` (registered in every converter set) emits it as `(sitofp(codes) - zp) * scale` over `[nB, 32, N]` `i8` / `i4` codes with `bf16` scales, then `linalg.batch_matmul`, all constants inline. A packed weight left unfolded fails the export. On an SL2610, Torq 2.3.0 with `--iree-input-type=stablehlo-torq` compiles the emitted Q8_0 and Q4_0 projections to its `mixed_quant_matmul` kernel and matches the dequantized weight (cosine 0.999996); `TorqBlockQuantGoldenTest` checks the match when `SKAINET_TORQ_COMPILE` names the compiler. `HloGenerator.generate` takes `packedConstants`.
- **StableHLO-phase target passes** (`skainet-compile-opt`, `skainet-compile-hlo`): `TargetOptimizer.stableHloPasses()` and `TargetOptimizers.registerStableHloPasses(target) { … }` / `stableHloPassesFor(target)` wire the `STABLE_HLO` phase. A `StableHloPass` rewrites the emitted module's text into valid StableHLO; `toStableHlo(graph, …, passes = …)` and `StableHloModule.applyPasses(passes)` apply them in order. The existing `toStableHlo` signatures are unchanged.
- **TOSA preparation passes** (`sk.ainet.compile.hlo.tosa`): `DivideByBroadcastPass` (a divide by a broadcast becomes a multiply by the broadcast reciprocal, or by `rsqrt` for a `sqrt`) and `RankTwoWeightMatmulPass` (`[1, M, K] · [K, N]` as a rank-2 dot) shape a module for StableHLO's own `--stablehlo-prepare-for-tosa --stablehlo-legalize-to-tosa`; `TosaFinish` translates what that legalization leaves (`broadcast_in_dim`, `rsqrt`, batched attention `dot_general`s) and throws `TosaFinishException` on anything else. Moonshine v2 tiny streaming encoder (64 frames) on an SL2610 with the Torq NPU's `tosa-torq` input: one dispatch per call, golden transcripts identical to the recipe-side rewrite they replace.
- **Mixed precision per layer** (`skainet-lang-core`, SKEEP-008 phase A1, #1469): `PrecisionBoundary` (`sk.ainet.lang.nn`) wraps a module built in another dtype and casts in and out with `TensorOps.convert` (free when the dtypes match, recorded by the tracing wrappers, lowered to `stablehlo.convert`); the DSL takes the dtype as a value, `dense(64, dtype = BF16) { }` and `stage("head", dtype = FP16) { }`. Inference only for now. Explanation article `explanation/mixed-precision.adoc`.
- **Lossless `repackTo` and explicit lossy `requantizeTo`** (`skainet-lang-core`, SKEEP-008 phase B1, #1470): `TensorStorage.repackTo` decodes any packed encoding with a dequantizer, and narrow dense FP16 / BF16, to `Dense(4)` bit-exactly instead of throwing; `TensorStorage.requantizeTo` is the visible lossy call, accepting exactly `Requantizers.supported`. `TensorStorageFactory.toTensorData` decodes Q4_0, Q5_0, Q5_1, Q5_K, Q6_K, `BITNET_PLANES` and `BITNET_B1_58` too.
- **`convTranspose2d`** (`skainet-lang-core`, `skainet-backend-cpu`, `skainet-compile-dag`, `skainet-compile-hlo`, #1464): an abstract `@Diff` op with PyTorch `conv_transpose2d` semantics (weight `[C_in, C_out/groups, kH, kW]`, groups, dilation, output padding), a validated CPU scatter-add kernel, input / weight / bias backward, and the StableHLO lowering as a `stablehlo.convolution` with `lhs_dilate = stride` over the spatially reversed kernel. `TransposedConv2d` works end to end.
- **ONNX import executes more graphs** (`skainet-io-onnx`, `skainet-compile-dag`, #1466, #1468): `LeakyRelu`, `Silu`, `Clip`, `BatchNormalization` (inference mode), `Pad` (constant, zero, spatial) and `ConvTranspose` (1D / 2D, `auto_pad=NOTSET`, symmetric pads) map to real operations instead of placeholders that threw on execution; forms that cannot be mapped faithfully keep the placeholder and still fail loudly. `ComputeGraphExecutor` dispatches `leakyRelu`, `clamp`, `pad2d`, `batchNorm`, `convTranspose1d` and `convTranspose2d`, and binds `onnx_initializer` nodes by id or tensor name.
- **Streaming `OnnxModelReader`** (`skainet-io-onnx`, #1465): `OnnxModelReader.open(source)` / `open(filePath)` reads the metadata and an index of every initializer, and `loadTensor(name)` reads and decodes one tensor lazily (`FLOAT`, `INT32`, `INT64`); unsupported dtypes, `external_data` and size mismatches fail instead of returning padded data. It replaces the no-arg constructor, which could only throw.
- **`GroupNormalization` forward pass** (`skainet-lang-core`, #1463): per-group biased mean and variance in f32 with `eps`, per-channel affine broadcast, matching `torch.nn.GroupNorm`; `groupNorm {}` no longer crashes on the first forward.
- **`Tensor.sliceCopy {}`** (`skainet-lang-core`, #1467): builds the slice like `sliceView {}` and materializes an independent copy instead of throwing `NotImplementedError`.
- **Precision, recall and F1 metrics** (`skainet-lang-core`, #1458, #1460, #1461, #1462): `Precision`, `Recall` and `F1Score` with `precision()` / `binaryPrecision()`, `recall()` / `binaryRecall()` and `f1Score()` / `binaryF1Score()`, the `Averaging` strategy (`BINARY`, `MACRO`, `MICRO`) and a shared confusion-matrix accumulator; zero denominators give 0.0.

### Changed

- **Dependencies**: Ktor client 3.6.1 (#1472), kotlinx-io 0.9.2 (#1474), JMH plugin 0.7.4 (#1473), AGP 9.5.1 (#1457, #1482), logback 1.6.6 (#1456), asciidoctor PDF 5.1.0 (#1452), CodeQL upload-sarif 4.39.2 (#1453, #1481).

### Deprecated

- **`dense<TLayer>()` and `stage<TStage>()`** (`skainet-lang-core`, #1469): the generic overloads could never work (an interface method's type parameter cannot be `reified`) and are now `@Deprecated(level = ERROR)`; use `dense(outputDimension, dtype = FP16) { }` / `stage(id, dtype = FP16) { }`.

### Removed

- The unused `TensorDefineDslImpl.create()` stub (`skainet-lang-core`, #1471).

### Fixed

- **Block-quant export reads loader-backed weights** (`skainet-compile-hlo`): `BlockQuantMatmulConverter` read Q8_0 / Q4_0 blocks only from heap `packedData`, so a weight from the GGUF loader (`BufferPackedTensorData`, whose `packedData` throws) failed the export. It now reads the ggml blocks through the weight's packed view, which works for both.
- **StableHLO export maps `Int4` to `i4` and fails on a dtype without an element type** (`skainet-compile-hlo`): `TypeMapper.mapDType` turned every name it did not know, `Int4` and `Ternary` among them, into `f32`, so a typed `Int4` or `Ternary` tensor exported as a module whose types disagreed with its data. `Int4`/`I4` now map to `i4`; an unset dtype (`""`, `unknown`, what the DSL records when none is given) stays `f32`; any other unknown name throws `IllegalArgumentException` naming the dtype. Quantized weights are unaffected: they reach the emitter as dense tensors with a storage encoding.
- **Tokenizer `Split` regex follows Unicode on the JVM** (`skainet-frontend-text`, #1477): the pattern compiles with `UNICODE_CHARACTER_CLASS`, so `\w`, `\s` and friends match non-ASCII characters as Hugging Face tokenizers do; a no-break space split wrongly before and gave wrong token ids (found on LFM2.5-350M).
- **`randn` never returns `Inf` / `NaN`** (`skainet-lang-core`, #1455): Box-Muller draws `u1` from (0, 1] instead of [0, 1), so `ln(u1)` stays finite, and the transform stays in double precision until the final cast.

## [0.62.4] - 2026-10-25

Expand Down
4 changes: 2 additions & 2 deletions CITATION.cff
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ authors:
orcid: https://orcid.org/0009-0006-7981-4075
- name: SKaiNET-developers contributors
website: https://github.com/SKaiNET-developers
version: "0.62.4"
date-released: "2026-10-21"
version: "0.63.0"
date-released: "2026-11-06"
license: MIT
repository-code: https://github.com/SKaiNET-developers/SKaiNET
url: https://skainet-developers.github.io/SKaiNET/
Expand Down
111 changes: 39 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
```kotlin
dependencies {
// Recommended: import the umbrella BOM and drop versions on the engine modules.
implementation(platform("sk.ainet:skainet-bom:0.62.4"))
implementation(platform("sk.ainet:skainet-bom:0.63.0"))

implementation("sk.ainet.core:skainet-lang-core")
implementation("sk.ainet.core:skainet-backend-cpu")
Expand Down Expand Up @@ -308,76 +308,32 @@ val withoutLabel = dataPipeline<RawDataset>()

---

## What's New in 0.62.4

An arm64 release. On macOS the JVM's dense FP32 products — `A · B` on heap or segment-backed operands and `x · Wᵀ`
on a resident dense weight — now go to Accelerate's SGEMM by default, which runs on the matrix units of Apple
silicon and threads itself; `JvmBlas` had never reached a BLAS before (the framework lookup and the heap downcall
both failed silently), and `-Dskainet.cpu.blas.enabled=false` restores the bundled kernels. Elsewhere the default
stays off and `skainet.cpu.blas.library` names an OpenBLAS or reference BLAS to use. The native dense FP32 kernels
get register tiles on AArch64 (Apple silicon, Android arm64, Linux arm64): an 8 × 8 tile of lane-indexed FMAs over a
packed panel of B for `A · B` and a four-by-three tile for `x · Wᵀ`, bit-identical across slices, threads and whole
calls, exercised on Apple silicon by the macOS CI leg. Both are unmeasured on arm64 in this release; the pull
requests (#1449, #1450) say how to run the bench.

### Carried from 0.62.3

A load-time and kernel release. `StreamingGGUFReader` decodes the GGUF metadata from 64 KB blocks instead of one
source read per value, and `TextFrontEnds.fromGguf` builds a SentencePiece tokenizer in one pass over the vocabulary
instead of three to four copies of it; `fields`, the public API and the token ids stay the same. On a Gemma-3
tokenizer GGUF a file-source parse drops from 1.0 s to 21–45 ms on the JVM and from 23.3 s to 1.2 s on a 32-bit ARM
Android device, where read + build on the first load in a process goes from 8.4–11.2 s to 6.0–6.4 s. The native
dense FP32 `A · B` kernel gets an AVX2/FMA register-tiled lane on x86-64 (63 × 1024 × 1024: 21 → 281 GFLOP/s), and
no longer overwrites the parts of a sub-block caller's output buffer outside the block.

### Carried from 0.62.2

A performance release for the JVM native lane. The dense FP32 `x · Wᵀ` kernel — what every resident dense
weight resolves to once `skainet-backend-native-cpu` is on the classpath — now splits its work across the
context `Schedule` (SKEEP-005) instead of a fixed four-thread C pool, and on x86-64 hosts with AVX2 and FMA
runs a runtime-selected eight-lane register-tiled lane; threaded, sliced and single calls stay bit-identical.
A single 63 × 1024 × 1024 call goes from 29.6 to 131.7 GFLOP/s on a six-core laptop, and a ModernBERT-large
encoder request in the LayaMoE example from 13.0 s to 6.4 s with unchanged answers. The text front-end
indexes added tokens by first character and length, so a Gemma chat turn encodes in 36 µs instead of 2.0 ms.
`CoroutineSchedule.forRange` never hands a body an empty range any more.

### Carried from 0.62.1

A fix release on top of 0.62.0: the native FFM bindings of the Q8_0, Q4_0, Q5_0 and Q5_1 kernels
copied a zero-filled staging buffer back from index 0, so a product served row by row — any
quantize-at-load encoder on the JVM with `skainet-backend-native-cpu` on the classpath — kept only
its last row. Single-row decode steps were never affected. A regression test now covers every packed
format for one, seven and fifty rows. Also: the YOLOv8 model code moves out of core into its cartridge
blueprint (`skainet-model-yolo` is gone; `Letterbox`, `BoundingBox` and `nonMaxSuppression` stay in
`skainet-data-media`), the StableHLO `softmax` axis and `logSoftmax` lowering are fixed, and the docs
gain *Flexibility at every compile stage* and the SKEEP-008 draft on precision and encoding conversion.

### Carried from 0.62.0

JVM ↔ Kotlin/Wasm parity: the browser tier runs the same kernels as the JVM, keeps its activations
in the sidecar, and reads models the way the JVM does — with every remaining difference documented.

- **Every kernel on the wasm tier** — explicit SIMD128 bodies for every matmul format (K-quants,
Q5_0/Q5_1, dense FP32/BF16/FP16), the ternary BitNet kernels, elementwise, activation and
reduction kernels, fused softmax, normalisation and banded attention, and convolutions as
im2col + GEMM, all threaded over the sidecar's row pool. Chains of ops stay resident in the
sidecar's linear memory (`Storage.OffHeap` in a browser) and come back to the Kotlin heap only when
something outside needs them.
- **Both directions** — the JVM binds the same C kernels through FFM: the direct `x · Wᵀ` kernel
(no transposed weight copy), fused softmax, layer and RMS normalisation, and SDPA with
key lengths, windows and grouped-query heads. One C source per kernel serves every native build.
- **Browser IO** — `Blob` sources address `Long` positions (shards over 2 GB), copy in bulk, and
serve blocking reads inside a Web Worker; sharded SafeTensors load from Blobs;
`PlannerProfile.BROWSER`; `MemoryProbe` reports the JavaScript heap, linear memory and the
sidecar's resident bytes; `skainet-data-source` resolves HTTP and Hugging Face URIs over `fetch`
on Kotlin/JS and Kotlin/Wasm.
- **One benchmark table** — the `engine-*` scenarios of the JVM publish harness run with identical
shapes in `wasm-bench`, and the kernel support matrix names what each tier serves and why the
rest is N/A. The operator coverage matrix is generated from backend data, with StableHLO, backward
and NN-layer columns and a CI leg that rejects unmarked stubs.
- **JVM decode allocation** — four fixes from the BitNet 2B4T profile: the ternary GEMV on the
shared row pool (+19 % decode), intrinsified Vector API elementwise paths, a dense `split` fast
path and bulk copies from mapped tensors.
## What's New in 0.63.0

Block-quantized weights reach an NPU through StableHLO, and the engine gains mixed precision, explicit re-encoding,
`convTranspose2d` and a working ONNX import path.

- **Q8_0 / Q4_0 weights to an NPU** — `PackedConstantHandling.KEEP_PACKED` keeps a packed weight in the traced
graph, `BlockQuantMatmulFusionPass` folds every `x · Wᵀ` over one, and the StableHLO export emits the
dequantize-then-`batch_matmul` chain with `i8` / `i4` codes and `bf16` scales inline. Torq 2.3.0
(`--iree-input-type=stablehlo-torq`) compiles it to its `mixed_quant_matmul` kernel: the whole FunctionGemma 270M
decode step runs on an SL2610 NPU in one dispatch at 162 ms per step, with the same next token as the f32
reference. Weights from the GGUF loader (off-heap) export the same as heap ones.
- **StableHLO and TOSA passes** — the `STABLE_HLO` phase of a target optimizer rewrites the emitted module, and
`DivideByBroadcastPass`, `RankTwoWeightMatmulPass` and `TosaFinish` prepare a module for StableHLO's own TOSA
legalization (Moonshine's streaming encoder runs on an NPU through `tosa-torq` in one dispatch). `Int4` maps to
`i4`.
- **Mixed precision per layer** — `dense(64, dtype = BF16) { }` and `stage("head", dtype = FP16) { }` build a layer
in another dtype behind a `PrecisionBoundary` that casts in and out (inference only). The generic
`dense<T>()` / `stage<T>()` overloads, which could only throw, are deprecated at `ERROR` level.
- **Re-encoding storages** — `repackTo(Dense(4))` decodes every packed encoding with a dequantizer bit-exactly, and
`requantizeTo` is the explicit lossy direction.
- **New ops and layers** — `convTranspose2d` with backward and StableHLO lowering, the `GroupNormalization` forward
pass, `sliceCopy {}`, and precision, recall and F1 metrics with binary, macro and micro averaging.
- **ONNX** — `LeakyRelu`, `Silu`, `Clip`, `BatchNormalization`, `Pad` and `ConvTranspose` import as executable ops,
and `OnnxModelReader.open` reads metadata up front and each tensor on demand.
- **Fixes** — the JVM tokenizer's `Split` regex follows Unicode character classes, and `randn` never returns
`Inf` / `NaN`.

See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release.

Expand All @@ -403,6 +359,17 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:

Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).

### Contributors (0.63.0)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the block-quant StableHLO export and its
fusion pass with the Torq board checks, the StableHLO-phase and TOSA preparation passes, mixed precision and
`repackTo` / `requantizeTo` (SKEEP-008), `convTranspose2d`, the ONNX op mappings and streaming reader,
`GroupNormalization`, `sliceCopy`, the tokenizer and `randn` fixes, and the release
- **Edwin** ([@EdinEd](https://github.com/EdinEd)) — the precision, recall and F1 metrics with the `Averaging`
strategy and the shared confusion-matrix accumulator (#1222)
- [@S0L0-R00T-DEV](https://github.com/S0L0-R00T-DEV) — the removal of the unused `TensorDefineDslImpl.create()`
stub (#1471)

### Contributors (0.62.4)

- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the Accelerate/BLAS routing of the JVM's
Expand Down Expand Up @@ -544,7 +511,7 @@ from it). Ready-to-paste BibTeX:
author = {Harakal, Michal and {SKaiNET-developers contributors}},
title = {{SKaiNET}: A {Kotlin} Multiplatform {AI} Framework},
year = {2026},
note = {Version 0.62.4. MIT License},
note = {Version 0.63.0. MIT License},
howpublished = {\url{https://github.com/SKaiNET-developers/SKaiNET}}
}
```
Expand Down
2 changes: 1 addition & 1 deletion docs/antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ asciidoc:
framework_name: SKaiNET
# Current SKaiNET release — bump once per release; referenced as
# {skainet_version} in dependency snippets (blocks need subs="attributes+").
skainet_version: 0.62.4
skainet_version: 0.63.0
ksp_version: 2.2.21-2.0.5
dokka_version: 2.1.0
asciidoctorj_version: 3.0.0
Loading
Loading