From 185f847878477dbfc5dc51e7ac73b5cbe101ef9b Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Wed, 11 Nov 2026 14:20:59 +0100 Subject: [PATCH] docs(skeep): SKEEP-003 status matches the code (#1500) An audit at 0.64.0 found the old storage layer still in place beside the new one. The SKEEP goes back to Accepted and gains a status table per acceptance criterion and per open decision, linking the issues that track them. 003a, SKEEP-002, memory-model.adoc and the TensorStorage KDoc lose statements that later work made untrue. Co-Authored-By: Claude Fable 5.2 --- CHANGELOG.md | 4 ++ .../ROOT/pages/explanation/memory-model.adoc | 9 +-- .../002-android-offheap-tensor-storage.adoc | 33 +++++------ .../pages/003-unified-tensor-storage.adoc | 57 ++++++++++++++++++- ...03a-placement-and-planning-resolution.adoc | 8 +-- docs/modules/skeep/pages/index.adoc | 6 +- .../lang/tensor/storage/TensorStorage.kt | 13 +++-- 7 files changed, 95 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ac786a7..4652f488 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## [Unreleased] +### Changed + +- **SKEEP-003 is back at `Accepted`, with an implementation-status table** (docs, #1500): an audit at 0.64.0 found the new storage model in use and the old layer (`TensorStorage`, `BufferHandle`, `TensorData` downcasts in the ops) still in place, so three of the five acceptance criteria are open. The SKEEP now says which, and what tracks each (#932). SKEEP-003a no longer lists #1148 as open, SKEEP-002's status records the Q4_K / Q6_K mapped staging of #1190 and its device measurement, `memory-model.adoc` drops a limit that #1190 removed, and the `TensorStorage` KDoc stops telling new code to target it. + ## [0.64.0] - 2026-11-11 Headline: **every differentiable op has a backward rule, and mixed precision is finished: it trains, exports without diff --git a/docs/modules/ROOT/pages/explanation/memory-model.adoc b/docs/modules/ROOT/pages/explanation/memory-model.adoc index 8aa0e8d7..c6b72d02 100644 --- a/docs/modules/ROOT/pages/explanation/memory-model.adoc +++ b/docs/modules/ROOT/pages/explanation/memory-model.adoc @@ -204,7 +204,7 @@ ARMv8.2 Cortex-A55 reference board. [cols="2,3",options="header"] |=== | Delivered | Notes -| `Storage` / `Scope` / `TensorView` / `Format` / `Layout` | All targets; `TensorData` implementations are façades over views +| `Storage` / `Scope` / `TensorView` / `Format` / `Layout` | All targets; every `TensorData` family used by the kernels exposes a view. Scoped activations (`StorageFloatTensorData`) sit on a `Storage`; the other classes still own their bytes and wrap the view around them (xref:skeep:003-unified-tensor-storage.adoc[SKEEP-003], implementation status) | `KernelKey` dispatch for matmul | Reference kernel plus packed and ternary kernel packs | Ternary encodings and `bitnet_gemv` | `TQ1_0`, `TQ2_0`, BitNet b1.58; reference kernel plus a NEON pack on Android | Memory plan, device profiles, fit check | `skainet-plan`, mobile/desktop/native profiles, two-pool device check @@ -247,9 +247,10 @@ anything is loaded rather than after an OOM. Known limits, stated rather than implied: -* *Packed weights still reach the managed heap under mapped staging.* The packed matmul SPI takes -`ByteArray`s, so a buffer-aware kernel is future work. Mapping therefore lifts the Android heap -ceiling for dense checkpoints, not yet for a Q4_K_M one. +* *Some packed weights still reach the managed heap under mapped staging.* Q4_K and Q6_K weights +stay in the file mapping (`BufferPackedTensorData`, read by row-major kernels through a direct +buffer, #1190), so a Q4_K_M checkpoint larger than the Android heap loads. The other packed +formats are still handed to their kernels as `ByteArray`s. * *Whole-file mapping only.* Files larger than 2 GB are refused rather than mapped in windows. * *Graph-level planning* (allocating a whole forward pass at once) stays in the downstream compiler by decision (#1134). Device placement itself was resolved in 0.49.0: `AllocationResolver` diff --git a/docs/modules/skeep/pages/002-android-offheap-tensor-storage.adoc b/docs/modules/skeep/pages/002-android-offheap-tensor-storage.adoc index 40f44773..44cd5988 100644 --- a/docs/modules/skeep/pages/002-android-offheap-tensor-storage.adoc +++ b/docs/modules/skeep/pages/002-android-offheap-tensor-storage.adoc @@ -232,14 +232,13 @@ keeps working unchanged and the overload is purely opt-in. Expected consumers: (packed quant blocks) and a synthesized SafeTensors file (dense F32/F16/BF16) produce bit-identical tensors under either placement. -== Implementation status (updated 2026-08-26, for 0.49.0) +== Implementation status (updated 2026-11-11, at 0.64.0) Tracked by SKEEP-003's M2 slice https://github.com/SKaiNET-developers/SKaiNET/issues/1038[#1038]. This SKEEP stays *Draft*: the mechanism is in `develop` (and since 0.49.0 the loader speaks one `WeightForm` — the legacy -`StagingPolicy`/`QuantPolicy`/`WeightOrientation` axes were removed in #1159), the acceptance -criteria that need a physical device are not met yet, and one of them cannot be met until an -unrelated contract is fixed. +`StagingPolicy`/`QuantPolicy`/`WeightOrientation` axes were removed in #1159), one acceptance +criterion has a device measurement behind it, and the others are not recorded. Landed: @@ -257,20 +256,22 @@ Landed: `AndroidGguf.fits(context, path, ctx)` answers from the GGUF header before a byte of payload is read, naming the pool that runs out and what to do about it. +* *Phase 4 for Q4_K and Q6_K* — `BufferPackedTensorData` keeps a packed weight in the file + mapping, and row-major kernels read it through a direct `ByteBuffer` + (https://github.com/SKaiNET-developers/SKaiNET/issues/1189[#1189], PR #1190). Measured on a Pixel 8a with the 256 MB heap cap + (https://github.com/SKaiNET-developers/SKaiNET/issues/1130[#1130]): Qwen2.5-1.5B Q4_K_M, 1.0 GB on disk, loads with 566 KB of + weights on the managed heap and decodes at 153 ms per step; before, it ran out of memory at + load. That is the acceptance criterion "a Q4_K model larger than the heap loads and generates". + Not landed, and why: -* *Packed weights still reach the managed heap.* Mapped staging serves dense F32 tensors as - zero-heap views, but every quantized tensor is still handed to its kernel as a `ByteArray`, - because the packed matmul SPI takes arrays. That is Phase 4, and it is blocked on - https://github.com/SKaiNET-developers/SKaiNET/issues/973[#973] — the packed-quant byte-order - contract — since a buffer-aware kernel first needs an unambiguous answer to *which* byte order it - is reading. Until then the heap ceiling is lifted for dense checkpoints, not for a Q4_K_M one, - which is exactly the acceptance criterion below that remains open. -* *The device numbers.* "≤ 40 MB managed heap for SmolLM2-135M Q8_0" and "a ~600 MB Q4_K model - loads on a 256 MB heap" are measurements on a physical device; the repository's CI has none. They - belong to the `skainet-decode` sample in SKaiNET-transformers, which owns a model and a device - lane. `androidHostTest` covers what can be proven without hardware: the Android compilation - builds, maps, loads, and produces bit-identical tensors under either staging. +* *Other packed formats still reach the managed heap.* Q8_0, Q4_0, Q5_0, Q5_1, Q5_K and the + ternary encodings are handed to their kernels as `ByteArray`s under mapped staging; only Q4_K + and Q6_K have the buffer-reading kernels. +* *The SmolLM2-135M Q8_0 heap figure and the mapped-versus-heap throughput comparison* are not + recorded. Q8_0 is one of the formats above, so the "less than 40 MB" criterion cannot hold yet. +* *Whole-file mapping only.* A file larger than 2 GB is refused instead of being mapped in + windows (https://github.com/SKaiNET-developers/SKaiNET/issues/1422[#1422] asks for the same in the IRPA writer). == Risks diff --git a/docs/modules/skeep/pages/003-unified-tensor-storage.adoc b/docs/modules/skeep/pages/003-unified-tensor-storage.adoc index 86a75a0c..09d7bc47 100644 --- a/docs/modules/skeep/pages/003-unified-tensor-storage.adoc +++ b/docs/modules/skeep/pages/003-unified-tensor-storage.adoc @@ -1,11 +1,64 @@ = SKEEP-003: Unifying the tensor storage model — one byte-owner, enforced ownership, coherent dtype/encoding :description: SKaiNET proposal to converge the TensorData and TensorStorage layers into one storage model with real ownership, a single view mechanism, and dtype/encoding coherence. -Status: Implemented (accepted 2026-08-22; M0/M1/M2 + the P7/P8 resolution shipped in 0.49.0) + +Status: Accepted (2026-08-22; M0/M1/M2 and the placement resolution shipped in 0.49.0; the removal of the old layer is open, see <>) + Audience: SKaiNET maintainers and contributors + Created: 2026-08-10 + Tracking issue: https://github.com/SKaiNET-developers/SKaiNET/issues/932[#932] (umbrella) → https://github.com/SKaiNET-developers/SKaiNET/issues/1001[#1001 M0] · https://github.com/SKaiNET-developers/SKaiNET/issues/1002[#1002 M1] · https://github.com/SKaiNET-developers/SKaiNET/issues/1003[#1003 M2] + -Implemented: the model as built is documented in xref:ROOT:explanation/memory-model.adoc[The memory model] and xref:ROOT:explanation/packed-weight-layout.adoc[Packed weight layout]; the proposal and milestone PRD that led there are in the pull requests and issues linked below (#932, #1001, #1002, #1003). +As built: the model is documented in xref:ROOT:explanation/memory-model.adoc[The memory model] and xref:ROOT:explanation/packed-weight-layout.adoc[Packed weight layout]; the proposal and milestone PRD that led there are in the pull requests and issues linked below (#932, #1001, #1002, #1003). + +[#implementation-status] +== Implementation status (audited 2026-11-11, at 0.64.0) + +The model is built and in use on every target: `Storage`, `TensorView`, `Format`, `Scope`, the +memory plan, the tracer, mapped staging. The old layer it was meant to replace is still in the +code, so this proposal is back at `Accepted` until the open rows below are closed. The umbrella +https://github.com/SKaiNET-developers/SKaiNET/issues/932[#932] lists the work. + +[cols="3,1,3",options="header"] +|=== +| Acceptance criterion | State | Where it stands + +| One abstraction owns bytes, ownership enforced +| Partly +| Enforcement is tested (`StorageTest`, `ScopeTest`, `TensorViewTest`, `JvmStorageTest`). `TensorStorage` and `BufferHandle` still exist beside `Storage`, and the streaming readers' `loadTensorStorage*` return the old type (https://github.com/SKaiNET-developers/SKaiNET/issues/1502[#1502], https://github.com/SKaiNET-developers/SKaiNET/issues/1507[#1507]). + +| One view mechanism +| Partly +| `OneViewMechanismTest` covers slice, transpose and unsqueeze on the new view. The public `slice()` family still returns the deprecated `SlicedTensorView` (https://github.com/SKaiNET-developers/SKaiNET/issues/1502[#1502]). + +| A packed tensor reports FP32 and its encoding; no dispatch site downcasts to a concrete `TensorData` class outside the data package +| Not met +| The first half holds (`FormatTest`). About 130 downcasts remain in the CPU, Accelerate and wasm ops, and matmul reaches `KernelKey` dispatch only as its last fallback (https://github.com/SKaiNET-developers/SKaiNET/issues/1501[#1501] for the guard, https://github.com/SKaiNET-developers/SKaiNET/issues/1507[#1507] for the migration). + +| No kernel regression against a baseline +| Partly +| The benchmarks exist; the recorded baseline was deleted with `docs/design/` in #1106 (https://github.com/SKaiNET-developers/SKaiNET/issues/1501[#1501]). + +| Flat direct memory under a forward scope on the JVM +| No test +| Slab reuse is tested (`ScopedOpOutputsTest`), direct memory only under the GC arena (`MemSegArenaLeakTest`) (https://github.com/SKaiNET-developers/SKaiNET/issues/1501[#1501]). +|=== + +Decisions 1, 2 (naming and package), 6, 9, 11, 12 and 13 are delivered; `LogicalDType` is +deprecated and its removal is https://github.com/SKaiNET-developers/SKaiNET/issues/1502[#1502]. The rest: + +[cols="1,4,3",options="header"] +|=== +| # | Open part | Tracked by + +| 1 | Façades: only `StorageFloatTensorData` sits on a `Storage`; the other `TensorData` classes own their bytes and wrap a view around them on each read | https://github.com/SKaiNET-developers/SKaiNET/issues/1507[#1507] +| 2 | `sk.ainet.lang.graph.Layout` is not deprecated; `BlockOrderLayout` implements it now | SKEEP-004, which owns the compile-side layout +| 3 | `get()` on a packed view decodes; the debug-mode warning on hot paths does not exist | https://github.com/SKaiNET-developers/SKaiNET/issues/1503[#1503] +| 4 | The `(head, tail)` pair and its gather fallback exist (`WindowedKV`); only the reference attention consumes them | https://github.com/SKaiNET-developers/SKaiNET/issues/1504[#1504], outside this SKEEP +| 5 | `BlockSpec.activation` drives the eager dispatcher; the StableHLO export does not read it | https://github.com/SKaiNET-developers/SKaiNET/issues/1505[#1505], outside this SKEEP +| 7 | `SuspendingRandomAccessSource` exists in `skainet-io-core`; there is no remote module | https://github.com/SKaiNET-developers/SKaiNET/issues/1506[#1506], outside this SKEEP +| 8 | Axis labels on `Shape` | https://github.com/SKaiNET-developers/SKaiNET/issues/1503[#1503] +| 10 | Activation ids exist when tracing, not in eager debug mode | https://github.com/SKaiNET-developers/SKaiNET/issues/1503[#1503] +|=== + +Compiled parity (P7 below) was closed as superseded: IREE validation lives in the IREE-tools and +conformity repositories (#1148), and this repository has no eager-versus-compiled parity test. == Decision (2026-08-22) diff --git a/docs/modules/skeep/pages/003a-placement-and-planning-resolution.adoc b/docs/modules/skeep/pages/003a-placement-and-planning-resolution.adoc index 082c289d..9ef9a4e4 100644 --- a/docs/modules/skeep/pages/003a-placement-and-planning-resolution.adoc +++ b/docs/modules/skeep/pages/003a-placement-and-planning-resolution.adoc @@ -98,9 +98,9 @@ The carry work landed as the #1178/#1179/#1180 arc (0.49.0): `TensorRef` carries encoding and block order; `TensorSpec` metadata accessors follow the untyped precedent; the structural `skainet.tensor_layouts` module attribute and `ExternalParameterRef.blockOrder` declare the facts to the consumer; and `LayoutAssignmentPass` runs on the production path via -`HloGenerator.generate(target = …)`, populating the pre-built `ResolvedComputeGraph` seams. Until -the conformity pipeline exists (#1148, the future IREE-validation SKEEP), the carriage tests are -the metadata's reader. +`HloGenerator.generate(target = …)`, populating the pre-built `ResolvedComputeGraph` seams. The +conformity pipeline that reads the metadata lives outside this repository (#1148 was closed as +superseded); here, the carriage tests are the metadata's reader. == Where everything landed @@ -113,7 +113,7 @@ the metadata's reader. | `memoryScope` wired into creation | #1145 | PR #1156 | Op outputs through the scope + offset-aware SIMD | #1146, #1173 | landed (PRs #1168/#1175) | Layout/placement carriage to StableHLO | #1147 (#1178/#1179/#1180) | landed (PRs #1182/#1183/#1184) -| `.vmfb` parity acceptance run | #1148 | open (follow-up) +| `.vmfb` parity acceptance run | #1148 | closed as superseded 2026-09-02: IREE validation lives in the IREE-tools and conformity repositories; no eager-versus-compiled parity test in this repository |=== The SKEEP-003 items "`StorageSpec`, `MemoryPlanner` and the annotations are deleted unless wired" diff --git a/docs/modules/skeep/pages/index.adoc b/docs/modules/skeep/pages/index.adoc index 1083211b..9c2f381d 100644 --- a/docs/modules/skeep/pages/index.adoc +++ b/docs/modules/skeep/pages/index.adoc @@ -116,12 +116,12 @@ Every proposal should include: | Off-heap tensor storage on Android | xref:skeep:003-unified-tensor-storage.adoc[SKEEP-003] -| Implemented +| Accepted | Unifying the tensor storage model | xref:skeep:003a-placement-and-planning-resolution.adoc[SKEEP-003a] -| Resolved -| Placement, eager lifetime and graph-level planning — the P7/P8 answers +| Implemented +| Addendum to SKEEP-003: placement, eager lifetime and graph-level planning | xref:skeep:004-virtual-tensor-layout.adoc[SKEEP-004] | Draft diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/storage/TensorStorage.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/storage/TensorStorage.kt index e6b57530..e4af9e76 100644 --- a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/storage/TensorStorage.kt +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/tensor/storage/TensorStorage.kt @@ -8,9 +8,9 @@ import sk.ainet.lang.types.DType /** * Runtime descriptor for a tensor's backing memory. * - * [TensorStorage] is the main architectural type that replaces ad-hoc - * array passing between loaders, planners, and backends. It carries enough - * information to handle a tensor without inspecting its bytes: + * [TensorStorage] describes a tensor read from a file before it becomes a + * tensor: the streaming GGUF and SafeTensors readers return it. It carries + * enough information to handle a tensor without inspecting its bytes: * * - **What** the values mean: [logicalType] * - **How** they are stored: [encoding] @@ -18,9 +18,10 @@ import sk.ainet.lang.types.DType * - **Layout**: [shape], [byteOffset], [strides], [isContiguous] * - **Ownership**: via [buffer]'s [BufferHandle] subtype * - * Existing [sk.ainet.lang.tensor.data.TensorData] remains as a - * compatibility façade. New loaders, planners, and backends should target - * [TensorStorage] directly. + * This is the older of two storage descriptions. The model new code targets is + * [sk.ainet.lang.memory.Storage] (owns bytes) with + * [sk.ainet.lang.memory.TensorView] (interprets them); SKEEP-003 replaces this + * type with a view once the readers return one (#1507). */ public data class TensorStorage( val shape: Shape,