[michalharakal]

Against engine 0.63.0, which adds block-quantized weights to StableHLO export (SKaiNET-developers/SKaiNET#1478, SKaiNET-developers/SKaiNET#1479) and fixes Unicode Split in the JVM tokenizer (SKaiNET-developers/SKaiNET#1477).

Headline: one decode step of a KV-cache decoder as an NPU graph with block-quantized weights. FunctionGemma 270M's decode step runs on the Torq NPU of an SL2610 in one dispatch: 162 ms per step, against about 610 ms per token on the CPU path, with the same top-1 (#529). Plus the LFM2 model family and its chat frontend (#527).

Added

• DecodeStepExport (llm-core, jvm):
  • traces a single-token step with a static past;
  • keeps Q8_0 / Q4_0 weights packed for BlockQuantMatmulFusionPass;
  • names every port.
• forwardEmbeddedWithPast on GemmaModel and DecoderKvModel.
• llm-inference:lfm2: the dense LFM2 hybrid (short conv + GQA), verified checkpoint loading, and a fixed-shape StableHLO export. Validated against LFM2.5-350M.
• llm-runtime:klfm2: the LFM2 chat frontend. Pinned metadata, verified chat templates, greedy streaming generation, and an IREE path.

Changed

• The with-past types are shared in sk.ainet.lang.nn.dsl.decoder: RopeTables, RopeCosSin, ChunkContext, KvStepOutput, buildRopeCosSin. They replace the Gemma and DecoderKvModel nested classes. Code that constructs ChunkContext directly needs one change; everything else is additive.
• Engine pin SKaiNET 0.63.0. The llama ForwardScopeSteadyStateTest now allows 4 ulp of each step's largest logit, because 0.63.0 changes the accumulation order.
• Dependencies: Ktor 3.6.1 (build(deps): Bump ktorClientCore from 3.6.0 to 3.6.1 #525), kotlinx-io 0.9.2 (build(deps): Bump org.jetbrains.kotlinx:kotlinx-io-core from 0.9.1 to 0.9.2 #526), logback 1.6.6 (build(deps): Bump ch.qos.logback:logback-classic from 1.6.5 to 1.6.6 #523).

Release steps

• Commit: version bump to 0.64.0 (gradle.properties, README, docs BOM snippets, LFM2 README engine line) and the CHANGELOG entry.
• After merge: tag 0.64.0 on develop.

🤖 Generated with Claude Code