release: prepare 0.38.0

CHANGELOG.md

@@ -2,8 +2,23 @@
 
 ## [Unreleased]
 
+## [0.38.0] - 2026-07-30
+
 ### Added
 
+- **Narrow-float (BF16 + FP16) weights kept packed.** A shared `NarrowFloatCodec` layer
+  (`Bf16Codec`/`Fp16Codec`) plus `NarrowFloatDenseTensorData`/`Fp16DenseTensorData` let SafeTensors F16
+  and GGUF F16/BF16 weights load `KEEP_NATIVE` — two bytes per element at rest instead of widening to
+  FP32 at load. `DefaultCpuOpsJvm` dispatches to format-specific matmul kernels by codec, so the packed
+  weight reaches the kernel rather than a widened copy. Narrow floats are a *storage* width only:
+  kernels widen to f32 lanes, accumulate in f32, and narrow on store.
+- **Native (FFM) FP16 matmul kernel.** `skainet_fp16_matmul` joins the existing BF16 kernel in
+  `skainet-backend-native-cpu`, wired through `NativeKernelProvider.matmulFp16()`. Until now the
+  provider carried `matmulBf16` but no FP16 counterpart, so BF16 resolved to the native kernel at
+  priority 100 while FP16 silently cascaded to the JVM Panama kernel at 50 — which read as a slow
+  kernel and was a missing one. `KernelProvider.supports` gains the matching `"Float16"` arm, absent
+  while every other matmul dtype was present.
+
 - **First-class dynamic dimensions (`Dim`).** A new `sk.ainet.lang.tensor.Dim` vocabulary makes "dynamic
   extent" explicit instead of an overloaded `-1`: `Dim.DYNAMIC` is a reserved sentinel (`Int.MIN_VALUE`)
   **distinct from reshape's `-1` = infer**, with the dynamic-aware shape arithmetic (`concat`, `compatible`,
@@ -53,6 +68,49 @@
   (`VoidTensorOps.calculateConcatShape`) and the emitter (`ShapeOperationsConverter` concat) now keep the
   concatenated axis dynamic when any operand's extent there is dynamic, instead of numerically summing it
   (which turned a growing cache `? ++ 1` into a bogus static `0`).
+- **Narrow-float weights transpose for free.** `NarrowFloatInputMajorTensorData` stores a rank-2 narrow
+  weight input-major, so the `[out, in]` → `[in, out]` transpose that `Linear.onForward` performs on
+  every call is a zero-copy reinterpretation of the same buffer. Projections are stored `[out, in]` but
+  the narrow matmul dispatch needs `[in, out]`, and transposing a row-major narrow tensor previously
+  walked it elementwise through boxed `get()` and widened to FP32 — 4.4 s for a 4096x11008 projection,
+  per weight, per token, which made `KEEP_NATIVE` slower than not using it. A row-major narrow buffer
+  deliberately still takes the generic path: swapping its shape would silently yield a different matrix
+  rather than the transpose.
+- **Native narrow-float kernels read the weight once per matmul.** Both the BF16 and FP16 kernels tile
+  the `j` dimension at `m > 1` and widen each weight row once per tile, instead of walking the whole
+  weight matrix once per row of the input. Accumulation into any output element stays `p` ascending, so
+  results are bit-identical to the previous formulation. At `m == 1` both keep the straight pass — there
+  is nothing to amortize and tiling costs ~15% there.
+- **`Fp16Codec.decode` is straight-line.** The subnormal arm no longer renormalizes in a data-dependent
+  loop; binary16 subnormals are `mant * 2⁻²⁴` with both factors exact in FP32, so one multiply lets the
+  hardware renormalize. A NaN now decodes **quiet**, matching `encode` (which already never emitted a
+  signaling binary16 NaN) and the hardware conversion the JVM kernel uses. This changes 1022 patterns —
+  the signaling NaNs — and nothing else; without it the JVM and every other target would disagree on
+  them.
+
+### Fixed
+
+- **FP16 matmul was 2–18x slower than the FP32 SGEMM it replaces.** The cause was dispatch, not
+  arithmetic: `NativeKernelProvider` had no `matmulFp16()`, so FP16 fell through to the JVM kernel while
+  BF16 ran natively. Head to head the two JVM kernels are within ~15% of each other. FP16 is now
+  1.5–1.7x *faster* than FP32.
+- **CI and supply-chain hardening.** Least-privilege permissions on the build workflow, the docs MathJax
+  npm install pinned by version, and a `logback-classic` bump.
+
+### Performance
+
+Measured on an i7-9750H (AVX2), OpenJDK 21, median ms per call, `4096x11008` projection:
+
+| | batch 1 | batch 16 |
+|---|---|---|
+| FP32 SGEMM | 108.7 | 271.1 |
+| BF16 | 57.4 | 143.5 |
+| FP16 | 71.2 | 159.3 |
+
+Both narrow formats now beat the FP32 SGEMM — BF16 by 1.8–1.9x, FP16 by 1.5–1.7x. Note these kernels
+are compute-bound on the FMA chain at batch 16, not bandwidth-bound: cutting weight traffic 16x bought
+only 9–19%, so the next win is a blocked microkernel or `bfdot`/`bfmmla` on ARMv8.6-A+, not more layout
+work.
 
 ## [0.37.0] - 2026-07-25
 

README.md

@@ -43,7 +43,7 @@ Add the core dependencies (Gradle Kotlin DSL):
 ```kotlin
 dependencies {
     // Recommended: import the umbrella BOM and drop versions on the engine modules.
-    implementation(platform("sk.ainet:skainet-bom:0.37.0"))
+    implementation(platform("sk.ainet:skainet-bom:0.38.0"))
 
     implementation("sk.ainet.core:skainet-lang-core")
     implementation("sk.ainet.core:skainet-backend-cpu")
@@ -287,20 +287,19 @@ val withoutLabel = dataPipeline<RawDataset>()
 
 ---
 
-## What's New in 0.37.0
+## What's New in 0.38.0
 
-- **`Lstm` layer** — single-layer, batch-first LSTM built from existing primitives only (no new `TensorOps` op, traces to StableHLO without a dedicated converter), with `torch.nn.LSTM`-compatible gate order and an explicit caller-owned `LstmState` + `step()` API for transducer prediction networks.
-- **Training essentials** — `Dropout` now performs real inverted dropout under a training-phase context (it was an identity placeholder), optimizers expose a mutable `lr` plus a `linearWarmupCosineDecay` LR schedule, `Linear` supports bias-less projections (`nn.Linear(bias=False)` equivalent) and is `open` for LoRA-style adapters.
-- **Attention scale fix** — `scaledDotProductAttention` at its default scale multiplied every score by zero on the CPU backend, collapsing softmax to a uniform average; it now resolves to `1/sqrt(headDim)` as documented.
-- **Autograd correctness** — `CrossEntropyLoss` no longer detaches the tape (gradients reached the predictions in neither target path), and `softmax`/`logSoftmax`/`variance` backward now work for rank ≥ 3.
-- **Android native IO** — `skainet-io-core` and `skainet-io-safetensors` gain `androidNative` targets (arm64 and arm32).
-- **Reproducible, hardened CI** — every GitHub Action pinned to a commit hash, the docs Docker image pinned by digest with exact npm package versions, and `allTests` split into parallel per-target jobs to end OOM flakes.
+- **Streaming KV-cache decode (dynamic dimensions)** — a first-class `Dim` vocabulary makes "dynamic extent" explicit instead of an overloaded `-1`, and the StableHLO emitter renders it as an MLIR `?`. One compiled vmfb now serves every autoregressive decode step with a growing cache, instead of one fixed cache length. Verified end-to-end: the full FunctionGemma `with_past` decode graph and the Moonshine v2 decoder (dynamic self *and* cross caches) self-compile from the DSL to a CPU vmfb — graphs that could not be compiled before. Static graphs are emitted byte-for-byte unchanged.
+- **Narrow-float (BF16 + FP16) weights kept packed** — SafeTensors F16 and GGUF F16/BF16 weights load `KEEP_NATIVE`, two bytes per element at rest instead of widening to FP32, and reach format-specific matmul kernels still packed. Narrow floats are a storage width only: kernels widen to f32 lanes and accumulate in f32.
+- **Both narrow formats now beat the FP32 SGEMM** — BF16 by 1.8–1.9x, FP16 by 1.5–1.7x on a 4096x11008 projection. Getting there took a zero-copy transpose for input-major weights (the per-token transpose previously widened the tensor elementwise, 4.4 s per projection), a native FFM FP16 kernel to match the existing BF16 one, and tiling both kernels so the weight is read once per matmul rather than once per input row.
+- **Allocation-free shape-only tracing** — `VoidTensorOps` propagates shapes through a `ShapeOnlyTensorData` that allocates no backing buffer, so a dynamic extent flows through a whole decode trace instead of throwing on a negative-size allocation.
 
-### Previously, in 0.36.0
+### Previously, in 0.37.0
 
-- **Kotlin 2.4.0 toolchain** — KSP 2.3.10 and Dokka 2.2.0 aligned to the new compiler. No public API changes.
-- **`permute` replay fix (`ComputeGraphExecutor`)** — a traced `permute(t, axes)` now replays with its recorded axes instead of being dispatched as a plain last-two-dims transpose.
-- **REUSE / SPDX license-compliance setup** — `REUSE.toml` + `LICENSES/`, a CI compliance workflow, and a REUSE status badge.
+- **`Lstm` layer** — single-layer, batch-first LSTM built from existing primitives only, with `torch.nn.LSTM`-compatible gate order and a caller-owned `LstmState` + `step()` API.
+- **Training essentials** — real inverted `Dropout`, mutable optimizer `lr` plus `linearWarmupCosineDecay`, bias-less and `open` `Linear`.
+- **Attention scale fix** — `scaledDotProductAttention` at its default scale multiplied every score by zero on the CPU backend; it now resolves to `1/sqrt(headDim)` as documented.
+- **Autograd correctness** — `CrossEntropyLoss` no longer detaches the tape, and `softmax`/`logSoftmax`/`variance` backward now work for rank ≥ 3.
 
 See [CHANGELOG.md](CHANGELOG.md) for details and the full release history.
 
@@ -326,6 +325,11 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:
 
 Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).
 
+### Contributors (0.38.0)
+
+- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — dynamic tensor dimensions for streaming KV-cache decode (#891), shared narrow-float BF16/FP16 layer (#886), zero-copy transpose for input-major narrow weights (#895), native FP16 matmul kernel (#896), read-once weight tiling in the native narrow kernels (#897)
+- **[@MacOS](https://github.com/MacOS)** — least-privilege permissions on the build workflow (#899), MathJax npm install pinned by version (#889)
+
 ### Contributors (0.37.0)
 
 - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — `Lstm` layer (#824), `Dropout` masking (#867), LR schedules (#866), optional/open `Linear` (#870, #875), SDPA scale fix (#880), autograd fixes (#877), `argMax` DAG spec (#878), tokenizer + `gather` fixes (#879), Android native IO targets (#836, #842, #845)

docs/antora.yml

@@ -15,7 +15,7 @@ asciidoc:
     framework_name: SKaiNET
     # Current SKaiNET release — bump once per release; referenced as
     # {skainet_version} in dependency snippets (blocks need subs="attributes+").
-    skainet_version: 0.37.0
+    skainet_version: 0.38.0
     ksp_version: 2.2.21-2.0.5
     dokka_version: 2.1.0
     asciidoctorj_version: 3.0.0

gradle.properties

@@ -1,5 +1,5 @@
 GROUP=sk.ainet.core
-VERSION_NAME=0.37.0
+VERSION_NAME=0.38.0
 POM_DESCRIPTION=SKaiNET
 
 POM_URL=https://github.com/SKaiNET-developers/skainet/