diff --git a/CHANGELOG.md b/CHANGELOG.md index 87961b0..f4c5702 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,11 +9,32 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] -Requires **SKaiNET engine 0.38.0** (narrow-float codec, `Fp16DenseTensorData`, FP16 matmul -kernels, codec-driven dispatch — engine PR #886). +## [0.38.0] — 2026-07-31 + +Ships against **SKaiNET engine 0.38.0**, which adds first-class dynamic tensor shapes (`Dim`) plus +the narrow-float codec (`Fp16DenseTensorData`, FP16 matmul kernels, codec-driven dispatch — engine +PR #886). Two headlines: **Moonshine v2 streaming ASR authored end-to-end in the SKaiNET NN DSL** +(the last vendor-ONNX graph is gone) and **narrow-float `KEEP_NATIVE` weights** across the LLM loaders. ### Added +- **Moonshine v2 — the complete streaming pipeline in the NN DSL**, self-compiled DSL → StableHLO → + IREE with no vendor neural binaries (`skainet-transformers-inference-moonshine`): + - **Audio frontend** (`MoonshineV2Frontend`): CMVN → `asinh` compression → filterbank matmul → + SiLU → two causal `Conv1d(k5,s2)` — the last vendor-ONNX graph, now DSL-authored (bit-exact vs + `frontend.onnx`, cos > 0.999). + - **Encoder** (position-free sliding-window local attention) and **adapter** (learned absolute + positional embedding, pos-embed add only) bridging the position-free memory to the decoder. + - **Decoder** authored in the DSL, reusing the shared KV-cache decoder. +- **True-dynamic KV-cache decode graphs.** `MOONSHINE_V2_TRUE_DYNAMIC` / `GEMMA_TRUE_DYNAMIC` trace + the cache seq dim as a real dynamic extent (`Dim.DYNAMIC`), so one compiled vmfb serves every + autoregressive position instead of a fixed-shape re-decode. Requires engine 0.38.0's `Dim`. +- **Fixed-max-pad cross-attention mask** for streaming decode: pad the encoder memory to a fixed MAX + and mask the padding, so one prefill + one `with_past` pair serve any encoder length ≤ MAX while the + self-cache stays dynamic (growing). `transformer-core`'s `MultiHeadAttention` gains an optional + trailing `crossMask` (default `null` → byte-identical for existing callers). +- **Gemma row-dequant of the packed `token_embd`** in the shared `Embedding`, cutting host memory at + load. - **FP16 KEEP_NATIVE on the SafeTensors path.** `DecoderSafeTensorsLoader` gains the F16 arm that BF16 has had since 0.25.0: with a `DTypePolicy` admitting FP16 (`Require(FP16)`, `Prefer(FP16)`, or `OneOf` containing FP16) it stops widening F16 tensors and wraps the @@ -68,6 +89,9 @@ kernels, codec-driven dispatch — engine PR #886). prevent. They now declare `keepNative = emptySet()` and reject it. **Callers relying on the old acceptance must switch to `Prefer(BF16)`** (a soft constraint, which still passes) until those chains grow a KEEP_NATIVE path. +- Moonshine v2 encoder sliding-window off-by-one (was cos 0.991 vs ONNX); the v2 config is set to the + real tiny-streaming dims; the adapter is pos-embed add only (no LayerNorm). +- kgemma heavy-trace test heap raised to 12 g, with honest skips. ## [0.36.1] — 2026-07-17 @@ -869,6 +893,7 @@ Version-aligned with **SKaiNET 0.21.0**. Last published transformers release before the engine-aligned version line. See `git log v0.16.0..0.18.0` for details. +[0.38.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.38.0 [0.36.1]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.36.1 [0.36.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.36.0 [0.31.0]: https://github.com/SKaiNET-developers/SKaiNET-transformers/releases/tag/0.31.0 diff --git a/README.md b/README.md index 8d2cb3b..ec57349 100644 --- a/README.md +++ b/README.md @@ -106,30 +106,36 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.36.1** (against **SKaiNET 0.36.0**) — a patch on 0.36.0 with two -additions, both purely additive for existing consumers. - -**BGE embedding models** (`BAAI/bge-small-en-v1.5` and siblings) now run on the BERT DSL path: - -- **CLS pooling** — `BertPooling { MEAN, CLS }`, auto-detected from the sentence-transformers - `1_Pooling/config.json`. Pooling stays outside the traced graph, so OPTIMIZED mode and - StableHLO export are unaffected. -- **Query/document asymmetry** — `EmbeddingModel` gains `embedQuery` / `embedDocument` / - `embedDocuments`, and `PrefixedEmbeddingModel` applies the per-model retrieval instruction - prefixes (E5 `query: `/`passage: `, BGE query instruction) that these models need to score - correctly. `fromHuggingFace` wires them automatically. -- Design notes: [embedding-model-coverage](docs/specs/embedding-model-coverage.md). - -**Beam search** for the T5 decoder and the vec2text inversion loop — vec2text's main quality lever: - -- `T5Runtime.generateBeam(...)` does token-level beam over the decoder, returning candidates - best-first by length-normalized log-probability. -- `Vec2TextInverter.invert(..., sequenceBeamWidth, tokenBeams)` adds a sequence-level beam that - keeps several hypotheses across correction rounds, ranked by cosine similarity to the target - embedding. -- Both are **off by default** — width 1 keeps the existing greedy behaviour, so this is a - drop-in upgrade. On the round-trip test's example sentence, one correction step with beam - (sequence width 3, token beams 3) improves cosine **0.765 → 0.818** over greedy. +The current release is **0.38.0** (against **SKaiNET 0.38.0**) — the release that completes the +**Moonshine v2 streaming ASR entirely in the SKaiNET NN DSL** and adds **narrow-float weights**. + +**Moonshine v2 — the whole pipeline is now DSL-authored and self-compiled** (DSL → StableHLO → IREE, +no vendor neural binaries) in `skainet-transformers-inference-moonshine`: the **audio frontend** +(CMVN → asinh → filterbank → SiLU → two causal `Conv1d`), the position-free sliding-window +**encoder**, the learned-positional-embedding **adapter**, and the KV-cache **decoder**. The last +vendor-ONNX graph — the frontend — is gone. + +- **Streaming decode over a growing cache.** The decoder exports **true-dynamic** KV-cache graphs + (`Dim.DYNAMIC`), so one compiled artifact serves every autoregressive position instead of a fixed + re-decode. A **fixed-max-pad cross-attention mask** lets one prefill + one `with_past` pair serve + any encoder-memory length ≤ MAX. `transformer-core`'s `MultiHeadAttention` gains an optional + trailing `crossMask` (default null — byte-identical for existing callers). + +**Narrow-float `KEEP_NATIVE` weights** — keep FP16/BF16 packed on the SafeTensors and GGUF paths +instead of widening to FP32 at load (half the weight bytes at rest), and relay matmul weights +input-major so the per-forward `.t()` becomes a zero-copy view: **BF16 1.8–1.9×, FP16 1.5–1.7×** over +FP32, and a 4.4 s-per-projection widening on `ffn_up` 8B removed. Wired across +llama/qwen/gemma/apertus/voxtral through the `DTypePolicyValidation` policy surface. + +**Gemma** now **row-dequants the packed `token_embd`** in the shared `Embedding`, cutting host memory +at load. + +This is the first release on **SKaiNET engine 0.38.0**, which ships first-class dynamic tensor shapes +(`Dim`) — the capability the true-dynamic decode export is built on. + +It builds on **0.36.1**, which added **BGE embedding models** on the BERT DSL path (CLS pooling + +retrieval prefixes) and **beam search** for the T5 decoder and the vec2text inversion loop — both +additive, drop-in for existing consumers. It builds on **0.36.0**, in which **BERT became completely defined in the SKaiNET NN DSL** and the deprecated hand-coded eager BERT stack was **removed (BREAKING)**: @@ -188,7 +194,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.36.1")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.38.0")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/gradle.properties b/gradle.properties index 8d70c4f..ad7c1b2 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.36.1 +VERSION_NAME=0.38.0 POM_DESCRIPTION=SKaiNET-transformers