feat(dtype): narrow-float KEEP_NATIVE weights end to end

CHANGELOG.md

@@ -9,6 +9,66 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+Requires **SKaiNET engine 0.38.0** (narrow-float codec, `Fp16DenseTensorData`, FP16 matmul
+kernels, codec-driven dispatch — engine PR #886).
+
+### Added
+
+- **FP16 KEEP_NATIVE on the SafeTensors path.** `DecoderSafeTensorsLoader` gains the F16 arm
+  that BF16 has had since 0.25.0: with a `DTypePolicy` admitting FP16 (`Require(FP16)`,
+  `Prefer(FP16)`, or `OneOf` containing FP16) it stops widening F16 tensors and wraps the
+  on-disk 2-bytes-per-element buffer in `Fp16DenseTensorData`. The arm was missing only
+  because no such storage type existed. `DefaultCpuOpsJvm` matches `NarrowFloatTensorData`
+  and picks the kernel by codec, so an F16 checkpoint now stays near its on-disk footprint
+  instead of inflating ~2× as FP32. Covers LLaMA, Qwen, and Voxtral, which share this loader.
+- **Narrow-float KEEP_NATIVE on the GGUF path — `DTypePolicy` is honored there at all now.**
+  `DecoderGgufWeightLoader` accepts a `dtypePolicy` and keeps F16 / BF16 source tensors packed
+  instead of widening every one to FP32. `LlamaNetworkLoader`, `QwenNetworkLoader`, and
+  `VoxtralNetworkLoader` plumb the policy attached via `withDtypePolicy` down into it; before
+  this the GGUF branches constructed the loader without the policy and silently ignored it.
+  This is the KEEP_NATIVE GGUF path the 0.25.0 notes parked, and it is what makes
+  `Require(BF16)` real on GGUF.
+
+  The packed path mirrors the FP32 path's layout handling exactly: for rank 2 it swaps the
+  shape to `[cols, rows]` and **moves no bytes**. GGUF header dims are reversed relative to the
+  logical row-major shape, so the "column-major → row-major" step is a reinterpretation, not a
+  permutation (`DequantOps.transposeColumnMajorToRowMajor` returns its input unchanged). An
+  actual element transpose here would have handed the matmul kernel a silently transposed
+  weight matrix. The result is genuinely zero-copy — the on-disk buffer becomes the storage.
+
+### Changed
+
+- **`DTypePolicyValidation` capability model is per-format.** `validate(policy, loaderName,
+  keepNative: Set<DType>)` replaces the BF16-only `allowBf16Require: Boolean` (kept as a
+  `@Deprecated` overload). A caller declares which narrow-float formats its chain actually
+  hands through packed, and a `Require` naming one is accepted only by a chain that can honor
+  it. The boolean could express neither "keeps FP16 but not BF16" nor the empty case.
+
+  The two formats are tracked separately and never interchangeably: `Require(BF16)` still
+  widens F16 sources, and vice versa. Both are 2 bytes per element, so mis-tagging F16 bytes as
+  BF16 decodes to plausible-looking garbage rather than throwing. `DTypePolicyValidation
+  .keepsNative(policy, native)` is the single decision point both loader chains share, mirroring
+  the engine's `mapPolicyToNarrow` / `keepsNative`.
+- **`Require(FP16)` is now accepted** by `LlamaNetworkLoader`, `QwenNetworkLoader`, and
+  `VoxtralNetworkLoader` (both GGUF and SafeTensors), and **`Require(BF16)` is now accepted on
+  their GGUF paths**. Both previously threw.
+- **Binary-breaking (source-compatible): `DecoderGgufWeightLoader` constructors** gain a
+  trailing `dtypePolicy: DTypePolicy = DTypePolicy.Any`, which changes their JVM descriptors.
+  Kotlin and Java callers compile unchanged; already-compiled callers must be rebuilt.
+  Behaviour with the default is identical to before.
+
+### Fixed
+
+- **`GemmaNetworkLoader` and `ApertusNetworkLoader` no longer accept a `Require(BF16)` they
+  ignore.** Both have their own weight chains (`Gemma4WeightLoader` /
+  `Gemma4SafeTensorsWeightLoader`, `ApertusWeightLoader` / `ApertusSingleSafeTensorsLoader`)
+  which widen every narrow float to FP32 and have no KEEP_NATIVE path. Their SafeTensors
+  entrypoints nevertheless passed `allowBf16Require = true`, so `Require(BF16)` validated and
+  was then silently disregarded at load — the exact failure the eager validator exists to
+  prevent. They now declare `keepNative = emptySet()` and reject it. **Callers relying on the
+  old acceptance must switch to `Prefer(BF16)`** (a soft constraint, which still passes) until
+  those chains grow a KEEP_NATIVE path.
+
 ## [0.36.1] — 2026-07-17
 
 Patch on **0.36.0** (same SKaiNET engine 0.36.0). Two additions: **BGE embedding models** on the

README.md

@@ -368,6 +368,12 @@ See `llm-test/llm-test-java/src/test/java/.../KLlamaJavaToolCallingTest.java` fo
 
 ## What's new in 0.25.0
 
+> **Superseded (unreleased, engine 0.38.0).** The narrow-float limits described below are gone:
+> the GGUF chain now honors `DTypePolicy` and keeps F16/BF16 packed, `Require(FP16)` is accepted
+> alongside `Require(BF16)`, and the two formats are resolved independently. Conversely, Gemma
+> and Apertus now *reject* `Require(BF16)` — their own weight chains never honored it. See the
+> `[Unreleased]` section of [CHANGELOG.md](CHANGELOG.md).
+
 - **`DTypePolicy` on every `*NetworkLoader.fromGguf` / `.fromSafeTensors`
   entry.** A sealed `DTypePolicy` type (`Any | Require | Prefer | OneOf`,
   upstream of SKaiNET 0.25.0) is now accepted on every loader companion in

llm-core/api/jvm/llm-core.api

@@ -1,6 +1,9 @@
 public final class sk/ainet/apps/llm/DTypePolicyValidation {
 	public static final field INSTANCE Lsk/ainet/apps/llm/DTypePolicyValidation;
+	public final fun keepsNative (Lsk/ainet/lang/types/DTypePolicy;Lsk/ainet/lang/types/DType;)Z
+	public final fun validate (Lsk/ainet/lang/types/DTypePolicy;Ljava/lang/String;Ljava/util/Set;)V
 	public final fun validate (Lsk/ainet/lang/types/DTypePolicy;Ljava/lang/String;Z)V
+	public static synthetic fun validate$default (Lsk/ainet/apps/llm/DTypePolicyValidation;Lsk/ainet/lang/types/DTypePolicy;Ljava/lang/String;Ljava/util/Set;ILjava/lang/Object;)V
 }
 
 public abstract class sk/ainet/apps/llm/DecoderRuntime : sk/ainet/apps/llm/InferenceRuntime {

llm-core/src/commonMain/kotlin/sk/ainet/apps/llm/DTypePolicyValidation.kt

@@ -13,69 +13,114 @@ import sk.ainet.lang.types.FP32
  * generalised execution-side dtype constraint surface. Its own loaders
  * (`StreamingGgufParametersLoader.withPolicy`, `SafeTensorsParametersLoader.withPolicy`)
  * validate the policy at construction so callers fail fast on impossible
- * requirements.
+ * requirements; this validator is the same boundary for the transformer-repo
+ * loader chains (`DecoderGgufWeightLoader`, `DecoderSafeTensorsLoader`, …).
  *
- * The transformer-repo loaders (`LlamaNetworkLoader`, `QwenNetworkLoader`, …) ship
- * their own weight-loading chain on top of `DecoderGgufWeightLoader` /
- * `DecoderSafeTensorsLoader`. Those chains do not yet plumb `DTypePolicy` through
- * to the underlying tensor producers — that's a separate follow-up. In the
- * meantime, accepting the policy on the public surface lets consumers express
- * intent today, and this validator ensures we reject impossible requirements at
- * the same boundary SKaiNET's own loaders do.
+ * ## What a chain can promise
  *
- * Today the transformer-repo loaders only produce FP32 (after Q4/Q8/BF16/F16
- * dequant on the SafeTensors path; native quantization preservation on the GGUF
- * path). That matches the SKaiNET 0.25.0 `StreamingGgufParametersLoader`
- * validator. The BF16 KEEP_NATIVE SafeTensors path (`Require(BF16)`) is allowed
- * here even though the transformer-repo `DecoderSafeTensorsLoader` does not yet
- * honor it — when wired through, no API change is needed.
+ * Every chain produces FP32, so `Require(FP32)` always passes. A `Require` naming a
+ * **narrow float** (BF16 / FP16) is a promise that the weights reach the kernel in their
+ * on-disk 2-bytes-per-element layout — the KEEP_NATIVE path. Only a chain that actually
+ * implements KEEP_NATIVE for that format can honour it, so each caller declares its
+ * capability via [keepNative] rather than the validator guessing from the format alone.
  *
- * Throws [IllegalArgumentException] on `Require(target)` for targets we cannot
- * produce. `Any`, `Prefer`, and `OneOf` always pass.
+ * The two narrow formats are tracked **separately and never interchangeably**. They are
+ * different bit layouts at the same width, so mis-tagging F16 bytes as BF16 decodes to
+ * plausible-looking garbage instead of throwing. A chain that keeps BF16 native but
+ * widens F16 declares exactly that, and `Require(FP16)` against it fails loudly.
+ *
+ * Note that a `Require(BF16)` chain still **widens F16 sources to FP32** — the policy
+ * names the format to preserve, not a conversion target. Neither narrow format can be
+ * re-encoded into the other without a lossy round-trip, and the loaders do not try.
+ *
+ * `Any`, `Prefer`, and `OneOf` always pass: they are soft constraints that a chain is
+ * free to satisfy or ignore per tensor.
+ *
+ * Throws [IllegalArgumentException] on `Require(target)` for targets the caller cannot
+ * produce.
  */
 public object DTypePolicyValidation {
 
     /**
-     * Validates a [DTypePolicy] for the transformer-repo loader chain.
+     * Validates a [DTypePolicy] for one transformer-repo loader chain.
      *
      * @param policy the policy supplied by the caller
      * @param loaderName loader name for error messages (e.g. `"LlamaNetworkLoader.fromGguf"`)
-     * @param allowBf16Require whether `Require(BF16)` is acceptable. SafeTensors-backed
-     *   loaders set this to `true` (matches SKaiNET's `SafeTensorsParametersLoader`); GGUF-only
-     *   loaders set it to `false` (matches SKaiNET's `StreamingGgufParametersLoader`).
+     * @param keepNative the narrow-float dtypes this chain hands through in their on-disk
+     *   packed layout. Empty (the default) means the chain widens every narrow float to
+     *   FP32, so any `Require` naming one is rejected.
      */
     public fun validate(
         policy: DTypePolicy,
         loaderName: String,
-        allowBf16Require: Boolean,
+        keepNative: Set<DType> = emptySet(),
     ) {
         when (policy) {
             DTypePolicy.Any -> Unit
             is DTypePolicy.Prefer -> Unit
             is DTypePolicy.OneOf -> Unit
-            is DTypePolicy.Require -> validateRequire(policy.target, loaderName, allowBf16Require)
+            is DTypePolicy.Require -> validateRequire(policy.target, loaderName, keepNative)
         }
     }
 
-    private fun validateRequire(target: DType, loaderName: String, allowBf16Require: Boolean) {
+    /**
+     * BF16-only capability flag.
+     *
+     * @param allowBf16Require whether `Require(BF16)` is acceptable.
+     */
+    @Deprecated(
+        "Narrow-float capability is per-format since engine 0.38.0 — a chain can keep FP16 " +
+            "native too. Pass the set of formats it keeps packed.",
+        ReplaceWith(
+            "validate(policy, loaderName, if (allowBf16Require) setOf(BF16) else emptySet())",
+            "sk.ainet.lang.types.BF16",
+        ),
+    )
+    public fun validate(
+        policy: DTypePolicy,
+        loaderName: String,
+        allowBf16Require: Boolean,
+    ): Unit = validate(policy, loaderName, if (allowBf16Require) setOf(BF16) else emptySet())
+
+    /**
+     * Whether [policy] asks for [native] source tensors to stay in their on-disk 16-bit layout.
+     *
+     * The single decision point every transformer-repo loader chain shares, mirroring the
+     * engine's `SafeTensorsParametersLoader.mapPolicyToNarrow` /
+     * `StreamingGgufParametersLoader.keepsNative`. Only the format the policy actually names
+     * is kept: `Require(BF16)` still widens F16 sources, because turning one narrow format
+     * into the other needs a lossy re-encode.
+     *
+     * [native] is expected to be [BF16] or [FP16]; any other dtype answers `false`.
+     */
+    public fun keepsNative(policy: DTypePolicy, native: DType): Boolean = when (policy) {
+        DTypePolicy.Any -> false
+        is DTypePolicy.Require -> policy.target == native
+        is DTypePolicy.Prefer -> policy.target == native
+        is DTypePolicy.OneOf -> native in policy.allowed
+    }
+
+    private fun validateRequire(target: DType, loaderName: String, keepNative: Set<DType>) {
         when (target) {
             FP32 -> Unit
-            BF16 -> if (!allowBf16Require) {
+            BF16, FP16 -> if (target !in keepNative) {
                 throw IllegalArgumentException(
-                    "$loaderName: Require(BF16) is not supported by the GGUF loader chain — " +
-                        "GGUF BF16 sources are dequanted to FP32 today (no KEEP_NATIVE GGUF path " +
-                        "yet). Use Any or Prefer(BF16) to accept the dequant fallback."
+                    "$loaderName: Require(${target.name}) is not supported by this loader chain — " +
+                        "${target.name} sources are widened to FP32 at load " +
+                        describeKeepNative(keepNative) + ". " +
+                        "Use Any or Prefer(${target.name}) to accept the widening fallback."
                 )
             }
-            FP16 -> throw IllegalArgumentException(
-                "$loaderName: Require(FP16) is not supported — the loader chain dequants F16 to " +
-                    "FP32 (no Fp16DenseTensorData backing yet). Use Any or Prefer(FP16)."
-            )
             else -> throw IllegalArgumentException(
                 "$loaderName: Require(${target.name}) is not satisfiable — the transformer-repo " +
-                    "loader chain produces FP32 (optionally BF16 on the SafeTensors KEEP_NATIVE " +
-                    "path). It cannot fabricate ${target.name} from arbitrary sources."
+                    "loader chain produces FP32 " + describeKeepNative(keepNative) + ". " +
+                    "It cannot fabricate ${target.name} from arbitrary sources."
             )
         }
     }
+
+    private fun describeKeepNative(keepNative: Set<DType>): String = when {
+        keepNative.isEmpty() -> "(this chain keeps no narrow float packed)"
+        else -> "(this chain keeps only ${keepNative.joinToString(" / ") { it.name }} packed)"
+    }
 }

llm-core/src/commonTest/kotlin/sk/ainet/apps/llm/DTypePolicyValidationTest.kt

@@ -0,0 +1,95 @@
+package sk.ainet.apps.llm
+
+import sk.ainet.lang.types.BF16
+import sk.ainet.lang.types.DTypePolicy
+import sk.ainet.lang.types.FP16
+import sk.ainet.lang.types.FP32
+import sk.ainet.lang.types.Int8
+import kotlin.test.Test
+import kotlin.test.assertFailsWith
+import kotlin.test.assertFalse
+import kotlin.test.assertTrue
+
+/**
+ * Pins the capability model behind [DTypePolicyValidation].
+ *
+ * A `Require` naming a narrow float is a promise that the weights reach the kernel packed, so it
+ * may only be accepted by a chain that actually implements KEEP_NATIVE for *that* format. The
+ * previous BF16-only boolean could not express "keeps FP16 but not BF16", nor the empty case
+ * (Gemma / Apertus) — which it papered over by accepting `Require(BF16)` and ignoring it.
+ */
+class DTypePolicyValidationTest {
+
+    private val bothNarrow = setOf(BF16, FP16)
+
+    @Test
+    fun `Require FP32 is always accepted — every chain produces FP32`() {
+        DTypePolicyValidation.validate(DTypePolicy.Require(FP32), "test", keepNative = emptySet())
+        DTypePolicyValidation.validate(DTypePolicy.Require(FP32), "test", keepNative = bothNarrow)
+    }
+
+    @Test
+    fun `soft policies never raise whatever they name`() {
+        for (keepNative in listOf(emptySet(), bothNarrow)) {
+            DTypePolicyValidation.validate(DTypePolicy.Any, "test", keepNative)
+            DTypePolicyValidation.validate(DTypePolicy.Prefer(BF16), "test", keepNative)
+            DTypePolicyValidation.validate(DTypePolicy.Prefer(FP16), "test", keepNative)
+            DTypePolicyValidation.validate(DTypePolicy.Prefer(Int8), "test", keepNative)
+            DTypePolicyValidation.validate(DTypePolicy.OneOf(setOf(FP32, BF16)), "test", keepNative)
+        }
+    }
+
+    @Test
+    fun `a chain that keeps nothing packed rejects both narrow Requires`() {
+        // The Gemma / Apertus position: their weight chains widen every narrow float, so a
+        // Require they cannot honor must fail loudly rather than be silently ignored.
+        assertFailsWith<IllegalArgumentException> {
+            DTypePolicyValidation.validate(DTypePolicy.Require(BF16), "test", keepNative = emptySet())
+        }
+        assertFailsWith<IllegalArgumentException> {
+            DTypePolicyValidation.validate(DTypePolicy.Require(FP16), "test", keepNative = emptySet())
+        }
+    }
+
+    @Test
+    fun `the two narrow formats are tracked independently`() {
+        // Keeps BF16 only.
+        DTypePolicyValidation.validate(DTypePolicy.Require(BF16), "test", keepNative = setOf(BF16))
+        assertFailsWith<IllegalArgumentException> {
+            DTypePolicyValidation.validate(DTypePolicy.Require(FP16), "test", keepNative = setOf(BF16))
+        }
+
+        // Keeps FP16 only — the mirror image, which the old boolean flag could not express.
+        DTypePolicyValidation.validate(DTypePolicy.Require(FP16), "test", keepNative = setOf(FP16))
+        assertFailsWith<IllegalArgumentException> {
+            DTypePolicyValidation.validate(DTypePolicy.Require(BF16), "test", keepNative = setOf(FP16))
+        }
+    }
+
+    @Test
+    fun `a dtype no chain produces is rejected even when both narrow floats are kept`() {
+        assertFailsWith<IllegalArgumentException> {
+            DTypePolicyValidation.validate(DTypePolicy.Require(Int8), "test", keepNative = bothNarrow)
+        }
+    }
+
+    @Test
+    fun `keepsNative names one format at a time`() {
+        assertFalse(DTypePolicyValidation.keepsNative(DTypePolicy.Any, BF16))
+        assertFalse(DTypePolicyValidation.keepsNative(DTypePolicy.Any, FP16))
+
+        // Require / Prefer keep exactly the format they name — never the sibling, which would
+        // mean reinterpreting one 16-bit layout as the other and decoding to silent garbage.
+        assertTrue(DTypePolicyValidation.keepsNative(DTypePolicy.Require(BF16), BF16))
+        assertFalse(DTypePolicyValidation.keepsNative(DTypePolicy.Require(BF16), FP16))
+        assertTrue(DTypePolicyValidation.keepsNative(DTypePolicy.Prefer(FP16), FP16))
+        assertFalse(DTypePolicyValidation.keepsNative(DTypePolicy.Prefer(FP16), BF16))
+
+        // OneOf may admit both at once.
+        val oneOf = DTypePolicy.OneOf(setOf(BF16, FP16))
+        assertTrue(DTypePolicyValidation.keepsNative(oneOf, BF16))
+        assertTrue(DTypePolicyValidation.keepsNative(oneOf, FP16))
+
+        assertFalse(DTypePolicyValidation.keepsNative(DTypePolicy.Require(FP32), BF16))
+    }
+}

llm-inference/apertus/src/commonMain/kotlin/sk/ainet/models/apertus/ApertusNetworkLoader.kt

@@ -40,8 +40,9 @@ public class ApertusNetworkLoader @PublishedApi internal constructor(
 
     /** See [sk.ainet.models.llama.LlamaNetworkLoader.withDtypePolicy]. */
     public fun withDtypePolicy(policy: DTypePolicy): ApertusNetworkLoader {
-        val allowBf16 = weightsProvider is WeightsProvider.SafeTensorsSingle
-        DTypePolicyValidation.validate(policy, "ApertusNetworkLoader.withDtypePolicy", allowBf16Require = allowBf16)
+        // As Gemma: `ApertusWeightLoader` / `ApertusSingleSafeTensorsLoader` widen every narrow
+        // float to FP32, so this loader keeps nothing packed and promises nothing.
+        DTypePolicyValidation.validate(policy, "ApertusNetworkLoader.withDtypePolicy", keepNative = emptySet())
         this.dtypePolicy = policy
         return this
     }

llm-inference/gemma/src/commonMain/kotlin/sk/ainet/models/gemma/GemmaNetworkLoader.kt

@@ -41,8 +41,11 @@ public class GemmaNetworkLoader @PublishedApi internal constructor(
 
     /** See [sk.ainet.models.llama.LlamaNetworkLoader.withDtypePolicy]. */
     public fun withDtypePolicy(policy: DTypePolicy): GemmaNetworkLoader {
-        val allowBf16 = weightsProvider is WeightsProvider.SafeTensorsIndex
-        DTypePolicyValidation.validate(policy, "GemmaNetworkLoader.withDtypePolicy", allowBf16Require = allowBf16)
+        // Gemma has its own weight chain (`Gemma4WeightLoader` / `Gemma4SafeTensorsWeightLoader`),
+        // which widens every narrow float to FP32 — it has no KEEP_NATIVE path yet, unlike the
+        // shared decoder chain LLaMA/Qwen/Voxtral use. So it promises nothing and `Require(BF16)`
+        // is rejected rather than accepted-and-ignored.
+        DTypePolicyValidation.validate(policy, "GemmaNetworkLoader.withDtypePolicy", keepNative = emptySet())
         this.dtypePolicy = policy
         return this
     }