fix(lang,backend-cpu): free transpose for narrow-float weights

[michalharakal]

Fixes #888.

Problem

Projections are stored [out, in], but chooseQuantizedMatmul needs [in, out], so Linear.onForward transposes the weight on every call. Transposing a row-major narrow-float tensor had no fast path: it walked the tensor elementwise through boxed get() and widened to FP32.

Measured on an i7-9750H that is 206 ms for a 2048x2048 projection and 4.4 s for 4096x11008 — per weight, per token. KEEP_NATIVE was therefore slower than not using it at all, which made the narrow-float matmul kernels unreachable in practice.

Change

Add NarrowFloatInputMajorTensorData: a rank-2 narrow weight whose bytes are stored input-major, built once via fromRowMajor(). Input-major storage of [rows, cols] is row-major storage of [cols, rows], so transposedView() hands back an ordinary NarrowFloatDenseTensorData over the same buffer — no copy, and element access stays correct on both sides, because each type indexes the shared bytes with the strides its own shape implies.

That is the improvement over the K-quant lazy transpose, where get() on a transposed tensor is meaningless and only the kernel's direct packedData read is valid.

Dispatch it from DefaultCpuOpsBase.transpose. Only the input-major type is reinterpreted; a row-major narrow buffer deliberately still falls through to the generic path, because swapping its shape would silently yield a different matrix rather than the transpose. The arm lives in the base class alone — DefaultCpuOpsJvm.transpose intercepts nothing that would shadow it, verified by disabling each arm in turn.

The second commit documents the resulting layout caveat on the SafeTensors KEEP_NATIVE policy: consumers that go through Linear should relay matmul weights with fromRowMajor, while gathered tensors — embedding tables above all — should stay row-major, since input-major storage would stride exactly the reads they serve.

Tests

NarrowFloatInputMajorTensorDataTest (lang-core) and NarrowFloatTransposeDispatchTest (backend-cpu, commonTest) cover:

• element access against the row-major original, and buffer sharing after transposedView()
• relayout round-trip, plus a square weight, where a wrong permutation still has the right byte count
• rank and size rejection
• matmul through the transpose against an FP32 reference

Both narrow codecs are exercised, with vacuity guards asserting they disagree on identical bytes — both formats are 2 bytes per element, so a codec mix-up produces plausible wrong numbers rather than an error. The dispatch test passes on linuxX64 as well as the JVM.

Measured effect

Validated downstream in SKaiNET-transformers, which relays matmul weights via fromRowMajor at load. On i7-9750H / OpenJDK 21, ffn_up 8B at batch 1, BF16: 4465 ms -> 58 ms (77x), and now 1.9x faster than the FP32 SGEMM it replaces.

Note this does not address #887 — FP16 remains slow in the kernel itself because Fp16Codec.decode dominates the inner loop. The two are independent; this PR is what makes BF16 KEEP_NATIVE usable at real model sizes.

Compatibility

API change is purely additive; skainet-lang-core.api updated accordingly.

[github-actions]

📖 Documentation Preview

The documentation has been built successfully for this PR.

Generated Files:

• Operator documentation: docs/modules/operators/_generated_/
• JSON schema output: operators.json

Artifacts:

• Download the documentation-preview-895 artifact to view the complete documentation locally.

This comment will be updated automatically when the PR is updated.