diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index fc6b8068..4d768e37 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,47 @@ +## 2026-07-19 — E-IMPLICIT-MORTON-TILE-1 — the 256:256=64k tile distance table is IMPLICIT, but the address does TWO jobs at TWO fidelities: pure shift/mask (address-only) is ROUTING (ρ≈0.51 hierarchical vs 0.41 flat — coarse is correct); the METRIC needs the stored 256-centroid codebook (ADC, ρ=1.0) — the 64k pairwise table is NEVER materialized in EITHER case (F11-adjacent named test, RUN) + +**Status:** FINDING (built + measured on committed data, hierarchical-vs-flat contrast). **Confidence:** High for the tier distinction + the 64k-never-materialized ruling; Medium-magnitude for the exact address-only ρ (8-genre committed data + rough deterministic 4-means, degenerate empty-branch leaves add noise — the *direction* hier>flat is robust, the absolute 0.51 is a floor). Runs the OGAR canon's F11-adjacent "hierarchical-4⁴ vs flat-256" named-but-unrun test. Operator: *"what about Morton-comma where the table is implicit 256:256=64k."* Deliverable: `implicit_morton_tile_probe.py` (scratchpad; std-only, deterministic, no RNG; buildable into `crates/deepnsm/examples/` on request). + +**The probe.** Build a 256-centroid codebook on the committed 8-genre semantic vectors as a **4-level 4-ary hierarchy** (byte = the four 2-bit ancestry groups). Compare two distances over all 32,640 leaf pairs: `d_mat` = materialized L2(centroid_a, centroid_b) (the 256×256 table, ground truth) vs `d_imp` = **address-only** distance from the two bytes (first-divergence-level Morton offset, scaled 4^−L — pure shift/mask, no centroid load). + +**Measured:** +- `ρ(d_imp, d_mat)` **HIERARCHICAL 4⁴ = 0.5083**; **FLAT-relabel = 0.4118** → the hierarchy→address→distance correspondence is REAL (+0.09, ~22% relative; the canon's "flat k-means-256 breaks this, hierarchical 4⁴ preserves it" holds in DIRECTION). +- All-levels weighted variant = 0.4168 (**worse** than divergence-level-only) → the address's real signal is the PREFIX, not the tail; post-divergence nibbles are arbitrary noise. +- **ADC-implicit = 1.0 by construction:** store the 256 centroids (2 KB texture), compute L2 on demand — the 64k pairwise table is NEVER materialized, ρ trivially perfect. + +**The ruling — the address does TWO jobs at TWO fidelities, neither needs the 64k table:** +| job | mechanism | fidelity | storage | +|---|---|---|---| +| **Routing / prefix-scoping** (`is_ancestor_of`, which codebook/basin, longest-prefix) | address-only Morton shift/mask | ρ≈0.51 — coarse is EXACTLY right | nothing, O(1) | +| **Metric** (fine cell↔cell distance ordering) | ADC over stored 256 centroids | ρ=1.0 | 2 KB codebook, never 64k | + +So "implicit 256:256=64k" is SOUND — the 64k pairwise table is a materialization *choice* you always decline (store the 256-centroid **texture**, regenerate the distance **shader** on demand). But the RADICAL address-only-*metric* reading (distance from bytes alone, no codebook load) is a coarse ρ≈0.51 proxy: it captures ROUTING, not the fine metric. The **comma** supplies fine structure for a *continuous perturbation field placed over the tile* (the shader — cf. `E-PERTURBATION-CONVERGENCE-1`), NOT for discrete cell↔cell metric (no sub-cell to interpolate between two quantized leaves). + +**Why this composes with the header/payload split (`E-DEEPNSM-FACET-BLIND-CONVERGENCE-1`):** the HEADER (classid/prefix) uses address-only ancestry → ROUTING, where coarse is correct — which is exactly why **frequency-as-header works** (a routing coordinate). The PAYLOAD (6×256:256) carries the METRIC → needs the stored centroids (ADC/Fisher-z, ρ=1.0) — which is exactly why **frequency-in-payload corrupts** (it perturbs a metric coordinate). Same address, two fidelities, two jobs — the two findings are one architecture. **Honest boundary:** 8-genre committed data with a rough k-means; the real 96D/6-subspace codebook (gitignored) may shift the address-only ρ, but the tier distinction (routing-coarse vs metric-needs-codebook; 64k never materialized) is structural. Additive: probe + board only, no core change. **Cross-ref:** E-DEEPNSM-FACET-BLIND-CONVERGENCE-1, E-FREQ-IS-COSINE-REPLACEMENT-1 (distance immaterializes because the centroid IS the cosine — the ADC ρ=1.0 leg), E-PERTURBATION-CONVERGENCE-1 (the comma as the continuous-field shader), OGAR `256×256 CENTROID TILE` canon + the F11 hierarchical-4⁴ condition, `soa_layout/le-contract.md` L4 `6×palette256:palette256`. + +## 2026-07-19 — E-DEEPNSM-FACET-BLIND-CONVERGENCE-1 — two INDEPENDENT blind implementations of PROBE-DEEPNSM-FACET (spec §3.2–3.5, `lemmas_5k.csv`) converged BYTE-IDENTICALLY (ρ_all=0.9992, NN-flip 12/256, anchor we/will −1915 ranks) → header-invariance CONFIRMED, PF=Header equilibrium CONFIRMED; and the pre-registered §3b double-decile falsifier is STRUCTURALLY MIS-SCOPED (empty even with full frequency range) — the SECOND criterion-vs-data miscalibration this arc after the Codex P1 + +**Status:** FINDING (diverse-redundancy replication, operator-adjudicated) + SPEC-CORRECTION (§3b). **Confidence:** High for header-invariance + convergence; High for the §3b mis-scope (proven on committed data, two independent runs + a rank-stratified widening). Provenance: the blind protocol the operator requested — *"you both test the outcome without telling the other session the results before I confirm that it was done"* — applied to the merged #758 design/spec handoff (`deepnsm-morton-comma-facet-v1.md`). Deliverables: the spec (merged #758); two independent Python impls (session A + operator-run session B, both from spec alone, neither importing the other); the decisive widened cell run this session. + +**The two blind runs (top-256 by rank, 32,640 pairs, deterministic — no RNG):** +| measurement | session A | session B | verdict | +|---|---|---|---| +| ρ_all (Spearman d_payload vs d_truth) | 0.9992 | 0.9992 | identical | +| HEADER freq-shuffle invariance | invariant (d_header=d_truth) | invariant (mean \|Δd_payload\|=0.2417) | identical | +| nearest-neighbor flip rate | 12/256 (4.7%) | 12/256 (4.7%) | identical | +| §3b corrupted set (near-decile ∩ freq-far-decile) | EMPTY (0 pairs) | EMPTY (0 pairs) | identical | +| named anchor (near decile, max Δf) | we/will dt=0.950→dp=1.175 (+0.224), −1915 ranks | we/will dt=0.950→dp=1.175 (+0.224), −1915 ranks | identical | + +**Two independent implementations from the spec text alone produced BYTE-IDENTICAL numbers.** That is the diverse-redundancy result working as designed: spec §3.2–3.5 is deterministic and unambiguous, so the empty-3b-vs-strong-anchor tension is a REAL property of the data, not either session's artifact. + +**The rulings (operator-adjudicated):** +1. **Header-invariance (orthogonality): CONFIRMED — the load-bearing half.** Frequency as a routing/attention HEADER does not leak into the semantic distance (`d_header=d_truth` by construction, freq-shuffle-invariant in both runs). This is the claim the whole facet design rests on. +2. **Collapse mechanism: PRESENT and consistent.** Folding frequency into the payload demotes near-synonym / frequency-divergent pairs by +1256..1915 rank positions (we/will, and county/am in the widened cell) and flips 4.7–6.2% of nearest neighbors. PF=Payload measurably corrupts near-neighbor structure. +3. **PF=Header is the game-matrix equilibrium: CONFIRMED.** (Address, Dist, Header) is the stable cell; (Address, Dist, Payload) is the defection trap the operator's game-matrix predicted. Frequency belongs in the `classid` header, the 6×(8:8) payload carries semantic distribution ONLY. +4. **§3b (the pre-registered double-decile AND falsifier) is STRUCTURALLY MIS-SCOPED — retire it.** The decisive widened cell this session (256 words RANK-STRATIFIED across the full 5,050 → frequency z-span −1.14…6.23, full dynamic range restored) STILL returns an empty near∩freq-far double-decile set. So the empty-3b is NOT a top-256 window artifact: semantic-nearness and frequency-nearness are POSITIVELY correlated in this data (near-decile semantic pairs are also frequency-near; the extreme intersection is ~disjoint), so the strict AND can never fire regardless of window. The correct falsifier is the **anchor + NN-flip signal** (which both runs and the widened cell agree fires cleanly), NOT the double-decile set. This is the SECOND criterion-vs-data miscalibration on this probe — cf. the Codex P1 (impossible `d_payload **⊘ SUPERSEDED (2026-07-19, `E-DEEPNSM-FACET-BLIND-CONVERGENCE-1`).** Two +> independent blind runs + a rank-stratified widening (freq z-span −1.14…6.23, +> full dynamic range) all return an **empty** near-decile∩freq-far-decile set +> (3b) — semantic-nearness and frequency-nearness are POSITIVELY correlated in +> this data, so the double-decile AND can never fire regardless of window. The +> 3b **set-membership** clause of §3.5 is therefore RETIRED as mis-scoped; the +> live falsifier is measurements **3a (NN-flip) + 4 (named anchor)**, which fire +> cleanly and identically across both blind runs. Verdict this arc: +> header-invariance CONFIRMED, PF=Header equilibrium CONFIRMED (localized tail +> collapse via anchor we/will −1915 ranks). See the EPIPHANY for the full +> adjudication. + ### 3.5 PASS / COLLAPSE criteria (pre-registered — fill with YOUR numbers only) - **HEADER lane** must be **frequency-shuffle-invariant** (measurement 2) — the lane-orthogonality proof (frequency didn't leak into semantics).