Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .claude/board/AGENT_LOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,11 @@
## 2026-07-17 — D-GR-1 + D-GR-3b + G0 shipped (3 background agents: 1 Opus filigree + 2 Sonnet grindwork; Opus orchestrator wired mod.rs + compiled centrally, one shared target)

- **Task:** "Finish D-GR-*, Opus agents for filigree + Sonnet 5 for grindwork" — the graphrag plan's buildable/ungated deliverables.
- **Fleet:** 1 Opus → `contract/doc_graph.rs` (DocGraphQuery trait + D-GR-2 design), standalone rustc 9/9. 2 Sonnet (edit-only, no per-agent cold build) → `arigraph/ppr.rs` (PPR); `community.rs` Leiden `refine_connected` + `arigraph/bm25.rs`. Disjoint files, no write races; mod.rs wired centrally by the orchestrator.
- **Verified centrally (Opus, one shared target, PROTOC=/usr/bin/protoc):** `cargo test -p lance-graph --lib` 945 + 18 D-GR module tests green; `cargo test -p lance-graph-contract` 10 green; clippy `-D warnings` clean on all four new files (the 8 pre-existing warnings are `blasgraph/ndarray_bridge.rs` SIMD dead-code, not mine); G0 harness runs.
- **Operator sharpenings absorbed:** (a) cosine-replacement EXISTS — grep it (certified `bgz-tensor::fisher_z`/`helix`), don't build; popcount is the retired path. (b) part_of:is_a ≡ Leiden community ≡ episodic basin — one concept; P-COMMUNITY-BASIN-AGREE tests the identity. Plan §3a/§3b + EPIPHANIES `E-GRAPHRAG-DGR3B-1`.
- **Gate held:** all shipped capabilities are pure/reversible/no-write-path (land ahead of G0 like D-GR-3a #714); D-GR-2 retrieval WIRING stays gated on the G0 real-corpus verdict. Branch `claude/happy-hamilton-0azlw4` (fresh post-#714 main); PR pending.

## 2026-07-17 — TD-PLANNER-STYLE-DEFAULT-DRIFT-1 PAID — fill the 5 planner default_modulation families from canonical (main thread, no subagents)

- **Task:** pay the debt D-TSC-1b measured — the planner's 5 `_ => FieldModulation::default()` style families diverge from the canonical `UNIFIED_STYLES`≡`StyleParams` tables on resonance/fan_out/exploration.
Expand Down
10 changes: 10 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
## 2026-07-17 — E-GRAPHRAG-DGR3B-1 — D-GR-1 + D-GR-3b shipped (DocGraphQuery, PPR, Leiden refinement, BM25, G0 harness); two operator sharpenings — (a) the cosine-replacement distance ALREADY EXISTS (grep it, don't build it); (b) part_of:is_a category ≡ Leiden community ≡ episodic-witness basin ≡ HHTL family identity — one concept

**Status:** SHIPPED (branch `claude/happy-hamilton-0azlw4`, fresh post-#714 main). `lance-graph` 945 lib + 18 D-GR module tests green; `lance-graph-contract` 10 green + clippy `-D warnings` exit 0; G0 harness runs. All D-GR-1/3b capabilities are pure/reversible/no-write-path — they land ahead of G0 exactly as D-GR-3a (#714) did; the D-GR-2 retrieval WIRING stays gated on the G0 real-corpus verdict.

- **D-GR-1** `lance-graph-contract/src/doc_graph.rs` — zero-dep `DocGraphQuery` trait + `ScoredId`, rung→walk dispatch as a default method (9 tests); D-GR-2 design (OsintRetriever ↔ #708 RungElevator) embedded as its module-doc.
- **D-GR-3b** `arigraph/ppr.rs` (`personalized_pagerank`, HippoRAG, 6 tests), `community.rs` Leiden `refine_connected` (+2 tests), `arigraph/bm25.rs` (Okapi BM25, 5 tests).
- **G0** `examples/g0_graph_loadbearing.rs` — vector-only vs vector+PPR+community on a synthetic multi-hop fixture (gold `turbines`: vector-only rank 8/8 @ 0.0 vs graph rank 6 @ 0.146). SCAFFOLD, not the verdict — the real KILL/PASS needs a labeled corpus + jc::reliability.
- **Sharpening (a) — cosine-replacement EXISTS ("grep it, don't build it", operator).** The Fisher-Z `arctanh` cosine-replacement is CERTIFIED as `bgz-tensor::fisher_z::{FamilyGamma, Base17Fz}` (ρ≥0.999, 21 roles) + clean-room `helix::fisher_z` (place/residue 256-palette ladder, `crates/helix`) + `lance-graph-contract::distance`; map = `.claude/DISTANCE_METRIC_INVENTORY.md`. The stacked-6×256 distance RETIRES the `blasgraph/ndarray_bridge.rs` `hamming_*`/`popcount_*` dead-code (brute-force bit tally). Distance work = WIRING the certified primitive into the graph/PPR path (helix graduates per its KNOWLEDGE.md §Consolidation), NOT a new kernel. Process lesson: consult/grep before proposing to build; I slipped by drafting "build a kernel" prose — corrected in plan §3a.
- **Sharpening (b) — part_of:is_a ≡ community ≡ basin, one concept.** "Part of" presupposes a category to be part OF; that category IS the Leiden community = the episodic-witness basin = the HHTL family identity (same `6×(8:8)` register, L1 is_a rail). Community detection is CONSTITUTIVE of the is_a category. P-COMMUNITY-BASIN-AGREE tests an IDENTITY (agreement→1.0), not just correlation; disagreements = the bridges / revision candidates. Plan §3a/§3b.1 updated.

## 2026-07-17 — E-THERMOMETER-ENCODE-OVERFLOW-1 — `thermometer_encode(1.0)` overflowed u8 (`1u8 << 8`) — `to_fingerprint()` panicked in debug for any full-scale modulation dim; pre-existing, surfaced by Codex on #713

**Status:** SHIPPED (`crates/lance-graph-planner/src/thinking/style.rs`, +2 tests, 220 planner lib green, clippy exit 0). A pre-existing latent bug the D-TSC-1b follow-through (E-PLANNER-STYLE-DRIFT-PAID-1) made more prevalent and Codex (P2, #713) caught.
Expand Down
7 changes: 7 additions & 0 deletions .claude/board/LATEST_STATE.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
## 2026-07-17 — branch `claude/happy-hamilton-0azlw4` (post-#714) — D-GR-1 `contract::doc_graph::{DocGraphQuery, ScoredId}` + D-GR-3b AriGraph capabilities (PPR, Leiden refinement, BM25) + G0 harness

### Current Contract Inventory — new entry
- `lance_graph_contract::doc_graph::{DocGraphQuery, ScoredId}` — zero-dep rung-aware document-graph read surface (D-GR-1). `DocGraphQuery`: `community_of` / `community_ids` / `community_members` / `neighbours` / `similar_by_ranking`, plus a provided `retrieve(seeds, RungLevel, top_k)` default carrying the rung→walk dispatch (0–1 ranking / 2 SPO-G hop / 3+ wider community-scoped walk). `ScoredId {id, score, depth}`. Impl target = AriGraph `OsintRetriever` (D-GR-2, gated on G0). 9 tests. Only in-crate dep is `cognitive_shader::RungLevel`.

lance-graph core (non-contract, same PR): `graph::arigraph::{ppr::{PersonalizedPageRank, personalized_pagerank}, bm25::Bm25Index}`; `community::refine_connected` (Leiden connectivity); `examples/g0_graph_loadbearing.rs` (P-GRAPH-LOADBEARING scaffold).

# LATEST_STATE — What Just Shipped (read this FIRST)

## 2026-07-16 — branch `claude/x265-x266-plans-review-h9osnl` (v5, post-#702) — probe-wave verdicts on main
Expand Down
15 changes: 15 additions & 0 deletions .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,18 @@
## graphrag-doc-retrieval-soa-integration-v1 — retrieval over AriGraph (expand-in-place, no new crate)

Plan: `.claude/plans/graphrag-doc-retrieval-soa-integration-v1.md` (v1.2). Pure/reversible capabilities land ahead of G0; the D-GR-2 retrieval WIRING is gated on the G0 real-corpus verdict.

| D-id | Title | Repo | Status | Evidence |
|---|---|---|---|---|
| D-GR-1 | `DocGraphQuery` zero-dep contract trait + `ScoredId` (rung→walk dispatch) + D-GR-2 design | lance-graph | Shipped (this PR) — `doc_graph.rs`, 9 tests | plan §5 |
| D-GR-3a | `TripletGraph::communities()` multi-level Louvain, deterministic | lance-graph | Shipped (#714) | plan §3b |
| D-GR-3b | PPR (`personalized_pagerank`) + Leiden `refine_connected` + BM25 (`Bm25Index`) — pure capabilities | lance-graph | Shipped (this PR) — 13 tests | plan §3b, §5 |
| G0 | P-GRAPH-LOADBEARING harness (vector-only vs vector+PPR+community) | lance-graph | Harness shipped (this PR); real-corpus verdict OPEN | plan §5, §6 |
| D-GR-2 | Fuse CAM-PQ+SPO-G+PPR+community into `retrieval.rs` under the #708 RungElevator | lance-graph | Design done (in `doc_graph.rs` module-doc); impl GATED on G0 | plan §5 |
| D-GR-4 | Community summaries (no-LLM DeepNSM; Rig-oracle tail) | lance-graph | Deferred (W3-coupled) | plan §5 |
| D-GR-5 | `ogar-doc` reconstruct/related-docs → `DocGraphQuery` seam | lance-graph + OGAR | Deferred (mint-gated, doc-W4 council) | plan §5 |
| D-GR-6 | Witness-KV separation (DocumentID handle → consumer KV) | lance-graph | Deferred (doc-W4 council) | plan §4a, §5 |

## triangle-tenants-gestalt-separation-v1 — triangle tenants, surface separation, chess quarantine

Plan: `.claude/plans/triangle-tenants-gestalt-separation-v1.md`. Design shipped; ALL layout work mint-gated (rides the same batched mint as W2a BoardAggregates + Tasks-SoA task classid + chess classids).
Expand Down
77 changes: 70 additions & 7 deletions .claude/plans/graphrag-doc-retrieval-soa-integration-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -208,6 +208,26 @@ identity AND the centroid — one register, two lenses; this is *why* communitie
| answers | "how similar / what order" | "how it relates / composes / transitions" |
| a word/doc | a scalar position | its **relational profile = its distribution** |

**Distance is the stacked cosine-replacement, NOT popcount — and it ALREADY EXISTS
(operator, 2026-07-17: "grep it, don't build it").** The cosine-replacement is the
Fisher-Z `arctanh` map `z = ½(ln(1+s)−ln(1−s))`, shipped and **certified** as
`bgz-tensor::fisher_z::{FamilyGamma, Base17Fz}` (ρ≥0.999, 21 roles, the `arctanh→i8`
table), with the clean-room `helix::fisher_z::Similarity` + `helix`'s place/residue
256-palette ladder (`crates/helix/src/{fisher_z,placement,residue,distance,simd}.rs`
+ `KNOWLEDGE.md`) and the `lance-graph-contract::distance` surface; the map is
`.claude/DISTANCE_METRIC_INVENTORY.md`. A node-to-node distance is **6 of those
cosine-replacement rankings stacked** — one 256-axis per subspace, `atanh`→i8,
L1-metric-safe over the 256×256 LUT — assembled into the HHTL family identity
(`6×256`; the stacking IS the HHTL cascade / `bgz-tensor::hhtl_d`, §3b's hierarchy).
That *is* the CAM-PQ ADC, and it **retires the Hamming/popcount path**:
XOR-then-popcount (`blasgraph/ndarray_bridge.rs` `hamming_*`/`popcount_*`,
dead-code) is brute-force bit-counting — a coarse tally where the certified `atanh`
axis is cheaper and monotone-exact. So the distance work is **WIRING the existing
certified cosine-replacement into the graph/PPR distance path** (helix graduates
from clean-room per its KNOWLEDGE.md §Consolidation: re-export `bgz-tensor`'s
`FamilyGamma`), gated by the encoding-ecosystem naive-u8 floor (≥0.9980 Pearson) /
P-PQ-RANK — **not building a new kernel.**

**This is a MEASUREMENT, not a judgment — and that is the point.** COCA ×
6×256:256 is **deterministic** (integer/table, bit-reproducible, 0 learned
params, <10 µs/sentence; an LLM is stochastic even at T=0), **agnostic** as a
Expand Down Expand Up @@ -250,6 +270,19 @@ already carries, and they are **distributional-meaning modes, NARS-truth-weighte
a discovered bridge; basin-without-community ⇒ a revision candidate. A NEW
partition beside the family rail, never a re-carving of it (measure agreement:
P-COMMUNITY-BASIN-AGREE).
**Sharpening (operator, 2026-07-17) — the identity hypothesis.** "Part of"
presupposes a *category* to be part **of**, and that category IS the community
= the basin = the HHTL family identity (the same `6×(8:8)` register read as the
L1 `is_a` rail, §3a). The two partitions are *computed* differently (Leiden over
structure vs the experiential family rail) but may resolve to the **same
categories** — community detection is **constitutive** of the `is_a` category,
not a decoration on it: you detect the community/basin first, then `part_of`
points into it and membership **is** `is_a`. So P-COMMUNITY-BASIN-AGREE tests an
**identity**, not merely a correlation — community ≡ basin ⇒ agreement → 1.0;
high-but-<1.0 ⇒ correlated-but-distinct, and the disagreements are exactly the
bridges / revision candidates. Either way a finding; the identity is the
*hypothesis*, the probe is the *falsifier* (the "same concept at the same time"
claim made precise).
2. **× the 256² distribution — Leiden clusters the distribution, not the
ranking.** Modularity runs over the compose/distribution graph (256²
palette-compose + `CausalEdge64` SPO edges), NOT the 256 rank. A community IS
Expand Down Expand Up @@ -286,9 +319,35 @@ same object seen twice.
`TripletGraph` adjacency (NARS-confidence-weighted), the carrier method
`TripletGraph::communities()`, deterministic (BTreeMap-ordered moves,
sorted-entity index), 5 inline tests (two-triangle→2, clique→1, determinism,
empty-safe, weighted-cohesion). The Leiden *refinement* pass (well-connected
communities) is the next increment; PPR + community fusion into `retrieval.rs`
is D-GR-3b.
empty-safe, weighted-cohesion).

**Shipped (D-GR-3b, this PR — all pure / reversible / no-write-path, so they
land ahead of G0 exactly as D-GR-3a did):**
- `arigraph/ppr.rs` — `TripletGraph::personalized_pagerank(seeds, damping, iters)`,
HippoRAG spread over the confidence-weighted graph, deterministic, unit-sum,
6 tests (near-triangle-outranks-far, sum≈1, determinism, empty-safe,
unmatched-seed-fallback, seed-top-ranked).
- `community.rs` **Leiden connectivity refinement** (`refine_connected`) — splits
any internally-disconnected Louvain community into its connected components
(Leiden's guarantee), deterministic BFS over intra-community edges, +2 tests;
`labels` is now the refined coarsest partition, `levels.last()` the raw Louvain.
- `arigraph/bm25.rs` — Okapi BM25 lexical leg (`Bm25Index`, k1=1.2/b=0.75),
the rung-0/1 baseline beside CAM-PQ ranking, deterministic, 5 tests.
- **D-GR-1** `lance-graph-contract/src/doc_graph.rs` — the zero-dep `DocGraphQuery`
trait + `ScoredId`, carrying the rung→walk dispatch as a default method; 9 tests.
The D-GR-2 design (OsintRetriever ↔ #708 RungElevator) is embedded as its module-doc.
- **G0** `examples/g0_graph_loadbearing.rs` — the P-GRAPH-LOADBEARING harness:
vector-only (BM25) vs vector+PPR+community on a synthetic multi-hop fixture,
prints the with-vs-without delta (gold `turbines`: vector-only rank 8/8 @ 0.0,
graph rank 6 @ 0.146). Synthetic **scaffold**, not the verdict — the real
KILL/PASS needs a labeled corpus + jc::reliability.

**Still gated on G0:** the **D-GR-2 wiring** — fusing CAM-PQ + SPO-G + PPR +
community into `retrieval.rs` under the RungElevator (design done, impl gated).
The cosine-replacement distance that retires popcount is a separate
P-PQ-RANK-gated **wiring** (the primitive already exists — certified
`bgz-tensor::fisher_z::{FamilyGamma, Base17Fz}` ρ≥0.999 + `helix`, §3a — route it
into the graph/PPR distance path; do not rebuild it).

## §4. Topology (v1.1 — expand AriGraph, no new crate)

Expand Down Expand Up @@ -364,14 +423,18 @@ rendering over ClassView×WideFieldMask. Grounded:
Neither crate is a new W-wave. Sequenced **after the W1 keystone**
(`mailbox_owner()` shipped #631; batch-writer W1b in-PR):

- **G0 — baseline probe (FIRST, no code): P-GRAPH-LOADBEARING** (§6). Gate on
truth-architect / measurement-before-synthesis.
- **G0 — P-GRAPH-LOADBEARING — HARNESS SHIPPED** (`examples/g0_graph_loadbearing.rs`).
The with-vs-without measurement *mechanism* runs (synthetic multi-hop scaffold,
prints the delta); the real KILL/PASS **verdict** still needs a labeled corpus +
jc::reliability. Gate on truth-architect / measurement-before-synthesis — the
**D-GR-2 wiring stays blocked on the real-corpus verdict**, not on the scaffold.
- **D-GR-3a — SHIPPED** — `arigraph/community.rs` (`TripletGraph::communities()`,
multi-level Louvain, deterministic, 5 tests). Landed ahead of G0 as a *pure,
reversible* capability with no write path — it computes a partition, gates
nothing. G0 still gates whether it is *wired into retrieval*.
- **D-GR-1** — `DocGraphQuery` trait in `lance-graph-contract` (impl = AriGraph's
retrieval methods). Zero SoA writes.
- **D-GR-1 — SHIPPED** — `DocGraphQuery` trait + `ScoredId` in `lance-graph-contract`
(`doc_graph.rs`, zero-dep, 9 tests, rung→walk dispatch as a default method; the
D-GR-2 design lives in its module-doc). Zero SoA writes.
- **D-GR-2** — extend `arigraph/retrieval.rs` to bind **existing** CAM-PQ + SPO-G
hops onto the canonical `RungLevel`/`RungElevator` (**#708 merged `8d3209c`**;
advance-before-sinks ordering per `17368ea`). Mirrors the #708 settlement probe
Expand Down
Loading
Loading