graphrag + doc-retrieval on the V3 SoA — plan v1.2 + arigraph community detection (D-GR-3a)#714
Merged
Merged
Conversation
|
Warning Review limit reached
Next review available in: 32 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_4b5f699e-d810-4306-b68d-8758adb13ef2) |
Evaluation + wave plan for feeding a new query-side crates/graphrag and OGAR's ogar-doc from the existing SoA + CausalEdge64 + (queued) EpisodicWitness64 substrate, aligned to #708 / D-TRI-6 (the rung-ascent loop) and the v3 waves. Key rulings the plan encodes: - Direction reframe (assembler-vs-storage boundary): OGAR owns doc ingestion + the document 0x080B mint (ogar-from-docv1 -> ogar-doc-ir -> ogar-doc, built); lance-graph provides contract types OGAR consumes + query. graphrag is query-side ONLY, reads the calcified substrate, never ingests. A lance-graph doc-transcode crate is REJECTED (duplicates ogar-from-docv1, inverts the dep). - REUSE the substrate, BUILD only 3 gaps: hierarchical Leiden, HippoRAG-PPR reset-distribution, BM25. Extraction/SPO/fact-store/vector-CAM-PQ already exist; causal_edge::CausalEdge64 is the SPO edge (not the thinking-engine variant); EpisodicWitness64 is a queued column (today WitnessTable + EpisodicEdges64). graphrag-rs = REUSE-AS-REFERENCE (E-V3-GRAPHRAG-INV-1). - Retrieval = rung ascent (#708): dispatches through RungElevator + canonical RungLevel; BLOCK ascends and widens the CAUSES..BECOMES predicate-plane mask, FLOW relaxes to base. The graph is load-bearing because surprise ascends the elevator. Maps onto the triangle-tenants Maslow pyramid (D-TRI). - Probe-first: G0 P-GRAPH-LOADBEARING (graph vs vector-only) gates all Leiden/PPR code. Composes atop oxigraph-arigraph-cognitive-shader-soa-merge-v1. No bytes land; D-GR-5 (OGAR seam) mint-gated + baton-audited. Board hygiene: INTEGRATION_PLANS prepend in the same commit. Co-Authored-By: Claude <noreply@anthropic.com>
…ate (§10) Verified every reuse/build assumption against the tree at 8d3209c (#708 merged). Verdict: FEASIBLE, two small non-blocking caveats, no circular deps. Confirmed reachable (pub): RungElevator (pub struct/fields + new + on_gate) and RungLevel/GateDecision/MailboxSoaView + CausalEdge64 all from ZERO-DEP crates; TripletGraph exposes the exact walk (get_associated multi-hop, intervene_on Pearl-2 apex, detect_contradictions = the BLOCK trigger, revise_with_evidence), spo_bridge::promote_to_spo pub, hdr_pagerank + ScentCsr::spmv pub, cam_pq pub. The rung loop maps 1:1 onto existing methods. Caveat 1 (drives a feature gate): the graph legs live in lance-graph core = drags datafusion/lance/arrow (19 heavy-dep lines); the rung-ascent core is zero-dep. => default graphrag = LIGHT (rung ascent + BM25 + in-memory PPR + G0 probe); feature graphrag-lance = HEAVY graph legs. §4 crate topology updated. Caveat 2: rung_widened_layer_mask is private (driver.rs:701) => make pub / move to contract, or replicate the pure fn (secondary leg; ascent works without it). Genuine BUILD confirmed: hierarchical Leiden (jc louvain is example-only), HippoRAG-PPR reset-distribution atop hdr_pagerank, BM25. Next step needs no upstream change: scaffold the light crate + cargo-check compile-proof, then G0. Co-Authored-By: Claude <noreply@anthropic.com>
…lone crate Operator-directed architecture correction. Instead of a query-side crates/graphrag, expand AriGraph, which already owns the retrieval brain + episodic-witness basins. Grounding (verified against the tree): AriGraph already has retrieval.rs (OsintRetriever = BFS+episodic fusion), witness_corpus.rs, episodic.rs, spo_bridge.rs, markov_soa.rs — and has NO community/basin/cluster/Leiden/partition type (grep-confirmed). So Leiden fills the one structural-partition gap that COMPLEMENTS the episodic-witness partition AriGraph owns; it does not clutter or duplicate. A separate crate is a free-function-on-carrier's-state (the CLAUDE.md litmus reject) and a parallel retrieval layer beside an existing one. v1.1 topology: - arigraph/community.rs (NEW): hierarchical Leiden over the TripletGraph adjacency (triplets + entity_index; backed by in-core blasgraph CSR). - arigraph/retrieval.rs (EXTEND): + PPR/HippoRAG atop blasgraph::hdr_pagerank; community becomes a THIRD fusion signal beside BFS + episodic; driven by the #708 RungElevator (detect_contradictions -> BLOCK -> wider community/PPR walk). - DocGraphQuery trait stays in lance-graph-contract (impl = AriGraph methods). - BM25/lexical stays OUT of AriGraph (text-index, not a fact graph). - NO new crate. This dissolves the §10 dep-weight feature-gate (the graph capabilities live where the graph lives — core). New design constraint: no-singleton + write-on-behalf — community detection reads the graph the mailbox owns; a persisted membership is a born-stamped value-tenant lane. The #708 rung-ascent alignment, the reuse census, the OGAR boundary, and the probes all carry over unchanged. Board hygiene: INTEGRATION_PLANS entry annotated with the v1.1 scope. Co-Authored-By: Claude <noreply@anthropic.com>
IMPLEMENT — arigraph/community.rs: TripletGraph::communities() — multi-level Louvain modularity over the triplet adjacency (NARS-confidence-weighted edges, self-loops dropped), a carrier method (litmus-compliant), deterministic (BTreeMap-ordered moves + sorted-entity index), 5 inline tests. The STRUCTURAL partition complementing the episodic/family (part_of:is_a) basins AriGraph owns. Louvain core verified standalone before the in-crate build (two-triangle -> 2 comms Q=0.357, clique -> 1, determinism, empty-safe, weighted-cohesion); built against jc/examples/splat_louvain_modularity.rs (parallel-session reference). Leiden refinement pass + PPR/community fusion into retrieval.rs are D-GR-3b. WRITE — plan v1.2: - The base is a MEANING MEASUREMENT: COCA + NARS are co-equal core faculties; agnosticism is scoped to raw data + consumer modelling only (NOT meaning). The 12-byte facet is 6x2x8bit (CLAM), never f32 (f32 is build-time source only); 256 = ranking (needs cosine-replacement), 256^2 = distribution (as accurate as content-blind f32); the 6x(8:8) register is polymorphically part_of:is_a family-identity OR palette256^2 centroid (one register, two lenses) (S3a). - Leiden community synergies: distributional-meaning modes, NARS-truth-weighted (6 axes; the rung-3 layer of the ascent) (S3b). - DocumentID-KV / witness-handle seam: a stable DocumentID keys the consumer KV; the witness/node hold handles only, never raw bytes (S4a, D-GR-6). - New probes: P-COMMUNITY-BASIN-AGREE, P-HIER-LEIDEN-HHTL, P-MEASUREMENT-DETERMINISM, P-PQ-RANK (all via jc::reliability, merged #709/#710). - Meaning-substrate hazard (S7.6): do not reintroduce "keep language out"; the only real constraint is crate-layering (core can't dep the downstream deepnsm crate; use the in-core nsm/ copy). Board hygiene: INTEGRATION_PLANS entry updated to v1.2. Co-Authored-By: Claude <noreply@anthropic.com>
AdaWorldAPI
marked this pull request as ready for review
July 17, 2026 14:54
AdaWorldAPI
force-pushed
the
claude/happy-hamilton-0azlw4
branch
from
July 17, 2026 14:55
f9ef30f to
2d45279
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9ef30f9f6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// Dense entity index: a stable `Vec<String>` (sorted for determinism) and | ||
| /// an entity → dense-id map. | ||
| fn entity_index_dense(&self) -> (Vec<String>, HashMap<String, usize>) { | ||
| let mut names: Vec<String> = self.entity_index.keys().cloned().collect(); |
There was a problem hiding this comment.
When delete_triplets soft-deletes edges, entity_index is intentionally left intact, so building the dense node list from these keys keeps endpoints whose only triplets are now deleted. Since build_adjacency then skips those deleted triplets, a graph after deleting its only x-y edge reports x and y as singleton communities (and community_of returns Some) even though the rest of TripletGraph's read APIs treat deleted triplets as absent; this inflates num_communities and returns stale members whenever community detection runs after deletions. Build the dense entity list from non-deleted triplets (or otherwise drop zero-degree stale nodes) instead.
Useful? React with 👍 / 👎.
AdaWorldAPI
pushed a commit
that referenced
this pull request
Jul 17, 2026
#714 (feat(arigraph): community.rs, D-GR-3a) merged without `cargo fmt`, so `cargo fmt --manifest-path crates/lance-graph/Cargo.toml -- --check` fails on main and blocks CI on every subsequent PR. Pure formatting — zero semantic change (block-body vs single-line closures, struct-literal wrapping, test tuple wrapping). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Awg6TXocHcwTtc6eGsHcdD
AdaWorldAPI
pushed a commit
that referenced
this pull request
Jul 17, 2026
…arness Ship the buildable, ungated graphrag deliverables (plan graphrag-doc-retrieval-soa-integration-v1 §5). All pure / reversible / no-write-path capabilities land ahead of G0 exactly as D-GR-3a (#714); the D-GR-2 retrieval wiring stays gated on the G0 real-corpus verdict. D-GR-1 (contract, zero-dep): - doc_graph::{DocGraphQuery, ScoredId} - rung-aware document-graph read surface with a provided retrieve(seeds, RungLevel, top_k) default carrying the rung->walk dispatch (0-1 ranking / 2 SPO-G hop / 3+ wider community-scoped walk); the D-GR-2 design (OsintRetriever <-> #708 RungElevator) lives in the module-doc. 9 tests. D-GR-3b (lance-graph core, AriGraph): - arigraph::ppr - TripletGraph::personalized_pagerank (HippoRAG spread, confidence-weighted, deterministic, unit-sum). 6 tests. - arigraph::community - Leiden refine_connected (splits internally- disconnected Louvain communities into connected components). +2 tests. - arigraph::bm25 - Okapi BM25 lexical leg (Bm25Index, k1=1.2/b=0.75). 5 tests. G0: - examples/g0_graph_loadbearing.rs - the P-GRAPH-LOADBEARING harness: vector-only (BM25) vs vector+PPR+community over a synthetic multi-hop fixture, prints the with-vs-without delta. Scaffold, not the verdict - the real KILL/PASS needs a labeled corpus + jc::reliability. Plan sharpened per operator (§3a/§3b): (a) the cosine-replacement distance ALREADY EXISTS (certified bgz-tensor::fisher_z::{FamilyGamma, Base17Fz} rho>=0.999, clean-room helix) - wire it, retire the blasgraph popcount dead-code; do not rebuild. (b) part_of:is_a category = Leiden community = episodic-witness basin - one concept; P-COMMUNITY-BASIN-AGREE tests the identity. Verified: lance-graph 945 lib + 18 D-GR module tests; lance-graph-contract 10 tests; clippy -D warnings clean on new files; fmt clean. Board: EPIPHANIES E-GRAPHRAG-DGR3B-1, STATUS_BOARD graphrag section, LATEST_STATE contract inventory, AGENT_LOG, plan v1.2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
AdaWorldAPI
added a commit
that referenced
this pull request
Jul 17, 2026
#714 (feat(arigraph): community.rs, D-GR-3a) merged without `cargo fmt`, so `cargo fmt --manifest-path crates/lance-graph/Cargo.toml -- --check` fails on main and blocks CI on every subsequent PR. Pure formatting — zero semantic change (block-body vs single-line closures, struct-literal wrapping, test tuple wrapping). Claude-Session: https://claude.ai/code/session_01Awg6TXocHcwTtc6eGsHcdD Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
graphrag + doc-retrieval on the V3 SoA — plan v1.2 + first code
An integration plan for giving lance-graph graphrag-style retrieval by expanding
AriGraph in place (communities / PPR / retrieval fusion) rather than adding a
standalone crate — plus the first shipped increment.
Shipped code (D-GR-3a)
crates/lance-graph/src/graph/arigraph/community.rs—TripletGraph::communities():multi-level Louvain modularity over the triplet adjacency (NARS-confidence-weighted
edges, self-loops dropped), a carrier method (litmus-compliant), deterministic
(BTreeMap-ordered moves + sorted-entity index). The structural partition
complementing the episodic / family (
part_of:is_a) basins AriGraph already owns.5 passed; 0 failedin-crate (two-triangle→2 comms Q=0.357, clique→1,determinism, empty-safe, weighted-cohesion); the Louvain core was verified standalone
first. Built against
jc/examples/splat_louvain_modularity.rs(the reference, per aparallel session's suggestion). Persists nothing → clear of the mint gate.
Plan (v1.2) —
.claude/plans/graphrag-doc-retrieval-soa-integration-v1.mdagnosticism is scoped to raw data + consumer modelling only. The 12-byte facet is
6×2×8bit(CLAM), never f32; 256 = ranking (needs cosine-replacement),256² = distribution (as accurate as content-blind f32); the
6×(8:8)register ispolymorphically
part_of:is_afamily-identity or palette256² centroid.rung-3 layer of the P3: wire the rung ascent loop + dedup RungLevel (D-TRI-6) #708 rung-ascent loop).
witness / node hold handles only, never raw bytes.
document 0x080Bmint; lance-graphprovides contract + query. graphrag-rs (external) = reference only.
jc::reliabilitybattery (P5a: jc reliability/validity metric battery (Pearson / Spearman / Cronbach α / ICC) #709/jc reliability: enforce the no-NaN contract on non-finite input (follow-up to #709) #710): P-COMMUNITY-BASIN-AGREE,P-HIER-LEIDEN-HHTL, P-MEASUREMENT-DETERMINISM, P-PQ-RANK.
Deferred (gated, not in this PR)
The
typed_field 0x080A/document 0x080Bmints +persist/reconstruct_documentActionDefs are operator-mint-gated behind the doc-W4 council (D-GR-5/6). This PR
lands only the design + the pure, reversible community-detection capability.
Board hygiene:
INTEGRATION_PLANS.mdupdated to v1.2 in the same commit.🤖 Generated with Claude Code
https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1