Graph structured store: replace single parent branches with multi parent entity nodes

[ram0verflow]

Problem

A fact currently lives in exactly one place. MemoryStore is a BTreeMap<String, Branch>, each Branch owns a Vec<VersionedValue> of details, and a detail belongs to that branch and to nothing else. That is a single parent tree, and the parent is a topic name chosen by an 8B classifier at write time.

Retrieval into generation does not use that tree at all (see #8): it is the driver's message index, which is lexical, BM25 plus embeddings over raw message text. The consequence is structural, not a tuning problem: a question can only reach a fact whose words overlap the question's words.

When an answer needs two facts and the question shares vocabulary with only one of them, the other is unreachable no matter how good the reranker is.

Why it matters

This is the failure that survived every other fix in the repo, and we now have it measured twice in different clothing (see #13 and 9ece598):

• Composition, "am I over my monthly API allowance", needs the plan limit and the month's usage. 1 of 7 at scale.
• Date shift, "when is my dentist appointment", needs "October 14th" and "pushing everything back a week". The second mention shares no token with "dentist", so it usually does not load and the model answers the stale date. 3 of 7 at scale.

Meanwhile the parts that are not retrieval bound are fine: contradiction chains 7 of 7, arithmetic through the calculator 7 of 7. The answer model is not the weak link. Reachability is.

Hypothesis

Replace single parent branches with multi parent entity nodes: a fact attaches to every entity it mentions rather than being filed under one classifier chosen topic. A statement like "took the cat to the vet on the 3rd" becomes reachable from cat, from vet, and from the date, instead of living under whichever single branch won.

Retrieval then becomes: resolve the entities in the question, then traverse edges, rather than match tokens. Composition becomes a walk between two nodes rather than a lexical coincidence.

Related work

• AriGraph (arXiv 2407.04363), the closest structural precedent: a memory graph integrating semantic memory (facts as KG vertices and edges) with episodic memory (specific experiences as episodic vertices and edges), supporting associative retrieval of interconnected concepts for planning.
• HeLa-Mem (arXiv 2604.16839): models conversation history as a dynamic Hebbian graph where co-activated memories strengthen their connections, retrieves by base activation then spreading activation, and consolidates hub clusters to prevent graph explosion. Evaluated on LoCoMo, the same benchmark we use, so its numbers are directly comparable to ours.
• HyMem (arXiv 2602.13933): dual granular storage with a two tier retrieval scheduler, a cheap summary path plus an LLM based deep path activated only for complex queries. Relevant to keeping graph traversal off the hot path, since our retrieval budget is about 50ms against 2 to 3s of generation.

The cold start caveat we have to plan around

HeLa-Mem states the limitation directly: "Hebbian weights require sufficient interaction history to accumulate, meaning the benefits of associative retrieval are less pronounced in early conversation stages." Their proposed remedy, initializing edges from semantic similarity as a prior, is left as future work.

This lands on us specifically. Every discriminate run is a 34 to 54 turn transcript, which is exactly the regime where a graph whose edges are learned has not warmed up. If we adopt an association learned graph and test it on our current harness, we will measure the cold start, not the mechanism. Two implications:

1. Seed edges deterministically at ingest from entity co-occurrence, which is available on turn one, rather than relying only on learned weights.
2. Any evaluation of this has to include long transcripts, which is the same blocker as Scale the discriminate harness to dozens of runs per condition #17.

[ram0verflow]

Design notes and the cheapest first experiment.

What our own data already says. Retrieval is the binding constraint. The strongest single piece of evidence: the one composition pass out of seven came from store_context putting the plan size directly into the prompt (9ece598). When the fact was reachable, the model composed it correctly without help. That is the whole argument for this issue in one data point, and it is also why #11 is worth revisiting rather than treating as settled.

Why the existing tree is not salvageable for this. The ablation in #1 showed tree routing contributes nothing to QA (0.455 without it against 0.449 with). A single parent hierarchy over classifier chosen topic names is the wrong index for "reach this fact from any of its entities". We keep the tree because online ingestion builds it for free, not because it retrieves well.

Sketch. Entity extraction at ingest should be deterministic and LLM free, which we already do in two places: symbol extraction in codegraph.rs and tokenization in hierarchical.rs. Node per entity, edge per co-mention, fact nodes carry the versioned value and provenance the store already tracks. Keep the message index as a parallel path so this can be A/B tested rather than swapped in.

Cheapest first experiment, and it should come before any prompt shape change. Use the graph only to select which messages load, keeping the prompt identical to today. That isolates "does graph reachability find the right facts" from "does the model use a new context format", which is the same discipline that made #11 interpretable. If graph selection moves composition off 1 of 7 with an unchanged prompt, the case is made.

Note on provenance for this issue: the framing above is reconstructed from repo evidence (the store layout, the ablation table, the scale results in 9ece598) and from verified published work. The original multi parent tagging example from the research discussion was not available when this was filed and should be added here.

[ram0verflow]

The n of 1 signal, and the experiment now built to test it.

Recording the reason this issue should be decided by evidence before any rewrite is committed to.

Across every composition attempt in the repo, exactly one has ever succeeded: the long transcript run with store_context on, where the model answered "Yes, you are over your monthly API allowance by 12,000 calls." In that run the store block carried the plan size into the prompt, which is the fact the driver index structurally cannot reach from the question's vocabulary. Every other run, including all the store off runs, failed the same way.

That is the first and only evidence anywhere in this project that the store's runtime path does something the message index cannot. It is also n of 1, from a setting that was measured as useless and shipped default off in #11. Both things are true, which is exactly why it needs replication rather than a decision.

stress_disjoint.py (c2ab113) is that replication, and it is deliberately narrower than the discriminate harness. Eight composition cases, all in the shape of the allowance case: two facts, where the question shares vocabulary with one of them and the second is lexically disjoint from both the question and the first fact. The disjointness is verified programmatically before any model time is spent, and the harness prints the shared token check per case:

case               a/b shared             q/b shared
  api allowance      ['thousand']           none
  flat affordability none                   none
  drive capacity     ['gigabytes']          none
  grant deadline     ['march']              none
  eng headcount      none                   none
  laptop battery     ['hours']              none
  gift budget        none                   none
  storage tier       ['gigabytes']          none

Zero of eight questions can reach the second fact lexically (one case originally could, and was rewritten). Shared unit words between the two facts are left in place because they are realistic and weak.

Run store context on against off across all eight, same protocol, with per answer store_topics attribution logged as in the original #11 experiment.

How this decides the issue. If store context wins clearly on these cases specifically, that is the strongest argument this project has produced for prioritising the graph work here, because it demonstrates the value of a reachability path that does not depend on lexical overlap, which is precisely what multi parent entity nodes generalise. If it does not replicate, that gets stated plainly and this issue drops in priority behind the cheaper items in the milestone.

Blocked on AWS re-auth, staged to run.

[ram0verflow]

Replication result: the n of 1 store signal did not replicate. And the sweep produced a better argument than the one it retired (0630586).

stress_disjoint.py, eight cases, disjointness verified programmatically (0 of 8 questions can lexically reach the second fact), store context on vs off, fresh state each, per answer attribution.

Result: 5 of 8 both conditions. A dead tie. (One correction en route: the grader produced its first false NEGATIVE, a correct "you are over the engineering hiring budget" scored as a miss because the needle set was too narrow; fixtured and fixed, tallies corrected by hand.) Attribution makes the tie worse for the store than it looks: the composition passes in both conditions were fault-chain-plus-calculator work with zero store topics paged. Of the ON condition's answers, seven paged store topics and it won nothing net by having them. Per the pre-commitment in this issue's earlier comment: that gets stated plainly, and the full graph rewrite drops in priority.

But the sweep surfaced the sharpest reachability evidence yet, and it does not involve the store. OFF condition, eng headcount case: the model faulted correctly for "total number of engineers hired this year". The re-page found nothing. The stored fact says "15 people on the platform team": not one shared word with the fault topic. The fault chain names the missing thing in the model's vocabulary and searches in the user's, so lexical retrieval fails across that gap even when the trigger problem is solved. No prompt fixes this; it is a property of retrieval-by-shared-words.

Second observation, same sweep: cases that share unit vocabulary bleed into each other in both conditions. One answer computed 620 - 140, subtracting the storage tier case's number from the drive case's, because both facts mention gigabytes. So lexical reachability is too blunt in both directions at once: it misses what is phrased differently and conflates what is phrased alike.

Where this leaves the issue. Lower priority than the fault-chain and scale work, per the replication result. But the hypothesis is now cleaner than when filed: the case for entity level indexing no longer rests on store context (retired) or on the n of 1 (retired), it rests on the vocabulary mismatch trace, which is exactly the failure an entity node for platform team / engineers would bridge. The cheapest next probe is correspondingly smaller than a graph rewrite: at fault time, expand the re-page query through the embedder (semantic neighbours of the fault topic) before giving up, and see if the eng headcount trace flips. If semantic expansion at the fault boundary is enough, the full graph may be overkill; if it is not, that is one more measured argument for it.

[ram0verflow]

Probe result (3ee7bd2): the trace flipped. And the graph question changed shape because of it.

fault_block_semantic unions the fault topic's pure dense neighbours into the re-page, candidate gate bypassed (the original miss happened because the target had zero BM25 hits and was not in a beamed leaf, so the dense scorer never even saw it). Rerun of the full disjoint transcript, the one that deterministically produced the miss:

[fault] 'current number of engineers hired this year' -> repage 4326 chars, semantic
[calc]  '15 - 12' -> = 3
reply:  'Yes, you are over the engineering hiring budget by 3 engineers.
         You budgeted for 12 engineers but have hired 15.'

Miss to hit, both store conditions, with the semantic flag on the repage for attribution. Vocabulary mismatch on the fault path is bridgeable at retrieval time, no graph required.

The other hand, from the same sweep, as the review demanded both be measured together:

1. The store on/off tie did not break: hand adjudicated 6 of 8 (off) against 5 of 8 (on). Still no store context win, now even with expansion live.
2. The conflation got worse where it was already worst. Both conditions now import the storage tier's 140 gigabytes as the drive's current usage and conclude a 620 GB library fits a 500 GB drive, with both correct numbers present in the same sentence. The ON condition passed this case in the previous sweep and fails it now. The wider net bridges gaps and feeds conflation with the same motion; the tension named in the review is real and measured, not hypothetical.

Where this leaves the issue. The recall half of the original hypothesis is now covered by a 40 line retrieval change. What expansion cannot do, and demonstrably makes worse, is keep facts attached to their owners. So this issue's surviving thesis is narrower and sharper: a graph buys precision, not recall: an entity scoped edge means the storage tier's 140 cannot drift into the drive's arithmetic just because both facts say gigabytes. The cheapest next probe changes accordingly: not "does the graph find more", but "does entity scoping stop the drive/storage bleed while keeping the eng headcount fix", which the disjoint harness already measures from both sides. Also worth an isolating A/B first: expansion off/on with store fixed, drive case only, to pin how much of the new bleed is expansion versus retrieval variance (two variables moved in the ON condition).

[ram0verflow]

Entity-scoping probe complete (8704434, a558067). It failed, and the failure rules out every cheaper alternative to this issue, which is the strongest form the argument has taken.

Built: fault_block_semantic(topic, budget, scope_entities) gates the semantic expansion so a dense neighbour is kept only if it shares a content entity (a token that is not a stopword, unit word, or number) with the fault topic. content_entities() drops unit words specifically so gigabytes cannot bridge the drive and storage-tier facts. Behind fault_scope_entities, default off.

The unit test settles the theory before any model runs. The drive fault shares no content entity with the storage 140 fact (scope drops it, fixing the bleed), but the engineers fault also shares no content entity with the "platform team" fact (scope drops it too, re-breaking the fix from the last round). Both needed facts are content-disjoint from their fault in the identical way. A lexical filter that fixes the conflation necessarily breaks the reach: lexically they are the same situation.

The measurement said more than the theory. Attribution runs with loop_trace captured:

• Sparse (drive + storage only, expansion toggled): expansion off answers correctly (620 vs 500, will not fit); expansion on runs calc(620 - 140) and concludes it fits. Expansion is the sole cause of the bleed here.
• Dense (drive + storage + eng): the trace shows no page fault at all, only calc(620 - 140). The model never faulted, because base route_query on the original question had already placed the storage 140 in the initial working set next to the drive numbers. Three plausible gigabyte figures are enough for it to pick a wrong pair and compute with confidence. Scoping the expansion changes nothing here: there is nothing to scope.

Two conclusions.

1. The bleed has a density-dependent entry point (expansion when sparse, base retrieval when dense), so no fault-time fix can cover it. The fix has to be at base retrieval.
2. Conflation is invisible to the fault protocol. The whole fault design rests on the model knowing when it is missing something; a wrong-but-plausible number handed over by retrieval is exactly the case where it does not know, so it never faults. This is the same shape as the silent-staleness date case (Fine tune round 3: teach the model to fault when a chain is missing a link #18).

What this leaves for this issue. Ruled out, with evidence: retrieval widening (causes the bleed), lexical entity-scoping of the expansion (no effect in the dense case, and analytically cannot thread the needle). What survives is exactly this issue: an entity-scoped base retrieval where "platform team" and "engineers" are linked by a semantic/structural edge and "storage tier" and "drive" are not. Not a token filter and not a wider net. The cheapest honest next step is no longer a shortcut around the graph, it is a minimal version of the graph: entity nodes from deterministic extraction at ingest, base retrieval resolving question entities to nodes and walking edges, measured on this same disjoint harness with the judge grading. That is the experiment that either earns the full build or kills it.

[ram0verflow]

Graph MVP built and measured (10386ef, 8742c54). It does not earn the full build on its own, but it is the most useful kind of negative: it confirms the precision mechanism and closes off the shortcut with evidence.

What was built. At ingest each message attaches to its content entities (content_entities, units and numbers dropped) building entity_msgs, and each entity is embedded once into entity_emb. entity_route resolves a query's content entities to the nearest entity nodes by embedding and returns their messages. The kernel's page_in REPLACES the lexical route with the entity route when entity_routing is on (a union cannot fix precision; the graph must select a cleaner set, not a larger one), falling back to lexical only when the graph resolves nothing.

Result. Disjoint harness, entity vs lexical base, fault expansion off so the base index is the only variable, judge-graded: graph 4/8, lexical 5/8 — one worse. The two cases that moved are the entire finding:

case	lexical	graph	why
drive capacity	FAIL (loaded 30, bled 140, "480 free, will fit")	PASS (loaded 15, no 140, "620 vs 500, will not fit")	precision win, exactly the predicted mechanism
storage tier	PASS	FAIL ("I need to know your current storage usage")	same orphaning, opposite sign
eng headcount	FAIL (reached engineers-12 only)	FAIL (reached neither)	reach not achieved
laptop battery	PASS	FAIL (reached flight, not battery)	reach not achieved

The mechanism, both directions. The storage-140 message's only content entity is the vague "keeping", so it hangs off an orphan node. For the drive question that is precision: the 140 is a distractor and the graph never reaches it, so drive is fixed. For the storage-tier question that is a loss: the 140 is the answer, and orphaned from everything it is unreachable by the one question that needs it. The same property that fixes one case breaks the other. Reach failed identically for the cross-vocabulary hops (battery did not resolve to the laptop charge fact, engineering did not resolve to the platform team) because the only link on offer was raw embedding similarity between a query word and an entity string, which is too weak across vocabulary and too blunt across unit words.

Verdict. The cheap embedding-resolution graph trades precision for reach and nets slightly worse, so it does not earn the full build. But it is not a null result. It proves the precision half of this issue's thesis is real (drive fixed by orphaning the distractor, the loaded count drops 30 -> 15 and the wrong number is gone), and it localizes the two things the shortcut omitted, both load bearing:

1. Coreference-aware extraction. "140 gigabytes up there" must attach to storage, not to "keeping". Naive per-message content extraction orphans facts that refer to earlier context, which is what broke storage-tier.
2. Entity-to-entity edges. "engineers" and "platform team" have to become one reachable thing without sharing a word, which query->entity embedding similarity cannot do. That is edges learned from co-occurrence, i.e. the Hebbian association in Reachability driven eviction priority and Hebbian edge reinforcement from the same recall event #15, and it carries HeLa-Mem's cold-start caveat with it.

So the next measurable step is not the full store rewrite, it is those edges (plus coref extraction) on this same harness with the judge. That experiment decides the build. The shortcut around the graph is now closed with data. Caveat: one run per condition; the drive result is a single run, but the mechanism is legible in the loaded counts rather than inferred from the score. Cross-links: #15 (Hebbian edges, now directly motivated), #19 (extraction quality).

[ram0verflow]

Correction and the deterministic verdict (commit 7b611f0)

I need to retract part of my earlier comment. The claim that the graph "fixes drive by excluding the orphaned 140, precision mechanism confirmed" did not replicate and was an n=1 artifact. Re-running the four-condition set, the same conditions flipped verdicts (baseline drive PASS one run, FAIL the next, on identical retrieval), which meant the judge-graded end-to-end was noise-dominated. So I cut the model and judge out entirely and measured retrieval composition directly (examples/retrieval_probe.rs, deterministic).

What the lexical route actually does: loads ALL 14 planted facts on every query (n=24 with distractors). Perfect recall, zero selectivity, because the corpus is tiny. The drive/storage conflation is therefore entirely model-side — handed every fact, the model picks the wrong same-unit pair about half the time. That is the coin-flip that made the end-to-end numbers unreadable.

What entity routing actually does: reaches only one of the two facts each synthesis question needs — drive.a (500) but not drive.b (620); eng.a (12) but not eng.b (platform team); storage.a (100) but not storage.b (140). Its apparent drive "precision" (no 140) is moot, because it also drops the 620 the question needs. It is not selecting a cleaner set; it is selecting a too-small, often-incomplete one.

The exact, deterministic killer. For the drive query "will the backup fit on the drive?" (entities {backup, drive}), ranked by entity-embedding similarity across the whole store:

entity	fact	score	rank
drive	drive.a = 500	1.000	0
keeping	storage.b = 140 (distractor)	0.546	3
external	drive.a = 500	0.520	11
photo	drive.b = 620 (needed)	0.465	35
library	drive.b = 620 (needed)	0.448	42
weighs	drive.b = 620 (needed)	0.399	63

The storage-tier distractor (140) is the 3rd-closest entity in the entire store; the drive question's own needed fact (620) sits at ranks 35-63. The wrong number is ~10x nearer than the right one, and no threshold admits the 620 without first admitting the 140 and thirty other entities. For disjoint synthesis, vocabulary proximity is anti-correlated with relevance.

Verdict: the cheap entity graph does not earn the build. Not because of a tuning miss but because the premise fails: the fact a question needs can be further in embedding space than the distractor that ruins it. A fix has to use structure that is not embedding proximity (the two facts of a case share a topic/event, not a vocabulary), which co-occurrence edges cannot supply here (see #15: the paired facts never co-occur).

Methodology note for the harness (relates to #23). Two things this exposed: (1) n=1 end-to-end with a generative model + LLM judge is too noisy to resolve these effects; the deterministic retrieval probe is the right instrument and should be the primary metric for retrieval changes. (2) The disjoint harness is too small to test precision at all — lexical just loads everything, so it never has to select. A precision test needs a corpus large enough that retrieval must discriminate. Both are worth fixing before the next retrieval experiment.

[ram0verflow]

HippoRAG / LightRAG evaluated against the deterministic probe — and the result says do NOT integrate, the fix is much cheaper

Environment correction first

My earlier rule-out of HippoRAG ("no torch 3.14 wheels") was wrong and is retracted. torch ships 3.14 wheels (2.9.0+). HippoRAG's real blocker was a hard pin to torch==2.5.1, which resolves cleanly on Python 3.12. Installed and ran real published HippoRAG 2.0.0a3 (OpenIE via local Ollama qwen2.5:14b, native facebook/contriever embeddings, its PPR retrieval) and LightRAG 1.5.4 (LLM entity/relation extraction, dual-level retrieval). No reimplementation.

Deterministic composition, four+ methods (rank of the SECOND operand each synthesis question needs, and the distractor)

case (needed 2nd operand)	our entity-graph (nomic on entity tokens)	nomic PASSAGE dense	contriever PASSAGE dense	HippoRAG (contriever + PPR)	LightRAG
drive.b (620)	~rank 40	2	2	1	returns all 16
eng.b (platform team)	unreachable	2	2	5 (graph hurt)	returns all 16
storage.b (140)	missed	1	4	2	returns all 16
laptop.b (6h)	1	1	1	1	returns all 16

What this shows

1. Plain passage-level dense retrieval, with the nomic model we already run, ranks both operands into the top-3 on all four cases. It matches or beats HippoRAG.
2. Our entity-graph (Graph structured store: replace single parent branches with multi parent entity nodes #14) failed because it embedded entity tokens, not passages. Embedding "My photo library weighs in at 620 gigabytes" as a passage puts it at rank 2 for the drive query; embedding the token "library" put it at rank ~40. The graph destroyed the context dense retrieval needs. That was the bug, not the embedding model and not the absence of a graph.
3. HippoRAG's PPR adds nothing positive here. Attribution control (pure contriever dense, no graph) ranks the operands the same or better than full HippoRAG; on eng the graph made it worse (rank 2 -> 5). And HippoRAG's own OpenIE triples show the two drive facts share no node (external drive/500 vs photo library/620), so PPR cannot bridge them — the good drive rank is dense-driven, not traversal-driven.
4. contriever is no better than nomic at the passage level (drive 2=2, eng 2=2, laptop 1=1; storage nomic 1 beats contriever 4). So there is not even an embedding-model case for switching.
5. LightRAG returns the whole corpus for every query (load-everything on 24 docs) — indistinguishable from the lexical baseline, inherits the same model-side conflation.

Integration verdict (Step 1 gate): NOT a graph win, so Step 2 (HippoRAG integration sketch) is not warranted

The retrieval improvement that fixes 3 of our 4 failures is available from passage-level dense retrieval we already have — no graph, no PPR, no new embedding model, no LLM-at-ingest OpenIE. Integrating HippoRAG would add a heavy ingestion cost (its OpenIE hit a 21/24 parse-failure rate with local qwen2.5:14b because the extractor is GPT-4o-format-tuned; a faithful deployment needs GPT-4o-class OpenIE, defeating local-first) for zero demonstrated retrieval gain over plain dense.

The actually-supported change (sketch only — NOT built, per your hold-for-review instruction)

• In hierarchical.rs, the entity-token graph route (entity_route) is a dead end; keep it disabled. The candidate-generation improvement is to ensure the existing BM25+embedding route ranks and selects a tight top-k of passages rather than loading everything within budget on small corpora (the load-everything behaviour is what hands the model the distractor).
• Kernel eviction, fault protocol, budget, dedup: unchanged, out of scope.
• Latency: zero new cost — reuses the nomic passage embeddings already computed at ingest; no query-time LLM.

Residual problem this does NOT fix (still #24)

Even perfect passage retrieval keeps the distractor near the operands: for the drive query the storage-140 ranks between drive.a and drive.b (rank 1, operands at 0 and 2). So a top-3 still hands the model the 140. The conflation is model-side and needs the answer-time attribution guard tracked in #24; retrieval ranking cannot eliminate it.

Honest caveats

• The harness is too small (24 docs) to test selectivity end-to-end; the rank comparison above is the reliable signal, not presence.
• HippoRAG's PPR was not tested on a fully healthy graph (21/24 OpenIE parse failures with the local model). But the attribution control shows plain passage dense already reaches the operands, so there is no headroom for PPR to prove value on this set even with perfect extraction. A GPT-4o OpenIE re-run could be done if we want to close that gap, but it would not change the integration decision.

Not wiring anything into the live driver pending review of this comparison.