diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 5526d7d98..dace457d1 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,274 @@ +## 2026-08-21 — E-ABBREVIATION-GREP-MANUFACTURED-AN-ABSENCE-1 — I reported a shipped 15-module subsystem as non-existent because `fn .*ppr` matches `approx`, and a `head` limit hid the real hits + +**Status:** FINDING (self-inflicted, caught by the operator pointing at +`E-ARIGRAPH-IS-AN-ISLAND`). **Confidence:** High — the mechanism is +reproducible in one line. + +**What I claimed, in a deliverable handed to the operator:** *"the assumption +is stale — no `ppr`, `personalized_page*`, `bm25`, `rrf`, or `community` +function exists in the workspace… AriGraph appears only as narrative in doc +comments."* + +**What is actually there:** `crates/lance-graph/src/graph/arigraph/` — **15 +modules, ~327 KB** — `ppr.rs` (`PersonalizedPageRank`, +`personalized_pagerank()`), `bm25.rs` (`Bm25Index::{build,score,rank}`), +`rrf.rs` (`reciprocal_rank_fusion()`), `community.rs` (`Communities`), +`markov_soa.rs`, `episodic.rs` (`EpisodicBasins`), `witness_corpus.rs`, +`retrieval.rs`, `triplet_graph.rs`, `orchestrator.rs`, and five more, all +re-exported from `mod.rs`. + +**The mechanism, in two compounding parts.** + +1. **The abbreviation collided with a common word.** My pattern was + `fn .*ppr\|fn .*bm25\|fn .*rrf\|fn .*community`. `"ppr" in "approx"` is + **`True`** — `a-p-p-r-o-x`. Every `fn approx(a, b, tol)` test helper in + `jc`, `holograph`, and `sigker` matched. The real functions matched + *nothing*, because they are spelled out — `personalized_pagerank`, + `reciprocal_rank_fusion` — and `bm25` lives in the TYPE name, not the + function name. +2. **`head -20` then converted noise into a false negative.** `community.rs` + genuinely had 5 matching lines. They were pushed off the end of the output + by the `approx` flood. I read "nothing relevant in the first 20 lines" as + "absent." + +Neither part is sufficient alone: a noisy pattern with full output would have +shown the real hits; a clean pattern with a `head` limit would have found them +first. **A pattern with a high false-positive rate and an output limit are +individually survivable and jointly a lie.** + +**The rule.** *Searching for a function's NAME is not searching for a +CAPABILITY.* Before reporting a subsystem absent, search the **filesystem** for +it — `find . -path "*arigraph*"` would have ended this in one call, needed no +guess about spelling, and cost less than the grep did. Names encode an author's +abbreviation preference; directories encode the subsystem. Corollaries: + +- An abbreviation of ≤4 characters is a **substring**, not a token. Check it + against ordinary vocabulary before trusting it (`ppr`⊂`approx`, + `rrf`, `cam`⊂`camera`, `ppr`⊂`suppress`). +- **Never conclude absence from a limited-output search.** Absence is a claim + about the whole set; a `head` reads a prefix of it. Re-run with a `-c` count + or no limit before writing "does not exist". +- **Absence and unwiring have opposite remedies.** Absent ⇒ build the organs. + Unwired ⇒ build nothing, close the seam. I recommended the expensive one. + +**This is the second instance of a named class.** The board already carries +*"a false negative manufactured by the intake"* (the sorted-histogram gate that +discarded which cell mapped to which offset, codex P1 on #876). Same class, +different mechanism: there the intake destroyed structure before comparison; +here the intake's own noise outran the output window. Two instances with +disjoint mechanisms make it a pattern rather than an anecdote: **the intake is +part of the measurement, and it fails silently in whichever direction nobody +checked.** + +**What the correct answer would have been**, and it was already on this board +under `E-ARIGRAPH-IS-AN-ISLAND`: every AriGraph module exists and tests green; +the chain is open at the joints; `HotWitness` is `todo!()`; +`Ee→EW64(hot)+WitnessCorpus(cold)` is the unwired task. *"The most expensive +kind of gap: invisible in green suites (every crate passes; the system doesn't +do the thing) because the integrating seam was never built."* I cited that +entry as prior art for a different claim **without reading it** — and it +contained the correction to the claim I was making in the same breath. + +## 2026-08-21 — E-ACADEMIC-CARVE-UNDERFILLS-ROWS-ARE-NOT-WORDS-1 — 20,845 COCA rows are 18,559 distinct words, so the 80×256 academic carve fills 90.6% and basins 73..79 are empty + +**Status:** FINDING (measured on the committed +`crates/deepnsm/word_frequency/academic_20k.csv`, produced while building the +deepnsm-v2 academic codebook for S3). **Confidence:** High — a count over a +committed file, reproducible with `csv.DictReader` and a `set`. + +**The falsified claim** is in `plans/causal-rung-standing-wave-v1.md:465`: + +> **Vocabulary 4096 → 20k academic:** **fits** the palette256² pair carve +> NATIVELY — 20480 = 80×256, `(basin, identity) = (id>>8, id&0xFF)` + +It does not fit. It under-fills. + +| | measured | +|---|---| +| rows in `academic_20k.csv` | 20,845 | +| **distinct surface forms** | **18,559** | +| duplicate rows (same word, different `Pos`) | **2,286 (11.0%)** | +| `PaletteVocab::ACADEMIC_20K` reserved slots | 20,480 | +| **carve fill** | **18,559 / 20,480 = 90.6%** | +| last id 18,558 lands at | basin **72**, slot 126 | +| **empty basins** | **73..79 (7 of 80)** | + +**The inference that produced the error is arithmetic on the wrong noun.** +`plans/deepnsm-morton-comma-facet-v1.md:140` records "20,845 rows" and calls +them "the 20k most-frequent COCA-Academic words". 20,845 > 20,480, so the carve +looks like it fits with 365 to spare. But `PaletteVocab::from_frequency_ranked` +admits by **surface form**, deduplicating (`duplicates_keep_first_id` is its own +pinned test), and COCA lists a word once per part of speech. The same plan file +even notes the multiplicity two paragraphs later — *"The `Pos='v'` rows in +`academic_20k.csv` (thousands of verbs)"* — without drawing the consequence for +the carve. **The fact was present, the subtraction was not.** + +**Why no test caught it, and this is the reusable half.** The crate's own +`academic_20k_carve_spans_80_basins` asserts exactly the property that fails on +real data — `pair("w20479") == Some((79, 255))`, i.e. basin 79 is reached — and +it passes, because its fixture is `(0..20480).map(|i| format!("w{i}"))`: +**20,480 synthetic words that are all-distinct by construction.** The real +input is 11% duplicates. A dedupe-sensitive property was tested on a +duplicate-free fixture, so the test could only ever confirm the arithmetic it +was written from. + +This is the fixture-shape failure this board already carries in other clothes +(a single-band fixture cannot see a multi-band defect; a justified page cannot +exercise the ragged branch). Stated for this class: **when the code under test +deduplicates, filters, or otherwise reduces its input, a fixture with nothing to +reduce measures the reservation, not the fill.** The falsifier is one line — +give the fixture a duplicate and assert the admitted count drops. + +**What is NOT claimed.** 90.6% fill is not by itself a defect: `RESERVE, DON'T +RECLAIM` is canon, and a partially-filled carve with `(basin, slot) = (id>>8, +id&0xFF)` is correct, addressable, and stable. Nothing needs to move. What must +change is the **claim**: the carve is under-filled by 1,921 slots, seven basins +are empty, and any design that assumed basin 79 is populated — or that reads +basin occupancy as a frequency signal — is reasoning from a number the data +never supported. Whether to fill the tail (a larger COCA slice, or admitting +`(word, Pos)` pairs rather than surface forms) is a deliberate decision with a +semantic cost either way, not a gap to be quietly padded. + +> **⊕ CORRECTED SAME-DAY (2026-08-21, codex P1 on #975 + the operator's +> "wieso pyyaml — hattest du nicht für dismech-rs schon was für Rust +> gebaut").** The counts above came from a throwaway Python line-scanner and +> were **wrong**. Codex caught it structurally without needing the corpus: +> the module documents `pathophysiology[].downstream[]` as a SUBSET with +> **2,497** mediator-bearing edges, and I had written **2,489** corpus-wide — +> *a subset cannot exceed the whole*. Re-measured with the parser that +> already existed (`dismech_oracle_census`, dismech-rs, over +> `graph::build_causal_graph`), cross-checked against an independent pyyaml +> structural parse — identical on every figure: +> +> | quantity | line-scan (wrong) | structural (authoritative) | +> |---|---:|---:| +> | label-KNOWN with ≥1 mediator | 2,489 | **2,512** (63.1%) | +> | label-KNOWN naming nothing | 1,489 (37.4%) | **1,466** (36.9%) | +> | distinct mediator strings | 3,048 | **3,095** | +> | exact node references | 45 (1.5%) | **45** (1.5%) — unchanged | +> | groundable by label alone | 59.4% | **59.3%** (1,834) | +> | ungrounded prose | 40.6% | **40.7%** (1,261) | +> | oracle diseases | not measured | **549** | +> | `UNKNOWN_INT` that DO name mediators | **missed entirely** | **92** | +> +> There is no contradiction once the number is right: 2,512 corpus-wide vs +> 2,497 in the downstream subset, the 15 sitting in `influences_mechanisms` +> (115) and `sequelae` (19). **Every qualitative claim survives** — the +> mediators are prose, the oracle is ~⅔ of the label-KNOWN set, the +> label-only edges need a third bucket. The magnitudes moved by <1%. +> +> **Two lessons, and the second is the expensive one.** (1) A line-oriented +> YAML walk is a parser you did not write and cannot test; it fails silently +> on shapes you did not imagine. (2) **The repository already had the +> structural parser** — `model::parse_disorder_raw` is serde_yaml over a +> committed type and `graph::build_causal_graph` already carries +> `intermediate_mechanisms: Vec` on the edge. The right answer was +> one `cargo run` away, is sub-second against pyyaml's two minutes, and is +> now committed and re-runnable instead of living in a `/tmp` script. *A +> measurement a committed parser can make must not be made by an ad-hoc +> script.* This is the same family as +> `E-ABBREVIATION-GREP-MANUFACTURED-AN-ABSENCE-1` two entries up — reaching +> for an improvised tool over the one the repo already ships — twice in one +> session, which makes it the session's dominant failure mode rather than an +> incident. +> +> **A third finding neither pass had:** 92 `INDIRECT_UNKNOWN_INTERMEDIATES` +> edges DO name mediators. The source contradicts its own label. They are +> neither oracle nor restraint control, and left in the control they read as +> hallucinated closure by the benchmark's own definition. + +**Artifact.** The derived codebook was uploaded round-trip-verified to S3 at +`lance-graph/codebooks/deepnsm-v2-academic-coca-v1/` (TSV + source CSV + +manifest), with the shortfall stated in the file's own header so a consumer +cannot pin it without reading it. It reports 18,559 entries; it does not pad. + +## 2026-08-21 — E-DISMECH-KNOWN-INTERMEDIATES-ARE-PROSE-NOT-IDENTITIES-1 — the 3,978-edge "ORACLE population" is 2,489 edges, and its mediators are 5-word prose, not node references + +**Status:** FINDING (measured on upstream `monarch-initiative/dismech`, the same +2,100-file ephemeral `/tmp` corpus as `E-DISMECH-CORPUS-CENSUS-1`). +**Confidence:** High — every number is a count, and the first count was WRONG +in a way worth recording (below). + +**This CORRECTS one sentence of `E-DISMECH-CORPUS-CENSUS-1`** (append-only: that +entry is not edited). Its census is right and stands — 9,073 / 4,539 / 3,978 / +408, total 17,998. What it also said is this: + +> `IndirectKnownIntermediates` (3,978) is the ORACLE population — **the source +> names the mediators, so they can be hidden and recovery measured.** + +The source mostly does not name them, and where it does, they are not +identities. + +| | measured | +|---|---| +| `INDIRECT_KNOWN_INTERMEDIATES` edges | 3,978 | +| ...carrying >=1 named intermediate | **2,489** | +| ...with **no** `intermediate_mechanisms` key at all | **1,489 (37.4%)** | +| total intermediate strings | 3,424 (distinct 3,048) | +| strings per edge | mean 1.38, max 4 | +| **strings that are an exact node reference** | **45 / 3,048 (1.5%)** | +| string length | median 4 words, mean 5.2, max 39 | + +A representative mediator: *"Classical-pathway inhibition yields serum +resistance, permitting spirochete survival during hematogenous dissemination."* +That is a sentence, not an address into the 48,467 distinct mechanism/target +names the corpus carries. + +**Three consequences, in decreasing obviousness.** + +1. **The oracle population is 2,489, not 3,978** — a 37.4% overstatement. The + 1,489 key-less edges are `KNOWN_INTERMEDIATES` **in label only**: the authors + asserted mediators exist and did not write them down. They are neither + oracle nor restraint control. Scored as positives they are unrecoverable and + depress every metric; scored as negatives they punish a correct answer. They + need a **third bucket**, or they poison the gold set. (This is the standing + iron falsifier *"unknown intermediates are treated as negative examples"*, + reached from an unexpected direction — via the KNOWN label, not the unknown + one.) + +2. **`Recall@k` over a mediator-identity candidate list has nothing to score + against.** At 1.5% exact reference there is no identity-typed gold. A gold + set must be MADE, and the making must not leak: grounding the 3,048 prose + strings by **label matching alone** shares no machinery with DeepNSM or + AriGraph, so it cannot launder the answer into the evaluation. Measured + headroom against the 48,467 names: exact (normalized) 204 (6.7%), a corpus + name is a substring 1,190 (39.0%), token-Jaccard >= 0.5 415 (13.6%) — + **1,809 groundable (59.4%), 1,239 ungrounded prose (40.6%)**. ~1,800 gold + mediators over ~2,489 supervised edges is enough to separate ablation + levels. **The 40.6% residue must be reported, not dropped** — dropping it + silently inflates every Recall@k, which is the vacuous-fence pattern this + board already carries under a different name. + +3. **`DismechTopology::source_knows_intermediates()` is correctly implemented + and its second doc sentence is not.** The function answers *"does the source + CLAIM to know"*, which is a question about the label and is exactly right. + Its doc then says *"This is what separates the oracle population from the + restraint control"* — and that is the falsified claim, because the + label-KNOWN population is not the oracle population. `LATEST_STATE.md` + carries the same overstatement ("the 3,978-edge ORACLE population"). **No + bits move and no API changes**; the correction is to the claim, and the + consumer that builds the oracle set must additionally require a non-empty + `intermediate_mechanisms`, which the contract crate cannot see because it is + deliberately source-side only. + +**A free by-product, needing no grounding at all:** predicting the 4-way +`CausalTopology` of a masked edge is a benchmark over all **17,998** labelled +examples with a stated majority-class prior of 50.4% (DIRECT). It can run +before any grounding lands and calibrate the harness — which is worth having +precisely because the identity benchmark now needs a build step first. + +**The method note, which generalizes past DisMech.** The first run of this +measurement returned *"4 of 3,978 carry a named intermediate"* and I nearly +recorded that. It was false: YAML block-list items sit at the SAME indent as +their key, and the sibling-scan broke on `indent <= key_indent`, so it stepped +off the list before reading a single item. The tell was the shape of the +result — a 0.1% rate on a field the corpus populates 2,489 times is not a +finding, it is a parser that missed. **A count that collapses to near-zero on a +field the schema clearly uses is a claim about the reader, not the corpus** — +the same rule this board already states for null probe results, applied to +counting rather than to timing. Reading three raw blocks cost thirty seconds +and moved the answer by a factor of 622. + ## 2026-08-20 — E-DISMECH-CORPUS-CENSUS-1 — the DisMech corpus measured: 87.2 MB of strings, of which the entire causal semantics is bits + codebook ordinals **Status:** FINDING (measured on upstream `monarch-initiative/dismech`, 2,100 diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 5808e50a2..b5c150cc1 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -6,7 +6,15 @@ - `DismechTopology` — the four measured `causal_link_type` states, 2 bits, `from_source`/`as_source`/`to_bits_2`/`from_bits_2`, plus `source_knows_intermediates()` / `mediator_unresolved()` which separate the - 3,978-edge ORACLE population from the 4,539-edge RESTRAINT CONTROL. + label-KNOWN 3,978 edges from the 4,539-edge RESTRAINT CONTROL. ⚠ The + label-KNOWN set is NOT the oracle population: 1,466 of the 3,978 (36.9%) + name no mediator at all. The remaining **2,512** (549 diseases) is the + population REQUIRING GROUNDING, not a usable oracle — only 45 of 3,095 + distinct mediator strings are exact node references (1.5%); 40.7% stay + ungrounded prose after label matching. A further **92** + `INDIRECT_UNKNOWN_INTERMEDIATES` edges DO name mediators and belong to + neither population. Measured by `dismech_oracle_census`; + `E-DISMECH-KNOWN-INTERMEDIATES-ARE-PROSE-NOT-IDENTITIES-1` (2026-08-21). - `Supports` (4, 2 b), `EvidenceSource` (5, 3 b) — round-tripped exhaustively. - `CitationNamespace` (PMID/DOI/ORPHA/NCT/CGGV/URL), `CitationKey` (`Identified{namespace,id}` | `ContentAddressed(ContentId)`), diff --git a/.claude/plans/causal-rung-standing-wave-v1.md b/.claude/plans/causal-rung-standing-wave-v1.md index e30338c3d..3b80636d2 100644 --- a/.claude/plans/causal-rung-standing-wave-v1.md +++ b/.claude/plans/causal-rung-standing-wave-v1.md @@ -462,7 +462,14 @@ Mapping to shipped seams (each future item gets its own gated plan/probes): optional seed a consumer feeds (via OGAR) to lance-graph arm-discovery / DeepNSM (tesseract-rs consumer guide); spider-rs is the crawl ingress twin. Paper PDF/scan → doc.v1 → SPO stream = this plan's input, no model anywhere. -- **Vocabulary 4096 → 20k academic:** fits the palette256² pair carve +- **Vocabulary 4096 → 20k academic:** ⚠ **CORRECTED 2026-08-21** — it does NOT + fit; it UNDER-FILLS. `academic_20k.csv`'s 20,845 ROWS are only **18,559 + distinct surface forms** (2,286 duplicates: same word, different `Pos`), and + `from_frequency_ranked` admits by surface form. The carve fills + **18,559/20,480 = 90.6%**; basins **73..79 are empty**. The addressing below + is unaffected and correct. See + `E-ACADEMIC-CARVE-UNDERFILLS-ROWS-ARE-NOT-WORDS-1`. + Original claim, retained: fits the palette256² pair carve NATIVELY — 20480 = 80×256, `(basin, identity) = (id>>8, id&0xFF)` — the "larger codebooks" case already in the facet design; the codebook lives on the Morton tile pyramid / gridlake SoA (the 256=4⁴ hierarchical-centroid diff --git a/crates/deepnsm-v2/src/vocab.rs b/crates/deepnsm-v2/src/vocab.rs index dfcef5538..655bd3938 100644 --- a/crates/deepnsm-v2/src/vocab.rs +++ b/crates/deepnsm-v2/src/vocab.rs @@ -151,6 +151,52 @@ mod tests { assert_eq!(v.id("dog"), Some(2)); } + /// The companion the synthetic carve test cannot be: its fixture is + /// all-distinct by construction, so it measures the RESERVATION and can + /// never see a fill shortfall. + /// + /// Real `academic_20k.csv` is 20,845 rows but only **18,559 distinct + /// surface forms** (2,286 duplicates — COCA lists a word once per part of + /// speech), and `from_frequency_ranked` admits by surface form. So the + /// academic carve fills 90.6% and basins 73..79 stay empty + /// (`E-ACADEMIC-CARVE-UNDERFILLS-ROWS-ARE-NOT-WORDS-1`, 2026-08-21). + /// + /// This pins the mechanism at unit scale: duplicates consume RANKS but not + /// SLOTS, so the last admitted id is strictly below the input length and + /// the reserved tail is unreachable. Falsified by deleting the + /// `contains_key` guard in `from_frequency_ranked` — the admitted count + /// then equals the input length and both assertions fail. + #[test] + fn duplicates_consume_ranks_but_not_slots_so_the_carve_underfills() { + let mut v = PaletteVocab::new(); + // 300 ranks; the 100 multiples of 3 are all the SAME word, so the + // distinct set is 200 unique `w` forms + one `the` = 201. + let words: Vec = (0..300) + .map(|i| { + if i % 3 == 0 { + "the".to_string() + } else { + format!("w{i}") + } + }) + .collect(); + let admitted = v.from_frequency_ranked(words); + assert_eq!(admitted, 201, "duplicates must not consume slots"); + // The anti-vacuity half: the fixture really does contain duplicates, + // so a dedupe-free implementation would differ here. + assert!(admitted < 300, "fixture has nothing to deduplicate"); + // The consequence that matters: the tail of the reserved range is + // unreachable from this input. + assert_eq!(v.pair("the"), Some((0, 0))); + // 99 duplicate ranks were skipped before it, so the last input word + // lands 99 slots below its own rank. + assert_eq!(v.id("w299"), Some(200)); + assert!( + v.len() < 300, + "last admitted id sits below the input length -- the reserved tail is empty" + ); + } + #[test] fn academic_20k_carve_spans_80_basins() { // The 80×256 academic carve fills basins 0..80 in frequency order. diff --git a/crates/lance-graph-contract/src/dismech_evidence.rs b/crates/lance-graph-contract/src/dismech_evidence.rs index c89f76ad2..f85fa5a1e 100644 --- a/crates/lance-graph-contract/src/dismech_evidence.rs +++ b/crates/lance-graph-contract/src/dismech_evidence.rs @@ -123,9 +123,33 @@ impl DismechTopology { /// Does the SOURCE claim to know the intermediates? /// /// True only for [`IndirectKnownIntermediates`](Self::IndirectKnownIntermediates). - /// This is what separates the oracle population from the restraint control: - /// for the other three a recovered mediator is not a success, it is an + /// For the other three a recovered mediator is not a success, it is an /// unsupported claim. + /// + /// **This is NOT by itself the oracle population** (corrected 2026-08-21, + /// `E-DISMECH-KNOWN-INTERMEDIATES-ARE-PROSE-NOT-IDENTITIES-1`). Measured + /// over the 2,100-file corpus by `dismech_oracle_census` (dismech-rs, + /// `graph::build_causal_graph`), cross-checked against an independent + /// structural parse: **1,466 of the 3,978 label-KNOWN edges (36.9%) carry + /// no `intermediate_mechanisms` at all** — the source asserts mediators + /// exist and does not name them. Those are neither oracle nor restraint + /// control and need a third bucket, or they poison a gold set in both + /// directions. A consumer building the oracle population must + /// additionally require a non-empty `intermediate_mechanisms`, which this + /// crate cannot check because it is deliberately source-side only and + /// never sees the edge body. + /// + /// Consistent with the `pathophysiology[].downstream[]` figures below by + /// scope, not by disagreement: 2,512 corpus-wide against 2,497 in that + /// subset, the 15-edge difference sitting in `influences_mechanisms` + /// (115 label-KNOWN edges) and `sequelae` (19). + /// + /// **2,512 is the population REQUIRING GROUNDING, not a usable oracle.** + /// Only 45 of 3,095 distinct mediator strings are exact node references + /// (1.5%); label matching grounds 1,834 (59.3%) and leaves 1,261 (40.7%) + /// ungrounded prose. Ranking candidate identities against that population + /// without grounding it first either compares identities to prose or + /// silently drops the residue — both corrupt any reported recall. #[must_use] pub const fn source_knows_intermediates(self) -> bool { matches!(self, Self::IndirectKnownIntermediates)