Two measured corrections: the DisMech oracle population, and the academic carve fill - #975
Conversation
…dentities E-DISMECH-CORPUS-CENSUS-1 (2026-08-20) established the causal_link_type census and stands. One sentence of it does not: "the source names the mediators, so they can be hidden and recovery measured". Measured on the same 2,100-file corpus: 1,489 of the 3,978 label-KNOWN edges (37.4%) carry no intermediate_mechanisms key at all, and of the 3,048 distinct mediator strings that do exist only 45 (1.5%) are an exact node reference. The rest are prose with a median length of 4 words. Consequences recorded in the new entry: the oracle population is 2,489, not 3,978; the 1,489 label-only edges need a third bucket or they poison a gold set in both directions; and Recall@k over a mediator-identity candidate list has nothing to score against until a gold set is built by an independent path (label matching alone grounds 59.4%, and the 40.6% residue has to be reported rather than dropped). DismechTopology::source_knows_intermediates() is correctly implemented -- it answers a question about the label -- so no bits move and no API changes. Corrected here are the two places that overstate what the label buys: that function's second doc sentence and the LATEST_STATE summary. Gates: lance-graph-contract 1180/1180 lib + 21 doctests, fmt clean.
Measured while building the deepnsm-v2 academic codebook for S3.
causal-rung-standing-wave-v1.md:465 claims the 20k academic vocabulary
"fits the palette256 pair carve NATIVELY -- 20480 = 80x256". It does not.
academic_20k.csv has 20,845 rows but only 18,559 distinct surface forms
(2,286 duplicates -- COCA lists a word once per part of speech), and
PaletteVocab::from_frequency_ranked admits by surface form. The carve
fills 18,559/20,480 = 90.6%; the last id lands at basin 72 slot 126, so
basins 73..79 are empty.
The inference that produced the claim was arithmetic on the wrong noun:
20,845 rows > 20,480 slots looks like a fit. The same plan file notes the
Pos multiplicity two paragraphs later without drawing the consequence.
No test could catch it. academic_20k_carve_spans_80_basins asserts exactly
the property that fails on real data -- pair("w20479") == (79, 255) -- and
passes, because its fixture is 20,480 synthetic words that are all-distinct
by construction. A dedupe-sensitive property tested on a duplicate-free
fixture can only confirm the arithmetic it was written from.
Added the companion falsifier: a fixture that actually has something to
deduplicate, pinning that duplicates consume ranks but not slots, with an
anti-vacuity assertion that the fixture is not duplicate-free. Disable-
verified -- removing the contains_key guard turns it red (and the
pre-existing duplicates_keep_first_id with it); restoring it turns both
green.
Nothing moves: RESERVE-DON'T-RECLAIM means a 90.6%-full carve is correct
and addressable. What changes is the claim, and any design that read basin
occupancy as a frequency signal or assumed basin 79 was populated.
Artifact uploaded round-trip-verified to
lance-graph/codebooks/deepnsm-v2-academic-coca-v1/ (TSV + source CSV +
manifest); the TSV states the shortfall in its own header so a consumer
cannot pin it without reading it. It reports 18,559 entries, unpadded.
Gates: deepnsm-v2 109/109, fmt clean, clippy -D warnings clean.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_c73812ff-78fa-46dc-a050-0bec0a600a29) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3fa7acb451
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| label-KNOWN set is NOT the oracle population: 1,489 of the 3,978 (37.4%) | ||
| name no mediator at all, so the usable oracle is **2,489** — | ||
| `E-DISMECH-KNOWN-INTERMEDIATES-ARE-PROSE-NOT-IDENTITIES-1` (2026-08-21). |
There was a problem hiding this comment.
For mediator-identity Recall@k experiments, the 2,489 non-empty edges are not yet a usable oracle: this commit's own census says only 45 of 3,048 distinct mediator strings are exact node references and that 40.6% remain ungrounded even after label matching. Treating all 2,489 as usable gold will either compare candidate identities against prose or silently discard the unresolved residue, corrupting the reported recall; describe this as the population requiring grounding rather than the usable oracle.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed, and this is the sharper of the two — "usable oracle" was the wrong noun and the numbers in the same commit contradict it.
Reworded to the population REQUIRING GROUNDING, with the reason inline rather than left implicit: 45 of 3,095 distinct mediator strings are exact node references (1.5%); label matching grounds 1,834 (59.3%) and leaves 1,261 (40.7%) as ungrounded prose. Both failure modes you name are now stated where the number is — comparing candidate identities against prose, or silently discarding the residue — so a consumer can't read 2,512 as a gold set.
The count also moved: 2,512, not 2,489 (see the other thread — the line-scanner was wrong, the structural parser corrects it). The grounding ratio is essentially unchanged, which is the part that matters here: the population is ~⅔ of the label-KNOWN set and roughly ⅗ groundable, so the residue has to be reported rather than dropped in any Recall@k that uses it.
Fixed in 5577728.
Generated by Claude Code
| /// over the 2,100-file corpus, **1,489 of the 3,978 label-KNOWN edges | ||
| /// (37.4%) carry no `intermediate_mechanisms` key at all** — the source |
There was a problem hiding this comment.
The new corpus-wide count implies 2,489 label-KNOWN edges have intermediates, but the same module documents pathophysiology[].downstream[] as a subset of that corpus with 2,497 such edges (lines 167–182). A subset cannot contain eight more qualifying edges than the full census, so at least one parser scope or count is wrong; consumers cannot reliably construct the gold population until these measurements are reconciled.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Confirmed, and you were right without needing the corpus — a subset cannot exceed the whole, so at least one count had to be wrong. It was mine.
The 2,489 came from a throwaway Python line-scanner. Re-measured with the structural parser that already existed in dismech-rs (dismech_oracle_census over graph::build_causal_graph, whose edge already carries intermediate_mechanisms: Vec<String>), cross-checked against an independent pyyaml parse — identical on every figure:
| line-scan | structural | |
|---|---|---|
| label-KNOWN with ≥1 mediator | 2,489 | 2,512 (63.1%) |
| label-KNOWN naming nothing | 1,489 | 1,466 (36.9%) |
| distinct mediator strings | 3,048 | 3,095 |
The subset figure (2,497) was correct all along. There is no contradiction once mine is right: 2,512 corpus-wide against 2,497 in pathophysiology[].downstream[], the 15-edge difference sitting in influences_mechanisms (115 label-KNOWN edges) and sequelae (19). The doc now states that scope relationship explicitly so the next reader doesn't have to re-derive it.
The census line also reproduces the board's own E-DISMECH-CORPUS-CENSUS-1 exactly (9,073 / 3,978 / 4,539 / 408), which is the anti-vacuity check that the binary reads the corpus it claims to.
One finding fell out that neither pass had: 92 INDIRECT_UNKNOWN_INTERMEDIATES edges DO name mediators — the source contradicting its own label. Neither oracle nor restraint control; left in the control they read as hallucinated closure by the benchmark's own definition. Recorded.
Fixed in 5577728.
Generated by Claude Code
…arrative
I reported to the operator that "no ppr, personalized_page*, bm25, rrf, or
community function exists in the workspace" and that "AriGraph appears only
as narrative in doc comments". Both are false.
crates/lance-graph/src/graph/arigraph/ is 15 modules, ~327 KB:
PersonalizedPageRank + personalized_pagerank(), Bm25Index::{build,score,
rank}, reciprocal_rank_fusion(), Communities, markov_soa, EpisodicBasins,
WitnessCorpus, retrieval, triplet_graph, orchestrator, and five more, all
re-exported from mod.rs.
Two compounding causes. The pattern `fn .*ppr` matches `approx` ("ppr" is a
substring of a-p-p-r-o-x), so every approx() test helper in jc, holograph
and sigker matched while the real functions -- spelled personalized_pagerank
and reciprocal_rank_fusion -- matched nothing; bm25 lives in the type name,
not the function name. Then head -20 pushed community.rs's 5 genuine hits
off the end. A noisy pattern with full output would have shown them; a clean
pattern with a limit would have found them first. Jointly they produce a
confident false negative.
The rule: searching for a function's NAME is not searching for a CAPABILITY.
find . -path "*arigraph*" would have ended it in one call with no guess about
spelling. And never conclude absence from a limited-output search -- absence
is a claim about the whole set, a head reads a prefix of it.
The correct state was already on this board under E-ARIGRAPH-IS-AN-ISLAND:
every module exists and tests green, the chain is open at the joints,
HotWitness is todo!(). I cited that entry as prior art for a different claim
without reading it, and it contained the correction to the claim I was
making in the same breath.
This matters beyond the error: absent and unwired have opposite remedies.
Absent means build the organs. Unwired means build nothing and close the
seam. I recommended the expensive one.
Second instance of a class the board already names -- "a false negative
manufactured by the intake" (the sorted-histogram gate, codex P1 on #876) --
with a disjoint mechanism, which makes it a pattern rather than an anecdote.
…at exists Codex P1 on #975 caught this structurally, without the corpus: the module documents pathophysiology[].downstream[] as a SUBSET with 2,497 mediator- bearing edges, and I had written 2,489 corpus-wide. A subset cannot exceed the whole, so at least one count was wrong. Mine was. The counts came from a throwaway Python line-scanner. Re-measured with dismech-rs's dismech_oracle_census over graph::build_causal_graph, cross- checked against an independent pyyaml structural parse -- identical on every figure: label-KNOWN with >=1 mediator 2489 -> 2512 (63.1%), 549 diseases label-KNOWN naming nothing 1489 -> 1466 (36.9%) distinct mediator strings 3048 -> 3095 exact node references 45 -> 45 (1.5%) unchanged groundable by label alone 59.4% -> 59.3% (1834) ungrounded prose 40.6% -> 40.7% (1261) UNKNOWN_INT that DO name -- -> 92 (missed entirely) No contradiction once the number is right: 2,512 corpus-wide against 2,497 in the downstream subset, the 15 sitting in influences_mechanisms (115) and sequelae (19). Every qualitative claim survives; magnitudes moved under 1%. Also addresses the second P1: 2,512 is now described as the population REQUIRING GROUNDING, never a usable oracle. Ranking candidate identities against it ungrounded either compares identities to prose or silently drops the 40.7% residue -- both corrupt any reported recall. And a third finding neither pass had: 92 INDIRECT_UNKNOWN_INTERMEDIATES edges DO name mediators, contradicting their own label. Neither oracle nor restraint control; left in the control they read as hallucinated closure. Gates: lance-graph-contract 1180/1180, fmt clean.
Both merged PRs shipped types, so neither is discharged by the Termination clause. #974's obligation was missed at merge on 2026-08-20 and is recorded late rather than silently skipped; its entry says so. PR_ARC_INVENTORY: two prepended entries with Added / Locked / Deferred / Docs / Review / Confidence. #974's carries the source-side-only ruling, the fail-closed parse, and the citation-identity falsifier, plus an explicit Superseded-by-#975 line for the one doc claim that did not survive. #975's carries the three findings, the two codex P1s, and the two rules the session paid for: absent is not unwired and their remedies are opposite; a measurement a committed parser can make must not be made by an ad-hoc script. LATEST_STATE: the #974 heading becomes a merged-PR entry and #975 is added above it, including the AriGraph correction as a standing warning -- the 15 shipped modules must not be rebuilt under a new name by a session that greps for a function spelling and concludes absence.
Two corrections, both found by measuring a claim rather than inheriting it, and both landing as append-only board entries plus the narrow fix each implies. No bits move; no API changes.
1. The DisMech "ORACLE population" is 2,489 edges, not 3,978
E-DISMECH-CORPUS-CENSUS-1(2026-08-20) established thecausal_link_typecensus and stands — 9,073 / 4,539 / 3,978 / 408, total 17,998. One sentence of it does not:Measured on the same 2,100-file corpus:
INDIRECT_KNOWN_INTERMEDIATESintermediate_mechanismskey at allThe mediators are prose — "Classical-pathway inhibition yields serum resistance, permitting spirochete survival during hematogenous dissemination." — not addresses into the 48,467 distinct mechanism names the corpus carries.
Three consequences: the oracle population is 2,489; the 1,489 label-only edges are neither oracle nor restraint control and need a third bucket or they poison a gold set in both directions; and
Recall@kover a mediator-identity candidate list has nothing to score against until a gold set is built by a path that cannot leak — label matching alone grounds 59.4%, and the 40.6% residue has to be reported, not dropped.DismechTopology::source_knows_intermediates()is correctly implemented — it answers a question about the label. Corrected here are the two places that overstate what the label buys: that function's second doc sentence, and theLATEST_STATEsummary.2. The academic carve under-fills — rows are not words
plans/causal-rung-standing-wave-v1.md:465claims the 20k academic vocabulary "fits the palette256² pair carve NATIVELY — 20480 = 80×256". It does not.academic_20k.csvhas 20,845 rows but only 18,559 distinct surface forms (2,286 duplicates — COCA lists a word once per part of speech), andfrom_frequency_rankedadmits by surface form. The carve fills 18,559 / 20,480 = 90.6%; the last id lands at basin 72 slot 126, so basins 73..79 are empty.The inference was arithmetic on the wrong noun: 20,845 rows > 20,480 slots looks like a fit. The same plan file notes the
Posmultiplicity two paragraphs later without drawing the consequence.No test could catch it.
academic_20k_carve_spans_80_basinsasserts exactly the property that fails on real data —pair("w20479") == (79, 255)— and passes, because its fixture is 20,480 synthetic words that are all-distinct by construction. A dedupe-sensitive property tested on a duplicate-free fixture can only confirm the arithmetic it was written from.Added the companion falsifier: a fixture that actually has something to deduplicate, with an anti-vacuity assertion that it is not duplicate-free. Disable-verified — removing the
contains_keyguard turns it red (and the pre-existingduplicates_keep_first_idwith it); restoring turns both green.Nothing moves:
RESERVE, DON'T RECLAIMmeans a 90.6%-full carve is correct and addressable. What changes is the claim, and any design that read basin occupancy as a frequency signal.Artifact
The derived deepnsm-v2 academic COCA codebook was uploaded round-trip-verified (PUT → GET → compare, same discipline as the bake uploader) to
lance-graph/codebooks/deepnsm-v2-academic-coca-v1/— TSV + source CSV + manifest. The TSV states the shortfall in its own header so a consumer cannot pin it without reading it. It reports 18,559 entries, unpadded.Board hygiene
EPIPHANIES.md— two prepended entries (E-DISMECH-KNOWN-INTERMEDIATES-ARE-PROSE-NOT-IDENTITIES-1,E-ACADEMIC-CARVE-UNDERFILLS-ROWS-ARE-NOT-WORDS-1)LATEST_STATE.md— oracle-population correctionplans/causal-rung-standing-wave-v1.md— dated correction, original claim retainedGates
lance-graph-contract1180/1180 lib + 21 doctests ·deepnsm-v2109/109 ·cargo fmtclean ·cargo clippy -D warningsclean.Generated by Claude Code