diff --git a/.claude/harvest/r2il/PROVENANCE.md b/.claude/harvest/r2il/PROVENANCE.md new file mode 100644 index 0000000000000..f0ba387717639 --- /dev/null +++ b/.claude/harvest/r2il/PROVENANCE.md @@ -0,0 +1,26 @@ +# R2IL pass-1 harvest — PROVENANCE + +Hashes below are **FNV-1a 64** (not a cryptographic hash) over the raw file bytes, computed inline in this example — no hashing dependency. + +## Corpus + +| path | bytes | fnv1a64 | status | +|---|---|---|---| +| /home/user/ruff/crates/ruff_r2il/../../../r2sleigh/tests/e2e/stress_test | 52128 | d60c9fe34de17594 | harvested (71 functions) | +| /home/user/ruff/crates/ruff_r2il/../../../r2sleigh/tests/e2e/stress_test_opt | 83880 | 40d985a73341e4c8 | harvested (72 functions) | +| /bin/ls | 142312 | c3453ae463c7fa3c | skipped (no symtab) | +| /usr/bin/env | 48072 | 7c3bd066635cc085 | skipped (no symtab) | + +## Environment + +- `r2sleigh` commit: `60942f6` +- Architecture: `x86-64` +- `sleigh-config` = "1.0", feature `x86` (exact resolved patch pinned by the committed `Cargo.lock`) +- Convention: `R2ilConvention::from_arch(&spec, [Copy, IntAdd, Load, Store, CBranch, Call, Return])` — one convention, built once, reused for every harvested function +- Caps in force: `R2IL_HARVEST_MAX_FUNCS=200`, `R2IL_HARVEST_MAX_SECTION_BYTES=262144` + +## Invocation + +```sh +cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example harvest_r2il +``` diff --git a/.claude/harvest/r2il/README.md b/.claude/harvest/r2il/README.md new file mode 100644 index 0000000000000..dd202decbd6ba --- /dev/null +++ b/.claude/harvest/r2il/README.md @@ -0,0 +1,74 @@ +# R2IL pass-1 harvest — the intake arm's artifact set + +Produced by `crates/ruff_r2il/examples/harvest_r2il.rs`. **These artifacts are +evidence, never a re-ingest path — nothing in ruff parses them back.** + +## Where the bulk artifacts live (NOT in git) + +Following the escalation MedCare-rs's `.claude/harvest/README.md` already +names — *"If it ever grows past a few MB, move it to a GitHub Release asset and +keep only this provenance file in-tree"* — the two bulk files are out of the +tree entirely. Uncompressed they were **3.4 MB / 10,729 rows**, which made +generated data **75 % of this branch's diff** (29,682 of 39,652 insertions). + +**Canonical:** GitHub Release +[`r2il-harvest-pass1`](https://github.com/AdaWorldAPI/ruff/releases/tag/r2il-harvest-pass1) + +```sh +curl -sL -o r2il-pass1.ore.tsv.gz \ + https://github.com/AdaWorldAPI/ruff/releases/download/r2il-harvest-pass1/r2il-pass1.ore.tsv.gz +curl -sL -o r2il-convention.toml.gz \ + https://github.com/AdaWorldAPI/ruff/releases/download/r2il-harvest-pass1/r2il-convention.toml.gz +zcat r2il-pass1.ore.tsv.gz | head # read without unpacking +``` + +**Scratch mirror:** S3 (Tigris), `s3://$AWS_S3_BUCKET_NAME/r2il-arc/harvest/` +— the full set including the small files, plus `STAGED-CODEGEN-GUIDE.md`, for +cross-session scratch. Read via `AWS_ENDPOINT_URL` / `AWS_ACCESS_KEY_ID` / +`AWS_SECRET_ACCESS_KEY` from the environment; never hardcode an endpoint or a +key. The Release is authoritative; S3 is a working mirror that may be pruned. + +The bucket is **shared** with other work (`q2`, `MedCare-rs`, `OSM`, +`ontologies`, …), so everything this arc wrote lives under the single +`r2il-arc/` prefix and nothing is written at the root. The one exception is the +MedCare-rs config/harvest backup, which belongs to that repo's own prefix and +sits at `MedCare-rs/harvest/2026-08-18/` — datestamped, per the +`bakes/-/` convention already in use there. + +Both are gitignored, so a regenerate run leaves the tree clean. + +## What IS in git + +| file | why it stays | +|---|---| +| `TRIAGE-RESULT.md` | the pre-registered bars B1/B2/B3, stated **before** the measured section — the point of the whole run | +| `r2il-pass1-slag.tsv` | the addressed residual ledger; the artifact a reviewer actually reads | +| `r2il-pass1-census.md` | per-fact-kind and per-opcode counts | +| `PROVENANCE.md` | corpus manifest (FNV-1a 64 per input), r2sleigh commit pin, caps, exact invocation | + +Together under 32 KB. The Release assets are reproducible from these plus the +pinned corpus; these are not reproducible from the Release. + +## Regenerate + +```sh +cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift \ + --example harvest_r2il + +# gzip AFTER the run: `gzip -9` compresses in place and removes its source, so +# compressing first would leave the next run writing an uncompressed sibling +# beside a stale archive. +gzip -9 .claude/harvest/r2il/r2il-pass1.ore.tsv \ + .claude/harvest/r2il/r2il-convention.toml +``` + +Then re-upload to the Release (new tag per pass — assets are immutable +evidence, never overwritten in place) and to the S3 prefix. + +## Measured, this pass + +143 functions across 4 x86-64 binaries (2 with symtab; 2 stripped and skipped +with a printed note). Conservation `54304 / 17557 / 36747 / 0`. +**B1 PASS · B2 91.30 % INVESTIGATE · B3 PASS.** The entire remaining B2 gap is +one named reason — `memory_object_escaped`, 1670 rows — which is legitimate +slag, not a defect. See `TRIAGE-RESULT.md`. diff --git a/.claude/harvest/r2il/STAGED-CODEGEN-GUIDE.md b/.claude/harvest/r2il/STAGED-CODEGEN-GUIDE.md new file mode 100644 index 0000000000000..f44e3df057da7 --- /dev/null +++ b/.claude/harvest/r2il/STAGED-CODEGEN-GUIDE.md @@ -0,0 +1,141 @@ +# Staged codegen off the R2IL harvest — without breaking what exists + +> **Audience:** the sibling session that consumes this arc's output (the Ghidra +> console work, and any codegen/target-profile work downstream). +> **Status of the substrate:** PR 1 shipped the intake arm; PR 2 (routes → V3) +> has NOT landed. Everything below is written so you can start staging now and +> not have to unpick it when PR 2 changes the physicalization. + +## 0. The one rule + +**Every export is additive. A consumer written against version N must keep +working, unread, against version N+1.** + +That is not a style preference — it is the same rule the substrate already runs +on. lance-graph's V3 canon says *"RESERVE, DON'T RECLAIM: a zero tier means +`not consulted`, never `compacted away`"*, and `ruff_spo_triplet::ir::ModelGraph` +is schema-locked at the top level with per-language growth confined to sibling +`Vec`s carrying `skip_serializing_if`. Copy that discipline; do not invent a +migration story you will then have to run. + +## 1. What you may consume today, and what is still moving + +| artifact | stability | use it for | +|---|---|---| +| `TRIAGE-RESULT.md` | **stable** | the bars and their verdicts. Read `B1` before trusting anything else — if conservation ever fails, the run is void. | +| `PROVENANCE.md` | **stable** | corpus identity (FNV-1a per input), r2sleigh pin, exact invocation. Cite this, never re-derive it. | +| `r2il-pass1-slag.tsv` | **stable shape** | the residual work queue. New `reason` values WILL appear; treat an unknown reason as "not yet classified", never as an error. | +| `r2il-pass1-census.md` | **stable shape** | counts per fact-kind / opcode. | +| `r2il-pass1.ore.tsv.gz` | **shape stable, columns additive** | the melted rows. Read by the `#schema` header, never by column position. | +| `r2il-convention.toml.gz` | **stable** | the drill tree. Every row is `unmeasured` until something measures it. | +| `FlatFact`'s two payload slots (`a`, `b`) | ⚠ **NOT stable** | their per-kind meaning is documented in `furnace.rs` and may be re-carved in PR 2. Do not hardcode the bit layout; go through the accessor or re-read the module table. | +| `OpTag::as_str()` opcode tags | **stable, with one correction** | one tag shipped briefly as `int_scary` — a spellchecker rewrite of `int_scarry` (P-code `INT_SCARRY`, signed carry) that reached the enum's `as_str`. Corrected; the pass-1 artifacts never carried it (no SCARRY op classified in the corpus). If you pinned the misspelling, repin. | +| the 16-byte `VarnodeFacet` **as an address** | ⚠ **provisional** | `PROVISIONAL_R2IL_VARNODE = 0x0000` is a local placeholder. The real classid is minted in `ogar_codebook` (PR 3). Treat the facet as an opaque key today; do not persist it as a durable address. | + +## 2. Staging order + +Stage in this order; each step is independently useful and none of them blocks +on PR 2. + +```text +S1 read the ledger slag + census only. Answers "what does the arm not + yet explain?" Needs no codegen at all. +S2 read the ore rows per-function fact rows, joined back to native + addresses via ore::instruction_addr. Enables + navigation and evidence display. +S3 emit into a landing generated code goes to an ADDITIVE landing zone (a + zone, not in place new module/crate), never edited into existing files. + openproject-nexgen-rs's `op-generated` is the worked + precedent: 16 structs emitted beside hand code, with + a `// @generated` header. +S4 wire ONE consumer prove the seam on a single real call site before + scaling. That is what "no wave scales out before + P-REHOST is green" means in a2ui-rs. +S5 target profiles only here does "ordinary Java vs Valhalla/Panama + Java" become a real fork. Until S4 is green it is a + design conversation, not a code path. +``` + +**Do not skip to S3.** The MedCare and OpenProject transcodes both earned their +numbers by measuring at S1/S2 first (`99.6 %` recoverable, `98.4 %` recipe +coverage) — those figures are what made the later codegen defensible. + +## 3. The old/new SoA and `Va*` format question + +There are two independent axes here and conflating them is the failure mode. + +**Axis 1 — the physical row.** The V3 512-byte `NodeRow` (`16 | 16 | 480`) is +CANON and unchanged. What is V1-legacy is the *reading* of two fields: the +`NodeGuid` u24 tail (new mints go through the 4+12 content-blind facet) and the +`EdgeBlock`'s 12+4 carving (resolve `ClassView::edge_codec_flavor`). Neither +requires an `ENVELOPE_LAYOUT_VERSION` bump, and neither is something this +harvest emits. + +**Axis 2 — the `Va*` carrier family** (`Vsa16kF32` / `Vsa16kBF16` / `Vsa16kF16` +/ `Vsa16kI8` / `Binary16K`). These are *compute* formats, selected per workload, +not a schema. The relevant standing rulings: `Vsa16kF32` is deprecated **as a +cross-boundary carrier** (it never crosses a mailbox boundary), and VSA is +demoted to its `I-VSA-IDENTITIES` niche — lossless role superposition of +**identities**, `N ≤ √d/4 ≈ 32`, never of content or of quantized codes. + +**What that means for you concretely:** the R2IL harvest emits *neither*. It +emits flat facet-addressed rows and a residual ledger. If you find yourself +about to bundle R2IL facts into a `Va*` carrier, run the four +`I-VSA-IDENTITIES` tests first — in particular Test 0 (register laziness: does +this thing have a natural id? then use the id) and Test 1 (bundle size). R2IL +facts have natural ids (`FactId`, `InstId`, `ValueId`), so Test 0 short-circuits +and the answer is almost certainly "not a VSA workload". + +## 4. Additive export — the mechanics + +When you extend the export (and you will), obey these five: + +1. **Version the header, never the reader's assumptions.** Every row artifact + carries `#version N` and a `#schema` line naming its columns. Bump `N` when + you ADD; a reader that keys off `#schema` names needs no change at all. +2. **Append columns; never reorder or remove.** A column that becomes + meaningless gets an explicit empty value, not deletion — the same + RESERVE-DON'T-RECLAIM rule the node key follows. +3. **New enum variants are expected, not exceptional.** `ResidualReason` grows + as the furnace learns; a consumer must render an unknown reason as its raw + string and carry on. Never `match` exhaustively across a process boundary. +4. **Never widen a field to fit one outlier.** That is what route-local + overflow is for (`mint_factored`'s base-255 cascade; `MAX_SIBLINGS_PER_TIER + = 255` is a design smell, *not* a storage ceiling). One 900-way phi does not + get to change the ABI for everyone. +5. **A new pass gets a NEW release tag.** Harvest assets are immutable + evidence. Re-running and overwriting `r2il-harvest-pass1` in place destroys + the ability to diff pass N against pass N+1 — which is the whole point of + keeping the failing B2 run in history. + +## 5. What "without breaking existing" means in practice + +- **Generated code lands beside hand code, never inside it.** A `// @generated` + header and its own module. If a generated item must be specialised, the + specialisation lives in hand code that *calls* it. +- **The SPO projection stays optional and lossy.** `ModelGraph → expand() → + Vec` is a projection of *semantic* facts (calls, reads, writes). It is + not the behavioral roundtrip oracle and must never become the source you + reconstruct R2IL from. +- **Conservation is your regression test.** If a staged consumer starts + dropping facts, `harvested == classified + residual` with `dropped == 0` is + the invariant that catches it. Assert it in your own pipeline too, not just + in ours. +- **When the arm can't explain something, that is data.** Do not paper a gap + with a catch-all so your codegen compiles. A named residual with an address + is worth more than a generated stub that silently means nothing. + +## 6. Where things live + +- **Canonical bulk evidence:** GitHub Release `r2il-harvest-pass1` + (`r2il-pass1.ore.tsv.gz`, `r2il-convention.toml.gz`). +- **Scratch mirror + config backup:** `s3://$AWS_S3_BUCKET_NAME/r2il-arc/` + (`harvest/` and `config-backup//`), and this guide at + `r2il-arc/harvest/STAGED-CODEGEN-GUIDE.md`. The MedCare-rs backup is the one + thing NOT under `r2il-arc/` — it belongs to that repo and lives at + `MedCare-rs/harvest/2026-08-18/`, datestamped like the `bakes/` siblings + beside it. Credentials from `AWS_*` env only — the bucket is **shared** with + other work (`q2`, `MedCare-rs`, `OSM`, `ontologies`), so stay inside your own + prefix and never write at the root. +- **In-tree:** `.claude/harvest/r2il/` keeps the small readable artifacts; + the plan and impl spec are in `.claude/plans/`. diff --git a/.claude/harvest/r2il/TRIAGE-RESULT.md b/.claude/harvest/r2il/TRIAGE-RESULT.md new file mode 100644 index 0000000000000..76678752e7152 --- /dev/null +++ b/.claude/harvest/r2il/TRIAGE-RESULT.md @@ -0,0 +1,31 @@ +# R2IL pass-1 TRIAGE RESULT + +Caps in force: `R2IL_HARVEST_MAX_FUNCS=200`, `R2IL_HARVEST_MAX_SECTION_BYTES=262144` + +## Pre-registered bars (stated BEFORE the measured section below) + +- **B1 — conservation (absolute).** `dropped == 0` and `harvested == classified + residual`. Any violation **KILLS** the pass: the enumerator, not the corpus, is wrong. Not a percentage. +- **B2 — coverage of the declared seven.** Of ore facts whose parent opcode is one of `{Copy, IntAdd, Load, Store, CBranch, Call, Return}`, **>=99% classify -> PASS; <90% -> KILL.** The 90-99% band is INVESTIGATE (expected causes: operand rows with no convention row at their address, `CallSite` rows with no `direct_target` — both legitimate slag under a parent that classified). +- **B3 — the slag is named and addressed, not lumped.** `residual > 0`, distinct `shape_id` count **>= 5**, `dominant_share() < 0.60`, and **every** residual except `NoFacetCoordinate` carries `at.is_some()`. `residual == 0` is a **KILL** too — it means someone widened the ladder. + +Also **pre-register a prediction that is NOT a bar** (so it can be wrong without moving a goalpost): on an x86-64 corpus `Copy/IntAdd/Load/Store` dominate, so pass 1 is expected to classify roughly **60-80%** of all `Op` facts. Record the measured figure either way. + +--- + +## Measured + +Functions harvested: 143 + +Conservation line: harvested 54304 / classified 17557 / residual 36747 / dropped 0 + +**B1: PASS** — dropped == 0: true; harvested == classified + residual: true + +**B2: INVESTIGATE** — 17528 classified / 19198 total ore facts under a seven-opcode parent = 91.30%. + + Derivation note: `ResidualFact` does not carry its parent opcode directly, so the denominator's residual half is APPROXIMATED by summing residual reasons that can *only* fire on a row whose parent op is one of the seven (`no_convention_row_at_address`, `indirect_target`, `memory_object_escaped`, `op_site_join_mismatch`, `custom_space_not_in_convention`, `facet_overflow_at_key`) — reasons that can only fire on a non-seven or no-parent-op row (`opcode_not_in_convention`, `user_op_not_in_convention`, `phi_fan_in_exceeds_predecessors`, `variadic_arity`, `no_facet_coordinate`) are excluded. Labelled APPROXIMATION, not exact — see the module doc comment above `SEVEN_ELIGIBLE_RESIDUAL_REASONS`. + +**B3: PASS** — residual > 0: true (36747); distinct shape_id count: 43 (>=5: true); dominant_share: 0.215 (<0.60: true). + + Spot check: every grouped bucket except no_facet_coordinate reports an example facet address. + +**Non-bar prediction, measured:** 5340 / 37728 `Op` facts classified = 14.15% (predicted 60-80%; OUTSIDE the predicted band — recorded honestly, not a bar). diff --git a/.claude/harvest/r2il/r2il-pass1-census.md b/.claude/harvest/r2il/r2il-pass1-census.md new file mode 100644 index 0000000000000..632d54715755d --- /dev/null +++ b/.claude/harvest/r2il/r2il-pass1-census.md @@ -0,0 +1,27 @@ +# R2IL pass-1 census + +Total classified `FlatFact` rows: 17557 + +## By fact kind + +| kind | count | +|---|---| +| call_site | 108 | +| edge | 456 | +| op | 5340 | +| operand_in | 7416 | +| operand_out | 4237 | + +## By opcode + +| opcode | count | +|---|---| +| branch | 25 | +| call | 428 | +| call_ind | 4 | +| cbranch | 995 | +| copy | 6753 | +| int_add | 4768 | +| load | 2394 | +| return | 222 | +| store | 1968 | diff --git a/.claude/harvest/r2il/r2il-pass1-slag.tsv b/.claude/harvest/r2il/r2il-pass1-slag.tsv new file mode 100644 index 0000000000000..1088fbc4d6747 --- /dev/null +++ b/.claude/harvest/r2il/r2il-pass1-slag.tsv @@ -0,0 +1,489 @@ +# R2IL pass-1 slag — the addressed residual ledger. Evidence, never a re-ingest path. +#version 1 +#section grouped — ResidualLedger::grouped(), merged across every harvested function +#schema section shape_id reason count example_facet +grouped 134a977e75924abd opcode_not_in_convention 7888 00000000d01140000000000000000000 +grouped 67b2c44555057f14 opcode_not_in_convention 5228 00000000d01140000000000000000000 +grouped c71d53c68ab98180 opcode_not_in_convention 2732 00000000d01140000000000000000000 +grouped 4a95503569ad12a6 no_facet_coordinate 2588 - +grouped 3ddd2923509c7f99 opcode_not_in_convention 2358 00000000d01140000000000000000000 +grouped 98de717e2fe793ce opcode_not_in_convention 2100 00000000d01140000000000000000000 +grouped 55c017693f2e5091 opcode_not_in_convention 1830 00000000d01140000000000000000000 +grouped d363ef40e5e9d53b memory_object_escaped 1670 00000000d01140000000000000000000 +grouped 4195ea16a74b2541 opcode_not_in_convention 1288 00000000431340000000000000000000 +grouped 13b27a08b280dbf9 opcode_not_in_convention 1204 00000000ac1340000000000000000000 +grouped 38185eeb1240bc84 opcode_not_in_convention 1185 00000000431340000000000000000000 +grouped 915977fb8730d4b8 opcode_not_in_convention 932 00000000f31240000000000000000000 +grouped 7d168676ae3b0785 opcode_not_in_convention 924 00000000f31240000000000000000000 +grouped 5b7105f529b450ea opcode_not_in_convention 860 000000003a1440000000000000000000 +grouped 463c09f7ecce624b opcode_not_in_convention 476 000000000c1440000000000000000000 +grouped d4496632e0982a90 opcode_not_in_convention 476 000000000c1440000000000000000000 +grouped 3a97a07dfac9b131 opcode_not_in_convention 416 00000000d01140000000000000000000 +grouped e8be19db525f9406 opcode_not_in_convention 300 00000000e91740000000000000000000 +grouped 47817e3fee8e884a opcode_not_in_convention 276 00000000f31240000000000000000000 +grouped 4194a77cd51dd75a opcode_not_in_convention 272 000000003a1440000000000000000000 +grouped 32880f6bb3a05eec opcode_not_in_convention 267 000000003a1440000000000000000000 +grouped 26909380d5a943f2 opcode_not_in_convention 264 00000000431340000000000000000000 +grouped 60ef1844628ca77f opcode_not_in_convention 220 00000000431340000000000000000000 +grouped 7cd59576ae038a0d opcode_not_in_convention 200 00000000651440000000000000000000 +grouped fe0a9a5b56991c1a opcode_not_in_convention 174 00000000f51140000000000000000000 +grouped da868beaddbd5e04 opcode_not_in_convention 92 00000000df1640000000000000000000 +grouped 059d86fc4e8c7de3 variadic_arity 80 00000000731840000000000000000000 +grouped cf8e29e9adbc6f07 opcode_not_in_convention 72 00000000b61240000000000000000000 +grouped ad6ee9a28ee1d70d opcode_not_in_convention 56 00000000e91740000000000000000000 +grouped 331f4abeadff6702 opcode_not_in_convention 54 00000000431340000000000000000000 +grouped 82bcb1bedb8f09fc opcode_not_in_convention 40 00000000b61240000000000000000000 +grouped 1d154638c246df67 opcode_not_in_convention 36 00000000431340000000000000000000 +grouped 3d7a2af4bc932920 opcode_not_in_convention 36 00000000431340000000000000000000 +grouped 3a1e3a6bb79e6ca0 opcode_not_in_convention 28 00000000651440000000000000000000 +grouped e38c976b86994d5f opcode_not_in_convention 28 00000000651440000000000000000000 +grouped 081a6bbe95b68d3d opcode_not_in_convention 24 00000000b61240000000000000000000 +grouped a8ad31e12dbe9f80 variadic_arity 21 00000000633540000000000000000000 +grouped d58bae2a556c7dd4 opcode_not_in_convention 20 00000000d01140000000000000000000 +grouped 234f01aef71604c0 opcode_not_in_convention 16 000000006d1f40000000000000000000 +grouped 4f904662b361ccce opcode_not_in_convention 6 00000000703a40000000000000000000 +grouped 9d2f7aeabacb60f8 opcode_not_in_convention 4 00000000632740000000000000000000 +grouped 574fb9d8504e8da7 opcode_not_in_convention 3 00000000601d40000000000000000000 +grouped b5b8e47e410f229d opcode_not_in_convention 3 00000000e73a40000000000000000000 +#section by_address — ResidualLedger::by_address(), the proposer's work queue, merged across every harvested function +#schema section prefix shape_id count +by_address space:2 134a977e75924abd 3632 +by_address - 4a95503569ad12a6 2588 +by_address space:0 134a977e75924abd 1972 +by_address space:0 d363ef40e5e9d53b 1670 +by_address space:3 134a977e75924abd 1417 +by_address space:0 67b2c44555057f14 1307 +by_address space:3 67b2c44555057f14 1280 +by_address space:2 55c017693f2e5091 1220 +by_address space:0 13b27a08b280dbf9 1204 +by_address space:2 67b2c44555057f14 1094 +by_address space:0 3ddd2923509c7f99 786 +by_address space:2 38185eeb1240bc84 694 +by_address space:0 c71d53c68ab98180 683 +by_address space:3 c71d53c68ab98180 683 +by_address space:0 55c017693f2e5091 610 +by_address space:2 4195ea16a74b2541 608 +by_address space:1/offset:0x202/size:1 67b2c44555057f14 557 +by_address space:1/offset:0x206/size:1 67b2c44555057f14 557 +by_address space:1/offset:0x207/size:1 c71d53c68ab98180 557 +by_address space:1/offset:0x20/size:8 98de717e2fe793ce 548 +by_address space:0 98de717e2fe793ce 525 +by_address space:3 98de717e2fe793ce 442 +by_address space:2 c71d53c68ab98180 413 +by_address space:2 98de717e2fe793ce 408 +by_address space:0 38185eeb1240bc84 395 +by_address space:2 5b7105f529b450ea 377 +by_address space:1/offset:0x0/size:4 3ddd2923509c7f99 357 +by_address space:0 4195ea16a74b2541 322 +by_address space:1/offset:0x0/size:8 3ddd2923509c7f99 320 +by_address space:0 915977fb8730d4b8 233 +by_address space:1/offset:0x20b/size:1 915977fb8730d4b8 233 +by_address space:0 7d168676ae3b0785 231 +by_address space:1/offset:0x200/size:1 7d168676ae3b0785 231 +by_address space:0 60ef1844628ca77f 220 +by_address space:0 5b7105f529b450ea 215 +by_address space:1/offset:0x0/size:4 134a977e75924abd 179 +by_address space:0 fe0a9a5b56991c1a 174 +by_address space:3 5b7105f529b450ea 162 +by_address space:3 915977fb8730d4b8 152 +by_address space:3 7d168676ae3b0785 150 +by_address space:1/offset:0x10/size:4 3ddd2923509c7f99 146 +by_address space:2 7d168676ae3b0785 143 +by_address space:2 915977fb8730d4b8 143 +by_address space:2 e8be19db525f9406 137 +by_address space:1/offset:0x0/size:4 c71d53c68ab98180 134 +by_address space:1/offset:0x10/size:8 3ddd2923509c7f99 132 +by_address space:1/offset:0x0/size:4 67b2c44555057f14 127 +by_address space:0 463c09f7ecce624b 119 +by_address space:0 d4496632e0982a90 119 +by_address space:1/offset:0x200/size:1 463c09f7ecce624b 118 +by_address space:1/offset:0x20b/size:1 d4496632e0982a90 118 +by_address space:2 7cd59576ae038a0d 107 +by_address space:2 26909380d5a943f2 105 +by_address space:2 47817e3fee8e884a 105 +by_address space:0 3a97a07dfac9b131 104 +by_address space:1/offset:0x0/size:4 3a97a07dfac9b131 99 +by_address space:2 4194a77cd51dd75a 98 +by_address space:0 47817e3fee8e884a 92 +by_address space:1/offset:0x206/size:1 38185eeb1240bc84 91 +by_address space:0 32880f6bb3a05eec 89 +by_address space:0 059d86fc4e8c7de3 80 +by_address space:1/offset:0x10/size:4 134a977e75924abd 75 +by_address space:0 e8be19db525f9406 75 +by_address space:3 463c09f7ecce624b 74 +by_address space:3 d4496632e0982a90 74 +by_address space:1/offset:0x202/size:1 4195ea16a74b2541 71 +by_address space:2 da868beaddbd5e04 69 +by_address space:2 32880f6bb3a05eec 68 +by_address space:0 4194a77cd51dd75a 68 +by_address space:1/offset:0x200/size:1 4195ea16a74b2541 67 +by_address space:0 26909380d5a943f2 66 +by_address space:1/offset:0x20b/size:1 134a977e75924abd 64 +by_address space:1/offset:0x206/size:1 4195ea16a74b2541 62 +by_address space:1/offset:0x20/size:8 134a977e75924abd 60 +by_address space:1/offset:0x200/size:1 134a977e75924abd 58 +by_address space:1/offset:0x206/size:1 26909380d5a943f2 56 +by_address space:1/offset:0x20/size:8 67b2c44555057f14 56 +by_address space:1/offset:0x20/size:8 c71d53c68ab98180 56 +by_address space:3 4194a77cd51dd75a 54 +by_address space:1/offset:0x202/size:1 134a977e75924abd 53 +by_address space:1/offset:0x206/size:1 134a977e75924abd 53 +by_address space:1/offset:0x207/size:1 134a977e75924abd 53 +by_address space:1/offset:0x207/size:1 4195ea16a74b2541 53 +by_address space:1/offset:0x10/size:4 c71d53c68ab98180 53 +by_address space:2 3a97a07dfac9b131 52 +by_address space:1/offset:0x20b/size:1 4195ea16a74b2541 52 +by_address space:0 7cd59576ae038a0d 50 +by_address space:2 3ddd2923509c7f99 49 +by_address space:1/offset:0x10/size:4 67b2c44555057f14 48 +by_address space:1/offset:0x0/size:4 e8be19db525f9406 48 +by_address space:1/offset:0x20/size:8 7d168676ae3b0785 42 +by_address space:1/offset:0x20/size:8 915977fb8730d4b8 42 +by_address space:1/offset:0x10/size:4 3a97a07dfac9b131 41 +by_address space:1/offset:0x8/size:4 3ddd2923509c7f99 41 +by_address space:1/offset:0x8/size:8 3ddd2923509c7f99 41 +by_address space:1/offset:0x30/size:4 3ddd2923509c7f99 41 +by_address space:1/offset:0x30/size:8 3ddd2923509c7f99 41 +by_address space:1/offset:0x0/size:4 32880f6bb3a05eec 40 +by_address space:1/offset:0x0/size:4 98de717e2fe793ce 40 +by_address space:1/offset:0x38/size:4 3ddd2923509c7f99 39 +by_address space:1/offset:0x38/size:8 3ddd2923509c7f99 39 +by_address space:1/offset:0x207/size:1 5b7105f529b450ea 39 +by_address space:1/offset:0x20b/size:1 5b7105f529b450ea 39 +by_address space:1/offset:0x0/size:8 134a977e75924abd 33 +by_address space:1/offset:0x38/size:4 134a977e75924abd 33 +by_address space:1/offset:0x0/size:8 4194a77cd51dd75a 33 +by_address space:1/offset:0x0/size:1 134a977e75924abd 31 +by_address space:1/offset:0x38/size:8 134a977e75924abd 31 +by_address space:1/offset:0x0/size:4 463c09f7ecce624b 31 +by_address space:1/offset:0x0/size:4 d4496632e0982a90 31 +by_address space:1/offset:0x0/size:4 4195ea16a74b2541 29 +by_address space:1/offset:0x0/size:8 c71d53c68ab98180 29 +by_address space:1/offset:0x200/size:1 26909380d5a943f2 28 +by_address space:1/offset:0xa0/size:4 3ddd2923509c7f99 28 +by_address space:1/offset:0xa0/size:8 3ddd2923509c7f99 27 +by_address space:1/offset:0x0/size:8 67b2c44555057f14 27 +by_address space:1/offset:0x207/size:1 67b2c44555057f14 27 +by_address space:1/offset:0x20b/size:1 67b2c44555057f14 27 +by_address space:1/offset:0x0/size:4 7d168676ae3b0785 27 +by_address space:1/offset:0x0/size:4 915977fb8730d4b8 27 +by_address space:1/offset:0x28/size:4 3ddd2923509c7f99 26 +by_address space:1/offset:0x28/size:8 3ddd2923509c7f99 26 +by_address space:1/offset:0x200/size:1 5b7105f529b450ea 26 +by_address space:1/offset:0x30/size:4 134a977e75924abd 24 +by_address space:1/offset:0x18/size:4 3ddd2923509c7f99 24 +by_address space:1/offset:0x18/size:8 3ddd2923509c7f99 24 +by_address space:2 ad6ee9a28ee1d70d 24 +by_address space:1/offset:0xb0/size:4 3ddd2923509c7f99 23 +by_address space:1/offset:0xb0/size:8 3ddd2923509c7f99 23 +by_address space:0 da868beaddbd5e04 23 +by_address space:2 331f4abeadff6702 22 +by_address space:1/offset:0x0/size:8 463c09f7ecce624b 22 +by_address space:1/offset:0x10/size:4 47817e3fee8e884a 22 +by_address space:1/offset:0x38/size:4 c71d53c68ab98180 22 +by_address space:1/offset:0x0/size:8 d4496632e0982a90 22 +by_address space:2 3a1e3a6bb79e6ca0 21 +by_address space:1/offset:0x0/size:4 47817e3fee8e884a 21 +by_address space:0 a8ad31e12dbe9f80 21 +by_address space:2 e38c976b86994d5f 21 +by_address space:1/offset:0x8/size:4 134a977e75924abd 20 +by_address space:0 331f4abeadff6702 20 +by_address space:1/offset:0x200/size:1 3a97a07dfac9b131 20 +by_address space:1/offset:0xb8/size:4 3ddd2923509c7f99 20 +by_address space:1/offset:0xb8/size:8 3ddd2923509c7f99 20 +by_address space:1/offset:0x0/size:8 32880f6bb3a05eec 19 +by_address space:1/offset:0x8/size:4 3a97a07dfac9b131 18 +by_address space:1/offset:0x10/size:4 98de717e2fe793ce 18 +by_address space:0 cf8e29e9adbc6f07 18 +by_address space:1/offset:0x10/size:4 e8be19db525f9406 18 +by_address space:1/offset:0x38/size:4 67b2c44555057f14 17 +by_address space:2 463c09f7ecce624b 16 +by_address space:1/offset:0x8/size:4 67b2c44555057f14 16 +by_address space:1/offset:0x8/size:4 c71d53c68ab98180 16 +by_address space:2 d4496632e0982a90 16 +by_address space:0 d58bae2a556c7dd4 16 +by_address space:1/offset:0x30/size:4 3a97a07dfac9b131 14 +by_address space:1/offset:0x10/size:4 4195ea16a74b2541 14 +by_address space:1/offset:0x0/size:1 7d168676ae3b0785 14 +by_address space:1/offset:0x0/size:1 915977fb8730d4b8 14 +by_address space:1/offset:0x10/size:4 915977fb8730d4b8 14 +by_address space:1/offset:0x0/size:1 98de717e2fe793ce 14 +by_address space:1/offset:0x0/size:8 98de717e2fe793ce 14 +by_address space:0 ad6ee9a28ee1d70d 14 +by_address space:2 1d154638c246df67 13 +by_address space:2 3d7a2af4bc932920 13 +by_address space:1/offset:0x10/size:4 32880f6bb3a05eec 12 +by_address space:1/offset:0x80/size:4 3ddd2923509c7f99 12 +by_address space:1/offset:0x80/size:8 3ddd2923509c7f99 12 +by_address space:1/offset:0xa8/size:4 3ddd2923509c7f99 12 +by_address space:1/offset:0xa8/size:8 3ddd2923509c7f99 12 +by_address space:1/offset:0x20/size:8 463c09f7ecce624b 12 +by_address space:1/offset:0x10/size:4 7d168676ae3b0785 12 +by_address space:1/offset:0x1200/size:8 82bcb1bedb8f09fc 12 +by_address space:1/offset:0x1200/size:8 cf8e29e9adbc6f07 12 +by_address space:1/offset:0x1240/size:8 cf8e29e9adbc6f07 12 +by_address space:1/offset:0x20/size:8 d4496632e0982a90 12 +by_address space:1/offset:0xa0/size:4 134a977e75924abd 11 +by_address space:0 1d154638c246df67 11 +by_address space:0 3d7a2af4bc932920 11 +by_address space:1/offset:0x10/size:8 463c09f7ecce624b 11 +by_address space:1/offset:0x38/size:8 463c09f7ecce624b 11 +by_address space:1/offset:0x38/size:8 67b2c44555057f14 11 +by_address space:3 7cd59576ae038a0d 11 +by_address space:1/offset:0x0/size:4 7cd59576ae038a0d 11 +by_address space:1/offset:0x30/size:4 98de717e2fe793ce 11 +by_address space:1/offset:0x38/size:8 c71d53c68ab98180 11 +by_address space:1/offset:0x10/size:8 d4496632e0982a90 11 +by_address space:1/offset:0x38/size:8 d4496632e0982a90 11 +by_address space:1/offset:0x18/size:4 134a977e75924abd 10 +by_address space:1/offset:0x1200/size:8 1d154638c246df67 10 +by_address space:1/offset:0x30/size:4 32880f6bb3a05eec 10 +by_address space:1/offset:0x1200/size:8 331f4abeadff6702 10 +by_address space:1/offset:0x1200/size:8 3d7a2af4bc932920 10 +by_address space:1/offset:0x38/size:4 7cd59576ae038a0d 10 +by_address space:1/offset:0x30/size:4 7d168676ae3b0785 10 +by_address space:0 82bcb1bedb8f09fc 10 +by_address space:1/offset:0x30/size:4 915977fb8730d4b8 10 +by_address space:1/offset:0xb8/size:4 98de717e2fe793ce 10 +by_address space:1/offset:0x1200/size:4 cf8e29e9adbc6f07 10 +by_address space:3 e8be19db525f9406 10 +by_address space:1/offset:0xb0/size:4 134a977e75924abd 9 +by_address space:1/offset:0xb8/size:4 134a977e75924abd 9 +by_address space:1/offset:0x202/size:1 26909380d5a943f2 9 +by_address space:1/offset:0x28/size:4 3a97a07dfac9b131 9 +by_address space:1/offset:0x80/size:4 3a97a07dfac9b131 9 +by_address space:1/offset:0xa0/size:4 3a97a07dfac9b131 9 +by_address space:1/offset:0xb0/size:4 3a97a07dfac9b131 9 +by_address space:1/offset:0x10/size:8 4194a77cd51dd75a 9 +by_address space:1/offset:0xa0/size:4 67b2c44555057f14 9 +by_address space:1/offset:0xb0/size:4 67b2c44555057f14 9 +by_address space:1/offset:0xb8/size:4 67b2c44555057f14 9 +by_address space:1/offset:0x0/size:8 7d168676ae3b0785 9 +by_address space:1/offset:0xb8/size:4 7d168676ae3b0785 9 +by_address space:1/offset:0x0/size:8 915977fb8730d4b8 9 +by_address space:1/offset:0xb8/size:4 915977fb8730d4b8 9 +by_address space:1/offset:0x8/size:8 98de717e2fe793ce 9 +by_address space:1/offset:0x28/size:4 98de717e2fe793ce 9 +by_address space:1/offset:0xa0/size:4 c71d53c68ab98180 9 +by_address space:1/offset:0xb0/size:4 c71d53c68ab98180 9 +by_address space:1/offset:0xb8/size:4 c71d53c68ab98180 9 +by_address space:1/offset:0x1200/size:8 081a6bbe95b68d3d 8 +by_address space:1/offset:0x10/size:1 134a977e75924abd 8 +by_address space:1/offset:0x28/size:4 134a977e75924abd 8 +by_address space:0 234f01aef71604c0 8 +by_address space:3 32880f6bb3a05eec 8 +by_address space:1/offset:0x88/size:4 3ddd2923509c7f99 8 +by_address space:1/offset:0x88/size:8 3ddd2923509c7f99 8 +by_address space:1/offset:0x10/size:4 463c09f7ecce624b 8 +by_address space:1/offset:0xb8/size:4 463c09f7ecce624b 8 +by_address space:1/offset:0x8/size:8 67b2c44555057f14 8 +by_address space:1/offset:0x30/size:4 67b2c44555057f14 8 +by_address space:1/offset:0x8/size:4 98de717e2fe793ce 8 +by_address space:1/offset:0x0/size:4 ad6ee9a28ee1d70d 8 +by_address space:1/offset:0x30/size:4 c71d53c68ab98180 8 +by_address space:1/offset:0x10/size:4 d4496632e0982a90 8 +by_address space:1/offset:0xb8/size:4 d4496632e0982a90 8 +by_address space:1/offset:0x38/size:4 e8be19db525f9406 8 +by_address space:0 081a6bbe95b68d3d 7 +by_address space:0 3a1e3a6bb79e6ca0 7 +by_address space:1/offset:0x38/size:4 4195ea16a74b2541 7 +by_address space:1/offset:0x8/size:4 463c09f7ecce624b 7 +by_address space:1/offset:0xb0/size:4 463c09f7ecce624b 7 +by_address space:1/offset:0x10/size:4 7cd59576ae038a0d 7 +by_address space:1/offset:0x18/size:4 98de717e2fe793ce 7 +by_address space:1/offset:0x8/size:4 d4496632e0982a90 7 +by_address space:1/offset:0xb0/size:4 d4496632e0982a90 7 +by_address space:0 e38c976b86994d5f 7 +by_address space:1/offset:0x8/size:1 134a977e75924abd 6 +by_address space:1/offset:0x8/size:8 134a977e75924abd 6 +by_address space:1/offset:0x30/size:8 134a977e75924abd 6 +by_address space:1/offset:0x0/size:8 234f01aef71604c0 6 +by_address space:1/offset:0x18/size:4 3a97a07dfac9b131 6 +by_address space:1/offset:0x90/size:4 3a97a07dfac9b131 6 +by_address space:1/offset:0xb8/size:4 3a97a07dfac9b131 6 +by_address space:1/offset:0x1200/size:16 3a97a07dfac9b131 6 +by_address space:1/offset:0xa0/size:4 463c09f7ecce624b 6 +by_address space:1/offset:0x30/size:4 47817e3fee8e884a 6 +by_address space:1/offset:0x18/size:4 67b2c44555057f14 6 +by_address space:1/offset:0x28/size:4 67b2c44555057f14 6 +by_address space:1/offset:0x8/size:4 7d168676ae3b0785 6 +by_address space:1/offset:0x28/size:4 7d168676ae3b0785 6 +by_address space:1/offset:0x1200/size:4 82bcb1bedb8f09fc 6 +by_address space:1/offset:0x8/size:4 915977fb8730d4b8 6 +by_address space:1/offset:0x28/size:4 915977fb8730d4b8 6 +by_address space:1/offset:0x38/size:8 98de717e2fe793ce 6 +by_address space:1/offset:0xb0/size:4 98de717e2fe793ce 6 +by_address space:1/offset:0x8/size:8 c71d53c68ab98180 6 +by_address space:1/offset:0x18/size:4 c71d53c68ab98180 6 +by_address space:1/offset:0x28/size:4 c71d53c68ab98180 6 +by_address space:1/offset:0x1280/size:8 cf8e29e9adbc6f07 6 +by_address space:1/offset:0xa0/size:4 d4496632e0982a90 6 +by_address space:1/offset:0x1240/size:8 081a6bbe95b68d3d 5 +by_address space:1/offset:0x18/size:8 134a977e75924abd 5 +by_address space:1/offset:0x1240/size:4 82bcb1bedb8f09fc 5 +by_address space:1/offset:0x10/size:8 98de717e2fe793ce 5 +by_address space:1/offset:0xa0/size:4 98de717e2fe793ce 5 +by_address space:2 cf8e29e9adbc6f07 5 +by_address space:1/offset:0x1240/size:4 cf8e29e9adbc6f07 5 +by_address space:1/offset:0x10/size:8 134a977e75924abd 4 +by_address space:1/offset:0x90/size:4 134a977e75924abd 4 +by_address space:1/offset:0x8/size:4 32880f6bb3a05eec 4 +by_address space:1/offset:0x10/size:8 32880f6bb3a05eec 4 +by_address space:1/offset:0x30/size:8 32880f6bb3a05eec 4 +by_address space:1/offset:0x38/size:4 32880f6bb3a05eec 4 +by_address space:1/offset:0x207/size:1 38185eeb1240bc84 4 +by_address space:1/offset:0x90/size:4 3ddd2923509c7f99 4 +by_address space:1/offset:0x90/size:8 3ddd2923509c7f99 4 +by_address space:1/offset:0x30/size:8 4194a77cd51dd75a 4 +by_address space:1/offset:0x30/size:4 463c09f7ecce624b 4 +by_address space:1/offset:0x80/size:8 463c09f7ecce624b 4 +by_address space:1/offset:0x8/size:4 47817e3fee8e884a 4 +by_address space:1/offset:0x28/size:4 47817e3fee8e884a 4 +by_address space:1/offset:0x38/size:4 47817e3fee8e884a 4 +by_address space:1/offset:0xb8/size:4 47817e3fee8e884a 4 +by_address space:1/offset:0x10/size:4 4f904662b361ccce 4 +by_address space:1/offset:0x10/size:8 67b2c44555057f14 4 +by_address space:1/offset:0x0/size:8 7cd59576ae038a0d 4 +by_address space:1/offset:0x18/size:4 7d168676ae3b0785 4 +by_address space:1/offset:0x38/size:8 7d168676ae3b0785 4 +by_address space:1/offset:0xa0/size:4 7d168676ae3b0785 4 +by_address space:1/offset:0xb0/size:4 7d168676ae3b0785 4 +by_address space:1/offset:0x18/size:4 915977fb8730d4b8 4 +by_address space:1/offset:0x38/size:8 915977fb8730d4b8 4 +by_address space:1/offset:0xa0/size:4 915977fb8730d4b8 4 +by_address space:1/offset:0xb0/size:4 915977fb8730d4b8 4 +by_address space:1/offset:0x10/size:8 c71d53c68ab98180 4 +by_address space:1/offset:0x30/size:4 d4496632e0982a90 4 +by_address space:1/offset:0x80/size:8 d4496632e0982a90 4 +by_address space:2 d58bae2a556c7dd4 4 +by_address space:1/offset:0x8/size:4 e8be19db525f9406 4 +by_address space:1/offset:0x1280/size:8 081a6bbe95b68d3d 3 +by_address space:1/offset:0x80/size:4 134a977e75924abd 3 +by_address space:1/offset:0x80/size:8 134a977e75924abd 3 +by_address space:1/offset:0xa8/size:4 3a97a07dfac9b131 3 +by_address space:1/offset:0x1240/size:16 3a97a07dfac9b131 3 +by_address space:1/offset:0x0/size:1 3ddd2923509c7f99 3 +by_address space:1/offset:0x8/size:8 463c09f7ecce624b 3 +by_address space:1/offset:0x18/size:4 463c09f7ecce624b 3 +by_address space:1/offset:0x38/size:4 463c09f7ecce624b 3 +by_address space:1/offset:0x18/size:4 47817e3fee8e884a 3 +by_address space:1/offset:0x18/size:8 67b2c44555057f14 3 +by_address space:1/offset:0x80/size:4 67b2c44555057f14 3 +by_address space:1/offset:0x80/size:8 67b2c44555057f14 3 +by_address space:1/offset:0x8/size:8 7d168676ae3b0785 3 +by_address space:1/offset:0x10/size:8 7d168676ae3b0785 3 +by_address space:1/offset:0x80/size:4 7d168676ae3b0785 3 +by_address space:1/offset:0x1280/size:8 82bcb1bedb8f09fc 3 +by_address space:1/offset:0x8/size:8 915977fb8730d4b8 3 +by_address space:1/offset:0x10/size:8 915977fb8730d4b8 3 +by_address space:1/offset:0x80/size:4 915977fb8730d4b8 3 +by_address space:1/offset:0x30/size:8 98de717e2fe793ce 3 +by_address space:1/offset:0x38/size:4 98de717e2fe793ce 3 +by_address space:1/offset:0x80/size:4 98de717e2fe793ce 3 +by_address space:1/offset:0x18/size:8 c71d53c68ab98180 3 +by_address space:1/offset:0x80/size:4 c71d53c68ab98180 3 +by_address space:1/offset:0x80/size:8 c71d53c68ab98180 3 +by_address space:1/offset:0x1280/size:4 cf8e29e9adbc6f07 3 +by_address space:1/offset:0x8/size:8 d4496632e0982a90 3 +by_address space:1/offset:0x18/size:4 d4496632e0982a90 3 +by_address space:1/offset:0x38/size:4 d4496632e0982a90 3 +by_address space:1/offset:0xa0/size:1 134a977e75924abd 2 +by_address space:1/offset:0xa8/size:4 134a977e75924abd 2 +by_address space:1/offset:0x1240/size:8 1d154638c246df67 2 +by_address space:1/offset:0x38/size:8 32880f6bb3a05eec 2 +by_address space:1/offset:0x1240/size:8 331f4abeadff6702 2 +by_address space:1/offset:0x38/size:4 3a97a07dfac9b131 2 +by_address space:1/offset:0x1240/size:8 3d7a2af4bc932920 2 +by_address space:1/offset:0x98/size:4 3ddd2923509c7f99 2 +by_address space:1/offset:0x98/size:8 3ddd2923509c7f99 2 +by_address space:1/offset:0x20a/size:1 3ddd2923509c7f99 2 +by_address space:1/offset:0x18/size:8 4194a77cd51dd75a 2 +by_address space:1/offset:0x80/size:8 4194a77cd51dd75a 2 +by_address space:1/offset:0x18/size:8 463c09f7ecce624b 2 +by_address space:1/offset:0x30/size:8 463c09f7ecce624b 2 +by_address space:1/offset:0x90/size:8 463c09f7ecce624b 2 +by_address space:1/offset:0x80/size:4 47817e3fee8e884a 2 +by_address space:1/offset:0xa0/size:4 47817e3fee8e884a 2 +by_address space:1/offset:0xa8/size:4 47817e3fee8e884a 2 +by_address space:1/offset:0xb0/size:4 47817e3fee8e884a 2 +by_address space:0 4f904662b361ccce 2 +by_address space:1/offset:0x1200/size:4 574fb9d8504e8da7 2 +by_address space:1/offset:0x10/size:4 5b7105f529b450ea 2 +by_address space:1/offset:0x30/size:8 67b2c44555057f14 2 +by_address space:1/offset:0x90/size:4 67b2c44555057f14 2 +by_address space:1/offset:0xa8/size:4 67b2c44555057f14 2 +by_address space:1/offset:0x30/size:8 7d168676ae3b0785 2 +by_address space:1/offset:0x38/size:4 7d168676ae3b0785 2 +by_address space:1/offset:0x1240/size:8 82bcb1bedb8f09fc 2 +by_address space:1/offset:0x30/size:8 915977fb8730d4b8 2 +by_address space:1/offset:0x38/size:4 915977fb8730d4b8 2 +by_address space:1/offset:0x18/size:8 98de717e2fe793ce 2 +by_address space:1/offset:0x20b/size:1 9d2f7aeabacb60f8 2 +by_address space:3 ad6ee9a28ee1d70d 2 +by_address space:1/offset:0x0/size:8 ad6ee9a28ee1d70d 2 +by_address space:1/offset:0x8/size:8 ad6ee9a28ee1d70d 2 +by_address space:1/offset:0x30/size:4 ad6ee9a28ee1d70d 2 +by_address space:1/offset:0x38/size:4 ad6ee9a28ee1d70d 2 +by_address space:1/offset:0x0/size:4 b5b8e47e410f229d 2 +by_address space:1/offset:0x30/size:8 c71d53c68ab98180 2 +by_address space:1/offset:0x90/size:4 c71d53c68ab98180 2 +by_address space:1/offset:0xa8/size:4 c71d53c68ab98180 2 +by_address space:1/offset:0x18/size:8 d4496632e0982a90 2 +by_address space:1/offset:0x30/size:8 d4496632e0982a90 2 +by_address space:1/offset:0x90/size:8 d4496632e0982a90 2 +by_address space:2 081a6bbe95b68d3d 1 +by_address space:1/offset:0x38/size:1 134a977e75924abd 1 +by_address space:1/offset:0x88/size:8 134a977e75924abd 1 +by_address space:1/offset:0x90/size:8 134a977e75924abd 1 +by_address space:1/offset:0x98/size:8 134a977e75924abd 1 +by_address space:2 234f01aef71604c0 1 +by_address space:1/offset:0x10/size:8 234f01aef71604c0 1 +by_address space:1/offset:0x0/size:1 32880f6bb3a05eec 1 +by_address space:1/offset:0x18/size:8 32880f6bb3a05eec 1 +by_address space:1/offset:0x80/size:8 32880f6bb3a05eec 1 +by_address space:1/offset:0x202/size:1 38185eeb1240bc84 1 +by_address space:1/offset:0x10/size:1 3ddd2923509c7f99 1 +by_address space:1/offset:0x30/size:1 3ddd2923509c7f99 1 +by_address space:1/offset:0x200/size:1 3ddd2923509c7f99 1 +by_address space:1/offset:0x1200/size:16 3ddd2923509c7f99 1 +by_address space:1/offset:0x8/size:8 4194a77cd51dd75a 1 +by_address space:1/offset:0x38/size:8 4194a77cd51dd75a 1 +by_address space:3 4195ea16a74b2541 1 +by_address space:1/offset:0x0/size:2 4195ea16a74b2541 1 +by_address space:1/offset:0x30/size:4 4195ea16a74b2541 1 +by_address space:1/offset:0x88/size:8 463c09f7ecce624b 1 +by_address space:1/offset:0x98/size:8 463c09f7ecce624b 1 +by_address space:1/offset:0xa8/size:4 463c09f7ecce624b 1 +by_address space:1/offset:0x88/size:4 47817e3fee8e884a 1 +by_address space:1/offset:0x98/size:4 47817e3fee8e884a 1 +by_address space:1/offset:0x1240/size:16 47817e3fee8e884a 1 +by_address space:0 574fb9d8504e8da7 1 +by_address space:1/offset:0x88/size:8 67b2c44555057f14 1 +by_address space:1/offset:0x90/size:8 67b2c44555057f14 1 +by_address space:1/offset:0x98/size:8 67b2c44555057f14 1 +by_address space:1/offset:0x18/size:8 7d168676ae3b0785 1 +by_address space:1/offset:0x80/size:1 7d168676ae3b0785 1 +by_address space:1/offset:0x80/size:8 7d168676ae3b0785 1 +by_address space:1/offset:0x90/size:8 7d168676ae3b0785 1 +by_address space:1/offset:0xa0/size:8 7d168676ae3b0785 1 +by_address space:2 82bcb1bedb8f09fc 1 +by_address space:1/offset:0x1280/size:4 82bcb1bedb8f09fc 1 +by_address space:1/offset:0x18/size:8 915977fb8730d4b8 1 +by_address space:1/offset:0x80/size:1 915977fb8730d4b8 1 +by_address space:1/offset:0x80/size:8 915977fb8730d4b8 1 +by_address space:1/offset:0x90/size:8 915977fb8730d4b8 1 +by_address space:1/offset:0xa0/size:8 915977fb8730d4b8 1 +by_address space:1/offset:0x80/size:1 98de717e2fe793ce 1 +by_address space:1/offset:0x80/size:8 98de717e2fe793ce 1 +by_address space:1/offset:0x90/size:8 98de717e2fe793ce 1 +by_address space:1/offset:0xa0/size:8 98de717e2fe793ce 1 +by_address space:0 9d2f7aeabacb60f8 1 +by_address space:2 9d2f7aeabacb60f8 1 +by_address space:0 b5b8e47e410f229d 1 +by_address space:1/offset:0x88/size:8 c71d53c68ab98180 1 +by_address space:1/offset:0x90/size:8 c71d53c68ab98180 1 +by_address space:1/offset:0x98/size:8 c71d53c68ab98180 1 +by_address space:1/offset:0x12c0/size:8 cf8e29e9adbc6f07 1 +by_address space:1/offset:0x88/size:8 d4496632e0982a90 1 +by_address space:1/offset:0x98/size:8 d4496632e0982a90 1 +by_address space:1/offset:0xa8/size:4 d4496632e0982a90 1 diff --git a/.claude/plans/r2il-behavioral-ir-v1-impl-spec.md b/.claude/plans/r2il-behavioral-ir-v1-impl-spec.md new file mode 100644 index 0000000000000..2f3140d25089c --- /dev/null +++ b/.claude/plans/r2il-behavioral-ir-v1-impl-spec.md @@ -0,0 +1,1071 @@ +# R2IL Behavioral IR v1 — PR 1 implementation spec (`crates/ruff_r2il`) + +> Implements **PR 1** of `.claude/plans/r2il-behavioral-ir-v1.md` as revised by the +> **PIVOT (2026-08-18)** and the **OPERATOR RULING (2026-08-18): the V3-shaped varnode is the +> DRILL KEY**. R2IL is an **INTAKE ARM**, not a bypass: typed input earns no exemption from +> ore → furnace → slag → proposer. Slag is evidence. +> Non-negotiables: workspace-**EXCLUDED** crate, path deps on +> `../../../r2sleigh/crates/{r2il,r2ssa}`, non-default feature `lift`. +> Every type/field/signature below was read from source this session. +> `JUDGMENT` marks a call made where the plan was silent. + +**One line so a future session does not re-chase the name:** `lance-graph-arm-discovery` is +**Association Rule Mining** (Aerial+, arXiv 2504.19354), not "intake arm". It consumes a declared +`FeatureSpec` + discretised `Dataset` and emits `CandidateRule`. It provides **no** intake, ore, +furnace, slag or proposer machinery. Do not reference it as reusable here. + +## 1. Framing — why there is a furnace at all + +> **Operator, 2026-08-18: "Varnode in the first stage is pointer chasing stacked god objects — +> hence the ore furnace slag."** + +The upstream typed truth is **good ore but structurally still stage-1 pointer chasing**: +`SSAFunction`'s private `HashMap`, a petgraph CFG, `BTreeMap` keyed +by String-carrying vars, facts as nested `BTreeMap`s of structs. **Typed ≠ refined.** Mistaking the +cleanliness of r2il's Rust types for refinement *is* the privileged-direct-path the pivot forbids. + +- **(a) The ARM preserves that object graph untouched** — it never flattens at intake. +- **(b) The FURNACE melts it** into flat, facet-addressed, concern-separated rows + (`FactId + VarnodeFacet + Concern` — plain flat `Vec`s, **never another object graph**). +- **(c) The SLAG is what resisted flattening**, addressed at the facet coordinate where it resisted. + +| stage | module | role | +| -------------------------------- | --------------------------- | ---------------------------------------------------------------- | +| 1 · intake arm / **ore carrier** | `behavior.rs` | lossless zero-copy assembly. **Not** an invented "contract" | +| 2 · **ore** | `ore.rs` | deterministic typed fact enumeration over the object graph | +| 3 · **furnace** | `furnace.rs` | melt → **flat** `FlatFact` rows; conservation ledger | +| 3b · **slag** | `slag.rs` | addressed residual rows; shape id + reason; **no `Other`, ever** | +| — · drill key + config tree | `facet.rs`, `convention.rs` | the 16-byte address, and longest-prefix-wins config over it | +| 4 · DTO / codebook factoring | `vocab.rs` | feeds lance-graph `ogar_codebook` **read-only** | +| — · artifact set | `examples/harvest_r2il.rs` | the deliverable, per MedCare-rs / openproject-nexgen-rs | + +Refined concern *contracts* are a later, measured furnace output. This PR does not invent them. + +## 2. Verified upstream facts every worker must build against + +A worker that contradicts one of these is wrong, not the source. + +| Fact | Where | +| ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- | +| `Varnode { space: SpaceId, offset: u64, size: u32, meta: Option }`; `PartialEq`/`Hash` **ignore `meta`** | `r2il/src/varnode.rs:19,149-163` | +| `SpaceId::{Ram, Register, Unique, Const, Custom(u32)}` (`Copy, Eq, Hash`, default `Ram`) | `r2il/src/space.rs:10-23` | +| `R2ILBlock { addr, size, ops: Vec, switch_info, op_metadata: BTreeMap }` | `r2il/src/opcode.rs:1190-1203` | +| `R2ILOp::{inputs() -> Vec<&Varnode>, output() -> Option<&Varnode>, is_control_flow/_memory_read/_memory_write}` | `r2il/src/opcode.rs:499-804` | +| **`ArchSpec` public fields** `name: String`, `addr_size: u32`, `spaces: Vec`, `registers: Vec`, `userops: Vec` | `r2il/src/serialize.rs:97-140` | +| `RegisterDef { name: String, offset: u64, size: u32, parent: Option }`; `UserOpDef { index: u32, name: String }`; `AddressSpace { id: SpaceId, name: String, … }` | `serialize.rs:42-51,77-82`; `space.rs:61-90` | +| `SSAFunction`: `pub name`, `pub entry`; `cfg`/`domtree`/`blocks`/`block_order` **PRIVATE** — use `blocks()`, `block_addrs()`, `get_block()`, `predecessors()`, `successors()`, `cfg()`, `num_blocks()` | `function.rs:251-268, 946-1004` | +| `SSAFunction::from_blocks_raw(&[R2ILBlock], Option<&ArchSpec>) -> Option` — raw, **no optimization**. `from_blocks_with_arch` = raw **then constructor-time SCCP** (can rewrite ops) | `function.rs:829, 738-753` | +| `SsaArtifact::{from_blocks, raw, …}` → accessors `function/graph/facts/objects/memory/predicates/call_sites/mode/with_name`; `new` **private**; `Debug + Clone`; `Deref` | `function.rs:78-234` | +| `SsaGraph::from_function`; public `entry, block_order, blocks, insts, values, def_of, uses_of, block_by_addr, value_by_var, op_inst_by_site, op_site_by_inst`; `GraphInst{id, block, ordinal, inputs, output, payload}`; `InstPayload::{Phi{predecessors}, Op(SSAOp)}`; `GraphBlock{id, addr, size, predecessors, successors, insts}` | `graph.rs:30-72` | +| `PreparedFunctionFacts::collect(&SSAFunction, &SsaGraph)` → `{objects, memory, predicates, call_sites}` | `semantic.rs:188-208` | +| `MemoryUseFact{location, version}`, `MemoryDefFact{location, previous_version, next_version}`, `MemoryLocation{object: ObjectId, offset: i64, size: u32}`, `MemoryVersion{object, version: u32}` | `semantic.rs:73-111` | +| `PredicateFact{id, block_addr, condition: ValueId, comparison: Option, true_target: u64, false_target: u64}`; `CompareKind::{Equal,NotEqual,Less,SignedLess,LessEqual,SignedLessEqual}` | `semantic.rs:113-138` | +| `CallSiteFact{id, at: InstId, target: ValueId, direct_target: Option, fallthrough, memory_effect}`; `ObjectKind::{StackSlot,FrameObject,Global{space:String,address},HeapAlloc,EscapedUnknown}` | `semantic.rs:172-186, 29-36` | +| `InterprocFunctionInput<'a>{id, name, prepared: &'a SsaArtifact}`; `solve_interproc_summary_set(&[..], Option<&ArchSpec>, Option, &BTreeMap<..>, InterprocSolveConfig)` | `interproc.rs:232,451` | +| **`SSAVar{name: String, version: u32, size: u32}` — NO `offset`, NO `SpaceId`** | `var.rs:10-17` | +| `SSAOp` has **no `Multiequal`, no `Indirect`**; memory ops carry `space: String` | `op.rs:16-350` | +| Function-level rename stringifies spaces as **`format!("{:?}", space)`** → `"Ram"`, `"Custom(7)"`; `varnode_to_name`: `Register→"rax"\|"reg:{x}"`, `Unique→"tmp:{x}"`, `Const→"const:{x}"`, `Ram→"ram:{x}"`, `Custom(id)→"space{id}:{x}"` | `rename.rs:456+`; `naming.rs:120-135` | +| `rename_op` is **total, 1 SSAOp per R2ILOp**; `Multiequal → SSAOp::Phi`, `Indirect → SSAOp::Copy` | `rename.rs:433,1087,1101` | +| Construction partitions `SSAOp::Phi` out of `ops` into `block.phis`, **zipping sources with `cfg.predecessors(addr)`** → fan-in truncates to the predecessor count | `function.rs:888-911` | +| `CFG::from_blocks`: **one `R2ILBlock` = one CFG node**; terminator from a **reverse** op scan (last CF op wins); `Fallthrough{next: addr+size}` when none; branch/call targets need `Const`/`Ram` to be typed | `cfg.rs:252-274, 96-160` | +| `PredicateFact` needs terminator `ConditionalBranch` **and** the block's **last** op `SSAOp::CBranch` | `semantic.rs:670-709` | +| memory uses ← `{Load,LoadLinked,LoadGuarded,AtomicCAS,StoreConditional}` + calls; defs ← `{Store,StoreGuarded,StoreConditional,AtomicCAS}` + calls | `semantic.rs:397-471` | +| `MemoryOrdering::{Relaxed,Acquire,Release,AcqRel,SeqCst,Unknown}` | `r2il/src/memory.rs:10-18` | +| `Disassembler::from_sla(&[u8], &str, &str)` (**no `new`**), `lift`, `lift_block`, `set_userop_map`; `MIN_BYTES = 16`; `build_arch_spec(&[u8], &str, &str)`; `userop_map_for_arch(&str)` | `disasm.rs:149,261,314,301`; `sleigh.rs:106` | +| `r2ssa` has a **hard, non-optional** dep on `r2sleigh-lift` (libsla native build) | `r2ssa/Cargo.toml` | + +### ⚠ The load-bearing consequence: the facet coordinate is NOT recoverable from SSA + +`SSAVar` carries `name/version/size` and **no offset and no `SpaceId`**. The only SSA-side trace of +a varnode's offset is inside the *display name* (`"reg:10"`, `"space7:1000"`) — and parsing display +strings is forbidden as a data path. Therefore: + +- **`ore::enumerate` takes BOTH the `FunctionBehavior` AND the source `&[R2ILBlock]`.** Operand + coordinates come from the **typed** `Varnode`s in the source ops, joined to SSA instructions by + the `(block_addr, op_idx)` key `SsaGraph::op_inst_by_site` provides — the same key §8 test 6 proves + round-trips. +- The join is **verified, not assumed**: at each site compare `OpTag::from_op(ssa_op)` against the + R2IL op's tag; a mismatch (the `Multiequal` index shift, or a `CallDefine` insertion) emits + `ResidualReason::OpSiteJoinMismatch`, never a silently misattributed coordinate. +- Rows with no source varnode (phi inputs, `CallDefine`) get `at: None` and become the **named** + residual `ResidualReason::NoFacetCoordinate`. Nothing is dropped. + +**Honesty notes that must appear as doc comments in the code, not only here:** + +1. `lift` does **not** buy "zero system deps by default": `r2ssa` drags `r2sleigh-lift` → `libsla` in + unconditionally; `lift` gates only this crate's **direct** disassembler/`sleigh-config` use. +1. `Varnode.meta` and `R2ILBlock.op_metadata` **do not cross into SSA**; they stay addressable by the + same `(block_addr, op_idx)` key. That rejoin is the contract. +1. `r2il::SwitchInfo` (struct) vs `r2ssa::SwitchInfo` (type alias) — alias on import. +1. **Do NOT copy `#![expect(clippy::print_stderr, …)]` from `ruff_cpp_spo`'s examples.** That lint + comes from ruff's `[workspace.lints]`, which an **excluded** crate cannot inherit; the unfulfilled + expectation would itself fail `clippy -D warnings`. + +______________________________________________________________________ + +## 3. File inventory — `crates/ruff_r2il/` + +```text +crates/ruff_r2il/ +├── Cargo.toml +├── src/lib.rs # crate docs + module decls (W1, up front) +├── src/behavior.rs # stage 1 — ore carrier +├── src/facet.rs # the DRILL KEY: 16-byte address + config-key scheme +├── src/convention.rs # longest-prefix-wins config tree over facet space +├── src/ore.rs # stage 2 — deterministic typed fact enumeration +├── src/furnace.rs # stage 3 — melt to FLAT addressed rows + conservation +├── src/slag.rs # stage 3b — addressed residual ledger +├── src/vocab.rs # stage 4 — DTO/codebook factoring +├── tests/lossless_fixtures.rs # §14 fixtures + the stressor-slag proof +├── examples/harvest_r2il.rs # THE deliverable: the .claude/harvest artifact set +└── examples/r2il_corpus_profile.rs # §12 corpus profile +``` + +`Cargo.toml`, full intended contents. Excluded ⇒ **no `{ workspace = true }` inheritance anywhere**: + +```toml +[package] +name = "ruff_r2il" +version = "0.1.0" +publish = false +# Excluded from the ruff workspace (root `exclude`), so nothing here can inherit from +# [workspace.package] / [workspace.dependencies] / [workspace.lints]. edition/rust-version are +# pinned by hand to satisfy BOTH sides: ruff (edition 2024, rust-version 1.93, toolchain 1.97.1) +# and r2sleigh (edition 2024). +edition = "2024" +rust-version = "1.93" +description = "R2IL intake arm: a lossless ore carrier over r2sleigh's r2il/r2ssa object graph, a furnace that melts it into flat facet-addressed concern rows, and an addressed residual (slag) ledger. Ore in, named slag out — no catch-all, nothing dropped." + +[lib] +name = "ruff_r2il" + +[dependencies] +# The AdaWorldAPI fork checkout, consumed by path (P0 fork rule). DEFAULT deps: the typed +# behavioral surface is the crate's reason to exist. +r2il = { path = "../../../r2sleigh/crates/r2il" } +r2ssa = { path = "../../../r2sleigh/crates/r2ssa" } + +# Non-default `lift`, mirroring the ruff_cpp_spo shape. NOTE the honest difference: r2ssa depends +# on r2sleigh-lift unconditionally, so this feature does NOT keep libsla out of the build — it +# gates only this crate's DIRECT use of the disassembler and the .sla/.pspec data. +r2sleigh-lift = { path = "../../../r2sleigh/crates/r2sleigh-lift", optional = true } +sleigh-config = { version = "1.0", optional = true, features = ["x86"] } + +[features] +default = [] +lift = ["dep:r2sleigh-lift", "dep:sleigh-config"] + +[[example]] +name = "harvest_r2il" +required-features = ["lift"] + +[[example]] +name = "r2il_corpus_profile" +required-features = ["lift"] + +[lints.rust] +unsafe_code = "forbid" +unreachable_pub = "warn" +``` + +- **No `license` field** (`publish = false`); no licensing statement is encoded here or in a comment. +- **No `[lints.clippy] pedantic`** — `JUDGMENT`: an excluded crate cannot inherit ruff's lint table, + and hand-copying it creates an unmeasured lint surface for workers forbidden to compile. +- **No `[dev-dependencies]`**, no serde, no gz: fixtures and artifacts use only `r2il` + `r2ssa`. +- `crates/ruff_r2il/Cargo.lock` **is committed** (not gitignored; the excluded crate is its own + workspace root and an arm's numbers must be reproducible). + +______________________________________________________________________ + +## 4. `src/facet.rs` — the DRILL KEY (promoted; two roles, one shape) + +Module doc must open with both roles, explicitly: + +> **Role 1 (PR 1, load-bearing): the ADDRESS / CONFIG-KEY scheme.** `VarnodeFacet` is the 16-byte +> V3-shaped identity `classid(space-class) | offset_lo | offset_hi | size`, **prefix-routable by +> construction**. It is the key `convention.rs` drills on and the coordinate `slag.rs` residuals +> are addressed at. +> **Role 2 (PR 2, NOT committed here): V3 SoA persistence.** Promoting the shape as a key commits +> **no storage layout**. Same 16 bytes, two roles, no persistence decision yet. + +```rust +/// Provisional container concept. The REAL mint is a canon-high slot in +/// `lance_graph_contract::ogar_codebook` (plan PR 3, the `NETWORK_LAYER = 0x0804` analog). +/// Until then this is LOCAL and provisional — never persist it as an address. +/// +/// ⚠ Known tension, recorded rather than hidden: OGAR's consumer rule is "hi u16 = shared +/// concept, lo u16 = APP render prefix — NEVER a shape ordinal", and the space discriminant +/// below IS a shape ordinal in the lo half. PR 3 owns the real carving. +pub const PROVISIONAL_R2IL_VARNODE: u16 = 0x0000; +pub const SPACE_RAM: u16 = 0; pub const SPACE_REGISTER: u16 = 1; +pub const SPACE_UNIQUE: u16 = 2; pub const SPACE_CONST: u16 = 3; +pub const CUSTOM_ORDINAL_BASE: u16 = 4; +pub const MAX_CUSTOM_ORDINAL: u16 = u16::MAX - CUSTOM_ORDINAL_BASE; // 65531 +``` + +**Layout — 16 bytes, all little-endian:** `0..4` `classid: u32` = +`((PROVISIONAL_R2IL_VARNODE as u32) << 16) | space_discriminant as u32`; `4..8` offset low 32; +`8..12` offset high 32; `12..16` `size: u32`. `Varnode::meta` is **documented-excluded** (advisory +plane) — matching upstream, where `Varnode`'s `PartialEq`/`Hash` already ignore it. + +```rust +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct VarnodeFacet(pub [u8; 16]); +impl VarnodeFacet { + pub fn space_discriminant(&self) -> u16; + pub fn offset(&self) -> u64; // lo | hi << 32 + pub fn size(&self) -> u32; + /// The three prefix keys this facet resolves against, coarsest first. + pub fn prefixes(&self) -> [FacetPrefix; 3]; +} + +/// A config-tree key: a facet PREFIX. Ordered coarse → fine, exactly the three levels the +/// convention drills on. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum FacetPrefix { + Space { discriminant: u16 }, + SpaceOffset { discriminant: u16, offset: u64 }, + SpaceOffsetSize { discriminant: u16, offset: u64, size: u32 }, +} +impl FacetPrefix { pub fn depth(&self) -> u8; /* 1, 2, 3 */ } + +/// Deterministic interning of `SpaceId::Custom(u32)` raw ids → lo-u16 ordinals (sorted order). +pub struct CustomSpaceTable { ids: Vec } +impl CustomSpaceTable { + /// `Err(CustomOrdinalExhausted)` when the set exceeds the lo-u16 budget — the + /// `mint_factored` principle: overflow is EVIDENCE a route needs factoring, never silent + /// truncation. + pub fn from_ids>(ids: I) -> Result; + /// Bootstrap from upstream data that already exists (§5's read-don't-retype rule): + /// every `AddressSpace` in `ArchSpec::spaces` whose `id` is `SpaceId::Custom(n)`. + pub fn from_arch(arch: &ArchSpec) -> Result; + pub fn ordinal_of(&self, raw: u32) -> Option; + pub fn raw_of(&self, ordinal: u16) -> Option; + pub fn len(&self) -> usize; +} + +pub enum FacetOverflow { + /// A `Custom(raw)` the table does not know. Projection REFUSES; never `raw as u16` + /// (65541 and 5 both truncate to 5). + UnknownCustomSpace { raw: u32 }, + CustomOrdinalExhausted { count: usize }, +} + +/// Build the drill key from the TYPED r2il varnode. This is the ONLY constructor — a facet is +/// never derived from an SSAVar (which has no offset and no SpaceId; see §2's ⚠ note). +pub fn project(vn: &Varnode, spaces: &CustomSpaceTable) -> Result; +pub fn unproject(f: &VarnodeFacet, spaces: &CustomSpaceTable) -> Result; +``` + +**Config-key-time losslessness (the promoted `Custom(u32)` falsifier).** State in the module docs: +*a `Custom` id that overflows the interned-ordinal budget must fail **typed** at CONFIG-KEY +construction, not only at projection.* A config tree keyed by a truncated address would silently +attach rows to the wrong varnode family — the worst possible failure for a drill scheme. Hence +`CustomSpaceTable::from_ids`/`from_arch` return `Result`, and `convention.rs` (§5) propagates it. + +### Tests, each two-sided + +1. **`fixed_spaces_round_trip_byte_for_byte`** (+ the four classid words must be **distinct**, so a + constant-returning impl fails). +1. **`custom_space_within_budget_round_trips`** — table `{3,7,9}`; `Custom(7)`'s lo u16 == + `CUSTOM_ORDINAL_BASE + 1` (sorted position). +1. **`custom_space_outside_the_table_errors_and_never_truncates`** — plan item **O3**: `Custom(5)` + vs an empty table → `Err(UnknownCustomSpace{raw:5})`; then intern **both** `5` and `65541` (the + pair a `raw as u16` cast would collide) and assert distinct facets, each unprojecting to its own + raw id. +1. **`too_many_custom_spaces_is_a_typed_overflow_not_a_wrap`** — budget+2 errors; exactly budget + succeeds. Both sides pinned. +1. **`offsets_above_u32_max_survive_the_lo_hi_split`** — `0x1234_5678_9ABC_DEF0` round-trips; + anti-vacuity: `u32::from_le_bytes(f.0[4..8]) as u64 != offset`. +1. **`meta_is_excluded_from_the_projection`** — same varnode with/without metadata → **identical 16 + bytes**; doc comment names the upstream contract it mirrors. +1. **`prefixes_are_ordered_coarse_to_fine_and_share_their_ancestors`** — `prefixes()[0].depth()==1 … [2].depth()==3`; two facets in the same space share `prefixes()[0]` but differ at `[1]` when + their offsets differ. Falsifies a prefix builder that ignores a component. + +______________________________________________________________________ + +## 5. `src/convention.rs` — longest-prefix-wins config tree (NEW) + +Precedent: `ruff_spo_triplet::concept_split::ConceptConvention` (caller-supplied, **zero domain +vocabulary in the module**) and OGAR codebook scoping (*"longest-prefix wins — one rule, every +level"*). **This module ships ZERO architecture vocabulary** — no register names, no opcode +semantics, no userop table. Everything arrives as data. + +```rust +/// One config row, attached at a facet PREFIX. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ConventionRow { + pub at: FacetPrefix, + pub name: Option, // e.g. a register name, a space name + pub note: Option, // free provenance text; never branched on + pub state: ValidationState, +} + +/// Mirrors openproject-nexgen-rs's `orm-ar-backprojection.toml` +/// (`validation_states = [unmeasured|confirmed|corrected|retired]`, meta key +/// `measure_dont_claim`). EVERY row starts `Unmeasured`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ValidationState { Unmeasured, Confirmed, Corrected, Retired } + +/// The drilling convention: a radix tree over varnode identity space. +/// +/// NOT a flat table. Rows attach at prefixes (space-class; space-class+offset; +/// space-class+offset+size) and resolution is LONGEST MATCHING PREFIX. Residual rows are +/// addressed (§7), so the proposer emits proposed rows AT those addresses and pass N+1 drills +/// with them — the config accumulates as a radix tree, self-scaffolding. +#[derive(Debug, Clone, Default)] +pub struct R2ilConvention { + rows: BTreeMap, // BTreeMap ⇒ deterministic emission order + spaces: CustomSpaceTable, + userops: BTreeMap, + /// THE furnace ladder, as DATA. Pass 1 carries exactly seven entries. Widening the pass is + /// a CONFIG change with a measured before/after ledger — never a code edit to a match arm. + classified_opcodes: BTreeSet, + pub arch: Option, // provenance only; never branched on +} + +impl R2ilConvention { + /// Pass 1: `[Copy, IntAdd, Load, Store, CBranch, Call, Return]`, no rows, no userops, no + /// custom spaces. Deliberately minimal — the stressors MUST land in slag, and the ledger + /// NAMING them is the acceptance criterion, not a bigger match. + pub fn minimal_pass_one() -> Self; + + /// BOOTSTRAP — read, never retype. Populates from data that already exists on `ArchSpec`: + /// * `arch.registers: Vec` → one + /// `FacetPrefix::SpaceOffsetSize{ SPACE_REGISTER, offset, size }` row per register, + /// `name = Some(reg.name)`, `state = Unmeasured`; + /// * one coarse `FacetPrefix::Space{ SPACE_REGISTER }` fall-through row named after the + /// register space, so an UNKNOWN register offset still resolves — to the space, not to + /// nothing; + /// * `arch.userops: Vec` → the userop table; + /// * `arch.spaces` (`AddressSpace{id: SpaceId::Custom(n), name}`) → `CustomSpaceTable`. + /// Errors only through `FacetOverflow` (config keys must be lossless, §4). + pub fn from_arch(arch: &ArchSpec, classified: impl IntoIterator) + -> Result; + + /// Longest-prefix-wins resolution: try `SpaceOffsetSize`, then `SpaceOffset`, then `Space`; + /// first hit wins. `None` = the convention says nothing at this address (⇒ slag). + pub fn resolve(&self, facet: &VarnodeFacet) -> Option<&ConventionRow>; + /// The longest prefix that DID resolve — what an addressed residual reports so the proposer + /// knows where to attach the next row. + pub fn resolved_prefix(&self, facet: &VarnodeFacet) -> Option; + + pub fn classifies(&self, op: OpTag) -> bool; + pub fn userop_name(&self, index: u32) -> Option<&str>; + pub fn spaces(&self) -> &CustomSpaceTable; + pub fn insert(&mut self, row: ConventionRow); // proposer entry point + pub fn rows(&self) -> impl Iterator; // BTreeMap order + + /// Nested-TOML rendering, mirroring the harvest precedents. Hand-written (no serde dep): + /// a `[meta]` table (`measure_dont_claim = true`, `validation_states = [...]`, arch, + /// classified_opcodes), then one `[[row]]` per row in `BTreeMap` order with + /// `prefix_depth`, `space`, `offset`, `size`, `name`, `state`. Emission only — nothing in + /// ruff parses it back. + pub fn to_toml(&self) -> String; +} +``` + +### Tests + +1. **`arch_registers_bootstrap_the_register_branch`** — build an `ArchSpec` with **N = 3** + `RegisterDef`s (e.g. `("rax",0,8)`, `("eax",0,4)`, `("rbx",8,8)` — note two share an offset and + differ only in size, which is exactly why the finest prefix carries size). Assert each of the 3 + register varnodes resolves via `resolve()` to a row whose `name` is its own register name, and + that every row's `state == Unmeasured`. Anti-vacuity: assert the three resolved names are + **distinct** (a bootstrap that mapped everything to one row would otherwise pass). +1. **`an_unknown_register_offset_falls_through_to_the_space_prefix`** — resolve + `Varnode::register(0xDEAD, 8)`: `resolve()` returns the coarse `Space{SPACE_REGISTER}` row and + `resolved_prefix()` reports `depth() == 1`, **not** `None` and **not** a depth-3 row. Two-sided + with test 1 (a known register must report `depth() == 3`). +1. **`longest_prefix_wins_over_a_coarser_row`** — insert a `SpaceOffset` row and a + `SpaceOffsetSize` row at the same offset; the facet with the matching size resolves to the + finer row, a facet with a different size resolves to the coarser one. Falsifies a first-match + or coarsest-wins implementation. +1. **`custom_space_overflow_fails_at_config_key_time`** — `from_arch` over an `ArchSpec` whose + custom spaces exceed the budget returns `Err(CustomOrdinalExhausted)`; a within-budget spec + succeeds. Pins the §4 promotion. +1. **`toml_rendering_is_byte_stable_and_starts_every_row_unmeasured`** — render twice, compare; + assert every `state = "unmeasured"` and that `measure_dont_claim` is present. + +______________________________________________________________________ + +## 6. `src/ore.rs` — stage 2, the enumeration (NEW) + +Module header carries the operator one-liner verbatim: + +> **"Varnode in the first stage is pointer chasing stacked god objects — hence the ore furnace +> slag."** The upstream object graph (a private `HashMap`, a petgraph CFG, +> `BTreeMap` keyed by String-carrying vars, nested-`BTreeMap` facts) is GOOD ORE +> and structurally still stage-1 pointer chasing. **Typed ≠ refined.** This module *reads* that +> graph and emits typed ore rows; it does **not** flatten (that is `furnace.rs`) and it does not +> classify. + +```rust +/// One variant per `SSAOp` variant. Closed, exhaustive, stable `as_str()` — the discipline +/// `ruff_spo_triplet::Predicate` enforces ("frontends MUST NOT emit raw predicate strings"). +/// `from_op` is a total match; `format!("{:?}")` is FORBIDDEN. +pub enum OpTag { Phi, Copy, Load, Store, Fence, LoadLinked, StoreConditional, AtomicCAS, + LoadGuarded, StoreGuarded, IntAdd, IntSub, /* … one per SSAOp variant … */ } +impl OpTag { + pub fn from_op(op: &SSAOp) -> Self; + /// From the R2IL side, for the op-site join verification (§2 ⚠). `Multiequal → Phi`, + /// `Indirect → Copy`, matching `rename_op`'s documented mapping. + pub fn from_r2il(op: &R2ILOp) -> Self; + pub fn as_str(self) -> &'static str; +} + +pub enum EdgeTag { Normal, True, False, Back } +pub enum CompareTag { Equal, NotEqual, Less, SignedLess, LessEqual, SignedLessEqual } +pub enum OperandPos { Input(usize), Output } + +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub struct FactProvenance { + pub inst: Option, pub block: Option, + pub op_site: Option<(u64, usize)>, pub value: Option, +} + +pub enum OreFact { + Op { prov, opcode: OpTag, ordinal: usize, input_arity: usize, has_output: bool }, + /// The TYPED coordinate components, taken from the source `Varnode` via the verified + /// op-site join — never from an `SSAVar` name. + Operand { prov, position: OperandPos, value: Option, + space: SpaceId, offset: u64, size: u32 }, + Edge { from: BlockId, to: BlockId, kind: EdgeTag }, + PhiInput { prov, index: usize, pred: BlockId, value: ValueId }, + MemoryUse { prov, object: ObjectId, version: u32, size: u32 }, + MemoryDef { prov, object: ObjectId, previous: u32, next: u32, size: u32 }, + Predicate { prov, id: PredicateId, condition: ValueId, comparison: Option, + true_target: u64, false_target: u64 }, + CallSite { prov, id: CallSiteId, target: ValueId, direct_target: Option }, + /// The join failed at this site — emitted so the failure is enumerated, not skipped. + JoinFailure { prov, expected: OpTag, found: OpTag }, +} + +/// Enumerate the ore. Deterministic, total, lossless w.r.t. the SSA surface. +/// `blocks` is the SOURCE R2IL, required for the typed operand coordinates (§2 ⚠). +pub fn enumerate(behavior: &FunctionBehavior, blocks: &[R2ILBlock]) -> Vec; +``` + +**Enumeration order is fixed and documented** (this IS the determinism guarantee): blocks in +`SSAFunction::block_addrs()` order (reverse postorder, a `Vec` — *not* the `HashMap`); within a +block, phis then ops by `GraphInst::ordinal`; per op the `Op` row, then `Operand` rows in `inputs` +order then `Output`; then `Edge` rows from `GraphBlock::successors`; then `PhiInput`; then memory +uses/defs by `InstId`; then predicates by `PredicateId`; then call sites by `CallSiteId`. Every +container touched is ordered — **no `HashMap` iteration anywhere**. + +### Tests + +1. **`enumeration_is_deterministic`** — `enumerate` twice over the same behavior and over a + freshly re-ingested identical block list; both sequences compare equal element by element. + Falsified the moment `HashMap` order leaks in. +1. **`operand_coordinates_come_from_the_typed_source_not_from_names`** — a fixture with + `Varnode::register(0x1234_5678_9ABC_DEF0 & 0xFFFF, 8)` and a `Custom(7)` operand: the emitted + `Operand` rows carry the exact `SpaceId` and `offset`, and **no** `OreFact` construction path + reads an `SSAVar::name`. Anti-vacuity: assert at least one operand has `space == SpaceId::Custom(7)` (unreachable from the SSA side without parsing). +1. **`an_op_site_join_mismatch_is_enumerated_not_skipped`** — the `Multiequal` fixture (§8 test 8) + shifts source indices; assert at least one `OreFact::JoinFailure` is emitted with both tags + populated, and that the total ore row count still matches the independent expectation. + +______________________________________________________________________ + +## 7. `src/furnace.rs` — stage 3, the melt (NEW) + +Module doc, mandatory and explicit: + +> The furnace melts the stage-1 object graph into **flat, facet-addressed, concern-separated +> rows**. `FlatFact` is `FactId + VarnodeFacet + Concern + fixed scalar payload` — plain flat +> `Vec`s. **It must never grow back into a nested object graph.** No `Vec`, no `Box`, no map, no +> `String` inside a `FlatFact`; cardinality is handled by EMITTING MORE ROWS (a `CallOther` with +> four inputs is one `Op` row plus four `Operand` rows), never by nesting a collection. A +> `FlatFact` refers to another only by `FactId`. + +```rust +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct FactId(pub u32); + +/// The route the row belongs to — r2sleigh's own decomposition, named. +pub enum Concern { Control, Values, Objects, Memory, Predicates, Calls } +pub enum FactKind { Op, OperandIn, OperandOut, Edge, PhiInput, MemUse, MemDef, + Predicate, CallSite } + +/// ONE flat row. `const _: () = assert!(size_of::() <= 64);` — the compile-time guard +/// against the nested-object-graph regression. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct FlatFact { + pub id: FactId, + /// The drill key. Operand rows carry their own varnode's facet; rows with no varnode of + /// their own are BLOCK-ANCHORED — `project(&Varnode::ram(block_addr, 0))` — which is a + /// documented convention, never an implicit default. + pub at: VarnodeFacet, + pub concern: Concern, + pub kind: FactKind, + pub opcode: OpTag, + /// Two fixed typed payload slots (e.g. version/previous/next, index/arity, target addr). + /// Their meaning per `kind` is documented in a table in the module docs. + pub a: u64, + pub b: u64, + pub prov: FactProvenance, +} + +/// Melt one function. `Ok` rows are flat and addressed; everything else is addressed slag. +pub fn smelt(behavior: &FunctionBehavior, blocks: &[R2ILBlock], conv: &R2ilConvention) + -> (Vec, ResidualLedger, HarvestReport); + +/// Conservation ledger — the `harvested N / classified X / residual Y / dropped 0` line. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct HarvestReport { pub harvested: usize, pub classified: usize, + pub residual: usize, pub dropped: usize } +impl HarvestReport { + /// `harvested == classified + residual + dropped` AND `dropped == 0`. + pub fn is_conserved(&self) -> bool; +} +/// Per-fact-kind and per-`OpTag` counts — the census (the "108,548 triples by predicate" +/// analog). `BTreeMap`, so the census artifact is byte-stable across runs. +pub struct Census { pub by_fact_kind: BTreeMap<&'static str, usize>, + pub by_opcode: BTreeMap<&'static str, usize> } +pub fn census(rows: &[FlatFact]) -> Census; +``` + +**Pass-1 ladder (the whole thing — nothing else classifies).** An `OreFact::Op` melts iff +`conv.classifies(opcode)`. An `Operand` melts iff its parent op melted **and** `facet::project` +succeeds **and** `conv.resolve(&facet)` is `Some`. `Edge`/`PhiInput`/`MemoryUse`/`MemoryDef`/ +`Predicate`/`CallSite` melt iff their parent op (by `prov.inst`) melted and their own condition +holds (`CallSite` needs `direct_target` `Some`; `PhiInput` needs `index < predecessors`; +`MemoryUse`/`Def` need a non-`EscapedUnknown` object). `JoinFailure` never melts. Everything else +becomes a **named, addressed** residual. There is no third outcome, so `dropped == 0` holds by +construction of the `Result` signature. + +### Tests + +1. **`a_flat_fact_stays_flat`** — `assert_eq!(size_of::(), )` and + `assert!(FlatFact: Copy)`. Adding a `Vec`/`String`/`Box` breaks both. This is the + never-a-nested-graph guard, mechanised. +1. **`cardinality_is_rows_not_nesting`** — a `CallOther` with four inputs produces **1** `Op` row + and **4** `OperandIn` rows sharing its `FactId` via `prov.inst`; assert the counts exactly. +1. **`melt_is_conserved_and_drops_nothing`** — over the §8 fixture: `harvested == classified + residual`, `dropped == 0`, and `harvested` equals an independently recomputed expectation. + Anti-vacuity: `harvested >= 50`. +1. **`the_convention_is_the_knob_not_the_code`** — with `minimal_pass_one()` every melted `Op`'s + opcode is one of the seven and `classified > 0`; then add exactly `OpTag::AtomicCAS` to the + convention and assert `classified` rises by **exactly** the AtomicCAS fact count and `residual` + falls by the same. Proves widening happens in data. +1. **`block_anchored_rows_are_addressed_not_defaulted`** — every emitted `FlatFact` has an `at` + whose `space_discriminant()` and `offset()` are meaningful; no row carries the all-zero facet + unless its varnode genuinely is `Ram:0/size 0`. Falsifies a lazy `VarnodeFacet::default()`. + +______________________________________________________________________ + +## 8. `src/slag.rs` — stage 3b, the ADDRESSED residual ledger (NEW) + +```rust +/// FNV-1a 64 over the residual's SHAPE — the reason discriminant and its typed payload only, +/// NEVER the provenance and NEVER the address. Identical shapes at different sites therefore +/// group, exactly as MedCare-rs's `fnv1a:` class fingerprints do. +pub struct ShapeId(pub u64); + +/// Why the CURRENT convention could not melt a fact. +/// +/// The slag doctrine (`concept_split.rs` § "The slag doctrine"): the residual is not waste, it +/// is the empirical boundary of the current convention; a recurring reason NAMES the next +/// convention row to add. +/// +/// HARD RULE — there is NO catch-all. No `Other`, no `Opaque`, no `Unknown`, no `_ =>` arm that +/// manufactures a reason. A shape that fits no variant means ADD A VARIANT (and record the +/// before/after counts), never widen an existing one to swallow it. +pub enum ResidualReason { + OpcodeNotInConvention { opcode: OpTag }, + NoConventionRowAtAddress, // resolve() returned None at every prefix + UserOpNotInConvention { userop: u32 }, + CustomSpaceNotInConvention { raw: u32 }, + FacetOverflowAtKey { raw: u32 }, // config keys must be lossless (§4) + VariadicArity { arity: usize }, + PhiFanInExceedsPredecessors { inputs: usize, predecessors: usize }, + MemoryObjectEscaped, + IndirectTarget, + NoFacetCoordinate, // phi input / CallDefine — no source varnode + OpSiteJoinMismatch { expected: OpTag, found: OpTag }, +} +impl ResidualReason { + /// Every variant, for the no-catch-all test. Adding a variant without listing it here is + /// exactly what that test exists to catch. + pub const ALL: &'static [&'static str] = &[ /* one stable snake_case name per variant */ ]; + pub fn as_str(&self) -> &'static str; + pub fn shape_id(&self) -> ShapeId; +} + +/// An ADDRESSED residual: it records WHERE in varnode identity space the melt failed, so the +/// proposer can emit a proposed `ConventionRow` AT that address and pass N+1 drills with it. +pub struct ResidualFact { + pub shape_id: ShapeId, + pub reason: ResidualReason, + /// The facet coordinate where it occurred. `None` only for `NoFacetCoordinate`. + pub at: Option, + /// The longest convention prefix that DID resolve — where the proposer attaches the next, + /// finer row. `None` = nothing resolved, so the proposal attaches at `Space`. + pub at_prefix: Option, + pub provenance: FactProvenance, +} + +pub struct ResidualLedger { rows: Vec } +impl ResidualLedger { + pub fn push(&mut self, fact: ResidualFact); + pub fn len(&self) -> usize; + pub fn rows(&self) -> &[ResidualFact]; + /// Grouped and counted, sorted by count DESC then `shape_id` ASC — deterministic artifact + /// order. Each group reports one example address so the proposal has a coordinate. + pub fn grouped(&self) -> Vec<(ShapeId, &'static str, usize, Option)>; + /// Grouped by (shape, resolved prefix) — the proposer's actual work queue. + pub fn by_address(&self) -> Vec<(Option, ShapeId, usize)>; + /// The largest group's share. A ledger where one shape absorbs everything is a catch-all + /// wearing a reason's clothes; the pre-registered bar (§10) tests this. + pub fn dominant_share(&self) -> f64; +} +``` + +**Rule that must be written into the module docs:** `residual` is NOT to be driven to 0 by widening +a match arm. It falls only when the **convention** gains a row, and every such widening lands with +its measured before/after counts in the harvest ledger. + +### Tests + +1. **`shape_id_groups_identical_shapes_across_addresses`** — two residuals with the same reason + payload but different `at`/`provenance` share a `shape_id`; a different payload + (`OpcodeNotInConvention{AtomicCAS}` vs `{CallOther}`) differs. Two-sided. +1. **`residuals_carry_the_address_they_failed_at`** — every pushed residual except + `NoFacetCoordinate` has `at.is_some()`; `by_address()` groups two same-shape residuals at + different prefixes into **two** entries, and two at the same prefix into **one** with count 2. +1. **`grouping_is_exact_and_deterministically_ordered`** — counts 3/1/2 group to exactly + `[3, 2, 1]` in that order, twice in a row; the total equals `len()`. +1. **`there_is_no_catch_all_reason`** — `ResidualReason::ALL` has no duplicates, its length equals + the variant count exercised by an exhaustive `match` in the test, and no entry matches + `"other" | "opaque" | "unknown" | "misc"`. Falsified by a future catch-all. + +______________________________________________________________________ + +## 9. `src/behavior.rs` — stage 1, the ORE CARRIER + +Doc comments frame it as the **ore carrier**: a truthful, lossless, zero-copy assembly of upstream +values that **never flattens at intake**. It invents no vocabulary and decides nothing. + +```rust +pub struct FunctionIdentity { pub entry: u64, pub name: Option, pub arch: Option } +// entry ← SSAFunction::entry; name ← SSAFunction::name; arch ← ArchSpec::name (advisory +// provenance, never an address; None ⇒ registers carry `reg:` names). + +/// The ORE CARRIER for one function. NOT a "behavioral contract" — nothing here is refined, +/// classified or proposed. It holds r2sleigh's own `SSAFunction` / `SsaGraph` / +/// `PreparedFunctionFacts` (as one `SsaArtifact`) and NAMES that decomposition through borrowed +/// accessors. Refined concern contracts are a later, measured furnace output; this type must +/// never grow into one. +pub struct FunctionBehavior { identity: FunctionIdentity, artifact: SsaArtifact, + summary: Option } +``` + +**`JUDGMENT` — one `SsaArtifact` instead of three sibling fields.** The plan sketches +`{identity, ssa, graph, facts, summary}`. `SsaArtifact` *is* that triple upstream +(`function.rs:79-96`), its constructor `new` is **private**, and `InterprocFunctionInput.prepared` +requires `&SsaArtifact` — a hand-assembled triple could never produce the `summary` the same struct +declares. Holding the artifact keeps all five concerns verbatim as accessors. Nothing is copied. + +```rust +/// RAW ingest: `SsaArtifact::raw(blocks, arch)` → `SSAFunction::from_blocks_raw` → +/// `SsaGraph::from_function` → `PreparedFunctionFacts::collect`. `None` exactly when upstream +/// is: empty `blocks`, or `CFG::from_blocks` fails. +pub fn from_blocks_raw(blocks: &[R2ILBlock], arch: Option<&ArchSpec>) -> Option; +/// GENERIC ingest (`SsaArtifact::from_blocks`) — applies constructor-time SCCP and can REWRITE +/// ops. Never use it for anything claiming losslessness. +pub fn from_blocks(blocks: &[R2ILBlock], arch: Option<&ArchSpec>) -> Option; +pub fn with_name(self, name: impl Into) -> Self; +pub fn with_summary(self, summary: FunctionSemanticSummary) -> Self; +/// One `InterprocFunctionInput{id, name, prepared: self.artifact()}` → +/// `solve_interproc_summary_set(&[input], arch, Some(id), &BTreeMap::new(), Default::default())` +/// → `set.summaries.remove(&id)`. Single-function scope: `has_unknown_calls` true whenever the +/// function calls anything is CORRECT, not a defect. +pub fn solve_summary(&mut self, id: InterprocFunctionId, arch: Option<&ArchSpec>); +``` + +`from_blocks_raw` body, spelled out: + +```rust +let artifact = SsaArtifact::raw(blocks, arch)?; +let function = artifact.function(); +let identity = FunctionIdentity { entry: function.entry, name: function.name.clone(), + arch: arch.map(|spec| spec.name.clone()) }; +Some(Self { identity, artifact, summary: None }) +``` + +**Borrowed route accessors** (no clones, no wrappers): `control() -> &SSAFunction` (CONTROL — CFG, +`BlockTerminator`, `CFGEdge`, block order), `values() -> &SsaGraph` (VALUES + DEF/USE + +PROVENANCE), `objects() -> &ObjectModel`, `memory() -> &MemorySSAFacts`, +`predicates() -> &PredicateFacts`, `calls() -> &CallSiteFacts`. Plus `identity()`, `ssa()` (alias +of `control`, kept because the plan names the field `ssa`), `graph()`, `facts()`, `artifact()`, +`summary()`, and provenance helpers `op_site(InstId) -> Option<(u64, usize)>`, +`inst_at(u64, usize) -> Option`, `value_var(ValueId) -> Option<&SSAVar>`, +`def_inst(ValueId)`, `use_sites(ValueId) -> &[UseSite]`. + +**Tests.** 1. **`from_blocks_raw_names_the_upstream_decomposition`** — `identity().entry == 0x1000`; +`control().num_blocks() == 2` (exact); `values().insts.len()` equals the source op count; values +non-empty. 2. **`empty_block_list_is_none_not_a_panic`** — `&[]` is `None`, a one-block list is +`Some`. 3. **`op_site_round_trips_through_the_graph_provenance_map`** — for every `Op` inst, +`inst_at(op_site(inst)?) == Some(inst)`; anti-vacuity: checked count `>= 3`. + +______________________________________________________________________ + +## 10. `tests/lossless_fixtures.rs` + +Hand-built `R2ILBlock`s through `FunctionBehavior::from_blocks_raw(&blocks, None)` — **raw**, +because the generic path runs SCCP and a losslessness claim must not go through an optimizer. +Helpers `fn reg(off: u64, size: u32)`, `fn con(v: u64, size: u32)` over `Varnode::{register, constant}`. + +### `fn fixture_function() -> Vec` — 4 blocks, 3-way merge + +**B0 @ `0x1000` size 8** → `ConditionalBranch{true: 0x1018, false: 0x1008}` + +```text +0 IntAdd { dst: reg(0x00,8), a: reg(0x00,8), b: con(0x10,8) } +1 IntSub { dst: reg(0x58,8), a: reg(0x00,8), b: con(4,8) } +2 IntLess { dst: reg(0x40,1), a: reg(0x00,8), b: con(0x100,8) } +3 CBranch { target: con(0x1018,8), cond: reg(0x40,1) } // MUST be the last op +``` + +**B1 @ `0x1008` size 8** → `ConditionalBranch{true: 0x1018, false: 0x1010}` + +```text +0 Load { dst: reg(0x08,8), space: Ram, addr: reg(0x00,8) } +1 Store { space: Ram, addr: reg(0x00,8), val: reg(0x08,8) } +2 Copy { dst: reg(0x00,8), src: reg(0x08,8) } // 2nd def of reg:0 +3 IntSLess { dst: reg(0x44,1), a: reg(0x08,8), b: con(0,8) } +4 CBranch { target: con(0x1018,8), cond: reg(0x44,1) } +``` + +**B2 @ `0x1010` size 8** → `Fallthrough{next: 0x1018}` — **the stressor block** + +```text +0 AtomicCAS { dst: reg(0x00,8), space: Ram, addr: reg(0x20,8), expected: con(0,8), + replacement: con(1,8), ordering: SeqCst } +1 StoreConditional { result: Some(reg(0x28,1)), space: Ram, addr: reg(0x20,8), + val: reg(0x00,8), ordering: Release } +2 StoreConditional { result: None, space: Ram, addr: reg(0x20,8), val: reg(0x00,8), + ordering: Relaxed } // optional output = None +3 LoadGuarded { dst: reg(0x30,8), space: SpaceId::Custom(7), addr: reg(0x20,8), + guard: reg(0x28,1), ordering: Acquire } // CUSTOM SPACE +4 StoreGuarded { space: SpaceId::Custom(7), addr: reg(0x20,8), val: reg(0x30,8), + guard: reg(0x28,1), ordering: AcqRel } +5 CallOther { output: Some(reg(0x38,8)), userop: 42, + inputs: vec![reg(0x30,8), reg(0x28,1), con(1,4), con(2,4)] } // 4 inputs +6 CallOther { output: None, userop: 42, inputs: vec![reg(0x30,8)] } +7 Insert { dst: reg(0x48,8), src: reg(0x30,8), value: reg(0x38,8), position: con(3,4) } +8 Load { dst: reg(0x50,8), space: Ram, addr: Varnode::ram(0x1234_5678_9ABC_DEF0, 8) } +9 Fence { ordering: MemoryOrdering::Unknown } +10 Copy { dst: reg(0x00,8), src: reg(0x48,8) } // 3rd def of reg:0 +``` + +B2 also carries **op metadata** at index 0 — `set_op_metadata(0, OpMetadata{memory_ordering: Some(SeqCst), atomic_kind: Some(AtomicKind::CompareExchange), ..Default::default()})` — and +**varnode metadata** on op 3's `dst` (`ScalarKind::UnsignedInt` + `PointerHint::PointerLike`). + +**B3 @ `0x1018` size 8** — merge, 3 predecessors, terminator `Return` (the reverse scan hits +`Return` before `Call`; correct and intended) + +```text +0 Call { target: con(0x2000,8) } +1 Return { target: reg(0x00,8) } // uses reg:0 → the merged phi is live +``` + +### Tests + +1. **`every_mandated_op_shape_survives_ingest_as_a_typed_ssa_op`** — `OpTag::from_op` over + `values().insts`, **set equality** with `{Phi, Copy, Load, Store, Fence, StoreConditional, AtomicCAS, LoadGuarded, StoreGuarded, IntAdd, IntSub, IntLess, IntSLess, CBranch, Call, Return, CallOther, Insert}` — a missing op fails AND an unexpected extra fails. +1. **`op_order_within_a_block_is_preserved_in_ordinal_order`** — for B2 (no `Multiequal`, so the + 1:1 rename rule holds): `ordinal` strictly increasing and the `OpTag` sequence equals the R2IL + source sequence element by element. Anti-vacuity: length `== 11`. +1. **`facts_are_populated_and_counted_against_upstreams_own_classification`** — + `calls().by_id.len() == 1`, `direct_target == Some(0x2000)`; `predicates().predicates.len() == 2` + and the comparison-kind set equals `{Less, SignedLess}`; memory counts equal an expectation + computed **from upstream's own rule** (uses ← `{Load,LoadLinked,LoadGuarded,AtomicCAS, StoreConditional}` + calls; defs ← `{Store,StoreGuarded,StoreConditional,AtomicCAS}` + calls); + `objects().global_objects` contains `address == 0x1234_5678_9ABC_DEF0`. Anti-vacuity: + `expected_uses >= 5`. Do **not** assert the `Global.space` string (Debug-formatted upstream) — + record the observed value in a doc comment as a stage-4 measurement. +1. **`phi_fan_in_equals_the_predecessor_count_at_the_three_way_merge`** — + `control().predecessors(0x1018).len() == 3` (exact); ≥1 phi there; **every** phi has + `inputs.len() == 3` and `predecessors.len() == 3`. +1. **`custom_space_and_every_memory_ordering_survive_into_typed_ssa_ops`** — `LoadGuarded` / + `StoreGuarded` share a non-empty space string; their `ordering` is `Acquire`/`AcqRel` (typed); + the set of `MemoryOrdering` values across all SSA ops equals + `{Relaxed, Acquire, Release, AcqRel, SeqCst, Unknown}` — exact set equality. +1. **`op_metadata_rejoins_by_op_site_even_though_ssa_does_not_carry_it`** — `inst_at(0x1010, 0)` is + `Some(i)`; `op_site(i) == Some((0x1010, 0))`; that inst is the `AtomicCAS`; the source block's + `op_metadata(0)` equals the constructed value. +1. **`varnode_metadata_is_advisory_and_does_not_change_ingest`** — fixture with and without the + `with_meta` → identical `(name, version, size)` value sequences and `insts.len()`. +1. **`multiequal_ingest_becomes_a_phi_zipped_to_the_predecessor_count`** — own 2-block fixture: + B0 `@0x2000 sz 4` = `Branch{target: con(0x2004,8)}`; B1 `@0x2004 sz 4` = `Multiequal{dst: reg(0x00,8), inputs: vec![reg(0x08,8), reg(0x10,8), reg(0x18,8)]}` then `Return{target: reg(0x00,8)}`. Assert phi count `== 1`; `inputs.len() == control().predecessors(0x2004).len()` + (**state the rule, not the number**); no `Op`-payload inst in that block is phi-shaped. +1. **`sixty_four_bit_offsets_are_not_truncated_on_ingest`** — the global object address is exactly + `0x1234_5678_9ABC_DEF0`, **and** no `SSAVar::name` equals `"ram:9abcdef0"` (the name a 32-bit + truncation would produce). Two-sided anti-truncation. +1. **`stressors_land_in_slag_under_pass_one_and_are_named_and_addressed`** — ⭐ **the proof the + loop works.** `furnace::smelt(&behavior, &blocks, &R2ilConvention::minimal_pass_one())`. Assert + `dropped == 0`, `is_conserved()`, `residual > 0`; the ledger's reason set **contains** + `OpcodeNotInConvention{AtomicCAS}`, `{CallOther}`, `{StoreGuarded}`, `{Insert}`, + `CustomSpaceNotInConvention{raw: 7}` and `VariadicArity{arity: 4}` — each asserted individually + so no single absorbing group can satisfy the test; and **every** such residual carries + `at.is_some()` (the addressed-slag rule). +1. **`widening_the_convention_moves_a_stressor_out_of_slag`** — the two-sided partner: add + `OpTag::AtomicCAS`; its residual rows disappear, `classified` rises by exactly that count and + `residual` falls by the same. Proves the ledger tracks the convention, not noise. +1. **`a_bootstrapped_convention_resolves_register_operands_that_pass_one_cannot`** — smelt once + with `minimal_pass_one()` (no rows ⇒ register operands are `NoConventionRowAtAddress`) and once + with `R2ilConvention::from_arch(&spec, seven)` over an `ArchSpec` naming `reg 0x00/8` and + `reg 0x08/8`: those operand rows move from residual to classified, and an *unnamed* offset + (`0x58`) still resolves — to the coarse `Space` row, at `depth() == 1`. Ties §5 to the melt. + +______________________________________________________________________ + +## 11. `examples/harvest_r2il.rs` (feature `lift`) — THE artifact set + +Mirrors `ruff_cpp_spo/examples/harvest_network.rs` (env-driven output path, stderr progress, file +emission). Writes into **`/home/user/ruff/.claude/harvest/r2il/`** (create it; it does not exist +yet), overridable via `R2IL_HARVEST_OUT`. + +**Format (`JUDGMENT`):** deterministic line-oriented **TSV** with `#schema` and `#version` header +lines for the row files, and hand-written **nested TOML** for the convention — **not** ndjson/serde. +Reasons: no serde dep; the plan forbids `R2IL→JSON→Ruff` and an ndjson ore invites exactly that +confusion; TSV/TOML diff cleanly. State in every header: **these artifacts are evidence, never a +re-ingest path — nothing in ruff parses them back.** + +| file | content | +| ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- | +| `r2il-pass1.ore.tsv` | one row per `FlatFact`, columns per `#schema`, in `smelt` order, `at` rendered as 32 hex chars | +| `r2il-pass1-census.md` | `by_fact_kind` + `by_opcode` tables (BTreeMap order ⇒ byte-stable) | +| `r2il-pass1-slag.tsv` | `ResidualLedger::grouped()` — `shape_id`, `reason`, `count`, example facet — **plus** a `by_address` section (the proposer's work queue) | +| `r2il-convention.toml` | `R2ilConvention::to_toml()` — the bootstrapped tree, every row `unmeasured` | +| `PROVENANCE.md` | see below | +| `TRIAGE-RESULT.md` | the pre-registered bar (stated first) + the measured outcome | + +`PROVENANCE.md` pins, so the next session does not re-derive: each corpus path with byte length and +an **FNV-1a 64** of its bytes (labelled FNV, *not* called a sha — no hashing dep); `r2sleigh` commit +**`60942f6`**; arch `x86-64`; `sleigh-config` version + feature; the convention used +(`minimal_pass_one` and/or `from_arch`); env caps in force; and the **EXACT** invocation: + +```sh +cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example harvest_r2il +``` + +### The PRE-REGISTERED bar — stated in `TRIAGE-RESULT.md` BEFORE the first run + +MedCare-rs's `TRIAGE-RESULT.md` states "recoverable ≥85% PASS, \<50% KILL" before running. The R2IL +analog cannot be a rate over *all* facts — pass 1 classifies only seven opcodes by design, so a low +overall rate is the intended result, not a failure. Three bars, each falsifiable: + +- **B1 — conservation (absolute).** `dropped == 0` and `harvested == classified + residual`. Any + violation **KILLS** the pass: the enumerator, not the corpus, is wrong. Not a percentage. +- **B2 — coverage of the declared seven.** Of ore facts whose parent opcode is one of + `{Copy, IntAdd, Load, Store, CBranch, Call, Return}`, **≥99% classify → PASS; \<90% → KILL.** + Those seven are fully specified here; a shortfall means the classifier is wrong, not that the + corpus is hard. The 90–99% band is INVESTIGATE (expected causes: operand rows with no convention + row at their address, `CallSite` rows with no `direct_target` — both legitimate slag under a + parent that classified). +- **B3 — the slag is named and addressed, not lumped.** `residual > 0`, distinct `shape_id` count + **≥ 5**, `dominant_share() < 0.60`, and **every** residual except `NoFacetCoordinate` carries + `at.is_some()`. A ledger where one shape absorbs everything is a catch-all wearing a reason's + clothes; an unaddressed ledger cannot feed the proposer. `residual == 0` is a **KILL** too — it + means someone widened the ladder. + +Also **pre-register a prediction that is NOT a bar** (so it can be wrong without moving a +goalpost): on an x86-64 corpus `Copy/IntAdd/Load/Store` dominate, so pass 1 is expected to classify +roughly **60–80%** of all `Op` facts. Record the measured figure either way. + +Caps echoed in the header: `R2IL_HARVEST_MAX_FUNCS` (default 200), +`R2IL_HARVEST_MAX_SECTION_BYTES` (default 262144). + +______________________________________________________________________ + +## 12. `examples/r2il_corpus_profile.rs` (feature `lift`) — §12 profile + +Plain text to stdout. No serde, no file writes, no `unwrap`/`panic` on corpus input. + +**Corpus**, each skipped with a printed note if absent: `$R2IL_CORPUS` (colon-separated) or CLI args +→ `concat!(env!("CARGO_MANIFEST_DIR"), "/../../../r2sleigh/tests/e2e/stress_test")` and `…_opt` +(verified ELF64 x86-64, **not stripped** — symtab present) → `/bin/ls`, `/usr/bin/env` (**stripped** +— op-level only). Nothing is copied into ruff. + +**`mod elf` — minimal ELF64 LE reader (~80 lines, no deps).** Every read bounds-checked; anything +malformed returns `None` and the binary is skipped with a note. Header: magic `\x7FELF` @0, +`EI_CLASS==2` @4, `EI_DATA==1` @5, `e_machine u16 @18 == 62`, `e_shoff u64 @40`, +`e_shentsize u16 @58`, `e_shnum u16 @60`, `e_shstrndx u16 @62`. Section header (64 B): +`sh_name u32 @0`, `sh_type u32 @4`, `sh_flags u64 @8`, `sh_addr u64 @16`, `sh_offset u64 @24`, +`sh_size u64 @32`, `sh_link u32 @40`, `sh_entsize u64 @56`; names from section `e_shstrndx` at +`sh_offset + sh_name`, NUL-terminated; executable = `sh_flags & 0x4 != 0 && sh_type != 8`. +Symbols: section `sh_type == 2 (SHT_SYMTAB)`, strtab = section `sh_link`; `Elf64_Sym` (24 B) +`st_name u32 @0`, `st_info u8 @4`, `st_value u64 @8`, `st_size u64 @16`; functions = +`st_info & 0xF == 2 (STT_FUNC) && st_size > 0` inside an exec section. + +**Setup:** `build_arch_spec(SLA_X86_64, PSPEC_X86_64, "x86-64")`, `Disassembler::from_sla(…)`, +`set_userop_map(userop_map_for_arch("x86-64"))`. libsla needs **≥16 bytes**, so every lift gets a +16-byte zero-padded window. + +**Pass 1 — `== MEASURED EXACT (op level) ==`.** Linear sweep: `disasm.lift(&window, addr)`; on `Ok` +advance `block.size.max(1)`, on `Err` count `undecodable_instructions` and advance 1 byte. Metrics +from `R2ILOp`'s own accessors (never `Display`): opcode frequency; ops per native instruction +(mean/min/median/max); input-arity histogram; output-count histogram; memory-op %, control-op %, +atomic %; `CallOther` arity distribution; **% fitting `dst+src0+src1` inline** +(`output_count <= 1 && inputs().len() <= 2`) and **% needing Vec routing** (complement). + +**Pass 2 — `== HEURISTIC-DERIVED (function / CFG level) ==`**, symtab binaries only. Per `STT_FUNC` +symbol: sweep `[st_value, st_value+st_size)`; leaders = `{st_value}` ∪ intra-function const branch +targets ∪ `{addr after any control-flow instruction}`; basic blocks = maximal ranges between +leaders; one `disasm.lift_block(&padded, bb_addr, bb_len)` per block → +`FunctionBehavior::from_blocks_raw(&bbs, Some(&spec))`. Metrics: blocks/fn, ops/block, phi fan-in +distribution, values/fn, call sites/fn, predicates/fn, plus `vocab::VocabHarvest::stats()`, a +`facet::project` sweep (facet count, `FacetOverflow` count by variant), and the `furnace::smelt` +conservation line — feeding open items **O1** and **O3**. + +**Labelling rule (non-negotiable in the output):** `STT_FUNC` boundaries are **exact**; the leader +set is an approximation (indirect branches / jump tables unresolved), so every CFG-derived row +prints under the heuristic heading and the header states `function_boundaries: symtab (exact) | leaders: intra-function const targets (approximate)`. A stripped binary prints +`function-level stats: skipped (no symtab)` — never a silent omission. Caps +`R2IL_PROFILE_MAX_FUNCS` (200) / `R2IL_PROFILE_MAX_SECTION_BYTES` (262144) are echoed. + +______________________________________________________________________ + +## 13. `src/vocab.rs` — stage 4 (DTO / codebook factoring) + +Feeds lance-graph's `ogar_codebook`; we **read** that codebook and never construct a parallel one. +PR 1 produces the local interning table and the measurement; PR 2/3 wire the real mint. + +```rust +pub struct VocabId(pub u32); +/// Deterministic, ORDER-INDEPENDENT interning: built from a BTreeSet, ids assigned in sorted +/// order, so the table is a pure function of the name SET. `JUDGMENT`: there is deliberately NO +/// public incremental `intern(&mut self)` in PR 1 — two build paths would make ids depend on +/// which one ran. +pub struct VocabTable { by_name: BTreeMap, names: Vec } +// from_names / id_of / name_of / len / is_empty / iter / name_bytes + +pub struct VocabHarvest { + pub ssa_names: VocabTable, // SSAVar::name over SsaGraph::values + pub op_spaces: VocabTable, // SSAOp `space: String` (Debug-formatted upstream) + pub object_spaces: VocabTable, // ObjectKind::Global{space} + pub userops: BTreeMap, // numeric already — COUNTED, not interned + pub custom_spaces_from_strings: BTreeSet, +} +impl VocabHarvest { pub fn from_behavior(&FunctionBehavior) -> Self; pub fn stats(&self) -> VocabStats; } + +/// The TYPED oracle for custom space ids: walks `Varnode::space` and each op's `space: SpaceId` +/// on the R2IL side. No parsing, no strings. +pub fn custom_space_ids_from_blocks(blocks: &[R2ILBlock]) -> BTreeSet; +/// Recover a custom space id from an upstream space STRING (`"Custom(7)"` from the function +/// rename, `"space_7"` from `block.rs::space_name`). +/// ⚠ MEASUREMENT ONLY — never a data path. The plan forbids parsing display strings for +/// behavioral truth; this exists solely so `VocabStats` can put a NUMBER on the loss by +/// comparing against `custom_space_ids_from_blocks`. +pub fn parse_custom_space_id(space: &str) -> Option; +pub fn custom_space_id_from_var_name(name: &str) -> Option; + +pub struct VocabStats { pub unique_ssa_names: usize, pub unique_op_spaces: usize, + pub unique_object_spaces: usize, pub unique_userops: usize, pub userop_mentions: usize, + pub unique_custom_spaces_from_strings: usize, pub total_values: usize, + pub ssa_name_bytes: usize, pub interned_id_bytes: usize } +``` + +**Tests.** 1. **`vocab_table_is_order_independent`** — the same 5 names in two orders (one with +duplicates) build equal tables; anti-vacuity: `len() == 5` and at least two ids differ. +2\. **`harvest_counts_every_userop_mention_but_interns_names_once`** — two `CallOther`s sharing +`userop: 7` plus one with `9` → `unique_userops == 2`, `userop_mentions == 3` (both exact), reused +name interned once. 3. *(recommended)* **`typed_custom_space_ids_are_the_oracle_for_the_string_set`** +— `custom_space_ids_from_blocks == {7}`; the string-recovered set equals it **on this fixture**, +with a doc comment recording that the equality is a measurement, not a guarantee. + +______________________________________________________________________ + +## 14. Worker split — DISJOINT files, Sonnet fleet + +| worker | owns (exclusively) | spec section | +| ------- | --------------------------------------------- | ---------------------------------- | +| **W1** | `Cargo.toml`, `src/lib.rs`, `src/behavior.rs` | §3, §9 | +| **W2** | `src/facet.rs` | §4 | +| **W3** | `src/convention.rs` | §5 (consumes W2's types as spec'd) | +| **W4** | `src/ore.rs` | §6 | +| **W5** | `src/furnace.rs` | §7 | +| **W6** | `src/slag.rs` | §8 | +| **W7** | `src/vocab.rs` | §13 | +| **W8** | `tests/lossless_fixtures.rs` | §10 | +| **W9** | `examples/harvest_r2il.rs` | §11 | +| **W10** | `examples/r2il_corpus_profile.rs` | §12 | + +**W1 writes every module declaration up front** — `src/lib.rs` contains `pub mod behavior; pub mod convention; pub mod facet; pub mod furnace; pub mod ore; pub mod slag; pub mod vocab;` from its +first commit, so no later worker ever edits a shared file. W2–W10 code against the APIs **as +written in this spec**, never against another worker's output; if a signature here is wrong, STOP +and report — do not improvise a different one. Dependency direction: `facet → convention → {ore, furnace}`, `furnace → slag`. All type shapes are spec'd here, so no worker blocks on another. + +**Verbatim guardrail block — paste into EVERY worker brief:** + +> Do NOT run cargo (build/check/test/clippy/fmt) — the orchestrator compiles centrally in the +> shared target/. Do NOT run git. Do NOT create worktrees. Do NOT write to any .claude/ board file. +> Edit ONLY your assigned files. Do not claim it compiles or that tests pass — you did not run it. + +**Orchestrator (Opus), after the fleet lands:** + +1. Add the exclusion to `/home/user/ruff/Cargo.toml`: + + ```toml + [workspace] + members = ["crates/*"] + exclude = ["crates/ruff_r2il"] + resolver = "2" + ``` + + (`exclude` wins over the `crates/*` glob; the root `Cargo.lock` must stay untouched.) + +1. Run every gate in §15 centrally, once. Fix cross-file fallout itself — do not re-fan-out. + +1. Run the harvest example; commit the artifact set under `.claude/harvest/r2il/`, with + `TRIAGE-RESULT.md`'s bar section written **before** the run and the measured section after. + +1. Commit on `claude/ruff-r2il-lancegraph-3tdt8d` with the board update in the **same** commit: + plan open items **O1** and **O3** get measured values; the four honesty notes (§2) are recorded. + Push and open the PR. + +______________________________________________________________________ + +## 15. Gates and definition of done + +```sh +cargo fmt --manifest-path crates/ruff_r2il/Cargo.toml --check +cargo clippy --manifest-path crates/ruff_r2il/Cargo.toml --all-targets -- -D warnings +cargo test --manifest-path crates/ruff_r2il/Cargo.toml +cargo clippy --manifest-path crates/ruff_r2il/Cargo.toml --features lift --all-targets -- -D warnings +cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example harvest_r2il +cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example r2il_corpus_profile + +# workspace-isolation proof — the exclusion must be real, not asserted +cargo check -p ruff_graph # ruff workspace still builds +cargo metadata --no-deps --format-version 1 | grep -c ruff_r2il # must print 0 +git status --porcelain Cargo.lock # must be EMPTY +``` + +`cargo check` is not run separately — clippy compiles. The first `--features lift` build pays the +libsla native compile (~43 s measured upstream); if the sandbox is offline add `--offline`. + +### Definition of done for PR 1 + +1. `crates/ruff_r2il/` matches the §3 inventory; root `Cargo.toml` carries `exclude`; root + `Cargo.lock` unchanged; `crates/ruff_r2il/Cargo.lock` committed. +1. All gates green, including the three isolation checks. +1. Test names match §4–§10 and §13. For every guarded assertion (the stressor-slag proof, the + convention-widening partner, the register bootstrap + fall-through pair, the flat-row size + guard, facet overflow, the metadata exclusion, the phi zip, the anti-vacuity counts) the + orchestrator ran the **disable-the-guard** check and recorded that it went red. +1. `.claude/harvest/r2il/` contains all six artifacts; `TRIAGE-RESULT.md` states the §11 bar + **before** the measured section; the conservation line `harvested N / classified X / residual Y / dropped 0` is present with `dropped == 0`. +1. **B1 holds absolutely.** B2 and B3 are reported PASS / INVESTIGATE / KILL by their own stated + thresholds — a KILL is recorded honestly and blocks PR 2, never argued away. +1. The residual ledger contains **no** catch-all row, **every** residual (bar `NoFacetCoordinate`) + carries its facet address, and `residual` was reduced only by a recorded convention change with + before/after counts — never by widening a match arm. +1. The doc framing is in the code, not only here: `ore.rs`'s header carries the operator one-liner + ("Varnode in the first stage is pointer chasing stacked god objects — hence the ore furnace + slag") plus **typed ≠ refined**; `furnace.rs` states the flat-rows constraint; `behavior.rs` + frames `FunctionBehavior` as the **ore carrier**, never a contract; `facet.rs` states both roles + (config key now, V3 persistence not committed); the four §2 honesty notes appear as doc comments. +1. No new type duplicates an upstream one; no `serde_json`; no `Display`/`Debug` parsing on any data + path (`vocab::parse_custom_space_id` is measurement-only and says so in its doc comment). diff --git a/.claude/plans/r2il-behavioral-ir-v1.md b/.claude/plans/r2il-behavioral-ir-v1.md new file mode 100644 index 0000000000000..5c0c562f7e902 --- /dev/null +++ b/.claude/plans/r2il-behavioral-ir-v1.md @@ -0,0 +1,407 @@ +# R2IL Behavioral IR — v1 (Phase Zero audit + wave plan) + +> **Status:** ACTIVE. Branch `claude/ruff-r2il-lancegraph-3tdt8d` (all three repos). +> **Governing rule:** Preserve semantics once. Decompose by concern. Handle +> cardinality separately. Project many ways. Never stringify typed IR merely so +> another component can rediscover the types. +> **Board pattern:** this file is the plan-of-record (lance-graph +> `.claude/plans/` convention). The orchestrating main thread is the SOLE +> writer of this file and any shared board file; workers leave records only in +> their own tag files. Board entries land in the same commit as the deliverable. + +## Phase Zero audit — findings (2026-08-17, three parallel audits + probes) + +### Ruff + +1. **Frontend inventory.** Python (`ruff_python_ast/parser/semantic/codegen` — + native, richest; plus `ruff_python_spo` Odoo harvest, `ruff_sqlalchemy_spo`, + `ruff_python_dto_check` route contracts), C# (`ruff_csharp_spo`, out-of-process + Roslyn → NDJSON contract), C++ (`ruff_cpp_spo` libclang walker + + `ruff_cpp_codegen`), Ruby (`ruff_ruby_spo`). All `*_spo` frontends fill ONE + shared typed IR — `ruff_spo_triplet::ir::ModelGraph` (schema-locked top level, + per-language sibling `Vec`s on `Model`) — and `expand()` + (`ruff_spo_triplet/src/expand.rs:109`) is the SINGLE deliberate collapse + point to `Vec`. +1. **RouteContract** (`ruff_python_dto_check/src/contract.rs:67-108`): concern + groups `id / inputs / data / output / guards / provenance`, EMERGENT + `HandlerKind` classifier, 5 cross-layer lints failing loud on dropped facts. + Config-driven, no framework idioms hardcoded. This is the concern-decomposition + philosophy to apply to R2IL — the philosophy, not the type. +1. **Overflow.** `ruff_spo_address::mint_factored` (`lib.rs:447`): base-255 + positional cascade (`b255_width`), `MAX_SIBLINGS_PER_TIER = 255` explicitly a + DESIGN-SMELL lint (`soc.rs:7-8`), never a storage ceiling. Its 16-byte `Facet` + is layout-identical to `lance_graph_contract::facet::FacetCascade`. + **Reuse the principle** (overflow = evidence a route needs factoring), not a + copied constant. +1. **NDJSON** is canonical ONLY for the SPO layer (`Triple` mirrors lance-graph's + `OntologyTriple` field-for-field; `deny_unknown_fields`; closed `Predicate` + enum ~34 variants — "frontends MUST NOT emit raw predicate strings"). +1. **Native-dep wiring precedent:** `ruff_cpp_spo` — non-default feature + (`libclang = ["dep:clang"]`), `required-features` on examples, "workspace + builds with zero system deps by default." + +### r2sleigh (checkout `/home/user/r2sleigh` @ 60942f6) + +1. **`r2il` is cleanly standalone**: pure Rust, deps serde/postcard/thiserror, + compiles in 8.25 s, everything re-exported at crate root. `R2ILOp` = one flat + enum, 60+ variants, NAMED `Varnode` fields (`IntAdd{dst,a,b}`); variadic only + where the semantics are variadic (`CallOther{output:Option, userop:u32, inputs:Vec}`, `Multiequal{dst, inputs:Vec}`); atomics carry + `ordering: MemoryOrdering`. `Varnode{space, offset:u64, size:u32, meta:Option}` — meta excluded from Eq/Hash (advisory). + `SpaceId{Ram,Register,Unique,Const,Custom(u32)}`. + +1. **`r2ssa` is directly consumable**: builds in 43 s INCLUDING the transitive + libsla-sys native compile (verified in-env, exit 0). Its function-level API + (`SSAFunction::from_blocks(&[R2ILBlock], Option<&ArchSpec>)`) needs NO + Disassembler; `Disassembler` appears ONLY in `block.rs` (legacy single-block + `to_ssa()` register-naming convenience) — the sole reason for the hard + `r2sleigh-lift` dep. Feature-gating it upstream is a genuinely generic + improvement (any SSA-only consumer sheds the native dep) but is NOT required: + **direct consumption already works** (§17 success outcome). + +1. **r2sleigh already grew the concern decomposition** (`r2ssa/src/semantic.rs`, + `graph.rs`, `interproc.rs`). The correspondence, VERIFIED against source: + + | r2sleigh type | route | + | ---------------------------------------------------------------------------------- | ----------------------------- | + | `SSAFunction` + `BlockTerminator` + `CFGEdge` | CONTROL | + | `SsaGraph` (`ValueId/InstId/BlockId(u32)`, `def_of`, `uses_of`, `op_inst_by_site`) | VALUES + DEF/USE + PROVENANCE | + | `ObjectModel` (`StackSlot/FrameObject/Global/HeapAlloc/EscapedUnknown`) | OBJECTS / ALIAS | + | `MemorySSAFacts` (`MemoryVersion`, uses/defs by inst, memory phis) | MEMORY | + | `PredicateFacts` (`CompareProvenance`, `BlockAssumption`, switches) | PREDICATES / GUARDS | + | `CallSiteFacts` (`target`, `direct_target`, `CallMemoryEffect`) | CALLS | + | `FunctionSemanticSummary` (interproc arg/memory/return effects) | SUMMARY (derived) | + + `SsaGraph` is ALREADY SoA-shaped (parallel `Vec`s indexed by u32 IDs) and + ALREADY uses the descriptor/routed operand encoding + (`GraphInst{inputs:Vec, output:Option}`). + **Consequence: Ruff does not invent a behavioral ontology. It names + r2sleigh's own decomposition as routes.** + +1. **Known string-forest**: `SSAVar{name:String, version, size}` (documented + upstream limitation) and `ObjectKind::Global{space:String,..}` — this is the + DTO/codebook plane's work, Ruff-side. + +1. **Corpus mechanics**: no `.sla` in-repo; specs come from the registry crate + `sleigh-config` (features per arch, e2e uses `x86`). Compiled test binaries + exist (`tests/e2e/stress_test`, `stress_test_opt`). `/home/user/ghidra` has + `.slaspec` sources if ever needed. + +### lance-graph V3 + +1. **Physical ABI**: `NodeRow` 512 B = `NodeGuid`(16) + `EdgeBlock`(16) + + value(480), const-asserted; `GUIDS_PER_NODE = 32` — "32 × 16-byte GUID + slots", Tetris-across-slots doctrine (`canonical_node.rs:793-810`). + `FacetCascade{facet_classid:u32, tiers:[FacetTier{lo:u8,hi:u8};6]}` = 16 B; + `CascadeShape::{G6D2,G4D3,G3D4}`. Grammar dispatch = classid → ClassView + (slot purity: labels/positions NEVER from payload). `ENVELOPE_LAYOUT_VERSION=2`. +1. **The precedent**: `lance_graph_contract::network` sinks Tesseract's 27-class + C++ `Network` hierarchy onto ONE FacetCascade per node — + `facet_classid = compose_classid(NETWORK_LAYER=0x0804, ntype as u16)` + (container concept on canon-high, subclass ordinal on custom-low, "container + kinds, not content" mint discipline), `G6D2` payload, names/weight-blobs + OUT-OF-LINE (Lance table keyed by classid+identity). Zero physical changes. +1. **Overflow mechanisms that exist**: (a) Tetris-across-slots in-row; (b) + out-of-line Lance-table escape (proven: network weights, 4M-vertex FMA + mesh); (c) designed-not-built stream-window escape. A CFG-shaped + "many small typed rows keyed to one parent" consumer is new WIRING over + proven pattern (b), not a new primitive. +1. **Codebook plane**: `ogar_codebook::{compose_classid, canonical_concept_id}` + — name→u16 vocabulary registry, compile-time-drift-checked + (`network_layer_const_matches_codebook` pattern). classid capacity nowhere + near exhausted. +1. **Verdict: V3 is sufficient. No V4. The hypothesis stands un-falsified** + pending the corpus measurements (which gate only the per-route LAYOUT + choices, not the physical grammar). +1. **Honesty note**: network.rs's "byte-parity vs real Tesseract" is + designed-for (oracle named) but in-repo tests are synthetic round-trips + against pre-registered values. R2IL routes should meet the same standard + they claim — no overclaiming. + +## PIVOT (operator, 2026-08-18): R2IL is an INTAKE ARM, not a bypass + +Typed input does not mean bypassing the furnace. R2IL enters through the same +intake-arm → ore → furnace → slag → proposer discipline every other foreign +representation uses. Slag is evidence, not failure; never hidden in `Other`. + +### Delta audit — where the machinery actually lives (measured 2026-08-18) + +**`lance-graph-arm-discovery` is a FALSE FRIEND — "ARM" = Association Rule +Mining** (Aerial+ transcode, arXiv 2504.19354), not "intake arm". Audited in +full: it provides NONE of foreign-shape discovery / arm generation / DTO +generation / codebook generation / contract generation / target codegen / +residual clustering / schema proposal. It consumes a DECLARED +`FeatureSpec` + discretised `Dataset` (panics on arity mismatch) and emits +`CandidateRule` → `TruthU8` → SPO ndjson. Workspace-EXCLUDED, dormant, blocked +on D-ARM-7 (Jirak floor) and D-ARM-SYN-1 (`ruff_spo_triplet::from_ndjson` +REJECTS its `implies` predicate). Board banks the finding +"arm-discovery-is-a-proposer-not-the-SPO-AST" +(`PR_ARC_INVENTORY.md:1915`) and explicitly rules out clustering reuse +(`:1908`). **Verdict: do NOT reuse for intake. Possible far-downstream +analytics reuse only** (mining correlations ACROSS already-ingested R2IL +facts), itself blocked. + +**The real furnace machinery is in `ruff_spo_triplet`** — three shipped, +language-agnostic, data-as-config stages, each with an explicit residual +ledger: + +| module | stage | slag mechanism | +| ------------------- | --------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | +| `concept_split.rs` | Phase 1 re-key (fix CONCEPT=CLASS) | `ResidualMethod` rows + reason; "the residual is not waste: it is the empirical boundary of the current convention" | +| `surface_schema.rs` | Phase 3 config-as-schema (pull config-wearing-method-clothes OUT before action lifting) | concept/facet residue deferred to concept_split | +| `recipe.rs` | recipe centroid classifier over fact-sets ONLY (no language tokens) | `Compensate`/`WriteRaise` = essential residue, hand-ported | + +`concept_split` ships **zero domain vocabulary** — the `ConceptConvention` +(verbs, scope qualifiers, aliases) is caller-supplied. This is the seam R2IL's +architecture-specific semantics enter through: an `R2ilConvention` (userop +table, custom-space table, arch profile) is the exact analog. `recipe.rs` +states the arm contract outright: "a frontend adds the arm purely by +populating those `Vec`s from its own AST; this module runs unchanged." + +### The transcode evidence — `.claude/harvest/` is the arm's artifact contract + +Two worked, measured transcodes establish what an intake arm MUST produce. +Not prose — committed data. + +**MedCare-rs** (`AdaWorldAPI/MedCare` C#/WinForms → Rust): + +- `medcare-2.0-spo-triples.ndjson.gz` — 108,548 triples, per-predicate census + in the README, provenance pinned (corpus `429b577`, harvester `562964f`, + EXACT invocation flags saved so the next session doesn't re-derive them). +- `medcare-soc-split.config.json` — the SoC proposal ledger: per-class + `{fingerprint: fnv1a:…, members, data, funcs, verdict, branches[]}`; + `MainForm` = 330 members / 210 data / 120 funcs → `duplication_and_conflation`, + split into named branches. **This is route decomposition as committed data.** +- `TRIAGE-RESULT.md` — the furnace measurement with a **PRE-REGISTERED bar** + ("recoverable ≥85% PASS, \<50% KILL" stated BEFORE the run), full histogram, + the 5-method essential residue named individually, and an explicit + "do not over-claim" caveat section. +- `compiled/medcare-actiondefs.json.gz`, `generated/do_adapters.rs`. + +**openproject-nexgen-rs** (Rails → Rust): + +- `2026-07-06-transpile-ledger.md` — reproducible chain + (`ruff_ruby_spo::extract_app_with_schema → ogar_from_ruff::mint::compile_graph_ruby → ogar_render_askama::render_class_with_methods → committed generated Rust`), + one repro command, counts table (945 extracted / 18 curated / **16 emitted, 2 + dropped WITH NAMED REASONS**), recipe census (98.4% recoverable, + essential_residue = 1), classid scheme, DoD checklist with ⏳ items honest. +- `orm-ar-backprojection.toml` — data-not-code resolver config with + `validation_states = [unmeasured|confirmed|corrected|retired]`, ALL rules + starting `unmeasured` and a meta key `measure_dont_claim`. +- `c4-rename-seed.ndjson` — vocabulary drift table with an explicit + `identity-default` catch-all ROW (a declared rule, not a silent fallback). + +**Consequence for PR 1 — the arm's deliverable is an artifact set, not a +struct.** `ruff_r2il` must emit, into `.claude/harvest/`: + +1. an **ore file** (typed, lossless, deterministic) + provenance block + (corpus, r2sleigh commit, arch, EXACT invocation); +1. a **census** (per-opcode/per-fact counts — the `108,548 triples by predicate` analog); +1. a **residual ledger** (slag rows with a deterministic shape id + reason, + grouped and counted — the `ResidualMethod` analog); +1. a **conservation line**: `harvested N / classified X / residual Y / dropped 0` — dropped MUST be 0, Y MUST NOT be driven to 0 by a catch-all; +1. a **pre-registered bar** stated BEFORE the first run. + +## OPERATOR RULING (2026-08-18): the V3-shaped varnode is the DRILL KEY + +The prior intakes' method — data-as-config where drilling down produces the +nested config that in turn gives the drilling its structure ("what to bolt +where") — had ad-hoc nesting keys (class names, method-name conventions). +For R2IL the **VarnodeFacet provides the shape for the drilling**: + +- `VarnodeFacet` = the 16-byte V3-shaped identity + `classid(space-class) | offset_lo | offset_hi | size` — prefix-routable by + construction. +- `R2ilConvention` is therefore NOT a flat table: it is a + **longest-prefix-wins config tree over varnode identity space** (space + class → offset(-range) → size), the same resolution rule as OGAR's + codebook scoping ("longest-prefix wins — one rule, every level"). Rendered + as nested TOML/JSON like the harvest precedents. +- **Slag rows are addressed**: each residual carries the facet coordinate + where it occurred; the proposer emits proposed config rows AT those + addresses; pass N+1 drills with them. The config accumulates as a radix + tree, self-scaffolding. +- **Bootstrap is read, not typed**: `ArchSpec`'s register table + (`add_register(name, offset, size)`) already IS facet-address → name rows — + the Register-space branch of the convention is populated from upstream data + (data-as-config doctrine: data that exists must be READ). +- Scope guard: this promotes the facet as the ADDRESS/CONFIG-KEY shape now; + actual V3 SoA persistence stays stage 5 / PR 2. Same 16 bytes, two roles, + no storage-layout commitment yet. The `SpaceId::Custom(u32)` lossless-fit + falsifier becomes MORE central (config keys must be lossless). + +**Why the furnace, in one line (operator, 2026-08-18): "Varnode in the first +stage is pointer chasing stacked god objects — hence the ore furnace slag."** +The upstream typed truth is GOOD ORE but structurally still stage-1 pointer +chasing: `SSAFunction`'s private `HashMap`, a petgraph CFG, +`BTreeMap` keyed by String-carrying vars, facts as nested +BTreeMaps of structs. **Typed ≠ refined.** The arm preserves that truth +untouched (never flattens at intake); the FURNACE is what melts the object +graph into flat facet-addressed concern rows; the SLAG is what resisted +flattening. Do not mistake the cleanliness of r2il's Rust types for +refinement — that mistake is exactly the "privileged direct path" the pivot +forbids. + +## CARRIED FORWARD (2026-08-18) — the ruff/r2sleigh half of the console ruling + +The operator's Ghidra-console ruling belongs to a different session; the +console, the Java extension, and the forensic product families are **out of +scope here**. Four of its requirements are pure ruff/r2sleigh R2IL properties +and DO carry momentum into PR 1. Recorded so they are not re-derived: + +**C1 — the library seam must survive an external caller.** Falsifiable test: +*could a thin external caller analyze one function and obtain structured +ore/furnace/slag results without parsing CLI strings or NDJSON?* **Measured +YES** — every entry point is a typed Rust fn over library types +(`FunctionBehavior::from_blocks_raw`, `furnace::smelt`, `ResidualLedger::*`, +`HarvestReport::*`); the CLI/TSV/TOML surfaces exist ONLY inside the two +`lift`-gated examples, whose artifacts are declared *evidence, never a +re-ingest path*. **Do not seal the API around CLI-only assumptions** — no RPC +protocol now, just keep the typed seam public. + +**C2 — provenance must reach the native instruction, not just the block.** +`FactProvenance.op_site: (block_addr, op_idx)` → +`R2ILBlock::op_metadata[op_idx].instruction_addr: Option` +(r2il `metadata.rs:103`: *"Source instruction address for this operation when +lifted as part of a block"*). **SSA does not carry it**, so this sidecar +rejoin IS the anchor — the same `(block_addr, op_idx)` key +`SsaGraph::op_inst_by_site` uses. Landed as `ore::instruction_addr(prov, blocks)` so a caller finds a named API instead of rediscovering a convention. +Chain: `fact → concern route → SSA/R2IL fact → instruction address → artifact`. + +**C3 — the conservation ledger is load-bearing, not diagnostic decoration.** +Three readings of one number: *what did the target fail to represent* +(transcode) · *what have we not yet explained* (reconstruction) · *what still +needs attention* (any investigative use). `dropped == 0` is what makes it +evidence rather than a progress bar. Already the PR-1 invariant; this +elevates it from QA metric to product property. + +**C4 — transcode is TARGET-ARCHITECTURE PROJECTION, not syntax conversion.** +`behavioral truth → target recipe → generated implementation`. A recovered +pointer graph does NOT oblige a pointer graph in the target; the furnace emits +concern-separated facts and a target profile decides the layout. PR 1 contains +no codegen, and `FlatFact` rows are concern-tagged + architecture-neutral, so +nothing here binds a future emitter. Corollary for the eventual roundtrip +oracle: **success is semantic/behavioral parity, never textual or binary +equality** — which is exactly why §14's reconstruction oracle is specified as +`R2IL → routes → semantic-equivalent R2IL`, with SPO explicitly NOT the oracle. + +## SUBSTRATE RULING (operator, 2026-08-18): THREE backends, none privileged + +The refined-truth sink must support **all three**, chosen at the seam, never +baked into the furnace: + +1. **Offline V3 substrate** — file/artifact-shaped, self-contained, no service. + What the PR-1 harvest artifacts already are. +1. **lance-graph zero-copy SoA** — live storage AND audit layer, so a future + external consumer can drill on the fly instead of re-harvesting. Zero-copy is + the point: rows are read in place from the SoA, never re-materialized. +1. **S3 object storage via `AWS_*` env vars (Railway Tigris)** — the same V3 + artifacts addressed remotely. Tigris is S3-compatible, so this is credential + plumbing (`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` / + `AWS_ENDPOINT_URL_S3` / `AWS_REGION`), **not** a fourth format. Never hardcode + an endpoint or embed a key; read the environment, and if unset fall back to + local offline mode rather than failing. + +**PR-1 consequence — a door to keep open, not work to do now.** The furnace +already returns `Vec` + `ResidualLedger` + `HarvestReport` in memory +and persists nothing; the examples write artifacts, the library does not. That +is exactly the shape all three backends need, so **PR 1 requires no change** — +the constraint is: do NOT introduce a persistence assumption into +`furnace`/`ore`/`slag`. PR 2 adds a sink trait behind which offline / lance-graph +/ S3 are implementations. Provenance (C2, `ore::instruction_addr`) is what makes +backend 2 an *audit* layer rather than a cache — keep it. + +## Architecture (ratified by operator feedback 2026-08-17) + +```text +Ruff + │ + ┌────────────┴─────────────┐ + │ │ + StructuralContract BehavioralContract ← ruff_r2il (NEW) + │ │ + ModelGraph R2IL / SSA (r2il + r2ssa, typed, direct) + │ │ + expand() concern routes (r2ssa's OWN decomposition, named) + │ │ + Vec V3 SoA / overflow (FacetCascade grammar + mint_factored + │ │ principle + out-of-line escape) + └────────────┬─────────────┘ + │ + OGAR/ClassView +``` + +- `ModelGraph → expand() → Vec` stays what it is: ONE deliberate, + documented-lossy SPO projection. The `Predicate` enum describes facts we + INTENTIONALLY project to SPO; it never becomes a backdoor opcode vocabulary. +- Canonical behavioral truth = the typed r2il/r2ssa values, assembled by + `ruff_r2il` into a `FunctionBehavior` contract whose route accessors ARE + r2sleigh's own `SsaGraph`/`PreparedFunctionFacts` decomposition. +- DTO/ClassId plane: intern `SSAVar.name`, userop ids, space categories, arch + names, symbols — numeric identities + codebook resolution; strings never hang + off operational objects in the projected form. +- Forbidden and staying forbidden: R2IL→JSON→Ruff; serde_json::to_value(op); + parsing display strings; generic edge soup; per-R2IL-struct 16-byte dogma. + +## Wave plan + +**PR 1 — ruff: `crates/ruff_r2il` (typed ingest + fixtures + corpus profiler).** +Workspace-EXCLUDED standalone crate (root `Cargo.toml` exclude; bgz17/deepnsm +precedent) so bare `cargo check --workspace` in ruff stays sibling-free and +`Cargo.lock` untouched. Path deps `../../../r2sleigh/crates/{r2il,r2ssa}` +(AdaWorldAPI fork, P0 fork rule). Feature `lift` (non-default, ruff_cpp_spo +pattern) gates `r2sleigh-lift` + `sleigh-config/x86` for the profiler example. +Contents: + +- `behavior.rs`: `FunctionBehavior{identity, ssa, graph, facts, summary}` + + named route accessors (control/values/objects/memory/predicates/calls) — + a thin truthful assembly, zero copying, no parallel ontology. +- `vocab.rs`: deterministic vocabulary harvest (SSAVar names, userops, space + categories) → interning table; measures the string-forest collapse. +- `facet.rs`: `VarnodeFacet` 16-byte projection EXPERIMENT (classid=space + class, a=offset lo32, b=offset hi32, c=size) + the `SpaceId::Custom(u32)` + lossless falsifier (custom ids need codebook interning — measure the fit). + Documented as a projection probe, not an address system. +- `tests/`: §14 lossless fixtures — every mandated op (Copy, IntAdd, Load, + Store, cmp, CBranch, Branch, Call, Return, AtomicCAS, StoreConditional, + Load/StoreGuarded, CallOther >2 in, Multiequal >2, Insert, Custom space, + 64-bit offsets, ordering, optional outputs, metadata) through + `FunctionBehavior::from_blocks` with typed-preservation asserts. +- `examples/r2il_corpus_profile.rs` (feature `lift`): §12 profile over real + corpora (r2sleigh e2e binaries + in-container ELFs): opcode freq, arity + histograms, %fitting dst+src0+src1, blocks/fn, ops/block, phi fan-in, + bytes/op for candidate layouts, overflow frequency at 255-rank. + +**PR 2 — ruff: route→V3 projection + DTO hooks (gated on PR 1 numbers).** +Measured layout choice per §11 (inline vs descriptor vs hybrid — note SsaGraph +is already descriptor-shaped in memory), `mint_factored`-principle overflow per +route, codebook wiring (read ogar_codebook, never construct a parallel one), +optional SPO projection of SEMANTIC facts only (calls/objects), round-trip +reconstruction oracle (R2IL → routes → semantic-equivalent R2IL; SPO explicitly +NOT the oracle). + +**PR 3 — lance-graph: ONLY if proven owner.** Expected residual: the codebook +mint for R2IL container concept(s) (the `NETWORK_LAYER=0x0804` analog) — a +canon-high slot is minted in `ogar_codebook`, which lance-graph owns. Defer +until PR 2 proves the route set; provisional classids documented Ruff-side +until then. + +**PR 4 — r2sleigh: candidate, not required.** Feature-gate `r2sleigh-lift` in +`r2ssa` (only `block.rs` needs it) → SSA-only consumers shed the native dep. +Generic, roadmap-aligned, Ruff-free. File upstream only with operator consent; +direct consumption already succeeds without it. + +## Stop conditions hit so far + +- §22.1: direct r2il/r2ssa consumption solves the upstream seam — YES (43 s). +- §22.4: V3 represents R2IL via routes+overflow — YES per audit; corpus gates layout only. +- §22.5: no V4; variable arity already routed upstream (SsaGraph descriptor shape). + +## Open items + +- O1: corpus profile numbers (PR 1 gate for PR 2 layout). +- O2: def-use persist-vs-derive benchmark (§13) — SsaGraph derives def_of/uses_of + from SSAFunction cheaply; measure before persisting. +- O3: `SpaceId::Custom(u32)` fit in the 16-byte varnode projection (fixture). +- O4: function discovery for whole-ELF lifting (linear sweep vs CLI vs r2) — + profiler may start at instruction/op-level stats + e2e binaries. +- O5: classid mint request shape for lance-graph (PR 3 gate). diff --git a/.gitignore b/.gitignore index 488e0301b1c87..7e2323067c884 100644 --- a/.gitignore +++ b/.gitignore @@ -231,3 +231,8 @@ cython_debug/ !crates/ruff_python_resolver/resources/test/airflow/venv/lib !crates/ruff_python_resolver/resources/test/airflow/venv/lib/python3.11/site-packages/_watchdog_fsevents.cpython-311-darwin.so !crates/ruff_python_resolver/resources/test/airflow/venv/lib/python3.11/site-packages/orjson/orjson.cpython-311-darwin.so + +# R2IL harvest bulk artifacts: GitHub Release `r2il-harvest-pass1` + S3 scratch. +# Regenerate with `--example harvest_r2il`; see .claude/harvest/r2il/README.md. +.claude/harvest/r2il/r2il-pass1.ore.tsv* +.claude/harvest/r2il/r2il-convention.toml* diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 7a7876439a7d5..2ed42ce1e3007 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,5 +1,10 @@ fail_fast: false +# Paths the fixer hooks must not touch. The last three were added after a +# repo-wide run mangled data: `typos` dropped a hex digit from a harvest +# shape_id and rewrote the P-code tag `int_scarry`; `mdformat` collapsed the +# YAML frontmatter of every `.claude/agents/*.md` card (no frontmatter +# plugin is configured); `ruff-format` reformatted golden test INPUTS. exclude: | (?x)^( .github/workflows/release.yml| @@ -16,7 +21,10 @@ exclude: | crates/ruff_python_formatter/tests/snapshots/.*| crates/ruff_python_resolver/resources/.*| crates/ruff_python_resolver/tests/snapshots/.*| - crates/ty_completion_eval/truth/.* + crates/ty_completion_eval/truth/.*| + vendor/.*| + .claude/.*| + crates/ruff_python_dto_check/tests/golden/.* )$ repos: diff --git a/Cargo.toml b/Cargo.toml index a7613684d811d..cf709b7a2112a 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -1,5 +1,11 @@ [workspace] members = ["crates/*"] +# ruff_r2il is a workspace-EXCLUDED standalone crate (its own workspace root): +# it path-deps the r2sleigh sibling checkout (r2il/r2ssa), and exclusion keeps +# the main workspace and its Cargo.lock sibling-free. Build it via +# `cargo --manifest-path crates/ruff_r2il/Cargo.toml`. +# See .claude/plans/r2il-behavioral-ir-v1.md. +exclude = ["crates/ruff_r2il"] resolver = "2" [workspace.package] @@ -38,7 +44,6 @@ ruff_python_parser = { path = "crates/ruff_python_parser" } ruff_python_semantic = { path = "crates/ruff_python_semantic" } ruff_python_stdlib = { path = "crates/ruff_python_stdlib" } ruff_python_trivia = { path = "crates/ruff_python_trivia" } -ruff_ruby_spo = { path = "crates/ruff_ruby_spo" } ruff_server = { path = "crates/ruff_server" } ruff_source_file = { path = "crates/ruff_source_file" } ruff_spo_triplet = { path = "crates/ruff_spo_triplet" } diff --git a/_typos.toml b/_typos.toml index c9a9ef3c9682d..ff07e99b27e20 100644 --- a/_typos.toml +++ b/_typos.toml @@ -11,6 +11,10 @@ extend-exclude = [ "crates/ty_ide/src/completion.rs", # Same for "Did you mean...?" levenshtein tests. "crates/ty_python_semantic/src/diagnostic/levenshtein.rs", + # Generated harvest evidence: hashes, addresses and opcode tags, not prose. + # The hook read the hex `a8ad31e12dbe9f80` as a misspelling and dropped a + # digit, breaking the shape_id join between two sections of the ledger. + ".claude/harvest/**/*", ] [default.extend-words] @@ -26,6 +30,17 @@ BA = "BA" # acronym for "Bad Allowed", used in testing. jod = "jod" # e.g., `jod-thread` Numer = "Numer" # Library name 'NumerBlox' in "Who's Using Ruff?" CPY = "CPY" # it's a Ruff rule category +# P-code / R2IL opcode names, not prose. INT_SCARRY is signed-carry; the +# hook's "scary" correction silently corrupts an emitted wire tag. +scarry = "scarry" +# Tesseract really does ship `src/ccstruct/statistc.h` under that spelling. +# "Correcting" it points the harvest docs at a file that does not exist. +statistc = "statistc" +# `oce` is the C# harvester's local for an ObjectCreationExpressionSyntax +# pattern binding; `tage` is a legacy column name carried by a codegen fixture. +# Both are identifiers under test — renaming them changes what is exercised. +oce = "oce" +tage = "tage" [default] extend-ignore-re = [ diff --git a/crates/ruff/src/commands/config.rs b/crates/ruff/src/commands/config.rs index f6c00548394ab..751e15876c52a 100644 --- a/crates/ruff/src/commands/config.rs +++ b/crates/ruff/src/commands/config.rs @@ -16,7 +16,7 @@ pub(crate) fn config(key: Option<&str>, format: HelpFormat) -> Result<()> { } HelpFormat::Json => { - println!("{}", &serde_json::to_string_pretty(&metadata)?); + println!("{}", serde_json::to_string_pretty(&metadata)?); } } } @@ -30,7 +30,7 @@ pub(crate) fn config(key: Option<&str>, format: HelpFormat) -> Result<()> { } HelpFormat::Json => { - println!("{}", &serde_json::to_string_pretty(&entry)?); + println!("{}", serde_json::to_string_pretty(&entry)?); } }, }, diff --git a/crates/ruff/src/commands/version.rs b/crates/ruff/src/commands/version.rs index 3361071206843..d750e908153a9 100644 --- a/crates/ruff/src/commands/version.rs +++ b/crates/ruff/src/commands/version.rs @@ -11,7 +11,7 @@ pub(crate) fn version(output_format: HelpFormat) -> Result<()> { match output_format { HelpFormat::Text => { - writeln!(stdout, "ruff {}", &version_info)?; + writeln!(stdout, "ruff {version_info}")?; } HelpFormat::Json => { serde_json::to_writer_pretty(stdout, &version_info)?; diff --git a/crates/ruff/src/lib.rs b/crates/ruff/src/lib.rs index 9d25a0967494a..e99bb894108d3 100644 --- a/crates/ruff/src/lib.rs +++ b/crates/ruff/src/lib.rs @@ -394,32 +394,28 @@ pub fn check(args: CheckCommand, global_options: GlobalConfigArgs) -> Result { - let Some(change_kind) = change_detected(&event?) else { - continue; - }; - - if matches!(change_kind, ChangeKind::Configuration) { - pyproject_config = - resolve::resolve(&config_arguments, cli.stdin_filename.as_deref())?; - } - Printer::clear_screen()?; - printer.write_to_user("File change detected...\n"); - - let diagnostics = commands::check::check( - &files, - &pyproject_config, - &config_arguments, - cache.into(), - noqa.into(), - fix_mode, - unsafe_fixes, - )?; - printer.write_continuously(&mut writer, &diagnostics, preview)?; - } - Err(err) => return Err(err.into()), + let event = rx.recv()?; + let Some(change_kind) = change_detected(&event?) else { + continue; + }; + + if matches!(change_kind, ChangeKind::Configuration) { + pyproject_config = + resolve::resolve(&config_arguments, cli.stdin_filename.as_deref())?; } + Printer::clear_screen()?; + printer.write_to_user("File change detected...\n"); + + let diagnostics = commands::check::check( + &files, + &pyproject_config, + &config_arguments, + cache.into(), + noqa.into(), + fix_mode, + unsafe_fixes, + )?; + printer.write_continuously(&mut writer, &diagnostics, preview)?; } } else { // Generate lint violations. diff --git a/crates/ruff/tests/integration_test.rs b/crates/ruff/tests/integration_test.rs index 1c6cc0eb0c15c..73fd6ea544307 100644 --- a/crates/ruff/tests/integration_test.rs +++ b/crates/ruff/tests/integration_test.rs @@ -1790,7 +1790,7 @@ fn check_input_from_argfile() -> Result<()> { )?; // Generate the args with the argfile notation - let argfile = format!("@{}", &input_file_path.display()); + let argfile = format!("@{}", input_file_path.display()); let mut cmd = RuffCheck::default().filename(argfile.as_ref()).build(); insta::with_settings!({filters => vec![ (file_a_path.display().to_string().as_str(), "/path/to/a.py"), diff --git a/crates/ruff_annotate_snippets/src/renderer/display_list.rs b/crates/ruff_annotate_snippets/src/renderer/display_list.rs index b7011cee197ba..88f6b94ef19b4 100644 --- a/crates/ruff_annotate_snippets/src/renderer/display_list.rs +++ b/crates/ruff_annotate_snippets/src/renderer/display_list.rs @@ -1267,9 +1267,8 @@ fn format_snippet<'m>( } #[inline] -// TODO: option_zip fn zip_opt(a: Option, b: Option) -> Option<(A, B)> { - a.and_then(|a| b.map(|b| (a, b))) + a.zip(b) } fn format_header<'a>( diff --git a/crates/ruff_cpp_spo/Cargo.toml b/crates/ruff_cpp_spo/Cargo.toml index 298b9da40f72b..93aa184ee7380 100644 --- a/crates/ruff_cpp_spo/Cargo.toml +++ b/crates/ruff_cpp_spo/Cargo.toml @@ -45,6 +45,14 @@ required-features = ["libclang"] name = "harvest_tesseract_dict" required-features = ["libclang"] +[[example]] +name = "harvest_leptonica_scale" +required-features = ["libclang"] + +[[example]] +name = "harvest_textord" +required-features = ["libclang"] + [dev-dependencies] # End-to-end pipeline test only (harvest -> reassemble -> ruff_cpp_codegen emit). # The harvester does NOT depend on the codegen at runtime — this is the test diff --git a/crates/ruff_cpp_spo/src/clang_walker.rs b/crates/ruff_cpp_spo/src/clang_walker.rs index 507709ac8105e..ec59344f0edca 100644 --- a/crates/ruff_cpp_spo/src/clang_walker.rs +++ b/crates/ruff_cpp_spo/src/clang_walker.rs @@ -149,7 +149,7 @@ pub fn walk_tu_with_diagnostics( /// /// This is the missing arm for C libraries: [`walk_tu`] harvests C++ *classes*; /// a C library (leptonica, zlib, …) is free functions on pointer buffers, so -/// the AR/OO member body-arm ([`method_body_arm`]) captures nothing there — but +/// the AR/OO member body-arm (`method_body_arm`) captures nothing there — but /// the call graph IS the transcode-driving structure (which functions to port, /// in what dispatch order). Numeric kernel BODIES remain the essential-15% /// hand-port (the doctrine); this mints the 85% structure that classifies + orders @@ -326,10 +326,10 @@ fn find_overloaded_decl_ref(node: &Entity) -> Option { /// directly inside a namespace or at global scope). /// /// Nested class-body enums are NOT collected here — they are covered by the -/// extended [`build_class`], which pushes them onto the owning +/// extended `build_class`, which pushes them onto the owning /// [`CppClass::declarations`] as `Declaration::Enum` alongside its fields and /// methods. This split mirrors [`walk_tu`] vs the class-body arm of -/// [`build_class`]: a free-standing enum has no owning class to attach to, so +/// `build_class`: a free-standing enum has no owning class to attach to, so /// it needs its own top-level collection. /// /// # Errors @@ -411,7 +411,7 @@ fn build_enum(e: &Entity) -> Option { /// The key is the `EntityKind` `Debug` name (e.g. `"Method"`, `"FieldDecl"`, /// `"FriendDecl"`); the value is how many times that kind appears as a direct /// member. The caller computes the *mapped fraction* — `BaseSpecifier` + -/// `FieldDecl` + `Method` are exactly the kinds [`build_class`] turns into a +/// `FieldDecl` + `Method` are exactly the kinds `build_class` turns into a /// [`Declaration`] today — versus the total, so a real-corpus walk shows which /// constructs the walker silently drops (the walker-follow-up backlog: /// `FriendDecl`, `StaticAssert`, templates, …) rather than asserting coverage. @@ -434,7 +434,7 @@ pub fn class_body_cursor_histogram( Ok(hist) } -/// The kinds [`build_class`] maps to a [`Declaration`] today — the "covered" +/// The kinds `build_class` maps to a [`Declaration`] today — the "covered" /// set for the `CPP-SCHEMA-FIT` mapped-fraction. Kept beside the walker so the /// coverage probe and the actual extraction can never drift apart. The /// function-like kinds (`Method` / `Constructor` / `Destructor` / diff --git a/crates/ruff_cpp_spo/src/lib.rs b/crates/ruff_cpp_spo/src/lib.rs index 189776a1e9dd2..6f7fc68d9b0b8 100644 --- a/crates/ruff_cpp_spo/src/lib.rs +++ b/crates/ruff_cpp_spo/src/lib.rs @@ -35,7 +35,7 @@ //! methods with their flags, system-header classes filtered out). //! [`extract`] — the corpus-TREE orchestration over `walk_tu` — remains //! `todo!()` (per-TU include resolution + cross-TU dedup). The target -//! triple shape is locked by [`tests::locked_shape_expands_to_expected_triples`]. +//! triple shape is locked by `tests::locked_shape_expands_to_expected_triples`. //! //! # Iron rules this frontend respects //! @@ -195,14 +195,14 @@ pub enum Declaration { /// Top-level entry: walk a C++ corpus **tree** and produce the IR. /// /// **Still `todo!()` — what's missing now is only per-TU include -/// auto-detection.** Everything beneath it is done: [`walk_tu`] walks ONE -/// translation unit, and [`extract_tree`] already does the recursive +/// auto-detection.** Everything beneath it is done: `walk_tu` walks ONE +/// translation unit, and `extract_tree` already does the recursive /// enumeration + per-TU walk + cross-TU dedup for a SINGLE caller-supplied /// include-arg set. What `extract` adds is resolving the right include args /// *per TU automatically* (the `tesseract-ocr/tesseract@5.5.0` + leptonica /// include graph — the real remaining work), so a caller can point it at a /// corpus root without hand-supplying `-I` flags. Once that lands it is -/// essentially [`extract_tree`] with auto-derived args, plus the +/// essentially `extract_tree` with auto-derived args, plus the /// `CPP-SCHEMA-FIT` coverage gate (`.claude/plans/cpp-spo-probes-v1.md`). #[must_use] pub fn extract(source_tree: &Path) -> ModelGraph { diff --git a/crates/ruff_csharp_spo/README.md b/crates/ruff_csharp_spo/README.md index 5062f0e7bc82a..101c11c7a094a 100644 --- a/crates/ruff_csharp_spo/README.md +++ b/crates/ruff_csharp_spo/README.md @@ -57,11 +57,11 @@ dotnet run --project crates/ruff_csharp_spo/harvester/CSharpSpoHarvester.csproj All optional; defaults reproduce the original EF-Core-flavoured behaviour, so existing invocations are unaffected. -| flag | default | what it does | -| ------------------------------ | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `--ns ` | `csharp` | IRI namespace prefix for every subject/object | -| `--mutator-names a,b,c` | the EF Core set (`SaveChanges`, `Update`, `Add`, `Remove`, …) | exact method names that make a `calls` fact fire — **replaces** the default set when given | -| `--mutator-prefixes add_,del_` | none | method-name *prefixes* that also count as mutators — for bespoke ADO.NET DALs with a naming convention instead of a fixed method set | +| flag | default | what it does | +| ------------------------------ | ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `--ns ` | `csharp` | IRI namespace prefix for every subject/object | +| `--mutator-names a,b,c` | the EF Core set (`SaveChanges`, `Update`, `Add`, `Remove`, …) | exact method names that make a `calls` fact fire — **replaces** the default set when given | +| `--mutator-prefixes add_,del_` | none | method-name *prefixes* that also count as mutators — for bespoke ADO.NET DALs with a naming convention instead of a fixed method set | | `--mutator-receivers mysql` | none (any receiver) | restricts the `calls` fact to invocations whose receiver's last identifier segment is in this list — `main.mysql.add_x(...)` matches receiver `mysql`; a form's own `set_Foo(...)` does not. **Applies to every mutator match**, name-set or prefix-based alike, once set. | A name match is `--mutator-names` (exact) **OR** `--mutator-prefixes` (prefix) @@ -81,15 +81,15 @@ The scaffold walks the **syntax layer** and emits the structural facts every class carries. Subjects/objects are namespaced with the `--ns` prefix (`csharp:` by default). -| C# construct | SPO triple | -| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | -| `class Invoice` | `(csharp:Invoice, rdf:type, ogit:ObjectType)` | -| `: DbBase` / `: IFoo` | `(csharp:Invoice, inherits_from, csharp:DbBase)` | -| `string number { get; set; }` / field | `(csharp:Invoice, has_field, csharp:Invoice.number)` + `(…​.number, rdf:type, ogit:Property)` + `(…​.number, field_type, string)` | -| `void Save()` | `(csharp:Invoice, has_function, csharp:Invoice.Save)` + `(…​.Save, rdf:type, ogit:Function)` | -| `static` method | `(csharp:Invoice.Save, is_static, true)` | -| `int Foo(int x, string y)` | `(csharp:Invoice.Foo, returns_type, int)` + `(…​.Foo, has_param_type, "0:int")` + `(…​.Foo, has_param_type, "1:string")` | -| method access specifier (always present) | `(csharp:Invoice.Foo, has_visibility, "public"\|"protected"\|"private")` | +| C# construct | SPO triple | +| ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | +| `class Invoice` | `(csharp:Invoice, rdf:type, ogit:ObjectType)` | +| `: DbBase` / `: IFoo` | `(csharp:Invoice, inherits_from, csharp:DbBase)` | +| `string number { get; set; }` / field | `(csharp:Invoice, has_field, csharp:Invoice.number)` + `(…​.number, rdf:type, ogit:Property)` + `(…​.number, field_type, string)` | +| `void Save()` | `(csharp:Invoice, has_function, csharp:Invoice.Save)` + `(…​.Save, rdf:type, ogit:Function)` | +| `static` method | `(csharp:Invoice.Save, is_static, true)` | +| `int Foo(int x, string y)` | `(csharp:Invoice.Foo, returns_type, int)` + `(…​.Foo, has_param_type, "0:int")` + `(…​.Foo, has_param_type, "1:string")` | +| method access specifier (always present) | `(csharp:Invoice.Foo, has_visibility, "public"\|"protected"\|"private")` | All predicates above are in the closed vocabulary already (shared with the C++/Rails frontends). NARS truth is `(f=1.0, c=0.9)` — the declared/structural diff --git a/crates/ruff_csharp_spo/examples/recipe_census.rs b/crates/ruff_csharp_spo/examples/recipe_census.rs index 69ee4b1b898d8..3cc6611fae765 100644 --- a/crates/ruff_csharp_spo/examples/recipe_census.rs +++ b/crates/ruff_csharp_spo/examples/recipe_census.rs @@ -12,6 +12,12 @@ //! triples.ndjson --ns medcare --exclude-prefix Crypt --exclude-prefix Mime //! ``` +#![expect( + clippy::print_stdout, + clippy::print_stderr, + reason = "this is a CLI report example whose entire purpose is printing a census/validation report to stdout/stderr" +)] + use ruff_spo_triplet::{RecipeCentroid, classify, is_recoverable, reassemble_model_graph}; use std::collections::BTreeMap; @@ -93,6 +99,13 @@ fn main() { println!(); println!("centroid distribution:"); for (name, n) in &counts { + // Precision loss here is inconsequential: these counts are function + // tallies (far below f64's 52-bit mantissa range) rendered as a + // human-readable percentage. + #[expect( + clippy::cast_precision_loss, + reason = "function-tally counts are far below f64's exact-integer range; only used for a display percentage" + )] let pct = 100.0 * (*n as f64) / (total.max(1) as f64); println!(" {n:6} ({pct:5.1}%) {name}"); if let Some(ex) = examples.get(name) { diff --git a/crates/ruff_csharp_spo/examples/validate_load.rs b/crates/ruff_csharp_spo/examples/validate_load.rs index 7f5955a136a96..c71c753dccebe 100644 --- a/crates/ruff_csharp_spo/examples/validate_load.rs +++ b/crates/ruff_csharp_spo/examples/validate_load.rs @@ -1,3 +1,8 @@ +#![expect( + clippy::print_stdout, + reason = "this is a CLI validation example whose entire purpose is printing a load-status report to stdout" +)] + fn main() { let ndjson = std::fs::read_to_string(std::env::args().nth(1).unwrap()).unwrap(); match ruff_csharp_spo::load(&ndjson) { diff --git a/crates/ruff_csharp_spo/src/lib.rs b/crates/ruff_csharp_spo/src/lib.rs index ed9bc8141dd0c..cf32edcb9be08 100644 --- a/crates/ruff_csharp_spo/src/lib.rs +++ b/crates/ruff_csharp_spo/src/lib.rs @@ -126,7 +126,7 @@ mod tests { assert_eq!(triples[0].s, "csharp:Widget.SetDefaults"); } - /// The UI-navigation arm — the WinForms `navigates_to` Klickweg edge + /// The UI-navigation arm — the `WinForms` `navigates_to` Klickweg edge /// (`EmitNavArm`). Subject is the CLASS that navigates, object is the target /// screen class. Shaped exactly as the harvester emits it (verified by /// running the real harvester over `harvester/fixtures/nav_shapes.cs`): @@ -134,7 +134,7 @@ mod tests { /// idiom (`HostControl` field-instantiates `ChildControl`, no `.Show()`), /// one via NAMESPACE-QUALIFIED hosting (`new Nested.QualifiedChild(..)` — /// LastSegment-normalized to the bare screen node), and one `selects_view` - /// ribbon fact. The `SaveFileDialog` CommonDialog and the non-screen + /// ribbon fact. The `SaveFileDialog` `CommonDialog` and the non-screen /// `StringBuilder` are both excluded. A clean load is the standing proof /// the nav arm stays inside the closed vocabulary. #[test] @@ -193,7 +193,7 @@ mod tests { /// `surfaces_concept` / `handles_event` / `contains_control` /// (`EmitUiConfigArm` + the `--room-aliases` config binding). One line /// per predicate, shaped exactly as `harvester/Program.cs` emits it - /// (verified against a real WinForms corpus: screen-classified types, + /// (verified against a real `WinForms` corpus: screen-classified types, /// Designer `+=` wiring, `Controls.Add` containment, directory→concept /// alias rows). The Klickweg EDGES ride the nav-arm test above; this /// pins the room-map half. A clean load is the standing proof the diff --git a/crates/ruff_linter/src/rules/fastapi/rules/fastapi_unused_path_parameter.rs b/crates/ruff_linter/src/rules/fastapi/rules/fastapi_unused_path_parameter.rs index 24be9c37dfc5a..64f1fac64d09b 100644 --- a/crates/ruff_linter/src/rules/fastapi/rules/fastapi_unused_path_parameter.rs +++ b/crates/ruff_linter/src/rules/fastapi/rules/fastapi_unused_path_parameter.rs @@ -399,19 +399,17 @@ impl<'a> Dependency<'a> { .map(|name| name.id.as_str()) }) .collect() - } else if let Some(method_def) = class_def - .body - .iter() - .filter_map(|stmt| stmt.as_function_def_stmt()) - .find(|func_def| func_def.name.as_str() == method_name) - { + } else { + let method_def = class_def + .body + .iter() + .filter_map(|stmt| stmt.as_function_def_stmt()) + .find(|func_def| func_def.name.as_str() == method_name)?; // Skip `self` parameter non_posonly_non_variadic_parameters(method_def) .skip(1) .map(|param| param.name().as_str()) .collect() - } else { - return None; }; Some(Self::Class(parameter_names)) diff --git a/crates/ruff_linter/src/rules/flake8_pyi/rules/custom_type_var_for_self.rs b/crates/ruff_linter/src/rules/flake8_pyi/rules/custom_type_var_for_self.rs index b22a050ad0901..ef7e47ceb2175 100644 --- a/crates/ruff_linter/src/rules/flake8_pyi/rules/custom_type_var_for_self.rs +++ b/crates/ruff_linter/src/rules/flake8_pyi/rules/custom_type_var_for_self.rs @@ -101,14 +101,14 @@ impl Violation for CustomTypeVarForSelf { fn message(&self) -> String { format!( "Use `Self` instead of custom TypeVar `{}`", - &self.typevar_name + self.typevar_name ) } fn fix_title(&self) -> Option { Some(format!( "Replace TypeVar `{}` with `Self`", - &self.typevar_name + self.typevar_name )) } } diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/outdated_version_block.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/outdated_version_block.rs index dfa945fab1c7a..06ad8060e943a 100644 --- a/crates/ruff_linter/src/rules/pyupgrade/rules/outdated_version_block.rs +++ b/crates/ruff_linter/src/rules/pyupgrade/rules/outdated_version_block.rs @@ -294,11 +294,11 @@ fn fix_always_false_branch( range, .. }) => { - debug_assert!( + debug_assert_eq!( checker .locator() - .slice(TextRange::at(range.start(), "elif".text_len())) - == "elif" + .slice(TextRange::at(range.start(), "elif".text_len())), + "elif" ); let end_location = range.start() + ("elif".text_len() - "if".text_len()); Some(Fix::unsafe_edit(Edit::deletion( diff --git a/crates/ruff_linter/src/rules/pyupgrade/rules/unnecessary_encode_utf8.rs b/crates/ruff_linter/src/rules/pyupgrade/rules/unnecessary_encode_utf8.rs index a5605391bd90c..8c418ae40ced4 100644 --- a/crates/ruff_linter/src/rules/pyupgrade/rules/unnecessary_encode_utf8.rs +++ b/crates/ruff_linter/src/rules/pyupgrade/rules/unnecessary_encode_utf8.rs @@ -133,7 +133,7 @@ fn replace_with_bytes_literal(locator: &Locator, call: &ast::ExprCall, tokens: & let _ = write!( &mut replacement, "b{}", - &string.trim_start_matches('u').trim_start_matches('U') + string.trim_start_matches('u').trim_start_matches('U') ); } _ => { diff --git a/crates/ruff_linter/src/rules/ruff/rules/sort_dunder_slots.rs b/crates/ruff_linter/src/rules/ruff/rules/sort_dunder_slots.rs index 4038a9508a579..b5767e69dfabe 100644 --- a/crates/ruff_linter/src/rules/ruff/rules/sort_dunder_slots.rs +++ b/crates/ruff_linter/src/rules/ruff/rules/sort_dunder_slots.rs @@ -240,13 +240,10 @@ impl<'a> StringLiteralDisplay<'a> { ast::Expr::Dict(dict) => { let mut narrowed_keys = Vec::with_capacity(dict.len()); for key in dict.iter_keys() { - if let Some(key) = key { - // This is somewhat unfortunate, - // *but* using a dict for __slots__ is very rare - narrowed_keys.push(key.to_owned()); - } else { - return None; - } + // This is somewhat unfortunate, + // *but* using a dict for __slots__ is very rare + let key = key?; + narrowed_keys.push(key.to_owned()); } // If `None` was present in the keys, it indicates a "** splat", .e.g // `__slots__ = {"foo": "bar", **other_dict}` diff --git a/crates/ruff_linter/src/source_kind.rs b/crates/ruff_linter/src/source_kind.rs index 7b029ce137546..35e660190e783 100644 --- a/crates/ruff_linter/src/source_kind.rs +++ b/crates/ruff_linter/src/source_kind.rs @@ -235,8 +235,8 @@ impl std::fmt::Display for SourceKindDiff<'_> { || (format!("cell {idx}"), format!("cell {idx}")), |path| { ( - format!("{}:cell {}", &fs::relativize_path(path), idx), - format!("{}:cell {}", &fs::relativize_path(path), idx), + format!("{}:cell {}", fs::relativize_path(path), idx), + format!("{}:cell {}", fs::relativize_path(path), idx), ) }, ); diff --git a/crates/ruff_python_ast/src/helpers.rs b/crates/ruff_python_ast/src/helpers.rs index 19955d1506d23..f465b9524d0eb 100644 --- a/crates/ruff_python_ast/src/helpers.rs +++ b/crates/ruff_python_ast/src/helpers.rs @@ -310,18 +310,10 @@ where range: _, node_index: _, }) - | Expr::List(ast::ExprList { - elts, - range: _, - node_index: _, - .. - }) - | Expr::Tuple(ast::ExprTuple { - elts, - range: _, - node_index: _, - .. - }) => elts.iter().any(|expr| any_over_expr(expr, &mut *func)), + | Expr::List(ast::ExprList { elts, .. }) + | Expr::Tuple(ast::ExprTuple { elts, .. }) => { + elts.iter().any(|expr| any_over_expr(expr, &mut *func)) + } Expr::ListComp(ast::ExprListComp { elt, generators, @@ -380,18 +372,8 @@ where range: _, node_index: _, }) - | Expr::Attribute(ast::ExprAttribute { - value, - range: _, - node_index: _, - .. - }) - | Expr::Starred(ast::ExprStarred { - value, - range: _, - node_index: _, - .. - }) => any_over_expr(value, func), + | Expr::Attribute(ast::ExprAttribute { value, .. }) + | Expr::Starred(ast::ExprStarred { value, .. }) => any_over_expr(value, func), Expr::Yield(ast::ExprYield { value, range: _, diff --git a/crates/ruff_python_dto_check/README.md b/crates/ruff_python_dto_check/README.md index 7462633f4d197..5176666b06506 100644 --- a/crates/ruff_python_dto_check/README.md +++ b/crates/ruff_python_dto_check/README.md @@ -55,7 +55,7 @@ for the full config schema. ## Output layout -``` +```text out/ ├── bundles/ │ ├── .ndjson — one JSON object per line, one per matched function @@ -82,9 +82,9 @@ Current implementation: - One matcher kind: `function_with_decorator` - Flask-style route detection via `@.route(...)` in the default profile - Config-driven emit fields (`def.name`, `def.params`, `def.body.source`, - `decorator.args[0]`, `decorator.kwargs.`, etc.) + `decorator.args[0]`, `decorator.kwargs.`, etc.) - Per-family observation block with set-algebra comparisons and percentile - distributions (no advisory English strings — pure content encoding) + distributions (no advisory English strings — pure content encoding) - Preflight subcommand with decorator histogram + framework fingerprint Out of scope for this iteration (reserved for future work): diff --git a/crates/ruff_python_dto_check/src/bin/ruff_py_dto.rs b/crates/ruff_python_dto_check/src/bin/ruff_py_dto.rs index b6021571919e1..6c2cffad9ae5f 100644 --- a/crates/ruff_python_dto_check/src/bin/ruff_py_dto.rs +++ b/crates/ruff_python_dto_check/src/bin/ruff_py_dto.rs @@ -27,7 +27,7 @@ struct Cli { #[derive(Subcommand)] enum Command { - /// Walk a Python repo root, emit per-family ndjson bundles under . + /// Walk a Python repo root, emit per-family ndjson bundles under ``. Harvest { /// Path to the JSON config file. #[arg(long)] @@ -76,7 +76,7 @@ enum Command { #[arg(long, default_value = "ruff-py-dto-codegen")] out: PathBuf, }, - /// Write the JSON Schema to or stdout. + /// Write the JSON Schema to `` or stdout. Schema { /// Output path (default: stdout). #[arg(long)] diff --git a/crates/ruff_python_dto_check/src/codegen/columns.rs b/crates/ruff_python_dto_check/src/codegen/columns.rs index e9b8806ca4447..a4f1e4dc42dfe 100644 --- a/crates/ruff_python_dto_check/src/codegen/columns.rs +++ b/crates/ruff_python_dto_check/src/codegen/columns.rs @@ -11,12 +11,12 @@ //! This is the proven Python extraction logic re-expressed in Rust; it is NOT //! reinvented. The regex-driven Python functions become hand-rolled, //! panic-free string scans (matching the style of `jinja.rs`): -//! - [`find_table_block`] ← `find_table_block` -//! - [`extract_loop`] ← `extract_loop` -//! - [`extract_headers`] ← `extract_headers` -//! - [`extract_body_cells`] ← `extract_body_cells` (+ `_find_outer_else`) -//! - [`classify_cell`] ← `classify_cell` -//! - [`extract_empty_row`] ← `extract_empty_row` +//! - `find_table_block` ← `find_table_block` +//! - `extract_loop` ← `extract_loop` +//! - `extract_headers` ← `extract_headers` +//! - `extract_body_cells` ← `extract_body_cells` (+ `_find_outer_else`) +//! - `classify_cell` ← `classify_cell` +//! - `extract_empty_row` ← `extract_empty_row` use crate::codegen::jinja::Cell; diff --git a/crates/ruff_python_dto_check/src/codegen/mod.rs b/crates/ruff_python_dto_check/src/codegen/mod.rs index 54cd4b49b9a0b..835c8c95f5a8a 100644 --- a/crates/ruff_python_dto_check/src/codegen/mod.rs +++ b/crates/ruff_python_dto_check/src/codegen/mod.rs @@ -1,7 +1,7 @@ //! Target emitter: contract → target source (handler + view template). //! //! The emitter is **kind-generalized**: each [`HandlerKind`] resolves to a -//! [`KindRecipe`] that describes the shape (signature, query, response) in +//! `KindRecipe` that describes the shape (signature, query, response) in //! data, so the other 10 kinds slot in by adding a recipe entry — not new //! Rust per kind. `list_for_tenant` and `soft_delete` are implemented //! end-to-end (matching the woa-rs port-drafts oracle, with the corrected diff --git a/crates/ruff_python_dto_check/src/extractors/body.rs b/crates/ruff_python_dto_check/src/extractors/body.rs index 25342312bdb84..1f14fac628325 100644 --- a/crates/ruff_python_dto_check/src/extractors/body.rs +++ b/crates/ruff_python_dto_check/src/extractors/body.rs @@ -129,7 +129,7 @@ pub fn extract_body(func: &StmtFunctionDef, profile: &ExtractionProfile) -> Body soft_delete: false, }; - // Decorators feed guard predicates (auth, modul, tenant gates). + // Decorators feed guard predicates (auth, module, tenant gates). for dec in &func.decorator_list { if let Some(name) = guard_name(&dec.expression) { w.guards.push(name); @@ -245,7 +245,7 @@ impl Walker<'_> { } } // else/finally commonly carry the return + side-effects; - // skipping them mis-classifies OutputKind (Codex P2). + // skipping them misclassifies OutputKind (Codex P2). for s in &t.orelse { self.walk_stmt(s); } diff --git a/crates/ruff_python_formatter/src/range.rs b/crates/ruff_python_formatter/src/range.rs index 488b5585f4f88..50ab6900f102b 100644 --- a/crates/ruff_python_formatter/src/range.rs +++ b/crates/ruff_python_formatter/src/range.rs @@ -540,6 +540,10 @@ impl NarrowRange<'_> { // The challenge here is that the second line of the multiline string uses a 4 space indentation. Using `dedent` would // dedent the second line to 0 spaces and the `indent` then adds a 2 space indentation to match the indentation in the source. // This is incorrect because the leading whitespace is the content of the string and not indentation, resulting in changed string content. + #[expect( + clippy::question_mark, + reason = "the else branch documents why the None case must not narrow the range" + )] if let Some(indentation) = indentation_at_offset(first_child.start(), self.context.source()) { diff --git a/crates/ruff_python_formatter/tests/fixtures.rs b/crates/ruff_python_formatter/tests/fixtures.rs index 441a7f7a34749..0ee9bd0977122 100644 --- a/crates/ruff_python_formatter/tests/fixtures.rs +++ b/crates/ruff_python_formatter/tests/fixtures.rs @@ -425,7 +425,7 @@ Formatted once: Formatted twice: --- {reformatted}---"#, - options = &DisplayPyOptions(options), + options = DisplayPyOptions(options), reformatted = reformatted.as_code(), ); } diff --git a/crates/ruff_python_parser/src/error.rs b/crates/ruff_python_parser/src/error.rs index 9f3aede832229..9a0b2d3eb7fd0 100644 --- a/crates/ruff_python_parser/src/error.rs +++ b/crates/ruff_python_parser/src/error.rs @@ -30,7 +30,7 @@ impl std::error::Error for ParseError { impl fmt::Display for ParseError { fn fmt(&self, f: &mut std::fmt::Formatter) -> std::fmt::Result { - write!(f, "{} at byte range {:?}", &self.error, self.location) + write!(f, "{} at byte range {:?}", self.error, self.location) } } diff --git a/crates/ruff_python_semantic/src/cfg/visualize.rs b/crates/ruff_python_semantic/src/cfg/visualize.rs index 64c9b18106300..fe861c122bd27 100644 --- a/crates/ruff_python_semantic/src/cfg/visualize.rs +++ b/crates/ruff_python_semantic/src/cfg/visualize.rs @@ -24,7 +24,7 @@ trait MermaidGraph<'a>: DirectedGraph<'a> { let num_nodes = self.num_nodes(); for idx in 0..num_nodes { let node = Self::Node::new(idx); - graph.push(format!("\tnode{}{}", idx, &self.draw_node(node))); + graph.push(format!("\tnode{}{}", idx, self.draw_node(node))); } // Draw edges diff --git a/crates/ruff_python_spo/src/navigation.rs b/crates/ruff_python_spo/src/navigation.rs index 96113b81a7a7c..d1fefb93923c6 100644 --- a/crates/ruff_python_spo/src/navigation.rs +++ b/crates/ruff_python_spo/src/navigation.rs @@ -141,7 +141,7 @@ pub fn extract_nav_edges(root: &Path, vocab: &NavVocab) -> Vec { /// (skipping a file that fails to parse — the crate's silent-skip invariant), /// AST-visits for `url_for` / `reverse` / `redirect` calls with a string- /// literal first argument, and lifts each known-screen target to an edge whose -/// source is the file's [`source_screen`]. Results are deduped by +/// source is the file's `source_screen`. Results are deduped by /// `(source, target, call)` and sorted deterministically. #[must_use] pub fn extract_nav_edges_with_report( diff --git a/crates/ruff_python_spo/src/odoo_regions.rs b/crates/ruff_python_spo/src/odoo_regions.rs index b4cdc67e3ddbb..361ac92f3f246 100644 --- a/crates/ruff_python_spo/src/odoo_regions.rs +++ b/crates/ruff_python_spo/src/odoo_regions.rs @@ -393,11 +393,11 @@ mod tests { use super::*; use ruff_spo_triplet::{Predicate, Provenance}; - /// A neutral synthetic view exercising: a `
` button (top_bar), a - /// `` field (left_nav), a `` field + a `` - /// field (center), a `oe_chatter` div field (right_panel), a comodel field + /// A neutral synthetic view exercising: a `
` button (`top_bar`), a + /// `` field (`left_nav`), a `` field + a `` + /// field (center), a `oe_chatter` div field (`right_panel`), a comodel field /// nested inside a `line_ids` field (must NOT dock), and an - /// `type="action"` button (opens_popup). No corpus tokens — all invented. + /// `type="action"` button (`opens_popup`). No corpus tokens — all invented. const FIXTURE: &str = r#" @@ -501,7 +501,7 @@ mod tests { /// Fence for the known codec edge (collapse-spec §2): the shared subject /// codec ([`RegionSubject::from_iri`](ruff_spo_triplet::RegionSubject)) /// decodes on the LAST `.`, so a control that itself contains a `.` (a rare - /// Odoo related-field path like `currency_id.symbol`) would mis-split. On + /// Odoo related-field path like `currency_id.symbol`) would split wrongly. On /// this corpus controls are Python identifiers / `%(...)d` action refs — /// dotless. If a real dotted control ever appears this trips, and the /// canonical separator must escalate to `::` (with the C# harvester @@ -512,7 +512,7 @@ mod tests { for f in &facts { assert!( !f.control.contains('.'), - "dotted control {:?} would mis-rsplit through the shared subject codec", + "dotted control {:?} would rsplit wrongly through the shared subject codec", f.control ); } diff --git a/crates/ruff_r2il/Cargo.lock b/crates/ruff_r2il/Cargo.lock new file mode 100644 index 0000000000000..5cb7a820100c2 --- /dev/null +++ b/crates/ruff_r2il/Cargo.lock @@ -0,0 +1,547 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "atomic-polyfill" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8cf2bce30dfe09ef0bfaef228b9d414faaf7e563035494d7fe092dba54b300f4" +dependencies = [ + "critical-section", +] + +[[package]] +name = "byteorder" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fd0f2584146f6f2ef48085050886acf353beff7305ebd1ae69500e27c67f64b" + +[[package]] +name = "cc" +version = "1.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "509591b7bcd67f4ef775afad7662703b4935daaa6ec0e5605cfb1090b32a2b6d" +dependencies = [ + "find-msvc-tools", + "shlex", +] + +[[package]] +name = "clap" +version = "4.6.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "473c7e07f409a8d772161724aa8db6a765a2532a70f9667eeb7b49d3d02fbdca" +dependencies = [ + "clap_builder", +] + +[[package]] +name = "clap_builder" +version = "4.6.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7b48fea5a88e9ae728a2dcbedbfc0e730f7d60da42e1cb049a83c9fb8b789889" +dependencies = [ + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + +[[package]] +name = "cobs" +version = "0.3.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0fa961b519f0b462e3a3b4a34b64d119eeaca1d59af726fe450bbba07a9fc0a1" +dependencies = [ + "thiserror", +] + +[[package]] +name = "codespan-reporting" +version = "0.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "af491d569909a7e4dee0ad7db7f5341fef5c614d5b8ec8cf765732aba3cff681" +dependencies = [ + "serde", + "termcolor", + "unicode-width", +] + +[[package]] +name = "critical-section" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "790eea4361631c5e7d22598ecd5723ff611904e3344ce8720784c93e3d83d40b" + +[[package]] +name = "cxx" +version = "1.0.199" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824894a4a85dca76d4c95c2b9098c036f5a29f627b30c12780774f6654e60974" +dependencies = [ + "cc", + "cxx-build", + "cxxbridge-cmd", + "cxxbridge-flags", + "cxxbridge-macro", + "foldhash 0.2.0", + "link-cplusplus", +] + +[[package]] +name = "cxx-build" +version = "1.0.199" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1ae0b651ea5b0000b19513aef5a03f194d7e3486f2d9258b658da8677fe9036" +dependencies = [ + "cc", + "codespan-reporting", + "indexmap", + "proc-macro2", + "quote", + "scratch", + "syn", +] + +[[package]] +name = "cxxbridge-cmd" +version = "1.0.199" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fb05f91d3fb8435d9bab6ac5ce6ac1868be774325fb7fb2a91be39393b21388e" +dependencies = [ + "clap", + "codespan-reporting", + "indexmap", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "cxxbridge-flags" +version = "1.0.199" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf293202e0e3e98495785745389e8d0755b217e66f19194a5c695c25e03282ef" + +[[package]] +name = "cxxbridge-macro" +version = "1.0.199" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ca001d746947c7249ed9d332a10f7a59daedbafeb0ec68c5c18a7db7a93f6ccc" +dependencies = [ + "indexmap", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "embedded-io" +version = "0.4.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ef1a6892d9eef45c8fa6b9e0086428a2cca8491aca8f787c534a3d6d0bcb3ced" + +[[package]] +name = "embedded-io" +version = "0.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "edd0f118536f44f5ccd48bcb8b111bdc3de888b58c74639dfb034a357d0f206d" + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "find-msvc-tools" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890" + +[[package]] +name = "fixedbitset" +version = "0.5.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d674e81391d1e1ab681a28d99df07927c6d4aa5b027d7da16ba32d1d21ecd99" + +[[package]] +name = "foldhash" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d9c4f5dac5e15c24eb999c26181a6ca40b39fe946cbe4c263c7209467bc83af2" + +[[package]] +name = "foldhash" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77ce24cb58228fbb8aa041425bb1050850ac19177686ea6e0f41a70416f56fdb" + +[[package]] +name = "hash32" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b0c35f58762feb77d74ebe43bdbc3210f09be9fe6742234d573bacc26ed92b67" +dependencies = [ + "byteorder", +] + +[[package]] +name = "hashbrown" +version = "0.15.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9229cfe53dfd69f0609a49f65461bd93001ea1ef889cd5529dd176593f5338a1" +dependencies = [ + "foldhash 0.1.5", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "heapless" +version = "0.7.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cdc6457c0eb62c71aac4bc17216026d8410337c4126773b9c5daba343f17964f" +dependencies = [ + "atomic-polyfill", + "hash32", + "rustc_version", + "serde", + "spin", + "stable_deref_trait", +] + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown 0.17.1", +] + +[[package]] +name = "libsla" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "478334afca2c29ba021109a115610eb8555bb388f32140d298dcb12585e5b147" +dependencies = [ + "libsla-sys", + "thiserror", +] + +[[package]] +name = "libsla-sys" +version = "0.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f2662124969c0a607944ebff7402f50371751ea2329a9ce21e5ef9c6c7caf47" +dependencies = [ + "cxx", + "cxx-build", +] + +[[package]] +name = "link-cplusplus" +version = "1.0.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f78c730aaa7d0b9336a299029ea49f9ee53b0ed06e9202e8cb7db9bae7b8c82" +dependencies = [ + "cc", +] + +[[package]] +name = "lock_api" +version = "0.4.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "224399e74b87b5f3557511d98dff8b14089b3dadafcab6bb93eab67d3aace965" +dependencies = [ + "scopeguard", +] + +[[package]] +name = "petgraph" +version = "0.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8701b58ea97060d5e5b155d383a69952a60943f0e6dfe30b04c287beb0b27455" +dependencies = [ + "fixedbitset", + "hashbrown 0.15.5", + "indexmap", + "serde", +] + +[[package]] +name = "postcard" +version = "1.1.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6764c3b5dd454e283a30e6dfe78e9b31096d9e32036b5d1eaac7a6119ccb9a24" +dependencies = [ + "cobs", + "embedded-io 0.4.0", + "embedded-io 0.6.1", + "heapless", + "serde", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r2il" +version = "0.1.0" +dependencies = [ + "postcard", + "serde", + "thiserror", +] + +[[package]] +name = "r2sleigh-lift" +version = "0.1.0" +dependencies = [ + "libsla", + "r2il", + "sleigh-config", + "thiserror", +] + +[[package]] +name = "r2ssa" +version = "0.1.0" +dependencies = [ + "petgraph", + "r2il", + "r2sleigh-lift", + "serde", + "thiserror", +] + +[[package]] +name = "ruff_r2il" +version = "0.1.0" +dependencies = [ + "r2il", + "r2sleigh-lift", + "r2ssa", + "sleigh-config", +] + +[[package]] +name = "rustc_version" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cfcb3a22ef46e85b45de6ee7e79d063319ebb6594faafcf1c225ea92ab6e9b92" +dependencies = [ + "semver", +] + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "scratch" +version = "1.0.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d68f2ec51b097e4c1a75b681a8bec621909b5e91f15bb7b840c4f2f7b01148b2" + +[[package]] +name = "semver" +version = "1.0.28" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8a7852d02fc848982e0c167ef163aaff9cd91dc640ba85e263cb1ce46fae51cd" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "shlex" +version = "2.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba" + +[[package]] +name = "sleigh-compiler" +version = "2.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c7855c5d0f953269587bce3df2948ea7aa1ada7040603109b3d0628f85717ea" +dependencies = [ + "cxx", + "cxx-build", + "libsla-sys", + "thiserror", +] + +[[package]] +name = "sleigh-config" +version = "1.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "36c03500e884ff5312b820b4cdd329e119522ec109b7869333d4a889bdb62fe1" +dependencies = [ + "heck", + "sleigh-compiler", +] + +[[package]] +name = "spin" +version = "0.9.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3763264f6b73151db08c50ff20d7d8a0b8796e021cdea7ceedad07b80155fa0e" +dependencies = [ + "lock_api", +] + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "3.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53e9bae58849f64dfa4f5d5ae372c8341f7305f82a3868709269343628b659a3" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "termcolor" +version = "1.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06794f8f6c5c898b3275aebefa6b8a1cb24cd2c6c79397ab15774837a0bc5755" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "thiserror" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec86235f5fcc2a73650310756d2ac5b138a5780bbbdfae3eeccec992c435ba4f" +dependencies = [ + "thiserror-impl", +] + +[[package]] +name = "thiserror-impl" +version = "2.0.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc04cd3e1236dd4a98afca4569f2deb3f120e5422a4023be2cb683f8486292af" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unicode-width" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254" + +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] diff --git a/crates/ruff_r2il/Cargo.toml b/crates/ruff_r2il/Cargo.toml new file mode 100644 index 0000000000000..61743f6da23a0 --- /dev/null +++ b/crates/ruff_r2il/Cargo.toml @@ -0,0 +1,42 @@ +[package] +name = "ruff_r2il" +version = "0.1.0" +publish = false +# Excluded from the ruff workspace (root `exclude`), so nothing here can inherit from +# [workspace.package] / [workspace.dependencies] / [workspace.lints]. edition/rust-version are +# pinned by hand to satisfy BOTH sides: ruff (edition 2024, rust-version 1.93, toolchain 1.97.1) +# and r2sleigh (edition 2024). +edition = "2024" +rust-version = "1.93" +description = "R2IL intake arm: a lossless ore carrier over r2sleigh's r2il/r2ssa object graph, a furnace that melts it into flat facet-addressed concern rows, and an addressed residual (slag) ledger. Ore in, named slag out — no catch-all, nothing dropped." + +[lib] +name = "ruff_r2il" + +[dependencies] +# The AdaWorldAPI fork checkout, consumed by path (P0 fork rule). DEFAULT deps: the typed +# behavioral surface is the crate's reason to exist. +r2il = { path = "../../../r2sleigh/crates/r2il" } +r2ssa = { path = "../../../r2sleigh/crates/r2ssa" } + +# Non-default `lift`, mirroring the ruff_cpp_spo shape. NOTE the honest difference: r2ssa depends +# on r2sleigh-lift unconditionally, so this feature does NOT keep libsla out of the build — it +# gates only this crate's DIRECT use of the disassembler and the .sla/.pspec data. +r2sleigh-lift = { path = "../../../r2sleigh/crates/r2sleigh-lift", optional = true } +sleigh-config = { version = "1.0", optional = true, features = ["x86"] } + +[features] +default = [] +lift = ["dep:r2sleigh-lift", "dep:sleigh-config"] + +[[example]] +name = "harvest_r2il" +required-features = ["lift"] + +[[example]] +name = "r2il_corpus_profile" +required-features = ["lift"] + +[lints.rust] +unsafe_code = "forbid" +unreachable_pub = "warn" diff --git a/crates/ruff_r2il/examples/harvest_r2il.rs b/crates/ruff_r2il/examples/harvest_r2il.rs new file mode 100644 index 0000000000000..55efca3b4bcf9 --- /dev/null +++ b/crates/ruff_r2il/examples/harvest_r2il.rs @@ -0,0 +1,1142 @@ +//! THE R2IL pass-1 harvest — the deliverable artifact set (impl-spec §11). +//! +//! Lifts the same small, fixed x86-64 ELF64 corpus `r2il_corpus_profile.rs` (§12) profiles — +//! `r2sleigh/tests/e2e/{stress_test,stress_test_opt}` (symtab present) plus `/bin/ls` and +//! `/usr/bin/env` (stripped, function-level harvest skipped with a printed note) — but instead of +//! printing a profile, runs the real pipeline **per function**: a minimal ELF64 section/symtab +//! walk locates `STT_FUNC` boundaries, `Disassembler::lift`/`lift_block` (16-byte padded windows) +//! rebuilds basic blocks, `FunctionBehavior::from_blocks_raw` ingests them, and +//! `furnace::smelt(&behavior, &blocks, &conv)` melts them against ONE `R2ilConvention::from_arch` +//! built from the real x86-64 `ArchSpec` and the pass-1 seven (`Copy, IntAdd, Load, Store, +//! CBranch, Call, Return`). Results accumulate across every harvested function into six artifacts +//! written to `.claude/harvest/r2il/` (override via `R2IL_HARVEST_OUT`): +//! +//! - `r2il-pass1.ore.tsv` — one row per melted `FlatFact`, `smelt` order, `at` as 32 hex chars +//! - `r2il-pass1-census.md` — `furnace::census()` by fact kind / by opcode (`BTreeMap` order) +//! - `r2il-pass1-slag.tsv` — `ResidualLedger::grouped()` + `by_address()`, merged across +//! every harvested function +//! - `r2il-convention.toml` — `R2ilConvention::to_toml()`, the one convention every function used +//! - `PROVENANCE.md` — corpus manifest (FNV-1a 64, not a sha), commit pin, env caps +//! - `TRIAGE-RESULT.md` — the three pre-registered bars B1/B2/B3, stated BEFORE the measured +//! section, plus the non-bar 60-80% Op-classify prediction +//! +//! **These artifacts are evidence, never a re-ingest path — nothing in ruff parses them back.** +//! +//! No `unwrap`/`panic` on corpus input: every ELF field read is bounds-checked and returns +//! `Option`; a malformed, unreadable, non-x86-64, or stripped binary is skipped with a printed +//! note, never a crash. +//! +//! Run: +//! ```sh +//! cargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example harvest_r2il +//! ``` + +// The workspace `clippy.toml` disallows `std::fs::*` and `std::env::var` with the reason "Use +// System::… instead **in ty crates**" — that policy exists so ty routes filesystem access through +// its `System` virtual-filesystem abstraction for testability. This is not a ty crate: `ruff_r2il` +// is workspace-EXCLUDED and does not (and should not) depend on `ty`, so `System` is unreachable +// here. The writes below are the deliverable itself — real harvest artifacts on real disk — and +// the env reads are the documented `R2IL_HARVEST_*` overrides. Suppressed narrowly and visibly +// (`expect`, per the repo's "prefer expect over allow" rule) rather than evaded by reaching for a +// non-disallowed API, which would hide that this crate sits outside the policy. +#![expect( + clippy::disallowed_methods, + reason = "not a ty crate: `System` is unavailable to this workspace-excluded crate, and these are the example's real artifact writes and documented env overrides" +)] + +use std::collections::{BTreeMap, BTreeSet}; +use std::env; +use std::fs; +use std::path::PathBuf; + +use r2il::{ArchSpec, R2ILBlock, R2ILOp, SpaceId}; +use r2sleigh_lift::{Disassembler, build_arch_spec, userop_map_for_arch}; + +use ruff_r2il::behavior::FunctionBehavior; +use ruff_r2il::convention::R2ilConvention; +use ruff_r2il::facet::FacetPrefix; +use ruff_r2il::furnace::{self, Census, FactKind, FlatFact, HarvestReport}; +use ruff_r2il::ore::OpTag; + +// ================================================================================================ +// Caps + FNV-1a 64 (labelled FNV — never a cryptographic hash; no hashing dependency). +// ================================================================================================ + +const DEFAULT_MAX_FUNCS: usize = 200; +const DEFAULT_MAX_SECTION_BYTES: usize = 262_144; + +/// Minimum bytes libsla needs per lift call. Mirrors the private +/// `r2sleigh_lift::disasm::Disassembler::MIN_BYTES` (disasm.rs:301, see impl-spec §2) — every +/// window handed to `lift`/`lift_block` below is zero-padded up to this length. +const LIFT_MIN_BYTES: usize = 16; + +const FNV_OFFSET_BASIS_64: u64 = 0xcbf2_9ce4_8422_2325; +const FNV_PRIME_64: u64 = 0x0000_0100_0000_01b3; + +/// FNV-1a 64 over raw bytes. Not a sha; labelled FNV throughout `PROVENANCE.md`. +fn fnv1a64(bytes: &[u8]) -> u64 { + let mut hash = FNV_OFFSET_BASIS_64; + for &b in bytes { + hash ^= u64::from(b); + hash = hash.wrapping_mul(FNV_PRIME_64); + } + hash +} + +fn env_cap(name: &str, default: usize) -> usize { + env::var(name) + .ok() + .and_then(|v| v.trim().parse::().ok()) + .unwrap_or(default) +} + +/// The pass-1 seven — the exact `OpTag` set both `minimal_pass_one()` and this harvest's +/// `R2ilConvention::from_arch` classify (impl-spec §5, §11's B2/prediction). +fn pass_one_seven() -> [OpTag; 7] { + [ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + ] +} + +/// `ResidualReason` stable snake_case names (§8's `ResidualReason::ALL`/`as_str`) that can ONLY +/// fire on a row whose parent op is one of the pass-1 seven — see the derivation note this const +/// feeds into `TRIAGE-RESULT.md`'s B2 section. `ResidualFact` (§8) does not carry its parent's +/// `OpTag` directly, so B2's residual-side count is reconstructed from the furnace's own pass-1 +/// ladder (§7): only the seven ever melt as an `Op` row in the first place, so every row whose +/// residual reason is NOT one of `opcode_not_in_convention` / `user_op_not_in_convention` / +/// `phi_fan_in_exceeds_predecessors` / `variadic_arity` / `no_facet_coordinate` — reasons that can +/// only fire on a non-seven opcode or a row with no parent op at all — must belong to a +/// seven-opcode parent. Labelled an APPROXIMATION in `TRIAGE-RESULT.md`, not an exact count. +/// Merged `ResidualLedger::grouped()` rows, keyed by `(shape_id.0, reason)` -> `(count, example +/// facet bytes)`. Aliased because the tuple-in-map shape trips `clippy::type_complexity` at both +/// its definition and its render-fn parameter. +type ReasonBucket = BTreeMap<(u64, &'static str), (usize, Option<[u8; 16]>)>; + +/// Merged `ResidualLedger::by_address()` rows, keyed by `(resolved prefix, shape_id.0)` -> count. +type AddrBucket = BTreeMap<(Option, u64), usize>; + +const SEVEN_ELIGIBLE_RESIDUAL_REASONS: &[&str] = &[ + "no_convention_row_at_address", + "indirect_target", + "memory_object_escaped", + "op_site_join_mismatch", + "custom_space_not_in_convention", + "facet_overflow_at_key", +]; + +// ================================================================================================ +// mod elf — minimal, no-deps ELF64 LE reader. Every read bounds-checked; anything malformed +// returns `None` and the caller skips the whole binary with a printed note. Duplicated (not +// shared) against `r2il_corpus_profile.rs`'s own copy — W9 and W10 own disjoint files. +// ================================================================================================ + +mod elf { + const EI_CLASS_64: u8 = 2; + const EI_DATA_LE: u8 = 1; + const EM_X86_64: u16 = 62; + const SHT_SYMTAB: u32 = 2; + const SHT_NOBITS: u32 = 8; + const SHF_EXECINSTR: u64 = 0x4; + const STT_FUNC: u8 = 2; + + pub(crate) struct Section { + pub name: String, + pub sh_type: u32, + pub flags: u64, + pub addr: u64, + pub offset: u64, + pub size: u64, + } + + impl Section { + pub(crate) fn is_exec(&self) -> bool { + self.flags & SHF_EXECINSTR != 0 && self.sh_type != SHT_NOBITS + } + + pub(crate) fn contains_range(&self, addr: u64, size: u64) -> bool { + addr >= self.addr && addr.saturating_add(size) <= self.addr.saturating_add(self.size) + } + } + + pub(crate) struct FuncSym { + pub name: String, + pub value: u64, + pub size: u64, + } + + pub(crate) struct ElfInfo { + pub sections: Vec
, + /// `STT_FUNC` symbols with `st_size > 0`, each already verified to sit inside a single + /// executable section. Sorted, deduped by address. + pub functions: Vec, + pub has_symtab: bool, + } + + fn u16le(d: &[u8], off: usize) -> Option { + let end = off.checked_add(2)?; + Some(u16::from_le_bytes(d.get(off..end)?.try_into().ok()?)) + } + fn u32le(d: &[u8], off: usize) -> Option { + let end = off.checked_add(4)?; + Some(u32::from_le_bytes(d.get(off..end)?.try_into().ok()?)) + } + fn u64le(d: &[u8], off: usize) -> Option { + let end = off.checked_add(8)?; + Some(u64::from_le_bytes(d.get(off..end)?.try_into().ok()?)) + } + + fn cstr_at(strtab: &[u8], off: usize) -> String { + if off >= strtab.len() { + return String::new(); + } + let end = strtab[off..] + .iter() + .position(|&b| b == 0) + .map_or(strtab.len(), |p| off + p); + String::from_utf8_lossy(&strtab[off..end]).into_owned() + } + + struct RawShdr { + name: u32, + sh_type: u32, + flags: u64, + addr: u64, + offset: u64, + size: u64, + link: u32, + entsize: u64, + } + + /// Parse an ELF64 LE x86-64 binary's header, section headers, and `STT_FUNC` symbol table. + /// Returns `None` the instant any read is out of bounds or a fixed-field check fails. + pub(crate) fn parse(data: &[u8]) -> Option { + if data.len() < 64 { + return None; + } + if &data[0..4] != b"\x7FELF" { + return None; + } + if data[4] != EI_CLASS_64 || data[5] != EI_DATA_LE { + return None; + } + if u16le(data, 18)? != EM_X86_64 { + return None; + } + let e_shoff = u64le(data, 40)? as usize; + let e_shentsize = u16le(data, 58)? as usize; + let e_shnum = u16le(data, 60)? as usize; + let e_shstrndx = u16le(data, 62)? as usize; + if e_shentsize < 64 || e_shnum == 0 { + return None; + } + + let mut raw: Vec = Vec::with_capacity(e_shnum); + for i in 0..e_shnum { + let rel = i.checked_mul(e_shentsize)?; + let base = e_shoff.checked_add(rel)?; + raw.push(RawShdr { + name: u32le(data, base)?, + sh_type: u32le(data, base + 4)?, + flags: u64le(data, base + 8)?, + addr: u64le(data, base + 16)?, + offset: u64le(data, base + 24)?, + size: u64le(data, base + 32)?, + link: u32le(data, base + 40)?, + entsize: u64le(data, base + 56)?, + }); + } + + let shstrtab: &[u8] = if e_shstrndx < raw.len() { + let s = &raw[e_shstrndx]; + let start = s.offset as usize; + let end = start.checked_add(s.size as usize)?; + data.get(start..end).unwrap_or(&[]) + } else { + &[] + }; + + let sections: Vec
= raw + .iter() + .map(|s| Section { + name: cstr_at(shstrtab, s.name as usize), + sh_type: s.sh_type, + flags: s.flags, + addr: s.addr, + offset: s.offset, + size: s.size, + }) + .collect(); + + let mut functions: Vec = Vec::new(); + let mut has_symtab = false; + if let Some(symtab) = raw.iter().find(|s| s.sh_type == SHT_SYMTAB) { + has_symtab = true; + if let Some(strtab_sec) = raw.get(symtab.link as usize) { + let str_start = strtab_sec.offset as usize; + let str_end = str_start + .checked_add(strtab_sec.size as usize) + .unwrap_or(str_start); + let strtab = data.get(str_start..str_end).unwrap_or(&[]); + let entsize = if symtab.entsize == 0 { + 24 + } else { + symtab.entsize as usize + }; + if entsize > 0 { + let count = (symtab.size as usize).checked_div(entsize).unwrap_or(0); + for i in 0..count { + let Some(rel) = i.checked_mul(entsize) else { + break; + }; + let Some(base_u64) = symtab.offset.checked_add(rel as u64) else { + break; + }; + let base = base_u64 as usize; + let Some(st_name) = u32le(data, base) else { + break; + }; + let Some(&st_info) = data.get(base + 4) else { + break; + }; + let Some(st_value) = u64le(data, base + 8) else { + break; + }; + let Some(st_size) = u64le(data, base + 16) else { + break; + }; + if st_info & 0xF != STT_FUNC || st_size == 0 { + continue; + } + let in_exec = sections + .iter() + .any(|sec| sec.is_exec() && sec.contains_range(st_value, st_size)); + if in_exec { + functions.push(FuncSym { + name: cstr_at(strtab, st_name as usize), + value: st_value, + size: st_size, + }); + } + } + } + } + } + functions.sort_by_key(|f| f.value); + functions.dedup_by_key(|f| f.value); + + Some(ElfInfo { + sections, + functions, + has_symtab, + }) + } +} + +// ================================================================================================ +// Corpus resolution — mirrors r2il_corpus_profile.rs (§12): env override, then CLI args, then the +// fixed default four. +// ================================================================================================ + +fn corpus_paths() -> Vec { + if let Ok(v) = env::var("R2IL_CORPUS") { + let paths: Vec = v + .split(':') + .filter(|s| !s.is_empty()) + .map(PathBuf::from) + .collect(); + if !paths.is_empty() { + return paths; + } + } + let args: Vec = env::args().skip(1).collect(); + if !args.is_empty() { + return args.into_iter().map(PathBuf::from).collect(); + } + let base = concat!(env!("CARGO_MANIFEST_DIR"), "/../../../r2sleigh/tests/e2e/"); + vec![ + PathBuf::from(format!("{base}stress_test")), + PathBuf::from(format!("{base}stress_test_opt")), + PathBuf::from("/bin/ls"), + PathBuf::from("/usr/bin/env"), + ] +} + +// ================================================================================================ +// Leader / basic-block detection + lift — the §12 "function boundaries from symtab, leaders +// approximate" method, reused here to feed `FunctionBehavior::from_blocks_raw`. +// ================================================================================================ + +fn pad_window(bytes: &[u8], want: usize) -> Vec { + let want = want.max(LIFT_MIN_BYTES); + let mut w: Vec = bytes.iter().take(want).copied().collect(); + if w.len() < want { + w.resize(want, 0); + } + w +} + +/// leaders = `{func_addr}` ∪ intra-function const branch/call targets ∪ `{addr after any +/// control-flow op}`. A single-instruction linear sweep via `disasm.lift`, exactly like §12's +/// Pass 1 op-level sweep, just also recording leaders as it goes. +fn find_leaders(disasm: &Disassembler, code: &[u8], func_addr: u64) -> BTreeSet { + let func_end = func_addr + code.len() as u64; + let mut leaders: BTreeSet = BTreeSet::new(); + leaders.insert(func_addr); + + let mut offset = 0usize; + while offset < code.len() { + let addr = func_addr + offset as u64; + let window = pad_window(&code[offset..], LIFT_MIN_BYTES); + match disasm.lift(&window, addr) { + Ok(block) => { + let size = (block.size as usize).max(1); + let mut had_cf = false; + for op in &block.ops { + if op.is_control_flow() { + had_cf = true; + } + let target = match op { + R2ILOp::Branch { target } => Some(target), + R2ILOp::CBranch { target, .. } => Some(target), + R2ILOp::Call { target } => Some(target), + _ => None, + }; + if let Some(t) = target + && t.space == SpaceId::Const + && t.offset >= func_addr + && t.offset < func_end + { + leaders.insert(t.offset); + } + } + if had_cf { + let after = addr + size as u64; + if after < func_end { + leaders.insert(after); + } + } + offset += size; + } + Err(_) => { + offset += 1; + } + } + } + leaders +} + +/// Basic blocks = maximal ranges between leaders, the last one running to the function's end. +/// One `disasm.lift_block` per range — `R2ILBlock`s ready for `FunctionBehavior::from_blocks_raw`. +fn lift_function_blocks(disasm: &Disassembler, code: &[u8], func_addr: u64) -> Vec { + if code.is_empty() { + return Vec::new(); + } + let func_end = func_addr + code.len() as u64; + let leaders = find_leaders(disasm, code, func_addr); + let mut boundaries: Vec = leaders.into_iter().collect(); + if boundaries.last() != Some(&func_end) { + boundaries.push(func_end); + } + boundaries.dedup(); + + let mut blocks = Vec::new(); + for w in boundaries.windows(2) { + let bb_addr = w[0]; + let bb_end = w[1]; + if bb_end <= bb_addr { + continue; + } + let bb_len = (bb_end - bb_addr) as usize; + let start_off = (bb_addr - func_addr) as usize; + if start_off >= code.len() { + continue; + } + let end_off = (start_off + bb_len).min(code.len()); + let raw = &code[start_off..end_off]; + let window = pad_window(raw, bb_len); + if let Ok(block) = disasm.lift_block(&window, bb_addr, bb_len) + && !block.ops.is_empty() + { + blocks.push(block); + } + } + blocks +} + +// ================================================================================================ +// Accumulators +// ================================================================================================ + +struct OreRow { + binary: String, + func_hex: String, + fact: FlatFact, +} + +struct CorpusEntry { + path: String, + len: usize, + fnv: Option, + status: String, +} + +struct Accum { + rows: Vec, + /// Merged `ResidualLedger::grouped()` output across every harvested function, keyed by + /// `(shape_id.0, reason)`. + reason_bucket: ReasonBucket, + /// Merged `ResidualLedger::by_address()` output across every harvested function. + addr_bucket: AddrBucket, + combined: HarvestReport, + remaining_func_budget: usize, +} + +// ================================================================================================ +// Per-binary / per-function driver +// ================================================================================================ + +/// The three handles that stay INVARIANT across every function of every binary: one +/// `Disassembler`, one `ArchSpec`, one `R2ilConvention` (built once by `from_arch`, per §11). +/// Bundled so the per-binary driver takes a context rather than eight positional parameters — +/// the grouping is the readable shape, and it also settles `clippy::too_many_arguments`. +struct LiftCtx<'a> { + disasm: &'a Disassembler, + spec: &'a ArchSpec, + conv: &'a R2ilConvention, +} + +fn process_elf_symtab_functions( + ctx: &LiftCtx<'_>, + binary_label: &str, + data: &[u8], + info: &elf::ElfInfo, + max_section_bytes: usize, + accum: &mut Accum, +) -> usize { + let mut processed = 0usize; + for func in &info.functions { + if accum.remaining_func_budget == 0 { + eprintln!( + "[harvest] {binary_label}: function budget (R2IL_HARVEST_MAX_FUNCS) exhausted, stopping this binary" + ); + break; + } + + let Some(sec) = info + .sections + .iter() + .find(|s| s.is_exec() && s.contains_range(func.value, func.size)) + else { + eprintln!( + "[harvest] {binary_label}: fn @ {:#x} not inside a single exec section, skip", + func.value + ); + continue; + }; + + let sec_size = (sec.size as usize).min(max_section_bytes); + let sec_start = sec.offset as usize; + let Some(sec_end) = sec_start.checked_add(sec_size) else { + continue; + }; + let Some(sec_bytes) = data.get(sec_start..sec_end) else { + eprintln!( + "[harvest] {binary_label}: section '{}' out of file bounds, skip", + sec.name + ); + continue; + }; + + if func.value < sec.addr { + continue; + } + let func_off_in_sec = (func.value - sec.addr) as usize; + if func_off_in_sec >= sec_bytes.len() { + eprintln!( + "[harvest] {binary_label}: fn '{}' @ {:#x} falls outside the {max_section_bytes} byte section cap, skip", + func.name, func.value + ); + continue; + } + let func_end_in_sec = func_off_in_sec + .saturating_add(func.size as usize) + .min(sec_bytes.len()); + let code = &sec_bytes[func_off_in_sec..func_end_in_sec]; + if code.is_empty() { + continue; + } + + let blocks = lift_function_blocks(ctx.disasm, code, func.value); + if blocks.is_empty() { + eprintln!( + "[harvest] {binary_label}: fn '{}' @ {:#x} produced no liftable blocks, skip", + func.name, func.value + ); + continue; + } + + let Some(behavior) = FunctionBehavior::from_blocks_raw(&blocks, Some(ctx.spec)) else { + eprintln!( + "[harvest] {binary_label}: fn '{}' @ {:#x} — from_blocks_raw returned None, skip", + func.name, func.value + ); + continue; + }; + let behavior = behavior.with_name(func.name.clone()); + + let (facts, ledger, report) = furnace::smelt(&behavior, &blocks, ctx.conv); + + accum.combined.harvested += report.harvested; + accum.combined.classified += report.classified; + accum.combined.residual += report.residual; + accum.combined.dropped += report.dropped; + + let func_hex = format!("{:#x}", func.value); + for fact in &facts { + accum.rows.push(OreRow { + binary: binary_label.to_string(), + func_hex: func_hex.clone(), + fact: *fact, + }); + } + + for (shape_id, reason, count, example) in ledger.grouped() { + let entry = accum + .reason_bucket + .entry((shape_id.0, reason)) + .or_insert((0usize, None)); + entry.0 += count; + if entry.1.is_none() { + entry.1 = example.map(|f| f.0); + } + } + for (prefix, shape_id, count) in ledger.by_address() { + *accum.addr_bucket.entry((prefix, shape_id.0)).or_insert(0) += count; + } + + accum.remaining_func_budget -= 1; + processed += 1; + } + processed +} + +// ================================================================================================ +// Rendering — TSV / Markdown / TOML. Every header states "evidence, never a re-ingest path". +// ================================================================================================ + +fn hex16(bytes: &[u8; 16]) -> String { + let mut s = String::with_capacity(32); + for b in bytes { + s.push_str(&format!("{b:02x}")); + } + s +} + +fn opt_dbg(v: &Option) -> String { + match v { + Some(x) => format!("{x:?}"), + None => "-".to_string(), + } +} + +fn facet_prefix_str(p: &Option) -> String { + match p { + None => "-".to_string(), + Some(FacetPrefix::Space { discriminant }) => format!("space:{discriminant}"), + Some(FacetPrefix::SpaceOffset { + discriminant, + offset, + }) => format!("space:{discriminant}/offset:{offset:#x}"), + Some(FacetPrefix::SpaceOffsetSize { + discriminant, + offset, + size, + }) => format!("space:{discriminant}/offset:{offset:#x}/size:{size}"), + } +} + +fn render_ore_tsv(rows: &[OreRow]) -> String { + let mut out = String::new(); + out.push_str( + "# R2IL pass-1 ore — evidence, never a re-ingest path. Nothing in ruff parses this back.\n", + ); + out.push_str( + "#schema binary\tfunction\tfact_id\tat\tconcern\tkind\topcode\ta\tb\tprov_inst\tprov_block\tprov_op_site\tprov_value\n", + ); + out.push_str("#version 1\n"); + for row in rows { + let f = &row.fact; + let op_site = match f.prov.op_site { + Some((addr, idx)) => format!("{addr:#x}:{idx}"), + None => "-".to_string(), + }; + out.push_str(&format!( + "{}\t{}\t{}\t{}\t{:?}\t{:?}\t{}\t{}\t{}\t{}\t{}\t{op_site}\t{}\n", + row.binary, + row.func_hex, + f.id.0, + hex16(&f.at.0), + f.concern, + f.kind, + f.opcode.as_str(), + f.a, + f.b, + opt_dbg(&f.prov.inst), + opt_dbg(&f.prov.block), + opt_dbg(&f.prov.value), + )); + } + out +} + +fn render_census_md(all_facts: &[FlatFact]) -> String { + let census: Census = furnace::census(all_facts); + let mut out = String::new(); + out.push_str("# R2IL pass-1 census\n\n"); + out.push_str(&format!( + "Total classified `FlatFact` rows: {}\n\n", + all_facts.len() + )); + out.push_str("## By fact kind\n\n| kind | count |\n|---|---|\n"); + for (k, v) in &census.by_fact_kind { + out.push_str(&format!("| {k} | {v} |\n")); + } + out.push_str("\n## By opcode\n\n| opcode | count |\n|---|---|\n"); + for (k, v) in &census.by_opcode { + out.push_str(&format!("| {k} | {v} |\n")); + } + out +} + +fn render_slag_tsv(reason_bucket: &ReasonBucket, addr_bucket: &AddrBucket) -> String { + let mut grouped: Vec<(u64, &'static str, usize, Option<[u8; 16]>)> = Vec::new(); + for (&(shape_id, reason), &(count, example)) in reason_bucket.iter() { + grouped.push((shape_id, reason, count, example)); + } + grouped.sort_by(|a, b| b.2.cmp(&a.2).then_with(|| a.0.cmp(&b.0))); + + let mut by_addr: Vec<(Option, u64, usize)> = Vec::new(); + for (&(prefix, shape_id), &count) in addr_bucket.iter() { + by_addr.push((prefix, shape_id, count)); + } + by_addr.sort_by(|a, b| b.2.cmp(&a.2).then_with(|| a.1.cmp(&b.1))); + + let mut out = String::new(); + out.push_str( + "# R2IL pass-1 slag — the addressed residual ledger. Evidence, never a re-ingest path.\n", + ); + out.push_str("#version 1\n"); + out.push_str( + "#section grouped — ResidualLedger::grouped(), merged across every harvested function\n", + ); + out.push_str("#schema section\tshape_id\treason\tcount\texample_facet\n"); + for (shape_id, reason, count, example) in &grouped { + let facet = match example { + Some(bytes) => hex16(bytes), + None => "-".to_string(), + }; + out.push_str(&format!( + "grouped\t{shape_id:016x}\t{reason}\t{count}\t{facet}\n" + )); + } + out.push_str( + "#section by_address — ResidualLedger::by_address(), the proposer's work queue, merged across every harvested function\n", + ); + out.push_str("#schema section\tprefix\tshape_id\tcount\n"); + for (prefix, shape_id, count) in &by_addr { + out.push_str(&format!( + "by_address\t{}\t{shape_id:016x}\t{count}\n", + facet_prefix_str(prefix) + )); + } + out +} + +fn render_provenance_md( + entries: &[CorpusEntry], + arch: &str, + max_funcs: usize, + max_section_bytes: usize, +) -> String { + let mut out = String::new(); + out.push_str("# R2IL pass-1 harvest — PROVENANCE\n\n"); + out.push_str( + "Hashes below are **FNV-1a 64** (not a cryptographic hash) over the raw file bytes, computed inline in this example — no hashing dependency.\n\n", + ); + out.push_str("## Corpus\n\n| path | bytes | fnv1a64 | status |\n|---|---|---|---|\n"); + for e in entries { + let fnv = e + .fnv + .map_or_else(|| "-".to_string(), |h| format!("{h:016x}")); + out.push_str(&format!( + "| {} | {} | {} | {} |\n", + e.path, e.len, fnv, e.status + )); + } + out.push_str("\n## Environment\n\n"); + out.push_str("- `r2sleigh` commit: `60942f6`\n"); + out.push_str(&format!("- Architecture: `{arch}`\n")); + out.push_str( + "- `sleigh-config` = \"1.0\", feature `x86` (exact resolved patch pinned by the committed `Cargo.lock`)\n", + ); + out.push_str( + "- Convention: `R2ilConvention::from_arch(&spec, [Copy, IntAdd, Load, Store, CBranch, Call, Return])` — one convention, built once, reused for every harvested function\n", + ); + out.push_str(&format!( + "- Caps in force: `R2IL_HARVEST_MAX_FUNCS={max_funcs}`, `R2IL_HARVEST_MAX_SECTION_BYTES={max_section_bytes}`\n", + )); + out.push_str("\n## Invocation\n\n```sh\ncargo run --manifest-path crates/ruff_r2il/Cargo.toml --features lift --example harvest_r2il\n```\n"); + out +} + +#[allow(clippy::too_many_arguments)] +fn render_triage_md( + max_funcs: usize, + max_section_bytes: usize, + combined: &HarvestReport, + functions_processed: usize, + classified_seven: usize, + residual_seven_eligible: usize, + dominant_share: f64, + distinct_shapes: usize, + residual_addressed_note: &str, + op_classified: usize, + op_total: usize, +) -> String { + let mut out = String::new(); + out.push_str("# R2IL pass-1 TRIAGE RESULT\n\n"); + out.push_str(&format!( + "Caps in force: `R2IL_HARVEST_MAX_FUNCS={max_funcs}`, `R2IL_HARVEST_MAX_SECTION_BYTES={max_section_bytes}`\n\n", + )); + + out.push_str("## Pre-registered bars (stated BEFORE the measured section below)\n\n"); + out.push_str( + "- **B1 — conservation (absolute).** `dropped == 0` and `harvested == classified + residual`. Any violation **KILLS** the pass: the enumerator, not the corpus, is wrong. Not a percentage.\n", + ); + out.push_str( + "- **B2 — coverage of the declared seven.** Of ore facts whose parent opcode is one of `{Copy, IntAdd, Load, Store, CBranch, Call, Return}`, **>=99% classify -> PASS; <90% -> KILL.** The 90-99% band is INVESTIGATE (expected causes: operand rows with no convention row at their address, `CallSite` rows with no `direct_target` — both legitimate slag under a parent that classified).\n", + ); + out.push_str( + "- **B3 — the slag is named and addressed, not lumped.** `residual > 0`, distinct `shape_id` count **>= 5**, `dominant_share() < 0.60`, and **every** residual except `NoFacetCoordinate` carries `at.is_some()`. `residual == 0` is a **KILL** too — it means someone widened the ladder.\n\n", + ); + out.push_str( + "Also **pre-register a prediction that is NOT a bar** (so it can be wrong without moving a goalpost): on an x86-64 corpus `Copy/IntAdd/Load/Store` dominate, so pass 1 is expected to classify roughly **60-80%** of all `Op` facts. Record the measured figure either way.\n\n", + ); + out.push_str("---\n\n## Measured\n\n"); + + out.push_str(&format!("Functions harvested: {functions_processed}\n\n")); + out.push_str(&format!( + "Conservation line: harvested {} / classified {} / residual {} / dropped {}\n\n", + combined.harvested, combined.classified, combined.residual, combined.dropped + )); + + let b1_dropped = combined.dropped == 0; + let b1_conserved = combined.harvested == combined.classified + combined.residual; + let b1 = b1_dropped && b1_conserved; + out.push_str(&format!( + "**B1: {}** — dropped == 0: {b1_dropped}; harvested == classified + residual: {b1_conserved}\n\n", + if b1 { "PASS" } else { "KILL" } + )); + + let b2_denominator = classified_seven + residual_seven_eligible; + let b2_pct = if b2_denominator == 0 { + 100.0 + } else { + classified_seven as f64 / b2_denominator as f64 * 100.0 + }; + let b2_verdict = if b2_pct >= 99.0 { + "PASS" + } else if b2_pct < 90.0 { + "KILL" + } else { + "INVESTIGATE" + }; + out.push_str(&format!( + "**B2: {b2_verdict}** — {classified_seven} classified / {b2_denominator} total ore facts under a seven-opcode parent = {b2_pct:.2}%.\n\n", + )); + out.push_str( + " Derivation note: `ResidualFact` does not carry its parent opcode directly, so the denominator's residual half is APPROXIMATED by summing residual reasons that can *only* fire on a row whose parent op is one of the seven (`no_convention_row_at_address`, `indirect_target`, `memory_object_escaped`, `op_site_join_mismatch`, `custom_space_not_in_convention`, `facet_overflow_at_key`) — reasons that can only fire on a non-seven or no-parent-op row (`opcode_not_in_convention`, `user_op_not_in_convention`, `phi_fan_in_exceeds_predecessors`, `variadic_arity`, `no_facet_coordinate`) are excluded. Labelled APPROXIMATION, not exact — see the module doc comment above `SEVEN_ELIGIBLE_RESIDUAL_REASONS`.\n\n", + ); + + let b3_residual = combined.residual > 0; + let b3_shapes = distinct_shapes >= 5; + let b3_share = dominant_share < 0.60; + let b3 = b3_residual && b3_shapes && b3_share; + out.push_str(&format!( + "**B3: {}** — residual > 0: {b3_residual} ({}); distinct shape_id count: {distinct_shapes} (>=5: {b3_shapes}); dominant_share: {dominant_share:.3} (<0.60: {b3_share}).\n\n", + if b3 { "PASS" } else { "KILL" }, + combined.residual, + )); + out.push_str(&format!(" {residual_addressed_note}\n\n")); + + let op_pct = if op_total == 0 { + 0.0 + } else { + op_classified as f64 / op_total as f64 * 100.0 + }; + let within = (60.0..=80.0).contains(&op_pct); + out.push_str(&format!( + "**Non-bar prediction, measured:** {op_classified} / {op_total} `Op` facts classified = {op_pct:.2}% (predicted 60-80%; {}).\n", + if within { + "within the predicted band" + } else { + "OUTSIDE the predicted band — recorded honestly, not a bar" + } + )); + + out +} + +// ================================================================================================ +// main +// ================================================================================================ + +fn main() -> Result<(), Box> { + let max_funcs = env_cap("R2IL_HARVEST_MAX_FUNCS", DEFAULT_MAX_FUNCS); + let max_section_bytes = env_cap("R2IL_HARVEST_MAX_SECTION_BYTES", DEFAULT_MAX_SECTION_BYTES); + + let out_dir = env::var("R2IL_HARVEST_OUT").unwrap_or_else(|_| { + concat!(env!("CARGO_MANIFEST_DIR"), "/../../.claude/harvest/r2il").to_string() + }); + fs::create_dir_all(&out_dir)?; + eprintln!("[harvest] output directory: {out_dir}"); + eprintln!( + "[harvest] caps: R2IL_HARVEST_MAX_FUNCS={max_funcs} R2IL_HARVEST_MAX_SECTION_BYTES={max_section_bytes}" + ); + + let spec = build_arch_spec( + sleigh_config::processor_x86::SLA_X86_64, + sleigh_config::processor_x86::PSPEC_X86_64, + "x86-64", + )?; + let mut disasm = Disassembler::from_sla( + sleigh_config::processor_x86::SLA_X86_64, + sleigh_config::processor_x86::PSPEC_X86_64, + "x86-64", + )?; + disasm.set_userop_map(userop_map_for_arch("x86-64")); + + let conv = R2ilConvention::from_arch(&spec, pass_one_seven()) + .map_err(|e| format!("R2ilConvention::from_arch(x86-64) failed: {e}"))?; + eprintln!( + "[harvest] convention built from {} registers, {} userops", + spec.registers.len(), + spec.userops.len() + ); + + let mut entries: Vec = Vec::new(); + let mut accum = Accum { + rows: Vec::new(), + reason_bucket: BTreeMap::new(), + addr_bucket: BTreeMap::new(), + combined: HarvestReport::default(), + remaining_func_budget: max_funcs, + }; + let mut functions_processed = 0usize; + + for path in corpus_paths() { + let label = path.display().to_string(); + let data = match fs::read(&path) { + Ok(d) => d, + Err(e) => { + eprintln!("[harvest] {label}: skip — read failed: {e}"); + entries.push(CorpusEntry { + path: label, + len: 0, + fnv: None, + status: format!("skipped (read failed: {e})"), + }); + continue; + } + }; + let fnv = fnv1a64(&data); + eprintln!( + "[harvest] {label}: {} bytes, fnv1a64={fnv:016x}", + data.len() + ); + + let Some(info) = elf::parse(&data) else { + eprintln!("[harvest] {label}: skip — not a recognized ELF64 LE x86-64 binary"); + entries.push(CorpusEntry { + path: label, + len: data.len(), + fnv: Some(fnv), + status: "skipped (not ELF64 LE x86-64)".to_string(), + }); + continue; + }; + + if !info.has_symtab { + eprintln!( + "[harvest] {label}: skip — stripped (no symtab); function-level harvest needs symtab boundaries" + ); + entries.push(CorpusEntry { + path: label, + len: data.len(), + fnv: Some(fnv), + status: "skipped (no symtab)".to_string(), + }); + continue; + } + + if accum.remaining_func_budget == 0 { + eprintln!( + "[harvest] {label}: skip — function budget (R2IL_HARVEST_MAX_FUNCS={max_funcs}) already exhausted" + ); + entries.push(CorpusEntry { + path: label, + len: data.len(), + fnv: Some(fnv), + status: "skipped (function budget exhausted)".to_string(), + }); + continue; + } + + eprintln!( + "[harvest] {label}: {} STT_FUNC symbols in executable sections", + info.functions.len() + ); + + let processed = process_elf_symtab_functions( + &LiftCtx { + disasm: &disasm, + spec: &spec, + conv: &conv, + }, + &label, + &data, + &info, + max_section_bytes, + &mut accum, + ); + functions_processed += processed; + entries.push(CorpusEntry { + path: label, + len: data.len(), + fnv: Some(fnv), + status: format!("harvested ({processed} functions)"), + }); + } + + eprintln!( + "[harvest] done: {functions_processed} functions across {} binaries, {} FlatFact rows, conservation {} / {} / {} / {}", + entries.len(), + accum.rows.len(), + accum.combined.harvested, + accum.combined.classified, + accum.combined.residual, + accum.combined.dropped + ); + + let facts_only: Vec = accum.rows.iter().map(|r| r.fact).collect(); + + // ---- artifact 1: ore.tsv ---- + let ore_path = format!("{out_dir}/r2il-pass1.ore.tsv"); + fs::write(&ore_path, render_ore_tsv(&accum.rows))?; + eprintln!("[harvest] wrote {ore_path} ({} rows)", accum.rows.len()); + + // ---- artifact 2: census.md ---- + let census_path = format!("{out_dir}/r2il-pass1-census.md"); + fs::write(&census_path, render_census_md(&facts_only))?; + eprintln!("[harvest] wrote {census_path}"); + + // ---- artifact 3: slag.tsv ---- + let slag_path = format!("{out_dir}/r2il-pass1-slag.tsv"); + fs::write( + &slag_path, + render_slag_tsv(&accum.reason_bucket, &accum.addr_bucket), + )?; + eprintln!("[harvest] wrote {slag_path}"); + + // ---- artifact 4: convention.toml ---- + let conv_path = format!("{out_dir}/r2il-convention.toml"); + fs::write(&conv_path, conv.to_toml())?; + eprintln!("[harvest] wrote {conv_path}"); + + // ---- artifact 5: PROVENANCE.md ---- + let prov_path = format!("{out_dir}/PROVENANCE.md"); + fs::write( + &prov_path, + render_provenance_md(&entries, &spec.name, max_funcs, max_section_bytes), + )?; + eprintln!("[harvest] wrote {prov_path}"); + + // ---- artifact 6: TRIAGE-RESULT.md ---- + let seven = pass_one_seven(); + let mut classified_seven = 0usize; + let mut op_classified = 0usize; + for f in &facts_only { + if seven.contains(&f.opcode) { + classified_seven += 1; + } + if matches!(f.kind, FactKind::Op) { + op_classified += 1; + } + } + + let mut distinct_shape_ids: BTreeSet = BTreeSet::new(); + let mut total_residual_grouped = 0usize; + let mut max_group = 0usize; + let mut residual_seven_eligible = 0usize; + let mut op_residual = 0usize; + let mut missing_facet_reasons: Vec = Vec::new(); + for (&(shape_id, reason), &(count, example)) in accum.reason_bucket.iter() { + distinct_shape_ids.insert(shape_id); + total_residual_grouped += count; + if count > max_group { + max_group = count; + } + if SEVEN_ELIGIBLE_RESIDUAL_REASONS.contains(&reason) { + residual_seven_eligible += count; + } + if reason == "opcode_not_in_convention" { + op_residual += count; + } + if reason != "no_facet_coordinate" && example.is_none() { + missing_facet_reasons.push(format!("shape {shape_id:016x} ({reason})")); + } + } + let dominant_share = if total_residual_grouped == 0 { + 0.0 + } else { + max_group as f64 / total_residual_grouped as f64 + }; + let distinct_shapes = distinct_shape_ids.len(); + let op_total = op_classified + op_residual; + + let residual_addressed_note = if missing_facet_reasons.is_empty() { + "Spot check: every grouped bucket except no_facet_coordinate reports an example facet address.".to_string() + } else { + format!( + "Spot check FAILED for: {}. (grouped() reports one example per shape — see slag.rs's own row-level tests for the per-row invariant this does not, by itself, disprove.)", + missing_facet_reasons.join(", ") + ) + }; + + let triage_path = format!("{out_dir}/TRIAGE-RESULT.md"); + fs::write( + &triage_path, + render_triage_md( + max_funcs, + max_section_bytes, + &accum.combined, + functions_processed, + classified_seven, + residual_seven_eligible, + dominant_share, + distinct_shapes, + &residual_addressed_note, + op_classified, + op_total, + ), + )?; + eprintln!("[harvest] wrote {triage_path}"); + + Ok(()) +} diff --git a/crates/ruff_r2il/examples/r2il_corpus_profile.rs b/crates/ruff_r2il/examples/r2il_corpus_profile.rs new file mode 100644 index 0000000000000..8e184185aae0c --- /dev/null +++ b/crates/ruff_r2il/examples/r2il_corpus_profile.rs @@ -0,0 +1,1031 @@ +//! §12 corpus profile — plain text to stdout, no serde, no file writes. +//! +//! Two independent passes over a small, fixed x86-64 ELF64 corpus: +//! +//! - **Pass 1 — `== MEASURED EXACT (op level) ==`.** A linear byte sweep of every executable +//! section, one [`r2sleigh_lift::Disassembler::lift`] call per native instruction. This is +//! EXACT at the op level: every metric here is read straight off the `R2ILOp`s libsla actually +//! produced, never inferred. +//! - **Pass 2 — `== HEURISTIC-DERIVED (function / CFG level) ==`.** Symtab-bearing binaries +//! only. `STT_FUNC` boundaries come straight from the symbol table and are EXACT; the +//! basic-block leader set inside each function (intra-function constant branch targets, plus +//! the address after every control-flow op) is an APPROXIMATION — indirect branches and jump +//! tables are not resolved. **Non-negotiable labelling rule:** every row this pass prints is +//! under the `HEURISTIC-DERIVED` heading, and the heading is paired with the exact caveat +//! sentence below, every time. A stripped binary (no symtab) never contributes a heuristic row +//! at all — it prints the explicit skip line instead of a silent omission. +//! +//! Nothing here is copied into ruff: the corpus lives outside the repository (siblings under +//! `r2sleigh/tests/e2e/`, or system binaries), and this example only ever reads it. +//! +//! No `unwrap`/`panic` on corpus input: every ELF field read is bounds-checked and returns +//! `Option`; a malformed or unreadable binary is skipped with a printed note, never a crash. +//! Every metric in both passes is read from `R2ILOp`'s own accessors (`inputs()`, `output()`, +//! `is_control_flow()`, `is_memory_read()`, `is_memory_write()`, and a typed `match` on the op's +//! own variant for the atomic check) — never from `Display`/`Debug` formatting of an op. + +// The workspace `clippy.toml` disallows `std::env::var` with the reason "Use System::env_var +// instead **in ty crates**" — that policy exists so ty reads configuration through its `System` +// abstraction for testability. This is not a ty crate: `ruff_r2il` is workspace-EXCLUDED and does +// not depend on `ty`, so `System` is unreachable here. The two reads below are the documented +// `R2IL_CORPUS` / `R2IL_PROFILE_*` overrides this example advertises in its own module docs. +// Suppressed narrowly and visibly (`expect`, per the repo's "prefer expect over allow" rule). +#![expect( + clippy::disallowed_methods, + reason = "not a ty crate: `System` is unavailable to this workspace-excluded crate; these are the example's documented env overrides" +)] + +use std::collections::BTreeMap; +use std::env; +use std::fs; +use std::path::{Path, PathBuf}; + +use r2il::{ArchSpec, R2ILBlock, R2ILOp, SpaceId}; +use r2sleigh_lift::{Disassembler, build_arch_spec, userop_map_for_arch}; + +use ruff_r2il::behavior::FunctionBehavior; +use ruff_r2il::convention::R2ilConvention; +use ruff_r2il::facet; +use ruff_r2il::furnace; +use ruff_r2il::ore::OpTag; +use ruff_r2il::vocab::VocabHarvest; + +/// Minimum bytes libsla needs per lift call. Mirrors the private +/// `r2sleigh_lift::disasm::Disassembler::MIN_BYTES` — every window handed to `lift`/`lift_block` +/// is zero-padded up to this length. +const MIN_LIFT_BYTES: usize = 16; + +const DEFAULT_MAX_FUNCS: usize = 200; +const DEFAULT_MAX_SECTION_BYTES: usize = 262_144; + +const ELF_EXEC_FLAG: u64 = 0x4; +const ELF_SHT_NOBITS: u32 = 8; +const ELF_SHT_SYMTAB: u32 = 2; +const ELF_STT_FUNC: u8 = 2; + +// ============================================================================================ +// mod elf — minimal, no-deps ELF64 LE reader. Every read bounds-checked; anything malformed +// returns `None` and the caller skips the whole binary with a printed note. This never parses +// anything beyond the header / section headers / symtab needed for the two passes above. +// ============================================================================================ +mod elf { + use super::{ELF_EXEC_FLAG, ELF_SHT_NOBITS, ELF_SHT_SYMTAB, ELF_STT_FUNC}; + + pub(crate) struct Section { + pub(crate) name: String, + pub(crate) sh_type: u32, + pub(crate) flags: u64, + pub(crate) addr: u64, + pub(crate) offset: u64, + pub(crate) size: u64, + } + + impl Section { + pub(crate) fn is_exec(&self) -> bool { + (self.flags & ELF_EXEC_FLAG) != 0 && self.sh_type != ELF_SHT_NOBITS + } + + pub(crate) fn end_addr(&self) -> u64 { + self.addr.saturating_add(self.size) + } + } + + pub(crate) struct Symbol { + pub(crate) name: String, + pub(crate) value: u64, + pub(crate) size: u64, + } + + pub(crate) struct Info { + pub(crate) sections: Vec
, + /// `STT_FUNC` symbols with `st_size > 0`, each verified to sit inside an executable + /// section at parse time. + pub(crate) functions: Vec, + } + + fn read_u16(buf: &[u8], off: usize) -> Option { + let b = buf.get(off..off + 2)?; + Some(u16::from_le_bytes([b[0], b[1]])) + } + + fn read_u32(buf: &[u8], off: usize) -> Option { + let b = buf.get(off..off + 4)?; + Some(u32::from_le_bytes([b[0], b[1], b[2], b[3]])) + } + + fn read_u64(buf: &[u8], off: usize) -> Option { + let b = buf.get(off..off + 8)?; + Some(u64::from_le_bytes([ + b[0], b[1], b[2], b[3], b[4], b[5], b[6], b[7], + ])) + } + + fn read_u8(buf: &[u8], off: usize) -> Option { + buf.get(off).copied() + } + + /// NUL-terminated name from a string table section, at `strtab_off + name_off`. + fn read_name(bytes: &[u8], strtab_off: u64, strtab_size: u64, name_off: u32) -> Option { + let start = strtab_off.checked_add(u64::from(name_off))?; + let limit = strtab_off.checked_add(strtab_size)?; + if start >= limit { + return None; + } + let start = usize::try_from(start).ok()?; + let limit = usize::try_from(limit).ok()?.min(bytes.len()); + let slice = bytes.get(start..limit)?; + let end = slice.iter().position(|&b| b == 0).unwrap_or(slice.len()); + std::str::from_utf8(&slice[..end]).ok().map(str::to_string) + } + + /// Parse an ELF64 LE x86-64 binary's header, section headers, and `STT_FUNC` symbol table. + /// Every offset below is the field's byte offset per the ELF64 spec, as documented in the + /// impl-spec. Returns `None` the moment any read is out of bounds or a fixed-field check + /// fails — the caller treats that as "skip this binary, print a note", never a panic. + pub(crate) fn parse(bytes: &[u8]) -> Option { + let magic = bytes.get(0..4)?; + if magic != [0x7f, b'E', b'L', b'F'] { + return None; + } + if read_u8(bytes, 4)? != 2 { + // EI_CLASS: ELFCLASS64 + return None; + } + if read_u8(bytes, 5)? != 1 { + // EI_DATA: ELFDATA2LSB + return None; + } + if read_u16(bytes, 18)? != 62 { + // e_machine: EM_X86_64 + return None; + } + let e_shoff = read_u64(bytes, 40)?; + let e_shentsize = read_u16(bytes, 58)?; + let e_shnum = read_u16(bytes, 60)?; + let e_shstrndx = read_u16(bytes, 62)?; + if e_shentsize < 64 || e_shnum == 0 { + return None; + } + + let section_header_off = |index: u16| -> Option { + let off = e_shoff.checked_add(u64::from(index).checked_mul(u64::from(e_shentsize))?)?; + usize::try_from(off).ok() + }; + + let shstr_hdr = section_header_off(e_shstrndx)?; + let shstrtab_offset = read_u64(bytes, shstr_hdr + 24)?; + let shstrtab_size = read_u64(bytes, shstr_hdr + 32)?; + + let mut sections = Vec::with_capacity(usize::from(e_shnum)); + for index in 0..e_shnum { + let hdr = section_header_off(index)?; + let sh_name = read_u32(bytes, hdr)?; + let sh_type = read_u32(bytes, hdr + 4)?; + let sh_flags = read_u64(bytes, hdr + 8)?; + let sh_addr = read_u64(bytes, hdr + 16)?; + let sh_offset = read_u64(bytes, hdr + 24)?; + let sh_size = read_u64(bytes, hdr + 32)?; + let sh_link = read_u32(bytes, hdr + 40)?; + let sh_entsize = read_u64(bytes, hdr + 56)?; + let name = + read_name(bytes, shstrtab_offset, shstrtab_size, sh_name).unwrap_or_default(); + sections.push(( + Section { + name, + sh_type, + flags: sh_flags, + addr: sh_addr, + offset: sh_offset, + size: sh_size, + }, + sh_link, + sh_entsize, + )); + } + + let mut functions = Vec::new(); + if let Some((symtab, link, entsize)) = sections + .iter() + .find(|(s, _, _)| s.sh_type == ELF_SHT_SYMTAB) + { + let entsize = if *entsize == 0 { 24 } else { *entsize }; + if let Some((strtab, _, _)) = sections.get(*link as usize) { + let count = symtab.size / entsize; + for i in 0..count { + let off = symtab.offset.checked_add(i.checked_mul(entsize)?)?; + let off = usize::try_from(off).ok()?; + let st_name = read_u32(bytes, off)?; + let st_info = read_u8(bytes, off + 4)?; + let st_value = read_u64(bytes, off + 8)?; + let st_size = read_u64(bytes, off + 16)?; + let is_func = (st_info & 0xF) == ELF_STT_FUNC; + if !is_func || st_size == 0 { + continue; + } + let in_exec = sections.iter().any(|(s, _, _)| { + s.is_exec() && st_value >= s.addr && st_value < s.end_addr() + }); + if !in_exec { + continue; + } + // A malformed/unreadable name still yields a usable symbol (name just + // falls back to empty) — only value/size gate whether the symbol is used. + let name = + read_name(bytes, strtab.offset, strtab.size, st_name).unwrap_or_default(); + functions.push(Symbol { + name, + value: st_value, + size: st_size, + }); + } + } + } + + Some(Info { + sections: sections.into_iter().map(|(s, _, _)| s).collect(), + functions, + }) + } +} + +// ============================================================================================ +// Corpus resolution +// ============================================================================================ + +/// `$R2IL_CORPUS` (colon-separated) or CLI args, else the fixed fallback list: the two +/// not-stripped r2sleigh e2e binaries (symtab present, exercise Pass 2), then two stripped +/// system binaries (op-level only, exercise the Pass 2 skip line). +fn corpus_paths() -> Vec { + let args: Vec = env::args().skip(1).collect(); + if !args.is_empty() { + return args.into_iter().map(PathBuf::from).collect(); + } + if let Ok(value) = env::var("R2IL_CORPUS") { + let paths: Vec = value + .split(':') + .filter(|s| !s.is_empty()) + .map(PathBuf::from) + .collect(); + if !paths.is_empty() { + return paths; + } + } + let mut out = vec![ + PathBuf::from(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../../r2sleigh/tests/e2e/stress_test" + )), + PathBuf::from(concat!( + env!("CARGO_MANIFEST_DIR"), + "/../../../r2sleigh/tests/e2e/stress_test_opt" + )), + ]; + out.push(PathBuf::from("/bin/ls")); + out.push(PathBuf::from("/usr/bin/env")); + out +} + +fn env_usize(name: &str, default: usize) -> usize { + env::var(name) + .ok() + .and_then(|v| v.parse::().ok()) + .unwrap_or(default) +} + +// ============================================================================================ +// Small, dependency-free distribution helper: mean / min / median / max over usize samples. +// ============================================================================================ + +fn distribution(values: &[usize]) -> Option<(f64, usize, f64, usize)> { + if values.is_empty() { + return None; + } + let mut sorted = values.to_vec(); + sorted.sort_unstable(); + let min = sorted[0]; + let max = sorted[sorted.len() - 1]; + let sum: usize = sorted.iter().sum(); + #[allow(clippy::cast_precision_loss)] + let mean = sum as f64 / sorted.len() as f64; + let mid = sorted.len() / 2; + #[allow(clippy::cast_precision_loss)] + let median = if sorted.len() % 2 == 1 { + sorted[mid] as f64 + } else { + (sorted[mid - 1] as f64 + sorted[mid] as f64) / 2.0 + }; + Some((mean, min, median, max)) +} + +fn print_distribution(label: &str, values: &[usize]) { + match distribution(values) { + Some((mean, min, median, max)) => println!( + " {label}: mean={mean:.2} min={min} median={median:.1} max={max} n={}", + values.len() + ), + None => println!(" {label}: (no samples)"), + } +} + +#[allow(clippy::cast_precision_loss)] +fn percent(part: usize, total: usize) -> f64 { + if total == 0 { + 0.0 + } else { + (part as f64 / total as f64) * 100.0 + } +} + +fn is_atomic(op: &R2ILOp) -> bool { + matches!( + op, + R2ILOp::AtomicCAS { .. } + | R2ILOp::LoadLinked { .. } + | R2ILOp::StoreConditional { .. } + | R2ILOp::LoadGuarded { .. } + | R2ILOp::StoreGuarded { .. } + ) +} + +// ============================================================================================ +// Pass 1 — MEASURED EXACT (op level) +// ============================================================================================ + +#[derive(Default)] +struct Pass1Stats { + instructions_decoded: usize, + undecodable_instructions: usize, + ops_total: usize, + opcode_freq: BTreeMap<&'static str, usize>, + ops_per_instruction: Vec, + input_arity_hist: BTreeMap, + output_count_hist: BTreeMap, + memory_ops: usize, + control_ops: usize, + atomic_ops: usize, + call_other_arity: Vec, + inline_fit: usize, + needs_vec_routing: usize, +} + +impl Pass1Stats { + fn record_block(&mut self, block: &R2ILBlock) { + self.ops_per_instruction.push(block.ops.len()); + for op in &block.ops { + self.ops_total += 1; + *self + .opcode_freq + .entry(OpTag::from_r2il(op).as_str()) + .or_insert(0) += 1; + + let input_arity = op.inputs().len(); + *self.input_arity_hist.entry(input_arity).or_insert(0) += 1; + + let output_count = usize::from(op.output().is_some()); + *self.output_count_hist.entry(output_count).or_insert(0) += 1; + + if op.is_memory_read() || op.is_memory_write() { + self.memory_ops += 1; + } + if op.is_control_flow() { + self.control_ops += 1; + } + if is_atomic(op) { + self.atomic_ops += 1; + } + if let R2ILOp::CallOther { inputs, .. } = op { + self.call_other_arity.push(inputs.len()); + } + + if output_count <= 1 && input_arity <= 2 { + self.inline_fit += 1; + } else { + self.needs_vec_routing += 1; + } + } + } +} + +/// Build a `MIN_LIFT_BYTES`-long window starting at `bytes[offset..]`, zero-padded if the +/// section runs out of bytes before then. +fn lift_window(bytes: &[u8], offset: usize) -> Vec { + let avail = &bytes[offset..]; + if avail.len() >= MIN_LIFT_BYTES { + avail[..MIN_LIFT_BYTES].to_vec() + } else { + let mut window = avail.to_vec(); + window.resize(MIN_LIFT_BYTES, 0); + window + } +} + +/// Linear op-level sweep of one section's bytes, capped at `cap` bytes. On `Ok`, advances by +/// `block.size.max(1)`; on `Err`, counts one undecodable instruction and advances one byte. +fn pass1_sweep( + disasm: &Disassembler, + bytes: &[u8], + base_addr: u64, + cap: usize, + stats: &mut Pass1Stats, +) { + let limit = bytes.len().min(cap); + let mut offset = 0usize; + while offset < limit { + let addr = base_addr.saturating_add(offset as u64); + let window = lift_window(bytes, offset); + match disasm.lift(&window, addr) { + Ok(block) => { + let advance = (block.size as usize).max(1); + stats.record_block(&block); + stats.instructions_decoded += 1; + offset += advance; + } + Err(_) => { + stats.undecodable_instructions += 1; + offset += 1; + } + } + } +} + +fn print_pass1(stats: &Pass1Stats) { + println!(" == MEASURED EXACT (op level) =="); + println!( + " instructions_decoded={} undecodable_instructions={} ops_total={}", + stats.instructions_decoded, stats.undecodable_instructions, stats.ops_total + ); + + print!(" opcode_freq:"); + if stats.opcode_freq.is_empty() { + println!(" (none)"); + } else { + println!(); + for (name, count) in &stats.opcode_freq { + println!(" {name}={count}"); + } + } + + print_distribution("ops_per_native_instruction", &stats.ops_per_instruction); + + print!(" input_arity_histogram:"); + if stats.input_arity_hist.is_empty() { + println!(" (none)"); + } else { + println!(); + for (arity, count) in &stats.input_arity_hist { + println!(" arity={arity} count={count}"); + } + } + + print!(" output_count_histogram:"); + if stats.output_count_hist.is_empty() { + println!(" (none)"); + } else { + println!(); + for (outputs, count) in &stats.output_count_hist { + println!(" outputs={outputs} count={count}"); + } + } + + println!( + " memory-op %={:.2} control-op %={:.2} atomic %={:.2} (of ops_total={})", + percent(stats.memory_ops, stats.ops_total), + percent(stats.control_ops, stats.ops_total), + percent(stats.atomic_ops, stats.ops_total), + stats.ops_total + ); + + print_distribution("call_other_arity", &stats.call_other_arity); + + println!( + " % fitting dst+src0+src1 inline={:.2} % needing Vec routing={:.2}", + percent(stats.inline_fit, stats.ops_total), + percent(stats.needs_vec_routing, stats.ops_total) + ); +} + +// ============================================================================================ +// Pass 2 — HEURISTIC-DERIVED (function / CFG level) +// ============================================================================================ + +/// One lifted native instruction inside a function, kept just long enough to compute leaders. +struct InstrInfo { + addr: u64, + size: usize, + is_control_flow: bool, + /// Constant branch targets this instruction's control-flow op(s) named — the approximate + /// half of the leader set (indirect targets are never resolved here). + const_targets: Vec, +} + +/// Sweep `[fn_start, fn_end)` inside one section's bytes at the instruction level (mirrors +/// `pass1_sweep`'s window/advance rule) to discover instruction boundaries and constant +/// control-flow targets. `section_bytes`/`section_addr` describe the CAPPED section slice this +/// function's bytes must fall within. +fn sweep_function_instructions( + disasm: &Disassembler, + section_bytes: &[u8], + section_addr: u64, + fn_start: u64, + fn_end: u64, +) -> Vec { + let mut infos = Vec::new(); + let mut addr = fn_start; + while addr < fn_end { + let Some(sec_off) = addr.checked_sub(section_addr) else { + break; + }; + let Ok(sec_off) = usize::try_from(sec_off) else { + break; + }; + if sec_off >= section_bytes.len() { + break; + } + let window = lift_window(section_bytes, sec_off); + match disasm.lift(&window, addr) { + Ok(block) => { + let size = (block.size as usize).max(1); + let is_control_flow = block.ops.iter().any(R2ILOp::is_control_flow); + let mut const_targets = Vec::new(); + for op in &block.ops { + if !op.is_control_flow() { + continue; + } + for input in op.inputs() { + if input.space == SpaceId::Const { + const_targets.push(input.offset); + } + } + } + infos.push(InstrInfo { + addr, + size, + is_control_flow, + const_targets, + }); + addr = addr.saturating_add(size as u64); + } + Err(_) => { + infos.push(InstrInfo { + addr, + size: 1, + is_control_flow: false, + const_targets: Vec::new(), + }); + addr = addr.saturating_add(1); + } + } + } + infos +} + +/// leaders = `{fn_start}` ∪ intra-function const branch targets ∪ `{addr after any +/// control-flow instruction}` — the approximate half of the labelling rule. +fn compute_leaders(fn_start: u64, fn_end: u64, infos: &[InstrInfo]) -> Vec { + let mut leaders = std::collections::BTreeSet::new(); + leaders.insert(fn_start); + for info in infos { + for &target in &info.const_targets { + if target >= fn_start && target < fn_end { + leaders.insert(target); + } + } + if info.is_control_flow { + let after = info.addr.saturating_add(info.size as u64); + if after > fn_start && after < fn_end { + leaders.insert(after); + } + } + } + leaders.into_iter().collect() +} + +/// Basic blocks = maximal ranges between leaders, the last one running to `fn_end`. +fn basic_block_ranges(fn_end: u64, leaders: &[u64]) -> Vec<(u64, u64)> { + let mut ranges = Vec::with_capacity(leaders.len()); + for (index, &start) in leaders.iter().enumerate() { + let end = leaders.get(index + 1).copied().unwrap_or(fn_end); + if end > start { + ranges.push((start, end)); + } + } + ranges +} + +/// One `disasm.lift_block(&padded, bb_addr, bb_len)` per basic block range. +fn lift_blocks( + disasm: &Disassembler, + section_bytes: &[u8], + section_addr: u64, + ranges: &[(u64, u64)], +) -> Option> { + let mut out = Vec::with_capacity(ranges.len()); + for &(bb_addr, bb_end) in ranges { + let bb_len = usize::try_from(bb_end - bb_addr).ok()?; + let sec_off = usize::try_from(bb_addr.checked_sub(section_addr)?).ok()?; + if sec_off >= section_bytes.len() { + return None; + } + let avail_len = (section_bytes.len() - sec_off).min(bb_len); + let mut padded = section_bytes[sec_off..sec_off + avail_len].to_vec(); + if padded.len() < MIN_LIFT_BYTES { + padded.resize(MIN_LIFT_BYTES, 0); + } + match disasm.lift_block(&padded, bb_addr, bb_len) { + Ok(block) => out.push(block), + Err(_) => return None, + } + } + Some(out) +} + +#[derive(Default)] +struct Pass2Stats { + functions_considered: usize, + functions_processed: usize, + functions_skipped: usize, + blocks_per_fn: Vec, + ops_per_block: Vec, + phi_fanin: Vec, + values_per_fn: Vec, + call_sites_per_fn: Vec, + predicates_per_fn: Vec, + vocab_unique_ssa_names: Vec, + vocab_unique_op_spaces: Vec, + vocab_unique_object_spaces: Vec, + vocab_unique_userops: Vec, + vocab_userop_mentions: Vec, + vocab_unique_custom_spaces_from_strings: Vec, + vocab_total_values: Vec, + vocab_ssa_name_bytes: Vec, + vocab_interned_id_bytes: Vec, + facet_ok: usize, + facet_unknown_custom_space: usize, + facet_ordinal_exhausted: usize, + smelt_harvested: usize, + smelt_classified: usize, + smelt_residual: usize, + smelt_dropped: usize, +} + +/// Record one skipped `STT_FUNC` symbol — named, addressed, never a silent omission — and bump +/// the ledger. `symbol.name` may be empty (an unreadable/absent strtab entry); that is reported +/// as `` rather than left blank. +fn skip_function(stats: &mut Pass2Stats, symbol: &elf::Symbol, reason: &str) { + let label = if symbol.name.is_empty() { + "" + } else { + symbol.name.as_str() + }; + println!(" skipped {label} @0x{:x}: {reason}", symbol.value); + stats.functions_skipped += 1; +} + +/// Everything Pass 2 needs that does not change per binary: the disassembler, the arch spec +/// used to build it, the pass-1-seven-opcode convention, and the two echoed caps. +struct Setup { + disasm: Disassembler, + spec: ArchSpec, + conv: R2ilConvention, + max_funcs: usize, + max_section_bytes: usize, +} + +fn print_pass2(stats: &Pass2Stats) { + println!(" == HEURISTIC-DERIVED (function / CFG level) =="); + println!( + " function_boundaries: symtab (exact) | leaders: intra-function const targets (approximate)" + ); + println!( + " functions_considered={} functions_processed={} functions_skipped={}", + stats.functions_considered, stats.functions_processed, stats.functions_skipped + ); + print_distribution("blocks_per_fn", &stats.blocks_per_fn); + print_distribution("ops_per_block", &stats.ops_per_block); + print_distribution("phi_fanin", &stats.phi_fanin); + print_distribution("values_per_fn", &stats.values_per_fn); + print_distribution("call_sites_per_fn", &stats.call_sites_per_fn); + print_distribution("predicates_per_fn", &stats.predicates_per_fn); + + print_distribution("vocab.unique_ssa_names", &stats.vocab_unique_ssa_names); + print_distribution("vocab.unique_op_spaces", &stats.vocab_unique_op_spaces); + print_distribution( + "vocab.unique_object_spaces", + &stats.vocab_unique_object_spaces, + ); + print_distribution("vocab.unique_userops", &stats.vocab_unique_userops); + print_distribution("vocab.userop_mentions", &stats.vocab_userop_mentions); + print_distribution( + "vocab.unique_custom_spaces_from_strings", + &stats.vocab_unique_custom_spaces_from_strings, + ); + print_distribution("vocab.total_values", &stats.vocab_total_values); + print_distribution("vocab.ssa_name_bytes", &stats.vocab_ssa_name_bytes); + print_distribution("vocab.interned_id_bytes", &stats.vocab_interned_id_bytes); + + let facet_total = + stats.facet_ok + stats.facet_unknown_custom_space + stats.facet_ordinal_exhausted; + println!( + " facet::project sweep: ok={} FacetOverflow(UnknownCustomSpace)={} FacetOverflow(CustomOrdinalExhausted)={} total={}", + stats.facet_ok, + stats.facet_unknown_custom_space, + stats.facet_ordinal_exhausted, + facet_total + ); + + let conserved = stats.smelt_dropped == 0 + && stats.smelt_harvested == stats.smelt_classified + stats.smelt_residual; + println!( + " furnace::smelt conservation: harvested {} / classified {} / residual {} / dropped {} (conserved: {})", + stats.smelt_harvested, + stats.smelt_classified, + stats.smelt_residual, + stats.smelt_dropped, + conserved + ); +} + +// ============================================================================================ +// Per-binary orchestration +// ============================================================================================ + +fn process_binary(setup: &Setup, path: &Path) { + println!("== binary: {} ==", path.display()); + + let bytes = match fs::read(path) { + Ok(bytes) => bytes, + Err(err) => { + println!(" skipped: cannot read file ({err})"); + println!(); + return; + } + }; + + let Some(info) = elf::parse(&bytes) else { + println!(" skipped: not a recognized ELF64 LE x86-64 binary (malformed or wrong arch)"); + println!(); + return; + }; + + let exec_sections: Vec<&elf::Section> = info.sections.iter().filter(|s| s.is_exec()).collect(); + if exec_sections.is_empty() { + println!(" skipped: no executable sections found"); + println!(); + return; + } + + print!(" exec sections:"); + for section in &exec_sections { + print!(" {}(size={})", section.name, section.size); + } + println!(); + + let mut p1 = Pass1Stats::default(); + for section in &exec_sections { + let Ok(start) = usize::try_from(section.offset) else { + continue; + }; + let len = usize::try_from(section.size).unwrap_or(0); + let Some(section_bytes) = bytes.get(start..start.saturating_add(len)) else { + continue; + }; + pass1_sweep( + &setup.disasm, + section_bytes, + section.addr, + setup.max_section_bytes, + &mut p1, + ); + } + print_pass1(&p1); + println!(); + + if info.functions.is_empty() { + println!(" function-level stats: skipped (no symtab)"); + println!(); + return; + } + + let exec_sections_owned: Vec = info + .sections + .into_iter() + .filter(elf::Section::is_exec) + .collect(); + let p2 = run_pass2_over(setup, &bytes, &exec_sections_owned, &info.functions); + print_pass2(&p2); + println!(); +} + +/// The real Pass 2 driver — reads section bytes out of the whole-file `bytes` buffer itself +/// (rather than pre-sliced section byte vectors), so each function's basic blocks can be lifted +/// straight from the file without an intermediate copy of the whole section. +fn run_pass2_over( + setup: &Setup, + bytes: &[u8], + exec_sections: &[elf::Section], + functions: &[elf::Symbol], +) -> Pass2Stats { + let mut stats = Pass2Stats { + functions_considered: functions.len(), + ..Pass2Stats::default() + }; + + let mut sorted: Vec<&elf::Symbol> = functions.iter().collect(); + sorted.sort_by_key(|symbol| symbol.value); + + for symbol in sorted.into_iter().take(setup.max_funcs) { + let Some(section) = exec_sections + .iter() + .find(|s| symbol.value >= s.addr && symbol.value < s.end_addr()) + else { + skip_function(&mut stats, symbol, "no containing executable section"); + continue; + }; + let Ok(sec_start) = usize::try_from(section.offset) else { + skip_function(&mut stats, symbol, "section offset does not fit usize"); + continue; + }; + let full_len = usize::try_from(section.size).unwrap_or(0); + let capped_len = full_len.min(setup.max_section_bytes); + let Some(section_bytes) = bytes.get(sec_start..sec_start.saturating_add(capped_len)) else { + skip_function(&mut stats, symbol, "section bytes out of file bounds"); + continue; + }; + + let fn_start = symbol.value; + let section_capped_end = section.addr.saturating_add(section_bytes.len() as u64); + let fn_end = symbol + .value + .saturating_add(symbol.size) + .min(section.end_addr()) + .min(section_capped_end); + if fn_end <= fn_start { + skip_function( + &mut stats, + symbol, + "empty range after capping to section bytes", + ); + continue; + } + + let infos = sweep_function_instructions( + &setup.disasm, + section_bytes, + section.addr, + fn_start, + fn_end, + ); + let leaders = compute_leaders(fn_start, fn_end, &infos); + let ranges = basic_block_ranges(fn_end, &leaders); + let Some(bbs) = lift_blocks(&setup.disasm, section_bytes, section.addr, &ranges) else { + skip_function(&mut stats, symbol, "lift_block failed on a basic block"); + continue; + }; + let Some(behavior) = FunctionBehavior::from_blocks_raw(&bbs, Some(&setup.spec)) else { + skip_function( + &mut stats, + symbol, + "FunctionBehavior::from_blocks_raw returned None", + ); + continue; + }; + + stats.functions_processed += 1; + stats.blocks_per_fn.push(bbs.len()); + for block in behavior.control().blocks() { + stats.ops_per_block.push(block.ops.len()); + for phi in &block.phis { + stats.phi_fanin.push(phi.sources.len()); + } + } + stats.values_per_fn.push(behavior.values().values.len()); + stats.call_sites_per_fn.push(behavior.calls().by_id.len()); + stats + .predicates_per_fn + .push(behavior.predicates().predicates.len()); + + let harvest = VocabHarvest::from_behavior(&behavior); + let vstats = harvest.stats(); + stats.vocab_unique_ssa_names.push(vstats.unique_ssa_names); + stats.vocab_unique_op_spaces.push(vstats.unique_op_spaces); + stats + .vocab_unique_object_spaces + .push(vstats.unique_object_spaces); + stats.vocab_unique_userops.push(vstats.unique_userops); + stats.vocab_userop_mentions.push(vstats.userop_mentions); + stats + .vocab_unique_custom_spaces_from_strings + .push(vstats.unique_custom_spaces_from_strings); + stats.vocab_total_values.push(vstats.total_values); + stats.vocab_ssa_name_bytes.push(vstats.ssa_name_bytes); + stats.vocab_interned_id_bytes.push(vstats.interned_id_bytes); + + for block in &bbs { + for op in &block.ops { + let mut varnodes = op.inputs(); + if let Some(output) = op.output() { + varnodes.push(output); + } + for vn in varnodes { + match facet::project(vn, setup.conv.spaces()) { + Ok(_) => stats.facet_ok += 1, + Err(facet::FacetOverflow::UnknownCustomSpace { .. }) => { + stats.facet_unknown_custom_space += 1; + } + Err(facet::FacetOverflow::CustomOrdinalExhausted { .. }) => { + stats.facet_ordinal_exhausted += 1; + } + } + } + } + } + + let (_rows, _ledger, report) = furnace::smelt(&behavior, &bbs, &setup.conv); + stats.smelt_harvested += report.harvested; + stats.smelt_classified += report.classified; + stats.smelt_residual += report.residual; + stats.smelt_dropped += report.dropped; + } + + stats +} + +// ============================================================================================ +// main +// ============================================================================================ + +fn main() { + let max_funcs = env_usize("R2IL_PROFILE_MAX_FUNCS", DEFAULT_MAX_FUNCS); + let max_section_bytes = env_usize("R2IL_PROFILE_MAX_SECTION_BYTES", DEFAULT_MAX_SECTION_BYTES); + + println!("r2il corpus profile (§12)"); + println!( + "caps: R2IL_PROFILE_MAX_FUNCS={max_funcs} R2IL_PROFILE_MAX_SECTION_BYTES={max_section_bytes}" + ); + println!( + "function_boundaries: symtab (exact) | leaders: intra-function const targets (approximate)" + ); + println!(); + + let spec = match build_arch_spec( + sleigh_config::processor_x86::SLA_X86_64, + sleigh_config::processor_x86::PSPEC_X86_64, + "x86-64", + ) { + Ok(spec) => spec, + Err(err) => { + println!("FATAL: build_arch_spec(x86-64) failed: {err}"); + return; + } + }; + + let mut disasm = match Disassembler::from_sla( + sleigh_config::processor_x86::SLA_X86_64, + sleigh_config::processor_x86::PSPEC_X86_64, + "x86-64", + ) { + Ok(disasm) => disasm, + Err(err) => { + println!("FATAL: Disassembler::from_sla(x86-64) failed: {err}"); + return; + } + }; + disasm.set_userop_map(userop_map_for_arch("x86-64")); + + // The pass-1 seven, bootstrapped from the real x86-64 ArchSpec so Pass 2's smelt/facet + // sweep can also resolve register operands (see impl-spec §10 test 12). Falls back to the + // unbootstrapped `minimal_pass_one` convention (still classifying the same seven opcodes, + // just without register rows or a CustomSpaceTable) if the arch's own custom-space count + // ever overflowed the lo-u16 budget — practically unreachable for x86-64, handled anyway + // rather than assumed away. + let seven = [ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + ]; + let conv = match R2ilConvention::from_arch(&spec, seven) { + Ok(conv) => conv, + Err(err) => { + println!( + "WARN: R2ilConvention::from_arch(x86-64) failed ({err}); falling back to minimal_pass_one" + ); + R2ilConvention::minimal_pass_one() + } + }; + + let setup = Setup { + disasm, + spec, + conv, + max_funcs, + max_section_bytes, + }; + + for path in corpus_paths() { + process_binary(&setup, &path); + } +} diff --git a/crates/ruff_r2il/src/behavior.rs b/crates/ruff_r2il/src/behavior.rs new file mode 100644 index 0000000000000..9dc7f3de712af --- /dev/null +++ b/crates/ruff_r2il/src/behavior.rs @@ -0,0 +1,297 @@ +//! Stage 1 — the **ore carrier**. +//! +//! [`FunctionBehavior`] is a truthful, lossless, zero-copy assembly of upstream values. It +//! **never flattens at intake** (that is [`crate::furnace`]'s job), it invents no vocabulary, and +//! it decides nothing. It is NOT a "behavioral contract" — nothing here is refined, classified, +//! or proposed. It holds r2sleigh's own `SSAFunction` / `SsaGraph` / `PreparedFunctionFacts` (as +//! one [`SsaArtifact`]) and *names* that decomposition through borrowed accessors. Refined +//! concern contracts are a later, measured furnace output; this type must never grow into one. + +use std::collections::BTreeMap; + +use r2il::{ArchSpec, R2ILBlock}; +use r2ssa::{ + CallSiteFacts, FunctionSemanticSummary, InstId, InterprocFunctionId, InterprocFunctionInput, + InterprocSolveConfig, MemorySSAFacts, ObjectModel, PredicateFacts, PreparedFunctionFacts, + SSAFunction, SSAVar, SsaArtifact, SsaGraph, UseSite, ValueId, solve_interproc_summary_set, +}; + +/// Identity metadata for one function, named apart from the ore itself. +/// +/// `entry` mirrors [`SSAFunction::entry`]; `name` mirrors [`SSAFunction::name`]; `arch` is +/// **advisory provenance only** — the `ArchSpec::name` supplied at ingest, when one was — never +/// an address, never branched on. When `arch` is `None`, register operands fall back to +/// r2sleigh's own `"reg:"` display convention rather than a resolved name. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FunctionIdentity { + pub entry: u64, + pub name: Option, + pub arch: Option, +} + +/// The ORE CARRIER for one function. +/// +/// NOT a "behavioral contract" — nothing here is refined, classified or proposed. It holds +/// r2sleigh's own `SSAFunction` / `SsaGraph` / `PreparedFunctionFacts` (as one [`SsaArtifact`]) +/// and NAMES that decomposition through borrowed accessors. Refined concern contracts are a +/// later, measured furnace output; this type must never grow into one. +/// +/// `JUDGMENT` — one [`SsaArtifact`] instead of three sibling fields. The plan sketches +/// `{identity, ssa, graph, facts, summary}`. `SsaArtifact` *is* that triple upstream, its +/// constructor `new` is **private**, and [`InterprocFunctionInput::prepared`] requires +/// `&SsaArtifact` — a hand-assembled triple could never produce the `summary` the same struct +/// declares. Holding the artifact keeps all five concerns verbatim as accessors. Nothing is +/// copied. +#[derive(Debug, Clone)] +pub struct FunctionBehavior { + identity: FunctionIdentity, + artifact: SsaArtifact, + summary: Option, +} + +impl FunctionBehavior { + /// RAW ingest: `SsaArtifact::raw(blocks, arch)` → `SSAFunction::from_blocks_raw` → + /// `SsaGraph::from_function` → `PreparedFunctionFacts::collect`. `None` exactly when + /// upstream is: empty `blocks`, or `CFG::from_blocks` fails. + /// + /// This is the losslessness-claiming ingest path — no optimization pass runs. + pub fn from_blocks_raw(blocks: &[R2ILBlock], arch: Option<&ArchSpec>) -> Option { + let artifact = SsaArtifact::raw(blocks, arch)?; + let function = artifact.function(); + let identity = FunctionIdentity { + entry: function.entry, + name: function.name.clone(), + arch: arch.map(|spec| spec.name.clone()), + }; + Some(Self { + identity, + artifact, + summary: None, + }) + } + + /// GENERIC ingest (`SsaArtifact::from_blocks`) — applies constructor-time SCCP and can + /// REWRITE ops. Never use it for anything claiming losslessness. + pub fn from_blocks(blocks: &[R2ILBlock], arch: Option<&ArchSpec>) -> Option { + let artifact = SsaArtifact::from_blocks(blocks, arch)?; + let function = artifact.function(); + let identity = FunctionIdentity { + entry: function.entry, + name: function.name.clone(), + arch: arch.map(|spec| spec.name.clone()), + }; + Some(Self { + identity, + artifact, + summary: None, + }) + } + + /// Attach a name, keeping the carrier's own [`FunctionIdentity`] and the wrapped + /// [`SsaArtifact`] (and thereby its [`SSAFunction::name`]) in sync. + pub fn with_name(mut self, name: impl Into) -> Self { + let name = name.into(); + self.artifact = self.artifact.with_name(name.clone()); + self.identity.name = Some(name); + self + } + + /// Attach a precomputed [`FunctionSemanticSummary`] (e.g. seeded, or solved elsewhere). + pub fn with_summary(mut self, summary: FunctionSemanticSummary) -> Self { + self.summary = Some(summary); + self + } + + /// Solve one [`FunctionSemanticSummary`] for this function in isolation: one + /// `InterprocFunctionInput { id, name, prepared: self.artifact() }` → + /// `solve_interproc_summary_set(&[input], arch, Some(id), &BTreeMap::new(), + /// InterprocSolveConfig::default())` → `set.summaries.remove(&id)`. + /// + /// Single-function scope: `has_unknown_calls` true whenever the function calls anything is + /// CORRECT under this scope, not a defect — a wider call graph is out of scope for the ore + /// carrier and belongs to a caller that assembles multiple `FunctionBehavior`s itself. + pub fn solve_summary(&mut self, id: InterprocFunctionId, arch: Option<&ArchSpec>) { + let input = InterprocFunctionInput { + id, + name: self.identity.name.clone(), + prepared: &self.artifact, + }; + let mut set = solve_interproc_summary_set( + &[input], + arch, + Some(id), + &BTreeMap::new(), + InterprocSolveConfig::default(), + ); + self.summary = set.summaries.remove(&id); + } + + /// Identity metadata (entry / name / advisory arch provenance). + pub fn identity(&self) -> &FunctionIdentity { + &self.identity + } + + /// CONTROL — CFG, `BlockTerminator`, `CFGEdge`, block order. + pub fn control(&self) -> &SSAFunction { + self.artifact.function() + } + + /// Alias of [`Self::control`] — kept because the plan names the field `ssa`. + pub fn ssa(&self) -> &SSAFunction { + self.control() + } + + /// VALUES + DEF/USE + PROVENANCE — insts, values, and the `(block_addr, op_idx)` join maps. + pub fn values(&self) -> &SsaGraph { + self.artifact.graph() + } + + /// Alias of [`Self::values`]. + pub fn graph(&self) -> &SsaGraph { + self.artifact.graph() + } + + /// The full prepared-facts bundle (objects, memory, predicates, call sites). + pub fn facts(&self) -> &PreparedFunctionFacts { + self.artifact.facts() + } + + pub fn objects(&self) -> &ObjectModel { + self.artifact.objects() + } + + pub fn memory(&self) -> &MemorySSAFacts { + self.artifact.memory() + } + + pub fn predicates(&self) -> &PredicateFacts { + self.artifact.predicates() + } + + pub fn calls(&self) -> &CallSiteFacts { + self.artifact.call_sites() + } + + /// The wrapped [`SsaArtifact`] itself — borrowed, never cloned out from under the carrier. + pub fn artifact(&self) -> &SsaArtifact { + &self.artifact + } + + /// The solved interprocedural summary, if [`Self::solve_summary`] or [`Self::with_summary`] + /// has been called. + pub fn summary(&self) -> Option<&FunctionSemanticSummary> { + self.summary.as_ref() + } + + // ---- provenance helpers ------------------------------------------------------------- + + /// `InstId` → `(block_addr, op_idx)`, via `SsaGraph::op_site_for_inst`. + pub fn op_site(&self, inst: InstId) -> Option<(u64, usize)> { + self.artifact.graph().op_site_for_inst(inst) + } + + /// `(block_addr, op_idx)` → `InstId`, via `SsaGraph::inst_id_for_op_site`. The inverse of + /// [`Self::op_site`] — see §8 test 6's round trip. + pub fn inst_at(&self, block_addr: u64, op_idx: usize) -> Option { + self.artifact + .graph() + .inst_id_for_op_site(block_addr, op_idx) + } + + /// `ValueId` → the `SSAVar` it interns, via `SsaGraph::value`. + pub fn value_var(&self, value: ValueId) -> Option<&SSAVar> { + self.artifact + .graph() + .value(value) + .map(|graph_value| &graph_value.var) + } + + /// `ValueId` → the `InstId` that defines it, via `SsaGraph::def_inst`. + pub fn def_inst(&self, value: ValueId) -> Option { + self.artifact.graph().def_inst(value) + } + + /// `ValueId` → every site that uses it, via `SsaGraph::use_sites`. + pub fn use_sites(&self, value: ValueId) -> &[UseSite] { + self.artifact.graph().use_sites(value) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use r2ssa::InstPayload; + + fn reg(offset: u64, size: u32) -> r2il::Varnode { + r2il::Varnode::register(offset, size) + } + + fn con(value: u64, size: u32) -> r2il::Varnode { + r2il::Varnode::constant(value, size) + } + + /// A minimal 2-block, no-merge, no-phi fixture: `0x1000` computes and unconditionally + /// branches to `0x1004`, which returns. Every op in the source survives 1:1 into SSA (no + /// `Multiequal` here, so the total inst count is exactly the source op count). + fn two_block_fixture() -> Vec { + let mut b0 = R2ILBlock::new(0x1000, 4); + b0.push(r2il::R2ILOp::IntAdd { + dst: reg(0x00, 8), + a: reg(0x00, 8), + b: con(1, 8), + }); + b0.push(r2il::R2ILOp::Branch { + target: con(0x1004, 8), + }); + + let mut b1 = R2ILBlock::new(0x1004, 4); + b1.push(r2il::R2ILOp::Return { + target: reg(0x00, 8), + }); + + vec![b0, b1] + } + + #[test] + fn from_blocks_raw_names_the_upstream_decomposition() { + let blocks = two_block_fixture(); + let expected_ops: usize = blocks.iter().map(|block| block.ops.len()).sum(); + + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("two linear blocks ingest"); + + assert_eq!(behavior.identity().entry, 0x1000); + assert_eq!(behavior.control().num_blocks(), 2); + assert_eq!(behavior.values().insts.len(), expected_ops); + assert!(!behavior.values().values.is_empty()); + } + + #[test] + fn empty_block_list_is_none_not_a_panic() { + assert!(FunctionBehavior::from_blocks_raw(&[], None).is_none()); + + let mut single = R2ILBlock::new(0x2000, 2); + single.push(r2il::R2ILOp::Return { target: con(0, 8) }); + assert!(FunctionBehavior::from_blocks_raw(&[single], None).is_some()); + } + + #[test] + fn op_site_round_trips_through_the_graph_provenance_map() { + let blocks = two_block_fixture(); + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("two linear blocks ingest"); + + let mut checked = 0usize; + for inst in &behavior.values().insts { + if !matches!(inst.payload, InstPayload::Op(_)) { + continue; + } + let site = behavior + .op_site(inst.id) + .expect("every Op-payload inst must carry a (block_addr, op_idx) site"); + assert_eq!(behavior.inst_at(site.0, site.1), Some(inst.id)); + checked += 1; + } + assert!(checked >= 3, "expected at least 3 op sites, got {checked}"); + } +} diff --git a/crates/ruff_r2il/src/convention.rs b/crates/ruff_r2il/src/convention.rs new file mode 100644 index 0000000000000..118e6578c73d9 --- /dev/null +++ b/crates/ruff_r2il/src/convention.rs @@ -0,0 +1,523 @@ +//! The longest-prefix-wins config tree over varnode identity space. +//! +//! Precedent: `ruff_spo_triplet::concept_split::ConceptConvention` (caller-supplied, **zero +//! domain vocabulary in the module**) and OGAR codebook scoping — *"longest-prefix wins — one +//! rule, every level."* +//! **This module ships ZERO architecture vocabulary.** No register names, no opcode semantics, +//! no userop table are hardcoded anywhere below. Everything the convention knows arrives as +//! data, either bootstrapped by reading it off an [`ArchSpec`] ([`R2ilConvention::from_arch`]) +//! or inserted by a caller ([`R2ilConvention::insert`]). +//! +//! # Not a flat table +//! +//! [`R2ilConvention`] is a radix tree over [`VarnodeFacet`] space, keyed by [`FacetPrefix`]. +//! Rows attach at one of three prefix depths — space-class alone, space-class+offset, or the +//! full space-class+offset+size — and [`R2ilConvention::resolve`] walks from the finest prefix +//! down to the coarsest, returning the first row it finds (**longest matching prefix wins**). +//! An address the convention says nothing about resolves to `None`, which is exactly the +//! addressed-residual case `slag.rs` names: the proposer emits a proposed [`ConventionRow`] AT +//! that address and the next drill pass melts it — the config accumulates as a radix tree, +//! self-scaffolding, one row at a time, never a code edit to a match arm. +//! +//! # The slag doctrine, restated for this module +//! +//! *"The residual is not waste: it is the empirical boundary of the current convention. A +//! recurring residual reason names the next convention fact to add."* This module is the thing +//! that boundary is measured against — every [`R2ilConvention::resolve`] miss is a fact about +//! what the convention does not yet know, not a defect in this module. + +use std::collections::{BTreeMap, BTreeSet}; + +use r2il::{ArchSpec, SpaceId}; + +use crate::facet::{ + CustomSpaceTable, FacetOverflow, FacetPrefix, SPACE_CONST, SPACE_RAM, SPACE_REGISTER, + SPACE_UNIQUE, VarnodeFacet, +}; +use crate::ore::OpTag; + +/// Mirrors openproject-nexgen-rs's `orm-ar-backprojection.toml` +/// (`validation_states = [unmeasured|confirmed|corrected|retired]`, meta key +/// `measure_dont_claim`). **Every row starts `Unmeasured`** — a convention row records where +/// the drill believes an address resolves, never a claim that the belief was checked. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ValidationState { + Unmeasured, + Confirmed, + Corrected, + Retired, +} + +impl ValidationState { + /// The stable, lowercase name used by [`R2ilConvention::to_toml`]. Never derived from + /// `Debug` — a rename of the variant must not silently change the emitted TOML. + pub fn as_str(self) -> &'static str { + match self { + ValidationState::Unmeasured => "unmeasured", + ValidationState::Confirmed => "confirmed", + ValidationState::Corrected => "corrected", + ValidationState::Retired => "retired", + } + } +} + +/// One config row, attached at a facet PREFIX. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ConventionRow { + pub at: FacetPrefix, + /// e.g. a register name, a space name. `None` for a row the proposer attached before it + /// had a better label than the address itself. + pub name: Option, + /// Free provenance text; never branched on — this is a note for a human, not a fact the + /// resolver reads. + pub note: Option, + pub state: ValidationState, +} + +/// The drilling convention: a radix tree over varnode identity space. +/// +/// See the module docs for the "not a flat table" / "longest matching prefix wins" framing. +#[derive(Debug, Clone, Default)] +pub struct R2ilConvention { + /// `BTreeMap` ⇒ deterministic emission order, both for `rows()` and for `to_toml()`. + rows: BTreeMap, + spaces: CustomSpaceTable, + userops: BTreeMap, + /// THE furnace ladder, as DATA. Pass 1 carries exactly seven entries. Widening the pass is + /// a CONFIG change with a measured before/after ledger — never a code edit to a match arm. + classified_opcodes: BTreeSet, + /// Provenance only; never branched on. + pub arch: Option, +} + +impl R2ilConvention { + /// Pass 1: `[Copy, IntAdd, Load, Store, CBranch, Call, Return]`, no rows, no userops, no + /// custom spaces. Deliberately minimal — the stressors in `tests/lossless_fixtures.rs` + /// MUST land in slag under this convention, and the ledger NAMING them is the acceptance + /// criterion, not a bigger match. + pub fn minimal_pass_one() -> Self { + Self { + rows: BTreeMap::new(), + spaces: CustomSpaceTable::default(), + userops: BTreeMap::new(), + classified_opcodes: BTreeSet::from([ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + ]), + arch: None, + } + } + + /// BOOTSTRAP — read, never retype. Populates from data that already exists on `ArchSpec`: + /// + /// * `arch.registers: Vec` → one + /// `FacetPrefix::SpaceOffsetSize{ SPACE_REGISTER, offset, size }` row per register, + /// `name = Some(reg.name)`, `state = Unmeasured`; + /// * one coarse `FacetPrefix::Space{ .. }` fall-through row per FIXED space (Register, Ram, + /// Unique, Const), each named after that space's own `arch.spaces` entry when the + /// architecture defines one, else a literal fallback, so an offset the corpus never named + /// still resolves — to the space, not to nothing. Custom spaces get NO blanket + /// fall-through: an unnamed `Custom(n)` must stay slag; + /// * `arch.userops: Vec` → the userop table; + /// * `arch.spaces` (`AddressSpace{id: SpaceId::Custom(n), name}`) → [`CustomSpaceTable`]. + /// + /// Errors only through [`FacetOverflow`] — config keys must be lossless (see `facet.rs`). + pub fn from_arch( + arch: &ArchSpec, + classified: impl IntoIterator, + ) -> Result { + let spaces = CustomSpaceTable::from_arch(arch)?; + + let mut rows = BTreeMap::new(); + + // A coarse fall-through row for EVERY fixed space — an offset the corpus never named + // still resolves to *something* rather than to nothing. + // + // MEASURED omission, fixed here: the first pass-1 harvest emitted a Register row and + // nothing else, so operands in Const / Unique / Ram could not resolve against ANY + // convention and every one of them became `no_convention_row_at_address` — 6828 rows, + // the single largest seven-eligible residual, and the reason bar B2 read 55.73% against + // a >=99% floor. That was a bootstrap gap, not evidence about the corpus: the very + // doctrine that gives Register a space-class fall-through applies identically to the + // other three. See `.claude/harvest/r2il/TRIAGE-RESULT.md` for the failing run, kept in + // history rather than overwritten. + // + // Custom spaces are deliberately NOT given a blanket fall-through: an unnamed + // `Custom(n)` is a genuine "the convention does not know this architecture's space" + // signal and must stay slag (`CustomSpaceNotInConvention`), which is exactly what the + // config-key falsifier in `facet.rs` pins. + for (discriminant, space_id, fallback_name) in [ + (SPACE_REGISTER, SpaceId::Register, "register"), + (SPACE_RAM, SpaceId::Ram, "ram"), + (SPACE_UNIQUE, SpaceId::Unique, "unique"), + (SPACE_CONST, SpaceId::Const, "const"), + ] { + let space_name = arch + .spaces + .iter() + .find(|space| space.id == space_id) + .map(|space| space.name.clone()) + .unwrap_or_else(|| fallback_name.to_string()); + let at = FacetPrefix::Space { discriminant }; + rows.insert( + at, + ConventionRow { + at, + name: Some(space_name), + note: None, + state: ValidationState::Unmeasured, + }, + ); + } + + // One finest-depth row per named register, read straight off `ArchSpec::registers`. + for register in &arch.registers { + let at = FacetPrefix::SpaceOffsetSize { + discriminant: SPACE_REGISTER, + offset: register.offset, + size: register.size, + }; + rows.insert( + at, + ConventionRow { + at, + name: Some(register.name.clone()), + note: None, + state: ValidationState::Unmeasured, + }, + ); + } + + let mut userops = BTreeMap::new(); + for userop in &arch.userops { + userops.insert(userop.index, userop.name.clone()); + } + + Ok(Self { + rows, + spaces, + userops, + classified_opcodes: classified.into_iter().collect(), + arch: Some(arch.name.clone()), + }) + } + + /// Longest-prefix-wins resolution: try `SpaceOffsetSize`, then `SpaceOffset`, then `Space`; + /// first hit wins. `None` means the convention says nothing at this address — an addressed + /// residual for `slag.rs`. + pub fn resolve(&self, facet: &VarnodeFacet) -> Option<&ConventionRow> { + facet + .prefixes() + .into_iter() + .rev() + .find_map(|prefix| self.rows.get(&prefix)) + } + + /// The longest prefix that DID resolve — what an addressed residual reports so the + /// proposer knows where to attach the next, finer row. + pub fn resolved_prefix(&self, facet: &VarnodeFacet) -> Option { + facet + .prefixes() + .into_iter() + .rev() + .find(|prefix| self.rows.contains_key(prefix)) + } + + pub fn classifies(&self, op: OpTag) -> bool { + self.classified_opcodes.contains(&op) + } + + pub fn userop_name(&self, index: u32) -> Option<&str> { + self.userops.get(&index).map(String::as_str) + } + + pub fn spaces(&self) -> &CustomSpaceTable { + &self.spaces + } + + /// The proposer entry point: attach (or overwrite) a row at its own `at` prefix. + pub fn insert(&mut self, row: ConventionRow) { + self.rows.insert(row.at, row); + } + + /// `BTreeMap` order — deterministic, matching `to_toml`'s row order. + pub fn rows(&self) -> impl Iterator { + self.rows.values() + } + + /// Nested-TOML rendering, mirroring the harvest precedents. Hand-written (no serde dep): + /// a `[meta]` table (`measure_dont_claim`, `validation_states`, `arch`, + /// `classified_opcodes`), then one `[[row]]` per row in `BTreeMap` order with + /// `prefix_depth`, `space`, `offset`, `size`, `name`, `state`. **Emission only** — nothing + /// in this crate (or in ruff) parses this back. + pub fn to_toml(&self) -> String { + let mut out = String::new(); + + out.push_str("[meta]\n"); + out.push_str("measure_dont_claim = true\n"); + out.push_str( + "validation_states = [\"unmeasured\", \"confirmed\", \"corrected\", \"retired\"]\n", + ); + if let Some(arch) = &self.arch { + out.push_str("arch = "); + out.push_str(&toml_quote(arch)); + out.push('\n'); + } + out.push_str("classified_opcodes = ["); + for (index, op) in self.classified_opcodes.iter().enumerate() { + if index > 0 { + out.push_str(", "); + } + out.push_str(&toml_quote(op.as_str())); + } + out.push_str("]\n"); + + for row in self.rows.values() { + push_row_toml(&mut out, row); + } + + out + } +} + +/// Render one `[[row]]` table. `prefix_depth`/`space`/`offset`/`size` follow the prefix's own +/// depth — a `Space` row has no `offset`/`size` key at all rather than a fabricated `0`. +fn push_row_toml(out: &mut String, row: &ConventionRow) { + out.push_str("\n[[row]]\n"); + match row.at { + FacetPrefix::Space { discriminant } => { + out.push_str("prefix_depth = 1\n"); + out.push_str(&format!("space = {discriminant}\n")); + } + FacetPrefix::SpaceOffset { + discriminant, + offset, + } => { + out.push_str("prefix_depth = 2\n"); + out.push_str(&format!("space = {discriminant}\n")); + out.push_str(&format!("offset = {offset}\n")); + } + FacetPrefix::SpaceOffsetSize { + discriminant, + offset, + size, + } => { + out.push_str("prefix_depth = 3\n"); + out.push_str(&format!("space = {discriminant}\n")); + out.push_str(&format!("offset = {offset}\n")); + out.push_str(&format!("size = {size}\n")); + } + } + if let Some(name) = &row.name { + out.push_str("name = "); + out.push_str(&toml_quote(name)); + out.push('\n'); + } + if let Some(note) = &row.note { + out.push_str("note = "); + out.push_str(&toml_quote(note)); + out.push('\n'); + } + out.push_str(&format!("state = \"{}\"\n", row.state.as_str())); +} + +/// Minimal TOML basic-string quoting (`"`, `\`, and `\n` escaped). Emission-only, matching +/// `to_toml`'s own contract — this never parses TOML back. +fn toml_quote(value: &str) -> String { + let mut quoted = String::with_capacity(value.len() + 2); + quoted.push('"'); + for ch in value.chars() { + match ch { + '"' => quoted.push_str("\\\""), + '\\' => quoted.push_str("\\\\"), + '\n' => quoted.push_str("\\n"), + _ => quoted.push(ch), + } + } + quoted.push('"'); + quoted +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::facet::project; + use r2il::serialize::UserOpDef; + use r2il::{AddressSpace, RegisterDef, Varnode}; + + #[test] + fn arch_registers_bootstrap_the_register_branch() { + let mut arch = ArchSpec::new("test-arch"); + // Two registers share an offset and differ only in size — exactly why the finest + // prefix must carry size, not just space+offset. + arch.registers = vec![ + RegisterDef::new("rax", 0, 8), + RegisterDef::new("eax", 0, 4), + RegisterDef::new("rbx", 8, 8), + ]; + let conv = R2ilConvention::from_arch(&arch, std::iter::empty()) + .expect("no custom spaces on this arch, must not overflow"); + + let mut resolved_names = Vec::new(); + for register in &arch.registers { + let vn = Varnode::register(register.offset, register.size); + let facet = project(&vn, conv.spaces()).expect("register varnode must project"); + let row = conv + .resolve(&facet) + .expect("a bootstrapped register must resolve"); + assert_eq!(row.state, ValidationState::Unmeasured); + let name = row + .name + .as_deref() + .expect("a bootstrapped register row must carry a name"); + assert_eq!(name, register.name); + resolved_names.push(name.to_string()); + } + + // Anti-vacuity: the three resolved names must be DISTINCT — a bootstrap that mapped + // every register onto one shared row would otherwise still pass the loop above. + let mut distinct = resolved_names.clone(); + distinct.sort(); + distinct.dedup(); + assert_eq!( + distinct.len(), + 3, + "expected 3 distinct register names, got {resolved_names:?}" + ); + } + + #[test] + fn an_unknown_register_offset_falls_through_to_the_space_prefix() { + let mut arch = ArchSpec::new("test-arch"); + arch.registers = vec![RegisterDef::new("rax", 0, 8)]; + let conv = R2ilConvention::from_arch(&arch, std::iter::empty()) + .expect("no custom spaces, must not overflow"); + + // Two-sided: a KNOWN register resolves at the finest depth. + let known = Varnode::register(0, 8); + let known_facet = project(&known, conv.spaces()).expect("must project"); + assert_eq!( + conv.resolved_prefix(&known_facet) + .map(|prefix| prefix.depth()), + Some(3), + "a known register must resolve to the depth-3 row" + ); + + // An UNKNOWN register offset falls through to the coarse Space row — not to `None` + // and not to a depth-3 row. + let unknown = Varnode::register(0xDEAD, 8); + let unknown_facet = project(&unknown, conv.spaces()).expect("must project"); + let row = conv + .resolve(&unknown_facet) + .expect("an unknown register offset must still fall through to the space row"); + assert_eq!(row.state, ValidationState::Unmeasured); + assert_eq!( + conv.resolved_prefix(&unknown_facet) + .map(|prefix| prefix.depth()), + Some(1), + "an unknown register offset must resolve at depth 1, not depth 3 and not None" + ); + } + + #[test] + fn longest_prefix_wins_over_a_coarser_row() { + let mut conv = R2ilConvention::minimal_pass_one(); + let discriminant = SPACE_REGISTER; + let offset: u64 = 0x10; + let coarse_prefix = FacetPrefix::SpaceOffset { + discriminant, + offset, + }; + let fine_prefix = FacetPrefix::SpaceOffsetSize { + discriminant, + offset, + size: 8, + }; + conv.insert(ConventionRow { + at: coarse_prefix, + name: Some("coarse".to_string()), + note: None, + state: ValidationState::Unmeasured, + }); + conv.insert(ConventionRow { + at: fine_prefix, + name: Some("fine".to_string()), + note: None, + state: ValidationState::Unmeasured, + }); + + // The facet whose size matches the finer row resolves to it, not to the coarser one. + let matching_size = Varnode::register(offset, 8); + let matching_facet = project(&matching_size, conv.spaces()).expect("must project"); + let matching_row = conv + .resolve(&matching_facet) + .expect("must resolve to the finer row"); + assert_eq!(matching_row.name.as_deref(), Some("fine")); + + // A facet at the SAME space+offset but a DIFFERENT size has no depth-3 row, so it + // falls back to the coarser SpaceOffset row — falsifies a first-match-wins or + // coarsest-wins implementation. + let other_size = Varnode::register(offset, 4); + let other_facet = project(&other_size, conv.spaces()).expect("must project"); + let other_row = conv + .resolve(&other_facet) + .expect("must resolve to the coarser row"); + assert_eq!(other_row.name.as_deref(), Some("coarse")); + } + + #[test] + fn custom_space_overflow_fails_at_config_key_time() { + // A within-budget spec succeeds. + let mut ok_arch = ArchSpec::new("ok-arch"); + ok_arch.spaces = (0..4u32) + .map(|raw| AddressSpace::new(SpaceId::Custom(raw), format!("custom{raw}"), 8)) + .collect(); + assert!(R2ilConvention::from_arch(&ok_arch, std::iter::empty()).is_ok()); + + // A spec whose custom spaces exceed the lo-u16 budget errors — typed, not a wrap. + let over_budget = crate::facet::MAX_CUSTOM_ORDINAL as u32 + 2; + let mut overflow_arch = ArchSpec::new("overflow-arch"); + overflow_arch.spaces = (0..over_budget) + .map(|raw| AddressSpace::new(SpaceId::Custom(raw), format!("custom{raw}"), 8)) + .collect(); + let err = R2ilConvention::from_arch(&overflow_arch, std::iter::empty()) + .expect_err("a budget-exceeding custom space table must error, not truncate"); + assert!( + matches!(err, FacetOverflow::CustomOrdinalExhausted { .. }), + "expected CustomOrdinalExhausted, got {err:?}" + ); + } + + #[test] + fn toml_rendering_is_byte_stable_and_starts_every_row_unmeasured() { + let mut arch = ArchSpec::new("test-arch"); + arch.registers = vec![RegisterDef::new("rax", 0, 8), RegisterDef::new("rbx", 8, 8)]; + arch.userops = vec![UserOpDef { + index: 42, + name: "syscall_helper".to_string(), + }]; + let conv = R2ilConvention::from_arch(&arch, [OpTag::Copy, OpTag::IntAdd]) + .expect("no custom spaces, must not overflow"); + + let rendered_once = conv.to_toml(); + let rendered_twice = conv.to_toml(); + assert_eq!( + rendered_once, rendered_twice, + "to_toml must be byte-stable across calls" + ); + + assert!(rendered_once.contains("measure_dont_claim = true")); + assert!(rendered_once.contains("validation_states")); + // Every row bootstrapped by from_arch starts Unmeasured — none of the other three + // states appear anywhere in the render. + assert!(rendered_once.contains("state = \"unmeasured\"")); + assert!(!rendered_once.contains("state = \"confirmed\"")); + assert!(!rendered_once.contains("state = \"corrected\"")); + assert!(!rendered_once.contains("state = \"retired\"")); + } +} diff --git a/crates/ruff_r2il/src/facet.rs b/crates/ruff_r2il/src/facet.rs new file mode 100644 index 0000000000000..20c9144afb9a9 --- /dev/null +++ b/crates/ruff_r2il/src/facet.rs @@ -0,0 +1,404 @@ +//! The DRILL KEY: a 16-byte, V3-shaped, prefix-routable address over r2il varnode identity +//! space. +//! +//! **Role 1 (PR 1, load-bearing): the ADDRESS / CONFIG-KEY scheme.** [`VarnodeFacet`] is the +//! 16-byte V3-shaped identity `classid(space-class) | offset_lo | offset_hi | size`, +//! **prefix-routable by construction**. It is the key `convention.rs` drills on and the +//! coordinate `slag.rs` residuals are addressed at. +//! +//! **Role 2 (PR 2, NOT committed here): V3 SoA persistence.** Promoting the shape as a key +//! commits **no storage layout**. Same 16 bytes, two roles, no persistence decision yet. + +use r2il::{ArchSpec, SpaceId, Varnode}; + +/// Provisional container concept. The REAL mint is a canon-high slot in +/// `lance_graph_contract::ogar_codebook` (plan PR 3, the `NETWORK_LAYER = 0x0804` analog). +/// Until then this is LOCAL and provisional — never persist it as an address. +/// +/// ⚠ Known tension, recorded rather than hidden: OGAR's consumer rule is "hi u16 = shared +/// concept, lo u16 = APP render prefix — NEVER a shape ordinal", and the space discriminant +/// below IS a shape ordinal in the lo half. PR 3 owns the real carving. +pub const PROVISIONAL_R2IL_VARNODE: u16 = 0x0000; + +/// Fixed-space discriminants, valid for every `VarnodeFacet` regardless of architecture. +pub const SPACE_RAM: u16 = 0; +pub const SPACE_REGISTER: u16 = 1; +pub const SPACE_UNIQUE: u16 = 2; +pub const SPACE_CONST: u16 = 3; + +/// The first ordinal handed out to an interned `SpaceId::Custom(u32)` raw id. +pub const CUSTOM_ORDINAL_BASE: u16 = 4; + +/// The maximum number of distinct custom spaces a single [`CustomSpaceTable`] can intern +/// without its highest ordinal (`CUSTOM_ORDINAL_BASE + count - 1`) overflowing the lo-u16 +/// budget. 65531. +pub const MAX_CUSTOM_ORDINAL: u16 = u16::MAX - CUSTOM_ORDINAL_BASE; + +/// The 16-byte, little-endian drill key over one [`Varnode`]'s identity. +/// +/// Layout, all little-endian: +/// - `0..4` `classid: u32` = `((PROVISIONAL_R2IL_VARNODE as u32) << 16) | space_discriminant` +/// - `4..8` offset, low 32 bits +/// - `8..12` offset, high 32 bits +/// - `12..16` `size: u32` +/// +/// `Varnode::meta` is **documented-excluded** (advisory plane) — matching upstream, where +/// `Varnode`'s `PartialEq`/`Hash` already ignore it (`r2il/src/varnode.rs:149-163`). Two +/// varnodes that differ only in `meta` project to byte-identical facets. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct VarnodeFacet(pub [u8; 16]); + +impl VarnodeFacet { + /// The lo-u16 half of the classid word — the space-class discriminant. + pub fn space_discriminant(&self) -> u16 { + let classid = u32::from_le_bytes([self.0[0], self.0[1], self.0[2], self.0[3]]); + (classid & 0xFFFF) as u16 + } + + /// The full 64-bit offset, recombined from the low/high 32-bit halves. + pub fn offset(&self) -> u64 { + let lo = u32::from_le_bytes([self.0[4], self.0[5], self.0[6], self.0[7]]) as u64; + let hi = u32::from_le_bytes([self.0[8], self.0[9], self.0[10], self.0[11]]) as u64; + lo | (hi << 32) + } + + /// The varnode's size in bytes. + pub fn size(&self) -> u32 { + u32::from_le_bytes([self.0[12], self.0[13], self.0[14], self.0[15]]) + } + + /// The three prefix keys this facet resolves against, coarsest first: space-class alone, + /// then space-class+offset, then the full space-class+offset+size. + pub fn prefixes(&self) -> [FacetPrefix; 3] { + let discriminant = self.space_discriminant(); + let offset = self.offset(); + let size = self.size(); + [ + FacetPrefix::Space { discriminant }, + FacetPrefix::SpaceOffset { + discriminant, + offset, + }, + FacetPrefix::SpaceOffsetSize { + discriminant, + offset, + size, + }, + ] + } +} + +/// A config-tree key: a facet PREFIX. Ordered coarse → fine, exactly the three levels +/// [`R2ilConvention`] (`convention.rs`) drills on. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum FacetPrefix { + Space { + discriminant: u16, + }, + SpaceOffset { + discriminant: u16, + offset: u64, + }, + SpaceOffsetSize { + discriminant: u16, + offset: u64, + size: u32, + }, +} + +impl FacetPrefix { + /// `1` for [`FacetPrefix::Space`], `2` for [`FacetPrefix::SpaceOffset`], `3` for + /// [`FacetPrefix::SpaceOffsetSize`]. + pub fn depth(&self) -> u8 { + match self { + FacetPrefix::Space { .. } => 1, + FacetPrefix::SpaceOffset { .. } => 2, + FacetPrefix::SpaceOffsetSize { .. } => 3, + } + } +} + +/// Why a [`VarnodeFacet`] could not be projected or unprojected. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum FacetOverflow { + /// A `SpaceId::Custom(raw)` the table does not know. Projection REFUSES; it never falls + /// back to `raw as u16` (`65541` and `5` both truncate to `5`, silently colliding two + /// distinct architecture-defined spaces). + UnknownCustomSpace { raw: u32 }, + /// The set of distinct custom-space raw ids exceeds [`MAX_CUSTOM_ORDINAL`]. Overflow is + /// EVIDENCE a route needs factoring, never silent truncation (the `mint_factored` + /// principle). + CustomOrdinalExhausted { count: usize }, +} + +impl core::fmt::Display for FacetOverflow { + fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result { + match self { + FacetOverflow::UnknownCustomSpace { raw } => { + write!(f, "custom space {raw} is not in the interned table") + } + FacetOverflow::CustomOrdinalExhausted { count } => { + write!( + f, + "{count} custom spaces exceed the {MAX_CUSTOM_ORDINAL}-space lo-u16 budget" + ) + } + } + } +} + +impl core::error::Error for FacetOverflow {} + +/// Deterministic interning of `SpaceId::Custom(u32)` raw ids → lo-u16 ordinals, in sorted +/// order. +/// +/// Config-key-time losslessness (the promoted `Custom(u32)` falsifier): a `Custom` id that +/// overflows the interned-ordinal budget must fail **typed** at CONFIG-KEY construction, not +/// only at projection. A config tree keyed by a truncated address would silently attach rows +/// to the wrong varnode family — the worst possible failure for a drill scheme. Hence +/// [`CustomSpaceTable::from_ids`] / [`CustomSpaceTable::from_arch`] return `Result`, and +/// `convention.rs` propagates it. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct CustomSpaceTable { + ids: Vec, +} + +impl CustomSpaceTable { + /// Intern a set of raw `Custom` space ids. Deduplicates and sorts first, so ordinal + /// assignment is deterministic regardless of input order. + /// + /// `Err(FacetOverflow::CustomOrdinalExhausted)` when the deduplicated set exceeds + /// [`MAX_CUSTOM_ORDINAL`] — the `mint_factored` principle: overflow is EVIDENCE a route + /// needs factoring, never silent truncation. + pub fn from_ids>(ids: I) -> Result { + let mut ids: Vec = ids.into_iter().collect(); + ids.sort_unstable(); + ids.dedup(); + if ids.len() > MAX_CUSTOM_ORDINAL as usize { + return Err(FacetOverflow::CustomOrdinalExhausted { count: ids.len() }); + } + Ok(Self { ids }) + } + + /// Bootstrap from upstream data that already exists (read, never retype): every + /// [`r2il::AddressSpace`] in `arch.spaces` whose `id` is `SpaceId::Custom(n)`. + pub fn from_arch(arch: &ArchSpec) -> Result { + let ids = arch.spaces.iter().filter_map(|space| match space.id { + SpaceId::Custom(raw) => Some(raw), + _ => None, + }); + Self::from_ids(ids) + } + + /// The lo-u16 ordinal assigned to `raw`, or `None` if it was never interned. + pub fn ordinal_of(&self, raw: u32) -> Option { + self.ids + .binary_search(&raw) + .ok() + .map(|pos| CUSTOM_ORDINAL_BASE + pos as u16) + } + + /// The raw `Custom` id an interned ordinal came from, or `None` if `ordinal` is out of + /// range (below [`CUSTOM_ORDINAL_BASE`] or past the interned set). + pub fn raw_of(&self, ordinal: u16) -> Option { + let pos = ordinal.checked_sub(CUSTOM_ORDINAL_BASE)?; + self.ids.get(pos as usize).copied() + } + + /// The number of distinct custom spaces interned. + pub fn len(&self) -> usize { + self.ids.len() + } + + /// `true` when no custom spaces have been interned. + pub fn is_empty(&self) -> bool { + self.ids.is_empty() + } +} + +/// Build the drill key from the TYPED r2il varnode. This is the ONLY constructor — a facet is +/// never derived from an `SSAVar` (which has no offset and no `SpaceId`; see the crate's +/// upstream-facts table). +pub fn project(vn: &Varnode, spaces: &CustomSpaceTable) -> Result { + let discriminant = match vn.space { + SpaceId::Ram => SPACE_RAM, + SpaceId::Register => SPACE_REGISTER, + SpaceId::Unique => SPACE_UNIQUE, + SpaceId::Const => SPACE_CONST, + SpaceId::Custom(raw) => spaces + .ordinal_of(raw) + .ok_or(FacetOverflow::UnknownCustomSpace { raw })?, + }; + let classid = ((PROVISIONAL_R2IL_VARNODE as u32) << 16) | discriminant as u32; + let offset_lo = (vn.offset & 0xFFFF_FFFF) as u32; + let offset_hi = (vn.offset >> 32) as u32; + + let mut bytes = [0u8; 16]; + bytes[0..4].copy_from_slice(&classid.to_le_bytes()); + bytes[4..8].copy_from_slice(&offset_lo.to_le_bytes()); + bytes[8..12].copy_from_slice(&offset_hi.to_le_bytes()); + bytes[12..16].copy_from_slice(&vn.size.to_le_bytes()); + Ok(VarnodeFacet(bytes)) +} + +/// Recover a `Varnode` from a drill key. The inverse of [`project`]; `meta` is always `None` +/// on the result (it was never part of the projection — see [`VarnodeFacet`]'s docs). +pub fn unproject(f: &VarnodeFacet, spaces: &CustomSpaceTable) -> Result { + let discriminant = f.space_discriminant(); + let space = match discriminant { + SPACE_RAM => SpaceId::Ram, + SPACE_REGISTER => SpaceId::Register, + SPACE_UNIQUE => SpaceId::Unique, + SPACE_CONST => SpaceId::Const, + other => { + let raw = spaces + .raw_of(other) + .ok_or(FacetOverflow::UnknownCustomSpace { raw: other as u32 })?; + SpaceId::Custom(raw) + } + }; + Ok(Varnode::new(space, f.offset(), f.size())) +} + +#[cfg(test)] +mod tests { + use super::*; + use r2il::{ScalarKind, VarnodeMetadata}; + + #[test] + fn fixed_spaces_round_trip_byte_for_byte() { + let table = CustomSpaceTable::default(); + let cases = [ + Varnode::ram(0x1000, 8), + Varnode::register(0x40, 4), + Varnode::unique(0x7, 2), + Varnode::constant(0xFF, 1), + ]; + let mut classids = Vec::new(); + for vn in &cases { + let facet = project(vn, &table).expect("fixed space must project"); + classids.push(u32::from_le_bytes([ + facet.0[0], facet.0[1], facet.0[2], facet.0[3], + ])); + let back = unproject(&facet, &table).expect("fixed space must unproject"); + assert_eq!(&back, vn); + } + // Anti-vacuity: a constant-returning `project` would produce four identical classids. + let mut sorted = classids.clone(); + sorted.sort_unstable(); + sorted.dedup(); + assert_eq!( + sorted.len(), + classids.len(), + "the four fixed-space classid words must be pairwise distinct: {classids:?}" + ); + } + + #[test] + fn custom_space_within_budget_round_trips() { + let table = CustomSpaceTable::from_ids([3, 7, 9]).expect("3 ids is within budget"); + let vn = Varnode::new(SpaceId::Custom(7), 0x20, 4); + let facet = project(&vn, &table).expect("interned custom space must project"); + // Sorted order is [3, 7, 9]; 7 is at position 1. + assert_eq!(facet.space_discriminant(), CUSTOM_ORDINAL_BASE + 1); + let back = unproject(&facet, &table).expect("must unproject"); + assert_eq!(back, vn); + } + + #[test] + fn custom_space_outside_the_table_errors_and_never_truncates() { + let empty = CustomSpaceTable::default(); + let vn = Varnode::new(SpaceId::Custom(5), 0, 1); + assert_eq!( + project(&vn, &empty), + Err(FacetOverflow::UnknownCustomSpace { raw: 5 }) + ); + + // 65541 = 5 + 65536: the exact pair a `raw as u16` cast would collide on. + let table = CustomSpaceTable::from_ids([5, 65541]).expect("2 ids is within budget"); + let vn_small = Varnode::new(SpaceId::Custom(5), 0x10, 4); + let vn_big = Varnode::new(SpaceId::Custom(65541), 0x10, 4); + let f_small = project(&vn_small, &table).expect("project raw=5"); + let f_big = project(&vn_big, &table).expect("project raw=65541"); + assert_ne!(f_small, f_big, "65541 must not truncate onto 5's facet"); + assert_eq!( + unproject(&f_small, &table).expect("unproject raw=5"), + vn_small + ); + assert_eq!( + unproject(&f_big, &table).expect("unproject raw=65541"), + vn_big + ); + } + + #[test] + fn too_many_custom_spaces_is_a_typed_overflow_not_a_wrap() { + let budget = MAX_CUSTOM_ORDINAL as u32; + + let exactly_budget: Vec = (0..budget).collect(); + assert!( + CustomSpaceTable::from_ids(exactly_budget).is_ok(), + "exactly the budget must succeed" + ); + + let over_budget: Vec = (0..(budget + 2)).collect(); + match CustomSpaceTable::from_ids(over_budget) { + Err(FacetOverflow::CustomOrdinalExhausted { count }) => { + assert_eq!(count, (budget + 2) as usize); + } + other => panic!("expected a typed CustomOrdinalExhausted, got {other:?}"), + } + } + + #[test] + fn offsets_above_u32_max_survive_the_lo_hi_split() { + let table = CustomSpaceTable::default(); + let offset: u64 = 0x1234_5678_9ABC_DEF0; + let vn = Varnode::ram(offset, 8); + let facet = project(&vn, &table).expect("ram varnode must project"); + assert_eq!(facet.offset(), offset); + + // Anti-vacuity: the low 32-bit word alone must NOT already equal the full offset — + // otherwise a lo-only implementation (silently dropping the high half) would still + // pass the assertion above. + let lo = u32::from_le_bytes([facet.0[4], facet.0[5], facet.0[6], facet.0[7]]) as u64; + assert_ne!(lo, offset); + + let back = unproject(&facet, &table).expect("must unproject"); + assert_eq!(back, vn); + } + + #[test] + fn meta_is_excluded_from_the_projection() { + // Mirrors r2il::Varnode, whose PartialEq/Hash already ignore `meta` + // (r2il/src/varnode.rs:149-163) — the facet projection preserves that contract. + let table = CustomSpaceTable::default(); + let plain = Varnode::register(0x18, 4); + let with_meta = plain.clone().with_meta(VarnodeMetadata { + scalar_kind: Some(ScalarKind::SignedInt), + ..Default::default() + }); + let f_plain = project(&plain, &table).expect("project plain"); + let f_meta = project(&with_meta, &table).expect("project with meta"); + assert_eq!(f_plain, f_meta, "meta must not change the projected bytes"); + } + + #[test] + fn prefixes_are_ordered_coarse_to_fine_and_share_their_ancestors() { + let table = CustomSpaceTable::default(); + let vn = Varnode::register(0x40, 8); + let facet = project(&vn, &table).expect("project"); + let prefixes = facet.prefixes(); + assert_eq!(prefixes[0].depth(), 1); + assert_eq!(prefixes[1].depth(), 2); + assert_eq!(prefixes[2].depth(), 3); + + // Falsifies a prefix builder that ignores a component: a facet in the same space but + // at a different offset must share the coarsest prefix and differ at the next one. + let vn_other_offset = Varnode::register(0x48, 8); + let facet_other = project(&vn_other_offset, &table).expect("project other offset"); + let prefixes_other = facet_other.prefixes(); + assert_eq!(prefixes[0], prefixes_other[0]); + assert_ne!(prefixes[1], prefixes_other[1]); + } +} diff --git a/crates/ruff_r2il/src/furnace.rs b/crates/ruff_r2il/src/furnace.rs new file mode 100644 index 0000000000000..dc8273a457eee --- /dev/null +++ b/crates/ruff_r2il/src/furnace.rs @@ -0,0 +1,1134 @@ +//! Stage 3 — the **furnace**: melts the stage-1/2 object graph into flat, facet-addressed, +//! concern-separated rows. +//! +//! [`FlatFact`] is `FactId + VarnodeFacet + Concern + fixed scalar payload` — plain flat +//! `Vec`. **It must never grow back into a nested object graph.** No `Vec`, no `Box`, +//! no map, no `String` inside a `FlatFact`; cardinality is handled by EMITTING MORE ROWS (a +//! `CallOther` with four inputs is one [`FactKind::Op`] row plus four [`FactKind::OperandIn`] +//! rows), never by nesting a collection. A `FlatFact` refers to another only by [`FactId`] (via +//! its shared `prov.inst`/`prov.op_site` — `furnace.rs` never invents a graph edge between rows). +//! +//! # The whole pass-1 ladder — nothing else classifies +//! +//! - An [`crate::ore::OreFact::Op`] melts iff `conv.classifies(opcode)` **and** a block-anchor +//! facet is derivable (`facet::project(&Varnode::ram(block_addr, 0), conv.spaces())`, from the +//! op's own `prov.op_site`). When it doesn't classify, the residual reason is +//! [`crate::slag::ResidualReason::VariadicArity`] when `input_arity` exceeds +//! [`VARIADIC_ARITY_THRESHOLD`] (a "this needs Vec-routing regardless of classification" signal +//! — CallOther's four-input stressor is the fixture that motivates the threshold value, see its +//! doc comment), else [`crate::slag::ResidualReason::OpcodeNotInConvention`]. +//! - An [`crate::ore::OreFact::Operand`] melts iff its OWN varnode facet-projects +//! **and** its parent op (by `prov.inst`) melted **and** `conv.resolve(&facet)` is `Some`. +//! **Facet-projection failure is checked FIRST, independent of the parent-melted gate** — an +//! operand referencing a `SpaceId::Custom` raw id the convention's space table doesn't know +//! fails with [`crate::slag::ResidualReason::CustomSpaceNotInConvention`] even when its parent +//! op also fails to classify (both residuals are real and both are pushed, from two distinct +//! ore facts: the `Op` row and the `Operand` row — never one row absorbing two reasons). +//! - [`crate::ore::OreFact::Edge`], [`crate::ore::OreFact::MemoryUse`], +//! [`crate::ore::OreFact::MemoryDef`], [`crate::ore::OreFact::Predicate`], +//! [`crate::ore::OreFact::CallSite`] are all BLOCK-ANCHORED (they carry no varnode of their +//! own): `at = facet::project(&Varnode::ram(block_addr, 0), conv.spaces())`, using the +//! enclosing op's `prov.op_site` (or, for `Edge`, the source block's own address — `Edge` +//! carries no `prov` at all, see its own arm below). +//! - `MemoryUse`/`MemoryDef` additionally need their `object` to resolve to a NON-`EscapedUnknown` +//! [`r2ssa::ObjectKind`] (`ResidualReason::MemoryObjectEscaped` otherwise). +//! - `CallSite` additionally needs `direct_target.is_some()` (`ResidualReason::IndirectTarget` +//! otherwise). +//! - `Predicate` and `Edge` have no additional per-row condition beyond their parent melting +//! (`Edge` has no parent at all — see below). +//! - All four (Edge excepted) gate on their parent op having melted, exactly like `Operand`. +//! - [`crate::ore::OreFact::PhiInput`] **never** becomes a [`FlatFact`] — `FlatFact.at` is a +//! non-optional [`VarnodeFacet`], and a phi input has no source varnode at all (this crate's +//! upstream-facts table: `SSAVar` carries no offset and no `SpaceId`, and a phi input is an +//! abstract SSA join edge, not a single varnode occurrence). It is ALWAYS residual: the +//! well-formed case (`index < predecessors_count` for the merge block) is +//! [`crate::slag::ResidualReason::NoFacetCoordinate`] (`at: None`, matching the spec text +//! naming phi inputs as the row kind whose facet is legitimately absent); the anomalous case +//! (`index >= predecessors_count`, which should not occur given `SSAFunction`'s own fan-in +//! zip — see this crate's upstream-facts table) is +//! [`crate::slag::ResidualReason::PhiFanInExceedsPredecessors`], best-effort block-anchored via +//! `prov.block` so it still carries an address. +//! - [`crate::ore::OreFact::JoinFailure`] never melts — +//! [`crate::slag::ResidualReason::OpSiteJoinMismatch`], block-anchored via `prov.op_site`. +//! +//! There is no third outcome besides melt/slag: every [`crate::ore::OreFact`] produces EXACTLY +//! one [`FlatFact`] or EXACTLY one [`crate::slag::ResidualFact`], so `dropped == 0` holds by +//! construction of [`smelt`]'s control flow, not by a separate bookkeeping check. +//! +//! # `FlatFact` payload table (`a`, `b` — both `u64`) +//! +//! | `kind` | `a` | `b` | +//! |---|---|---| +//! | [`FactKind::Op`] | the source instruction's `ordinal` within its block | `input_arity` in bits `0..32`, `has_output` (`0`/`1`) in bit `32` | +//! | [`FactKind::OperandIn`] | the input index (`OperandPos::Input(i)`) | the operand's `ValueId.0 + 1`, or `0` when the operand carries no SSA value | +//! | [`FactKind::OperandOut`] | unused, always `0` (an output has no index) | the operand's `ValueId.0 + 1`, or `0` | +//! | [`FactKind::Edge`] | the destination `BlockId.0`, widened | the [`crate::ore::EdgeTag`] ordinal: `Normal=0, True=1, False=2, Back=3` | +//! | [`FactKind::MemUse`] | the `ObjectId.0`, widened | `version` in bits `0..32`, `size` in bits `32..64` | +//! | [`FactKind::MemDef`] | `previous` in bits `0..32`, `next` in bits `32..64` | the `ObjectId.0` in bits `0..32`, `size` in bits `32..64` | +//! | [`FactKind::Predicate`] | `true_target` (a real code address) | `false_target` (a real code address) | +//! | [`FactKind::CallSite`] | `direct_target` (only ever melted when `Some`) | the call target's `ValueId.0`, widened | +//! +//! `PhiInput` never appears as a `FlatFact.kind` — see the ladder above. `condition` and +//! `comparison` on a melted `Predicate` row are NOT captured in the fixed two-slot payload (a +//! documented, honest PR-1 loss, not a silent drop — the source `OreFact::Predicate` retains full +//! fidelity; a future pass may split `Predicate` into a targets row and a comparison row rather +//! than widen the payload past two `u64` slots). `id` on `Predicate`/`CallSite` is likewise not +//! duplicated into the payload — it is recoverable from `prov.inst`. + +use std::collections::BTreeMap; + +use r2il::{R2ILBlock, Varnode}; +use r2ssa::{BlockId, ObjectKind}; + +use crate::behavior::FunctionBehavior; +use crate::convention::R2ilConvention; +use crate::facet::{self, FacetOverflow, VarnodeFacet}; +use crate::ore::{self, EdgeTag, FactProvenance, OpTag, OperandPos, OreFact}; +use crate::slag::{ResidualFact, ResidualLedger, ResidualReason}; + +/// An op's `input_arity` strictly above this threshold gets +/// [`crate::slag::ResidualReason::VariadicArity`] instead of +/// [`crate::slag::ResidualReason::OpcodeNotInConvention`] when it fails to classify. +/// +/// `JUDGMENT` — the plan names the residual but not the exact cutoff. Pinned against the §8/§10 +/// stressor block's own arities: `AtomicCAS`/`StoreGuarded`/`Insert` all have arity 3 and are +/// expected to surface as `OpcodeNotInConvention`; the 4-input `CallOther` is the ONLY op in that +/// fixture with arity 4 and is expected to surface as `VariadicArity{arity: 4}` — `3` is the +/// largest threshold consistent with both halves of that expectation. +const VARIADIC_ARITY_THRESHOLD: usize = 3; + +// ================================================================================================ +// FactId / Concern / FactKind +// ================================================================================================ + +/// The index of a [`FlatFact`] within one [`smelt`] call's output `Vec`. Assigned sequentially in +/// emission order; NOT a `prov.inst`/`prov.op_site` alias — those already carry the upstream +/// identity, this is purely the flat row's own position. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct FactId(pub u32); + +/// The route a [`FlatFact`] belongs to — r2sleigh's own decomposition, named. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Concern { + /// [`FactKind::Op`], [`FactKind::Edge`] — instruction identity and control-flow structure. + Control, + /// [`FactKind::OperandIn`], [`FactKind::OperandOut`] — SSA value flow. + Values, + /// Reserved for a future object-lifecycle `FactKind`; no pass-1 `FactKind` uses it. + Objects, + /// [`FactKind::MemUse`], [`FactKind::MemDef`] — memory SSA versioning. + Memory, + /// [`FactKind::Predicate`]. + Predicates, + /// [`FactKind::CallSite`]. + Calls, +} + +/// Which shape of ore fact a [`FlatFact`] flattens. See the module docs' payload table for the +/// `a`/`b` meaning per variant. `PhiInput` from [`crate::ore::OreFact`] has no corresponding +/// variant here — it never melts (module docs, "the whole pass-1 ladder"). +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum FactKind { + Op, + OperandIn, + OperandOut, + Edge, + MemUse, + MemDef, + Predicate, + CallSite, +} + +impl FactKind { + /// Stable `snake_case` name, for [`Census`]. Never derived from `Debug`. + pub fn as_str(self) -> &'static str { + match self { + FactKind::Op => "op", + FactKind::OperandIn => "operand_in", + FactKind::OperandOut => "operand_out", + FactKind::Edge => "edge", + FactKind::MemUse => "mem_use", + FactKind::MemDef => "mem_def", + FactKind::Predicate => "predicate", + FactKind::CallSite => "call_site", + } + } +} + +// ================================================================================================ +// FlatFact +// ================================================================================================ + +/// ONE flat row. No `Vec`, no `Box`, no map, no `String` — see the module docs' opening +/// paragraph. The compile-time guard below is the never-a-nested-graph regression test, +/// mechanised: a field addition that breaks `Copy` or grows the struct past the pinned budget +/// fails the BUILD, not merely a test run. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct FlatFact { + pub id: FactId, + /// The drill key. Operand rows carry their OWN varnode's facet; rows with no varnode of + /// their own are BLOCK-ANCHORED — `facet::project(&Varnode::ram(block_addr, 0), spaces)` — + /// a documented convention (see the module docs' ladder), never an implicit default. + pub at: VarnodeFacet, + pub concern: Concern, + pub kind: FactKind, + pub opcode: OpTag, + /// Two fixed typed payload slots. Meaning per `kind`: the module docs' payload table. + pub a: u64, + pub b: u64, + pub prov: FactProvenance, +} + +/// The never-a-nested-graph guard, mechanised. +/// +/// The property being guarded is **flatness**, not a byte count: no `Vec`, `Box`, `String`, or map +/// may appear inside a [`FlatFact`], cardinality is expressed by emitting MORE ROWS, and a row +/// refers to another only by [`FactId`]. `Copy` is the load-bearing half — a type containing any +/// heap indirection cannot derive it — and the size pin is what catches a field being added or a +/// nested type sneaking in. +/// +/// **The pin is MEASURED, not budgeted.** The spec drafted `<= 64` before anyone had computed the +/// real layout; on a 64-bit target the true size is 88, and the arithmetic is entirely accounted +/// for: `id` 4 + `at` 16 + `concern`/`kind`/`opcode` 1 each + `a` 8 + `b` 8 + +/// [`FactProvenance`] 48 = 87, rounded to 88 by the align-8 requirement. `FactProvenance` is 48 +/// because its four `Option`s have no niche to exploit — `Option`, `Option` and +/// `Option` are 8 each and `Option<(u64, usize)>` is 24. Nothing here is heap-allocated; +/// the type is exactly as flat as intended, and 64 was simply the wrong number. +/// +/// An EXACT pin is deliberately stronger than the spec's `<=`: growth AND shrinkage both fail, so +/// a future field addition or a layout change cannot pass silently. +/// +/// Deferred, and recorded rather than silently taken: `op_site: Option<(u64, usize)>` spends 8 +/// bytes on an op index that never needs 64 bits, so a `(u64, u32)` pair would bring the row to +/// 80. That is an `ore.rs` type change with test fallout, and the row's physical width only starts +/// to matter at stage-5 persistence (PR 2, where cache-line straddling is a real cost) — not for +/// an in-memory intermediate. Revisit it there, with a measurement. +const FLAT_FACT_SIZE: usize = 88; +const _: () = assert!(core::mem::size_of::() == FLAT_FACT_SIZE); + +// ================================================================================================ +// HarvestReport / Census +// ================================================================================================ + +/// The conservation ledger — the `harvested N / classified X / residual Y / dropped 0` line. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct HarvestReport { + pub harvested: usize, + pub classified: usize, + pub residual: usize, + pub dropped: usize, +} + +impl HarvestReport { + /// `harvested == classified + residual` AND `dropped == 0`. + #[must_use] + pub fn is_conserved(&self) -> bool { + self.dropped == 0 && self.harvested == self.classified + self.residual + } +} + +/// Per-fact-kind and per-opcode counts over one [`smelt`]'s `Vec` output — +/// the "N triples by predicate" analog. `BTreeMap`, so the census artifact is byte-stable across +/// runs (matching `FactKind`/`OpTag`'s own `as_str()` discipline: never a `Debug`-derived key). +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct Census { + pub by_fact_kind: BTreeMap<&'static str, usize>, + pub by_opcode: BTreeMap<&'static str, usize>, +} + +#[must_use] +pub fn census(rows: &[FlatFact]) -> Census { + let mut by_fact_kind: BTreeMap<&'static str, usize> = BTreeMap::new(); + let mut by_opcode: BTreeMap<&'static str, usize> = BTreeMap::new(); + for row in rows { + *by_fact_kind.entry(row.kind.as_str()).or_insert(0) += 1; + *by_opcode.entry(row.opcode.as_str()).or_insert(0) += 1; + } + Census { + by_fact_kind, + by_opcode, + } +} + +// ================================================================================================ +// smelt +// ================================================================================================ + +/// Melt one function's ore into flat, addressed rows under `conv`. `Ok` rows are flat and +/// addressed; everything else is addressed slag. See the module docs for the complete pass-1 +/// ladder — this is the ONLY place that ladder is implemented; widening classification is a +/// `R2ilConvention` data change, never a new match arm here. +#[must_use] +pub fn smelt( + behavior: &FunctionBehavior, + blocks: &[R2ILBlock], + conv: &R2ilConvention, +) -> (Vec, ResidualLedger, HarvestReport) { + let ore_facts = ore::enumerate(behavior, blocks); + let harvested = ore_facts.len(); + + let mut flat: Vec = Vec::new(); + let mut ledger = ResidualLedger::new(); + // `InstId.0` -> (did this instruction's Op row actually melt?, its OpTag). Populated as we + // go; every Op row for a given inst is guaranteed (by `ore::enumerate`'s documented + // enumeration order — "per op the Op row, then Operand rows...") to be processed before any + // dependent fact referencing the same `prov.inst`, so a single forward pass suffices. + let mut op_status: BTreeMap = BTreeMap::new(); + + for fact in ore_facts { + match fact { + OreFact::Op { + prov, + opcode, + ordinal, + input_arity, + has_output, + } => { + let at = any_anchor_facet(behavior, prov, conv); + let classifies = conv.classifies(opcode); + let melted = classifies && at.is_some(); + if let Some(inst) = prov.inst { + op_status.insert(inst.0, (melted, opcode)); + } + if !classifies { + let reason = if input_arity > VARIADIC_ARITY_THRESHOLD { + ResidualReason::VariadicArity { arity: input_arity } + } else { + ResidualReason::OpcodeNotInConvention { opcode } + }; + push_named_residual(&mut ledger, reason, at, conv, prov); + } else if let Some(at) = at { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Control, + kind: FactKind::Op, + opcode, + a: ordinal as u64, + b: pack_op_metadata(input_arity, has_output), + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } + + OreFact::Operand { + prov, + position, + value, + space, + offset, + size, + } => { + let (parent_melted, parent_opcode) = prov + .inst + .and_then(|inst| op_status.get(&inst.0).copied()) + // Structurally unreachable given `ore::enumerate`'s Op-before-Operand + // ordering guarantee (module docs) — `OpTag::Copy` is an arbitrary + // placeholder for the "not melted" fallback this can never actually take. + .unwrap_or((false, OpTag::Copy)); + let varnode = Varnode::new(space, offset, size); + match facet::project(&varnode, conv.spaces()) { + Ok(own_facet) => { + if !parent_melted { + push_named_residual( + &mut ledger, + ResidualReason::OpcodeNotInConvention { + opcode: parent_opcode, + }, + Some(own_facet), + conv, + prov, + ); + } else if conv.resolve(&own_facet).is_some() { + let kind = match position { + OperandPos::Input(_) => FactKind::OperandIn, + OperandPos::Output => FactKind::OperandOut, + }; + let a = match position { + OperandPos::Input(idx) => idx as u64, + OperandPos::Output => 0, + }; + let b = value.map(|v| v.0 as u64 + 1).unwrap_or(0); + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at: own_facet, + concern: Concern::Values, + kind, + opcode: parent_opcode, + a, + b, + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoConventionRowAtAddress, + Some(own_facet), + conv, + prov, + ); + } + } + Err(FacetOverflow::UnknownCustomSpace { raw }) => { + // Own-facet projection failed before the parent-melted gate is even + // consulted — see the module docs' ladder note on priority. Falls back + // to the enclosing op's block-anchor so the residual still carries an + // address (every reason but `NoFacetCoordinate` must). + let at = any_anchor_facet(behavior, prov, conv); + push_named_residual( + &mut ledger, + ResidualReason::CustomSpaceNotInConvention { raw }, + at, + conv, + prov, + ); + } + Err(FacetOverflow::CustomOrdinalExhausted { count }) => { + // Structurally unreachable for a validly-constructed `CustomSpaceTable` + // (the budget is enforced at `from_ids`/`from_arch` time) — handled + // rather than unwrapped, per the crate's no-panic discipline. + let at = any_anchor_facet(behavior, prov, conv); + push_named_residual( + &mut ledger, + ResidualReason::FacetOverflowAtKey { raw: count as u32 }, + at, + conv, + prov, + ); + } + } + } + + OreFact::Edge { from, to, kind } => { + // `Edge` carries no `prov` at all (`crate::ore::OreFact::Edge`'s own definition) + // — there is no parent op to gate on, so a CFG edge melts unconditionally given a + // derivable block-anchor facet. `JUDGMENT`: CFG topology is intrinsic model + // structure (a block's successor list exists independent of whether its + // terminating op happens to be classified), unlike every other fact kind, which + // is gated on its owning instruction. + let from_addr = block_addr_of(behavior, from); + let at = from_addr + .and_then(|addr| facet::project(&Varnode::ram(addr, 0), conv.spaces()).ok()); + let opcode = from_addr.and_then(|addr| terminator_opcode(blocks, addr)); + let synth_prov = FactProvenance { + inst: None, + block: Some(from), + op_site: None, + value: None, + }; + match (at, opcode) { + (Some(at), Some(opcode)) => { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Control, + kind: FactKind::Edge, + opcode, + a: to.0 as u64, + b: edge_tag_ordinal(kind), + prov: synth_prov, + }); + } + _ => { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + synth_prov, + ); + } + } + } + + OreFact::PhiInput { prov, index, .. } => { + // Never melts — see the module docs' ladder. Fan-in well-formedness (index vs. + // the merge block's real predecessor count) selects WHICH residual reason + // applies, not whether one applies at all. + let predecessor_count = prov + .block + .and_then(|block| block_addr_of(behavior, block)) + .map(|addr| behavior.control().predecessors(addr).len()); + match predecessor_count { + Some(count) if index >= count => { + let at = prov + .block + .and_then(|block| block_addr_of(behavior, block)) + .and_then(|addr| { + facet::project(&Varnode::ram(addr, 0), conv.spaces()).ok() + }); + push_named_residual( + &mut ledger, + ResidualReason::PhiFanInExceedsPredecessors { + inputs: index + 1, + predecessors: count, + }, + at, + conv, + prov, + ); + } + _ => { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } + } + + OreFact::MemoryUse { + prov, + object, + version, + size, + } => { + let (parent_melted, parent_opcode) = prov + .inst + .and_then(|inst| op_status.get(&inst.0).copied()) + .unwrap_or((false, OpTag::Copy)); + let at = any_anchor_facet(behavior, prov, conv); + let escaped = object_is_escaped(behavior, object); + if !parent_melted { + push_named_residual( + &mut ledger, + ResidualReason::OpcodeNotInConvention { + opcode: parent_opcode, + }, + at, + conv, + prov, + ); + } else if escaped { + push_named_residual( + &mut ledger, + ResidualReason::MemoryObjectEscaped, + at, + conv, + prov, + ); + } else if let Some(at) = at { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Memory, + kind: FactKind::MemUse, + opcode: parent_opcode, + a: object.0 as u64, + b: (version as u64) | ((size as u64) << 32), + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } + + OreFact::MemoryDef { + prov, + object, + previous, + next, + size, + } => { + let (parent_melted, parent_opcode) = prov + .inst + .and_then(|inst| op_status.get(&inst.0).copied()) + .unwrap_or((false, OpTag::Copy)); + let at = any_anchor_facet(behavior, prov, conv); + let escaped = object_is_escaped(behavior, object); + if !parent_melted { + push_named_residual( + &mut ledger, + ResidualReason::OpcodeNotInConvention { + opcode: parent_opcode, + }, + at, + conv, + prov, + ); + } else if escaped { + push_named_residual( + &mut ledger, + ResidualReason::MemoryObjectEscaped, + at, + conv, + prov, + ); + } else if let Some(at) = at { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Memory, + kind: FactKind::MemDef, + opcode: parent_opcode, + a: (previous as u64) | ((next as u64) << 32), + b: (object.0 as u64) | ((size as u64) << 32), + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } + + OreFact::Predicate { + prov, + true_target, + false_target, + .. + } => { + let (parent_melted, parent_opcode) = prov + .inst + .and_then(|inst| op_status.get(&inst.0).copied()) + .unwrap_or((false, OpTag::Copy)); + let at = any_anchor_facet(behavior, prov, conv); + if !parent_melted { + push_named_residual( + &mut ledger, + ResidualReason::OpcodeNotInConvention { + opcode: parent_opcode, + }, + at, + conv, + prov, + ); + } else if let Some(at) = at { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Predicates, + kind: FactKind::Predicate, + opcode: parent_opcode, + a: true_target, + b: false_target, + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } + + OreFact::CallSite { + prov, + target, + direct_target, + .. + } => { + let (parent_melted, parent_opcode) = prov + .inst + .and_then(|inst| op_status.get(&inst.0).copied()) + .unwrap_or((false, OpTag::Copy)); + let at = any_anchor_facet(behavior, prov, conv); + if !parent_melted { + push_named_residual( + &mut ledger, + ResidualReason::OpcodeNotInConvention { + opcode: parent_opcode, + }, + at, + conv, + prov, + ); + } else if let Some(target_addr) = direct_target { + if let Some(at) = at { + flat.push(FlatFact { + id: FactId(flat.len() as u32), + at, + concern: Concern::Calls, + kind: FactKind::CallSite, + opcode: parent_opcode, + a: target_addr, + b: target.0 as u64, + prov, + }); + } else { + push_named_residual( + &mut ledger, + ResidualReason::NoFacetCoordinate, + None, + conv, + prov, + ); + } + } else { + push_named_residual( + &mut ledger, + ResidualReason::IndirectTarget, + at, + conv, + prov, + ); + } + } + + OreFact::JoinFailure { + prov, + expected, + found, + } => { + let at = any_anchor_facet(behavior, prov, conv); + push_named_residual( + &mut ledger, + ResidualReason::OpSiteJoinMismatch { expected, found }, + at, + conv, + prov, + ); + } + } + } + + let report = HarvestReport { + harvested, + classified: flat.len(), + residual: ledger.len(), + dropped: 0, + }; + (flat, ledger, report) +} + +// ================================================================================================ +// helpers +// ================================================================================================ + +/// The block-anchored facet convention: `facet::project(&Varnode::ram(block_addr, 0), spaces)`, +/// using the enclosing op's `(block_addr, op_idx)` site. `None` when `op_site` itself is `None`, +/// or (defensively; should not occur for a `Ram` varnode under any convention) when projection +/// fails. +fn block_anchor_facet( + op_site: Option<(u64, usize)>, + conv: &R2ilConvention, +) -> Option { + let (block_addr, _) = op_site?; + facet::project(&Varnode::ram(block_addr, 0), conv.spaces()).ok() +} + +/// Anchor a row that has no `op_site` but DOES know its block. +/// +/// Measured gap this closes: an SSA **phi** is a real instruction with a `prov.block` but no +/// source `op_site` (it has no R2IL op behind it), and an [`crate::ore::OreFact::Edge`] carries no +/// `prov.inst` at all — so both fell through [`block_anchor_facet`] and produced an UNADDRESSED +/// residual. Under the addressed-slag rule that is a defect, not an acceptable default: a residual +/// the proposer cannot locate is a residual it cannot answer. Both are block-addressable, so they +/// are addressed here. +/// +/// This is strictly a fallback — a row with a real `op_site` still anchors on it. `None` survives +/// only for rows with neither coordinate, which under the pass-1 ladder means exactly the phi +/// INPUT rows whose reason is [`crate::slag::ResidualReason::NoFacetCoordinate`] (an SSA join edge +/// is not a varnode occurrence, so it legitimately has no facet). +fn any_anchor_facet( + behavior: &FunctionBehavior, + prov: FactProvenance, + conv: &R2ilConvention, +) -> Option { + if let Some(facet) = block_anchor_facet(prov.op_site, conv) { + return Some(facet); + } + let block_addr = block_addr_of(behavior, prov.block?)?; + facet::project(&Varnode::ram(block_addr, 0), conv.spaces()).ok() +} + +/// `BlockId` -> its real address, via [`crate::behavior::FunctionBehavior::values`]'s +/// `SsaGraph::block`. +fn block_addr_of(behavior: &FunctionBehavior, block: BlockId) -> Option { + behavior + .values() + .block(block) + .map(|graph_block| graph_block.addr) +} + +/// The `OpTag` of a block's terminating (last) source op — used as `Edge`'s `opcode`, since an +/// `Edge` has no parent instruction of its own to take one from. `None` when the block can't be +/// found in `blocks` or has no ops at all (both defensive — every real block has a terminator). +fn terminator_opcode(blocks: &[R2ILBlock], addr: u64) -> Option { + blocks + .iter() + .find(|block| block.addr == addr) + .and_then(|block| block.ops.last()) + .map(OpTag::from_r2il) +} + +/// Whether `object` resolves to `ObjectKind::EscapedUnknown` — or is simply unknown to the +/// object model, which is treated the same conservative way (not classifiable this pass). +fn object_is_escaped(behavior: &FunctionBehavior, object: r2ssa::ObjectId) -> bool { + behavior + .objects() + .object(object) + .map(|fact| matches!(fact.kind, ObjectKind::EscapedUnknown)) + .unwrap_or(true) +} + +/// Pack an `Op` row's `input_arity`/`has_output` into one `u64`: arity in bits `0..32`, +/// `has_output` in bit `32`. +fn pack_op_metadata(input_arity: usize, has_output: bool) -> u64 { + (input_arity as u64) | ((has_output as u64) << 32) +} + +/// `EdgeTag` -> its `u64` payload ordinal (see the module docs' payload table). +fn edge_tag_ordinal(tag: EdgeTag) -> u64 { + match tag { + EdgeTag::Normal => 0, + EdgeTag::True => 1, + EdgeTag::False => 2, + EdgeTag::Back => 3, + } +} + +/// Push one residual, computing `shape_id` from the reason and `at_prefix` (opportunistically, +/// from whatever facet we DO have — see [`crate::convention::R2ilConvention::resolved_prefix`]) +/// in one place, so every one of the nine `OreFact` arms above shares the exact same bookkeeping. +fn push_named_residual( + ledger: &mut ResidualLedger, + reason: ResidualReason, + at: Option, + conv: &R2ilConvention, + provenance: FactProvenance, +) { + let at_prefix = at.as_ref().and_then(|facet| conv.resolved_prefix(facet)); + ledger.push(ResidualFact { + shape_id: reason.shape_id(), + reason, + at, + at_prefix, + provenance, + }); +} + +// ================================================================================================ +// tests +// ================================================================================================ + +#[cfg(test)] +mod tests { + use super::*; + use r2il::{ArchSpec, MemoryOrdering, RegisterDef, SpaceId}; + + #[test] + fn a_flat_fact_stays_flat() { + // The compile-time guard above is the real contract; this pins the same MEASURED constant + // at runtime plus the `Copy` bound (the half that actually proves no heap indirection — + // a type owning a `Vec`/`Box`/`String`/map cannot derive `Copy`). + // + // The pin is exact rather than a bound, so both growth and shrinkage fail loudly. It is + // measured on a 64-bit target; a genuinely different ABI would need a re-measure, not a + // loosened assertion — and the arithmetic behind 88 is spelled out at the const assert. + assert_eq!(core::mem::size_of::(), FLAT_FACT_SIZE); + fn assert_copy() {} + assert_copy::(); + } + + #[test] + fn cardinality_is_rows_not_nesting() { + let arch = { + let mut arch = ArchSpec::new("test-arch"); + arch.registers = vec![ + RegisterDef::new("r0", 0, 8), + RegisterDef::new("r1", 8, 8), + RegisterDef::new("r2", 16, 8), + RegisterDef::new("r3", 24, 8), + RegisterDef::new("r4", 40, 8), + ]; + arch + }; + let mut block = R2ILBlock::new(0x3000, 4); + block.push(r2il::R2ILOp::CallOther { + output: Some(Varnode::register(40, 8)), + userop: 1, + inputs: vec![ + Varnode::register(0, 8), + Varnode::register(8, 8), + Varnode::register(16, 8), + Varnode::register(24, 8), + ], + }); + block.push(r2il::R2ILOp::Return { + target: Varnode::register(0, 8), + }); + let blocks = vec![block]; + let behavior = FunctionBehavior::from_blocks_raw(&blocks, Some(&arch)) + .expect("single-block CallOther+Return fixture must ingest"); + let conv = R2ilConvention::from_arch(&arch, [OpTag::CallOther, OpTag::Return]) + .expect("no custom spaces on this arch, must not overflow"); + + let (flat, _ledger, report) = smelt(&behavior, &blocks, &conv); + assert!(report.is_conserved()); + + let op_rows: Vec<&FlatFact> = flat + .iter() + .filter(|row| row.kind == FactKind::Op && row.opcode == OpTag::CallOther) + .collect(); + let operand_in_rows: Vec<&FlatFact> = flat + .iter() + .filter(|row| row.kind == FactKind::OperandIn && row.opcode == OpTag::CallOther) + .collect(); + assert_eq!( + op_rows.len(), + 1, + "expected exactly 1 Op row, got {op_rows:?}" + ); + assert_eq!( + operand_in_rows.len(), + 4, + "expected exactly 4 OperandIn rows (cardinality is rows, never nesting), got {operand_in_rows:?}" + ); + + let inst = op_rows[0].prov.inst; + assert!(inst.is_some()); + assert!( + operand_in_rows.iter().all(|row| row.prov.inst == inst), + "every operand row must trace back to the SAME instruction as the Op row" + ); + } + + #[test] + fn melt_is_conserved_and_drops_nothing() { + // A self-contained, linear (single-block, no branches) fixture reproducing the SHAPE of + // the plan's §8 stressor block (many diverse ops) without depending on + // `tests/lossless_fixtures.rs`, which this file does not own. 12 binary ops (2 inputs + 1 + // output each) plus Load/Store/Copy/Call/Return comfortably clear the >= 50 anti-vacuity + // bound on Op+Operand facts ALONE, independent of however many memory/call-site facts + // upstream additionally derives. + let mut block = R2ILBlock::new(0x4000, 40); + let reg_a = Varnode::register(8, 8); + let reg_b = Varnode::register(16, 8); + let binary_ops = [ + r2il::R2ILOp::IntAdd { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntSub { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntMult { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntAnd { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntOr { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntXor { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntLess { + dst: Varnode::register(0, 1), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntSLess { + dst: Varnode::register(0, 1), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntLeft { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntRight { + dst: Varnode::register(0, 8), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntEqual { + dst: Varnode::register(0, 1), + a: reg_a.clone(), + b: reg_b.clone(), + }, + r2il::R2ILOp::IntNotEqual { + dst: Varnode::register(0, 1), + a: reg_a.clone(), + b: reg_b.clone(), + }, + ]; + for op in binary_ops { + block.push(op); + } + block.push(r2il::R2ILOp::Load { + dst: Varnode::register(24, 8), + space: SpaceId::Ram, + addr: reg_a.clone(), + }); + block.push(r2il::R2ILOp::Store { + space: SpaceId::Ram, + addr: reg_a.clone(), + val: Varnode::register(24, 8), + }); + block.push(r2il::R2ILOp::Copy { + dst: Varnode::register(32, 8), + src: Varnode::register(24, 8), + }); + block.push(r2il::R2ILOp::Call { + target: Varnode::constant(0x9000, 8), + }); + block.push(r2il::R2ILOp::Return { + target: Varnode::register(0, 8), + }); + + let blocks = vec![block]; + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("linear fixture must ingest"); + let conv = R2ilConvention::minimal_pass_one(); + + let (flat, ledger, report) = smelt(&behavior, &blocks, &conv); + assert_eq!(report.harvested, flat.len() + ledger.len()); + assert_eq!(report.classified, flat.len()); + assert_eq!(report.residual, ledger.len()); + assert_eq!(report.dropped, 0); + assert!(report.is_conserved()); + assert!( + report.harvested >= 50, + "expected >= 50 ore facts (anti-vacuity bound), got {}", + report.harvested + ); + } + + #[test] + fn the_convention_is_the_knob_not_the_code() { + // §5's `R2ilConvention` API has no incremental "add one opcode" mutator — widening the + // convention is expressed as a second `from_arch` call with `AtomicCAS` added to the + // classified set, the only constructor spec'd that can grow the set at all. + let arch = { + let mut arch = ArchSpec::new("test-arch"); + arch.registers = vec![ + RegisterDef::new("dst", 0, 8), + RegisterDef::new("addr", 8, 8), + RegisterDef::new("expected", 16, 8), + RegisterDef::new("replacement", 24, 8), + ]; + arch + }; + let mut block = R2ILBlock::new(0x5000, 8); + block.push(r2il::R2ILOp::AtomicCAS { + dst: Varnode::register(0, 8), + space: SpaceId::Ram, + addr: Varnode::register(8, 8), + expected: Varnode::register(16, 8), + replacement: Varnode::register(24, 8), + ordering: MemoryOrdering::SeqCst, + }); + block.push(r2il::R2ILOp::Return { + target: Varnode::register(0, 8), + }); + let blocks = vec![block]; + let behavior = FunctionBehavior::from_blocks_raw(&blocks, Some(&arch)) + .expect("AtomicCAS+Return fixture must ingest"); + + let narrow = R2ilConvention::from_arch(&arch, [OpTag::Return]) + .expect("no custom spaces on this arch, must not overflow"); + let (narrow_flat, _narrow_ledger, narrow_report) = smelt(&behavior, &blocks, &narrow); + assert!( + narrow_flat.iter().all(|row| row.opcode != OpTag::AtomicCAS), + "AtomicCAS must not classify under the narrow convention" + ); + + let wide = R2ilConvention::from_arch(&arch, [OpTag::Return, OpTag::AtomicCAS]) + .expect("no custom spaces on this arch, must not overflow"); + let (wide_flat, _wide_ledger, wide_report) = smelt(&behavior, &blocks, &wide); + + // The AtomicCAS instruction's own facts: 1 Op row + 3 input operands (addr, expected, + // replacement — all registers, matched by `arch.registers` above, so all resolve once + // classified) + 1 output operand (dst). + let atomic_cas_facts = 1 + 3 + 1; + assert_eq!( + wide_report.classified, + narrow_report.classified + atomic_cas_facts, + "classified must rise by exactly the AtomicCAS fact count" + ); + assert_eq!( + narrow_report.residual, + wide_report.residual + atomic_cas_facts, + "residual must fall by exactly the same count" + ); + assert!( + wide_flat + .iter() + .any(|row| row.opcode == OpTag::AtomicCAS && row.kind == FactKind::Op) + ); + } + + #[test] + fn block_anchored_rows_are_addressed_not_defaulted() { + let mut block = R2ILBlock::new(0x6000, 8); + block.push(r2il::R2ILOp::IntAdd { + dst: Varnode::register(0, 8), + a: Varnode::register(8, 8), + b: Varnode::register(16, 8), + }); + block.push(r2il::R2ILOp::Return { + target: Varnode::register(0, 8), + }); + let blocks = vec![block]; + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None) + .expect("IntAdd+Return fixture must ingest"); + let conv = R2ilConvention::minimal_pass_one(); + let (flat, ledger, report) = smelt(&behavior, &blocks, &conv); + assert!(report.is_conserved()); + + // Every Op row (the only block-anchored FlatFact kind in this fixture) carries the REAL + // block address, never the all-zero facet a lazy `VarnodeFacet::default()`-style + // implementation would produce. + let mut op_rows_checked = 0; + for row in &flat { + if row.kind == FactKind::Op { + assert_eq!(row.at.offset(), 0x6000); + assert_eq!(row.at.space_discriminant(), facet::SPACE_RAM); + op_rows_checked += 1; + } + } + assert_eq!( + op_rows_checked, 2, + "expected both IntAdd and Return to classify under pass 1" + ); + + // And any residual that DID land a Ram-space address must likewise be the real block + // address, not zero. + for residual in ledger.rows() { + if let Some(at) = residual.at + && at.space_discriminant() == facet::SPACE_RAM + { + assert_eq!(at.offset(), 0x6000); + } + } + } +} diff --git a/crates/ruff_r2il/src/lib.rs b/crates/ruff_r2il/src/lib.rs new file mode 100644 index 0000000000000..86f21eaeb4e96 --- /dev/null +++ b/crates/ruff_r2il/src/lib.rs @@ -0,0 +1,70 @@ +//! # `ruff_r2il` — the R2IL intake arm +//! +//! > **Operator, 2026-08-18: "Varnode in the first stage is pointer chasing stacked god objects — +//! > hence the ore furnace slag."** +//! +//! `ruff_r2il` is an **INTAKE ARM**, not a bypass: typed input earns no exemption from +//! ore → furnace → slag → proposer. Slag is evidence. +//! +//! The upstream typed truth (`r2il`/`r2ssa`) is **good ore but structurally still stage-1 +//! pointer chasing**: `SSAFunction`'s private `HashMap`, a petgraph CFG, +//! `BTreeMap` keyed by String-carrying vars, facts as nested `BTreeMap`s of +//! structs. **Typed ≠ refined.** Mistaking the cleanliness of r2il's Rust types for refinement +//! *is* the privileged-direct-path the pivot forbids. +//! +//! - **(a) The ARM preserves that object graph untouched** — it never flattens at intake. +//! - **(b) The FURNACE melts it** into flat, facet-addressed, concern-separated rows +//! (`FactId + VarnodeFacet + Concern` — plain flat `Vec`s, **never another object graph**). +//! - **(c) The SLAG is what resisted flattening**, addressed at the facet coordinate where it +//! resisted. +//! +//! | stage | module | role | +//! |---|---|---| +//! | 1 · intake arm / **ore carrier** | [`behavior`] | lossless zero-copy assembly. **Not** an invented "contract" | +//! | 2 · **ore** | [`ore`] | deterministic typed fact enumeration over the object graph | +//! | 3 · **furnace** | [`furnace`] | melt → **flat** `FlatFact` rows; conservation ledger | +//! | 3b · **slag** | [`slag`] | addressed residual rows; shape id + reason; **no `Other`, ever** | +//! | — · drill key + config tree | [`facet`], [`convention`] | the 16-byte address, and longest-prefix-wins config over it | +//! | 4 · DTO / codebook factoring | [`vocab`] | feeds lance-graph `ogar_codebook` **read-only** | +//! | — · artifact set | `examples/harvest_r2il.rs` | the deliverable, per MedCare-rs / openproject-nexgen-rs | +//! +//! Refined concern *contracts* are a later, measured furnace output. This crate does not invent +//! them. +//! +//! ## One line so a future session does not re-chase the name +//! +//! `lance-graph-arm-discovery` is **Association Rule Mining** (Aerial+, arXiv 2504.19354), not +//! "intake arm". It consumes a declared `FeatureSpec` + discretised `Dataset` and emits +//! `CandidateRule`. It provides **no** intake, ore, furnace, slag or proposer machinery. Do not +//! reference it as reusable here. +//! +//! ## Honesty notes (crate-scoped — every one of these is load-bearing) +//! +//! 1. **`lift` does *not* buy "zero system deps by default".** `r2ssa` drags `r2sleigh-lift` → +//! `libsla` in unconditionally; the `lift` feature on *this* crate gates only its own +//! **direct** disassembler / `sleigh-config` use, not the transitive native build. +//! 2. **`Varnode::meta` and `R2ILBlock::op_metadata` do not cross into SSA.** They stay +//! addressable by the same `(block_addr, op_idx)` key that [`behavior::FunctionBehavior`] +//! exposes via its provenance helpers. That rejoin is the contract — never a parsed display +//! string. +//! 3. **`r2il::SwitchInfo` (a struct) and `r2ssa::SwitchInfo` (a type alias) are different +//! shapes with the same name.** Alias explicitly on import; never assume they unify. +//! 4. **Do NOT copy `#![expect(clippy::print_stderr, …)]` from `ruff_cpp_spo`'s examples.** That +//! lint comes from ruff's `[workspace.lints]`, which this **excluded** crate cannot inherit — +//! the unfulfilled expectation would itself fail `clippy -D warnings`. +//! +//! ## Verified upstream facts +//! +//! Every type and signature this crate builds against was read from the `r2sleigh` source this +//! session (commit `60942f6`), not inferred. See +//! `.claude/plans/r2il-behavioral-ir-v1-impl-spec.md` §2 for the full ledger. A worker or +//! reviewer who finds a signature here contradicting the actual upstream source should treat the +//! source as authoritative and file a correction — never silently improvise a replacement shape. + +pub mod behavior; +pub mod convention; +pub mod facet; +pub mod furnace; +pub mod ore; +pub mod slag; +pub mod vocab; diff --git a/crates/ruff_r2il/src/ore.rs b/crates/ruff_r2il/src/ore.rs new file mode 100644 index 0000000000000..46c3d3727a900 --- /dev/null +++ b/crates/ruff_r2il/src/ore.rs @@ -0,0 +1,1135 @@ +//! Stage 2 — the **ore**: deterministic typed fact enumeration over the object graph. +//! +//! > **"Varnode in the first stage is pointer chasing stacked god objects — hence the ore +//! > furnace slag."** (operator, 2026-08-18) +//! +//! The upstream object graph [`crate::behavior::FunctionBehavior`] carries — a private +//! `HashMap`, a petgraph CFG, `BTreeMap` keyed by +//! String-carrying vars, facts as nested `BTreeMap`s of structs — is GOOD ORE and structurally +//! still stage-1 pointer chasing. **Typed ≠ refined.** Mistaking the cleanliness of r2il's Rust +//! types for refinement *is* the privileged-direct-path the pivot forbids. This module *reads* +//! that graph and emits typed [`OreFact`] rows; it does **not** flatten (that is +//! [`crate::furnace`]) and it does **not** classify (that is [`crate::convention`], applied by +//! [`crate::furnace`]). +//! +//! # The load-bearing consequence: the facet coordinate is NOT recoverable from SSA +//! +//! `SSAVar` carries `name` / `version` / `size` and **no offset and no `SpaceId`**. The only +//! SSA-side trace of a varnode's offset is inside its *display name* (`"reg:10"`, +//! `"space7:1000"`) — and parsing display strings is forbidden as a data path +//! (`format!("{:?}")` is FORBIDDEN as a data path throughout this crate). Therefore +//! [`enumerate`] takes BOTH the [`crate::behavior::FunctionBehavior`] AND the source +//! `&[R2ILBlock]`: operand coordinates come from the **typed** [`Varnode`]s in the source ops, +//! joined to SSA instructions by the `(block_addr, op_idx)` key +//! `SsaGraph::op_inst_by_site` / `op_site_by_inst` provide. +//! +//! The join is **verified, not assumed**: at each `InstPayload::Op` site we compare +//! `OpTag::from_op(ssa_op)` (what SSA expects at this site) against `OpTag::from_r2il` of the +//! r2il op actually found at the joined `(block_addr, op_idx)` (what the source actually has). A +//! mismatch — the `Multiequal` index shift (a phi extracted from `.ops` into `.phis` leaves a +//! later r2il op's `.ops` index pointing at what SSA calls a different instruction), or a +//! `CallDefine` insertion (a synthetic op with no r2il counterpart at all, shifting every later +//! index) — emits [`OreFact::JoinFailure`], **never** a silently misattributed coordinate. +//! +//! Rows with no source varnode at all — phi inputs (routed through the dedicated +//! [`OreFact::PhiInput`], which carries no facet-coordinate fields to misattribute) and +//! `SSAOp::CallDefine` (a fresh unknown-register value with no source op whatsoever, so no join +//! is even attempted for it) — are the furnace's `NoFacetCoordinate` residual once melted; this +//! module simply never manufactures a coordinate for them. Nothing is dropped: every +//! `InstPayload` produces an [`OreFact::Op`] row regardless of whether its operand coordinates +//! could be recovered. +//! +//! # `FactProvenance.value` convention +//! +//! [`FactProvenance::value`] is "the single [`ValueId`] this particular row is most directly +//! about, if any": an [`OreFact::Op`] row carries the instruction's own output (the value it +//! defines); an [`OreFact::Operand`] row carries that operand's own value (duplicating the +//! variant's own `value` field so a generic consumer that only inspects `prov` still sees it); +//! an [`OreFact::PhiInput`] row carries that input's value; an [`OreFact::Predicate`] row +//! carries the branch condition; an [`OreFact::CallSite`] row carries the call target; +//! [`OreFact::MemoryUse`] / [`OreFact::MemoryDef`] leave it `None` (a memory fact's identity is +//! an `ObjectId`, a distinct fact-space from a single SSA value). +//! +//! # Enumeration order — fixed and documented (the determinism guarantee) +//! +//! 1. **Blocks** in [`SSAFunction::block_addrs`] order (reverse postorder, a `Vec` — *never* the +//! private `HashMap`). +//! 2. **Within a block**, `GraphBlock::insts` order (phis then ops, each in `GraphInst::ordinal` +//! order — already the construction order upstream; never re-sorted here): +//! - a `Phi` payload emits one [`OreFact::Op`] (`opcode = OpTag::Phi`) and nothing else at +//! this step (no join target exists for a phi); +//! - an `Op` payload emits one [`OreFact::Op`], then — unless the op is `SSAOp::CallDefine` +//! (no source op exists at all) — attempts the op-site join: on a tag match, one +//! [`OreFact::Operand`] per source-op input (in `R2ILOp::inputs()` order) then one for the +//! output if present; on a tag mismatch, one [`OreFact::JoinFailure`]; if no r2il op exists +//! at the joined site at all (an index-shift landed past the end of the source block), +//! neither is emitted — the [`OreFact::Op`] row already stands for the instruction. +//! - then, once every inst in the block has been visited: one [`OreFact::Edge`] per +//! `GraphBlock::successors` entry, in that `Vec`'s order; +//! - then one [`OreFact::PhiInput`] per phi source, for every phi in the block, in phi-then- +//! source order. +//! 3. **Then, function-wide** (every container touched is a `BTreeMap` — deterministic key +//! order, never a `HashMap`): +//! - [`OreFact::MemoryUse`] then [`OreFact::MemoryDef`], both by ascending [`InstId`] +//! (`MemorySSAFacts::uses_by_inst` / `defs_by_inst` are already `BTreeMap`); +//! - [`OreFact::Predicate`], by ascending [`PredicateId`] (`PredicateFacts::predicates`); +//! - [`OreFact::CallSite`], by ascending [`CallSiteId`] (`CallSiteFacts::by_id`). + +use r2il::{R2ILBlock, R2ILOp, SpaceId}; +use r2ssa::{ + BlockId, BlockTerminator, CallSiteId, CompareKind, InstId, InstPayload, ObjectId, PredicateId, + SSAOp, ValueId, +}; + +use crate::behavior::FunctionBehavior; + +// ============================================================================================ +// OpTag +// ============================================================================================ + +/// One variant per `SSAOp` variant. Closed, exhaustive, stable `as_str()` — the discipline +/// `ruff_spo_triplet::Predicate` enforces ("frontends MUST NOT emit raw predicate strings"). +/// `from_op` and `from_r2il` are TOTAL matches; `format!("{:?}")` is FORBIDDEN as a data path. +/// +/// `Phi` and `CallDefine` exist only on the SSA side (`r2il::R2ILOp` has neither: a phi arrives +/// as `Multiequal`, and `CallDefine` is synthesized during renaming with no r2il counterpart at +/// all). `r2il::R2ILOp::Multiequal` and `::Indirect` exist only on the r2il side and map onto +/// `OpTag::Phi` / `OpTag::Copy` respectively, mirroring `rename_op`'s documented mapping +/// (`Multiequal → SSAOp::Phi`, `Indirect → SSAOp::Copy`, `r2ssa/src/rename.rs:1087,1101`). +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum OpTag { + Phi, + Copy, + Load, + Store, + Fence, + LoadLinked, + StoreConditional, + AtomicCAS, + LoadGuarded, + StoreGuarded, + IntAdd, + IntSub, + IntMult, + IntDiv, + IntSDiv, + IntRem, + IntSRem, + IntNegate, + IntCarry, + IntSCarry, + IntSBorrow, + IntAnd, + IntOr, + IntXor, + IntNot, + IntLeft, + IntRight, + IntSRight, + IntEqual, + IntNotEqual, + IntLess, + IntSLess, + IntLessEqual, + IntSLessEqual, + IntZExt, + IntSExt, + BoolNot, + BoolAnd, + BoolOr, + BoolXor, + Piece, + Subpiece, + PopCount, + Lzcount, + Branch, + CBranch, + BranchInd, + Call, + CallInd, + CallDefine, + Return, + FloatAdd, + FloatSub, + FloatMult, + FloatDiv, + FloatNeg, + FloatAbs, + FloatSqrt, + FloatCeil, + FloatFloor, + FloatRound, + FloatNaN, + FloatEqual, + FloatNotEqual, + FloatLess, + FloatLessEqual, + Int2Float, + Float2Int, + FloatFloat, + Trunc, + CallOther, + Nop, + Unimplemented, + CpuId, + Breakpoint, + PtrAdd, + PtrSub, + SegmentOp, + New, + Cast, + Extract, + Insert, +} + +impl OpTag { + /// One arm per `SSAOp` variant (`r2ssa/src/op.rs`), total match — a variant added upstream + /// without a corresponding arm here fails to compile, which is the point. + pub fn from_op(op: &SSAOp) -> Self { + match op { + SSAOp::Phi { .. } => OpTag::Phi, + SSAOp::Copy { .. } => OpTag::Copy, + SSAOp::Load { .. } => OpTag::Load, + SSAOp::Store { .. } => OpTag::Store, + SSAOp::Fence { .. } => OpTag::Fence, + SSAOp::LoadLinked { .. } => OpTag::LoadLinked, + SSAOp::StoreConditional { .. } => OpTag::StoreConditional, + SSAOp::AtomicCAS { .. } => OpTag::AtomicCAS, + SSAOp::LoadGuarded { .. } => OpTag::LoadGuarded, + SSAOp::StoreGuarded { .. } => OpTag::StoreGuarded, + SSAOp::IntAdd { .. } => OpTag::IntAdd, + SSAOp::IntSub { .. } => OpTag::IntSub, + SSAOp::IntMult { .. } => OpTag::IntMult, + SSAOp::IntDiv { .. } => OpTag::IntDiv, + SSAOp::IntSDiv { .. } => OpTag::IntSDiv, + SSAOp::IntRem { .. } => OpTag::IntRem, + SSAOp::IntSRem { .. } => OpTag::IntSRem, + SSAOp::IntNegate { .. } => OpTag::IntNegate, + SSAOp::IntCarry { .. } => OpTag::IntCarry, + SSAOp::IntSCarry { .. } => OpTag::IntSCarry, + SSAOp::IntSBorrow { .. } => OpTag::IntSBorrow, + SSAOp::IntAnd { .. } => OpTag::IntAnd, + SSAOp::IntOr { .. } => OpTag::IntOr, + SSAOp::IntXor { .. } => OpTag::IntXor, + SSAOp::IntNot { .. } => OpTag::IntNot, + SSAOp::IntLeft { .. } => OpTag::IntLeft, + SSAOp::IntRight { .. } => OpTag::IntRight, + SSAOp::IntSRight { .. } => OpTag::IntSRight, + SSAOp::IntEqual { .. } => OpTag::IntEqual, + SSAOp::IntNotEqual { .. } => OpTag::IntNotEqual, + SSAOp::IntLess { .. } => OpTag::IntLess, + SSAOp::IntSLess { .. } => OpTag::IntSLess, + SSAOp::IntLessEqual { .. } => OpTag::IntLessEqual, + SSAOp::IntSLessEqual { .. } => OpTag::IntSLessEqual, + SSAOp::IntZExt { .. } => OpTag::IntZExt, + SSAOp::IntSExt { .. } => OpTag::IntSExt, + SSAOp::BoolNot { .. } => OpTag::BoolNot, + SSAOp::BoolAnd { .. } => OpTag::BoolAnd, + SSAOp::BoolOr { .. } => OpTag::BoolOr, + SSAOp::BoolXor { .. } => OpTag::BoolXor, + SSAOp::Piece { .. } => OpTag::Piece, + SSAOp::Subpiece { .. } => OpTag::Subpiece, + SSAOp::PopCount { .. } => OpTag::PopCount, + SSAOp::Lzcount { .. } => OpTag::Lzcount, + SSAOp::Branch { .. } => OpTag::Branch, + SSAOp::CBranch { .. } => OpTag::CBranch, + SSAOp::BranchInd { .. } => OpTag::BranchInd, + SSAOp::Call { .. } => OpTag::Call, + SSAOp::CallInd { .. } => OpTag::CallInd, + SSAOp::CallDefine { .. } => OpTag::CallDefine, + SSAOp::Return { .. } => OpTag::Return, + SSAOp::FloatAdd { .. } => OpTag::FloatAdd, + SSAOp::FloatSub { .. } => OpTag::FloatSub, + SSAOp::FloatMult { .. } => OpTag::FloatMult, + SSAOp::FloatDiv { .. } => OpTag::FloatDiv, + SSAOp::FloatNeg { .. } => OpTag::FloatNeg, + SSAOp::FloatAbs { .. } => OpTag::FloatAbs, + SSAOp::FloatSqrt { .. } => OpTag::FloatSqrt, + SSAOp::FloatCeil { .. } => OpTag::FloatCeil, + SSAOp::FloatFloor { .. } => OpTag::FloatFloor, + SSAOp::FloatRound { .. } => OpTag::FloatRound, + SSAOp::FloatNaN { .. } => OpTag::FloatNaN, + SSAOp::FloatEqual { .. } => OpTag::FloatEqual, + SSAOp::FloatNotEqual { .. } => OpTag::FloatNotEqual, + SSAOp::FloatLess { .. } => OpTag::FloatLess, + SSAOp::FloatLessEqual { .. } => OpTag::FloatLessEqual, + SSAOp::Int2Float { .. } => OpTag::Int2Float, + SSAOp::Float2Int { .. } => OpTag::Float2Int, + SSAOp::FloatFloat { .. } => OpTag::FloatFloat, + SSAOp::Trunc { .. } => OpTag::Trunc, + SSAOp::CallOther { .. } => OpTag::CallOther, + SSAOp::Nop => OpTag::Nop, + SSAOp::Unimplemented => OpTag::Unimplemented, + SSAOp::CpuId { .. } => OpTag::CpuId, + SSAOp::Breakpoint => OpTag::Breakpoint, + SSAOp::PtrAdd { .. } => OpTag::PtrAdd, + SSAOp::PtrSub { .. } => OpTag::PtrSub, + SSAOp::SegmentOp { .. } => OpTag::SegmentOp, + SSAOp::New { .. } => OpTag::New, + SSAOp::Cast { .. } => OpTag::Cast, + SSAOp::Extract { .. } => OpTag::Extract, + SSAOp::Insert { .. } => OpTag::Insert, + } + } + + /// One arm per `R2ILOp` variant (`r2il/src/opcode.rs`), total match. `Multiequal` and + /// `Indirect` are the two r2il-only variants and route to `Phi` / `Copy` respectively, + /// matching `rename_op`; every other variant maps to the identically-named `OpTag`. + pub fn from_r2il(op: &R2ILOp) -> Self { + match op { + R2ILOp::Copy { .. } => OpTag::Copy, + R2ILOp::Load { .. } => OpTag::Load, + R2ILOp::Store { .. } => OpTag::Store, + R2ILOp::Fence { .. } => OpTag::Fence, + R2ILOp::LoadLinked { .. } => OpTag::LoadLinked, + R2ILOp::StoreConditional { .. } => OpTag::StoreConditional, + R2ILOp::AtomicCAS { .. } => OpTag::AtomicCAS, + R2ILOp::LoadGuarded { .. } => OpTag::LoadGuarded, + R2ILOp::StoreGuarded { .. } => OpTag::StoreGuarded, + R2ILOp::IntAdd { .. } => OpTag::IntAdd, + R2ILOp::IntSub { .. } => OpTag::IntSub, + R2ILOp::IntMult { .. } => OpTag::IntMult, + R2ILOp::IntDiv { .. } => OpTag::IntDiv, + R2ILOp::IntSDiv { .. } => OpTag::IntSDiv, + R2ILOp::IntRem { .. } => OpTag::IntRem, + R2ILOp::IntSRem { .. } => OpTag::IntSRem, + R2ILOp::IntNegate { .. } => OpTag::IntNegate, + R2ILOp::IntCarry { .. } => OpTag::IntCarry, + R2ILOp::IntSCarry { .. } => OpTag::IntSCarry, + R2ILOp::IntSBorrow { .. } => OpTag::IntSBorrow, + R2ILOp::IntAnd { .. } => OpTag::IntAnd, + R2ILOp::IntOr { .. } => OpTag::IntOr, + R2ILOp::IntXor { .. } => OpTag::IntXor, + R2ILOp::IntNot { .. } => OpTag::IntNot, + R2ILOp::IntLeft { .. } => OpTag::IntLeft, + R2ILOp::IntRight { .. } => OpTag::IntRight, + R2ILOp::IntSRight { .. } => OpTag::IntSRight, + R2ILOp::IntEqual { .. } => OpTag::IntEqual, + R2ILOp::IntNotEqual { .. } => OpTag::IntNotEqual, + R2ILOp::IntLess { .. } => OpTag::IntLess, + R2ILOp::IntSLess { .. } => OpTag::IntSLess, + R2ILOp::IntLessEqual { .. } => OpTag::IntLessEqual, + R2ILOp::IntSLessEqual { .. } => OpTag::IntSLessEqual, + R2ILOp::IntZExt { .. } => OpTag::IntZExt, + R2ILOp::IntSExt { .. } => OpTag::IntSExt, + R2ILOp::BoolNot { .. } => OpTag::BoolNot, + R2ILOp::BoolAnd { .. } => OpTag::BoolAnd, + R2ILOp::BoolOr { .. } => OpTag::BoolOr, + R2ILOp::BoolXor { .. } => OpTag::BoolXor, + R2ILOp::Piece { .. } => OpTag::Piece, + R2ILOp::Subpiece { .. } => OpTag::Subpiece, + R2ILOp::PopCount { .. } => OpTag::PopCount, + R2ILOp::Lzcount { .. } => OpTag::Lzcount, + R2ILOp::Branch { .. } => OpTag::Branch, + R2ILOp::CBranch { .. } => OpTag::CBranch, + R2ILOp::BranchInd { .. } => OpTag::BranchInd, + R2ILOp::Call { .. } => OpTag::Call, + R2ILOp::CallInd { .. } => OpTag::CallInd, + R2ILOp::Return { .. } => OpTag::Return, + R2ILOp::FloatAdd { .. } => OpTag::FloatAdd, + R2ILOp::FloatSub { .. } => OpTag::FloatSub, + R2ILOp::FloatMult { .. } => OpTag::FloatMult, + R2ILOp::FloatDiv { .. } => OpTag::FloatDiv, + R2ILOp::FloatNeg { .. } => OpTag::FloatNeg, + R2ILOp::FloatAbs { .. } => OpTag::FloatAbs, + R2ILOp::FloatSqrt { .. } => OpTag::FloatSqrt, + R2ILOp::FloatCeil { .. } => OpTag::FloatCeil, + R2ILOp::FloatFloor { .. } => OpTag::FloatFloor, + R2ILOp::FloatRound { .. } => OpTag::FloatRound, + R2ILOp::FloatNaN { .. } => OpTag::FloatNaN, + R2ILOp::FloatEqual { .. } => OpTag::FloatEqual, + R2ILOp::FloatNotEqual { .. } => OpTag::FloatNotEqual, + R2ILOp::FloatLess { .. } => OpTag::FloatLess, + R2ILOp::FloatLessEqual { .. } => OpTag::FloatLessEqual, + R2ILOp::Int2Float { .. } => OpTag::Int2Float, + R2ILOp::Float2Int { .. } => OpTag::Float2Int, + R2ILOp::FloatFloat { .. } => OpTag::FloatFloat, + R2ILOp::Trunc { .. } => OpTag::Trunc, + R2ILOp::CallOther { .. } => OpTag::CallOther, + R2ILOp::Nop => OpTag::Nop, + R2ILOp::Unimplemented => OpTag::Unimplemented, + R2ILOp::CpuId { .. } => OpTag::CpuId, + R2ILOp::Breakpoint => OpTag::Breakpoint, + // The two r2il-only variants — see `rename_op` (`r2ssa/src/rename.rs:1087,1101`). + R2ILOp::Multiequal { .. } => OpTag::Phi, + R2ILOp::Indirect { .. } => OpTag::Copy, + R2ILOp::PtrAdd { .. } => OpTag::PtrAdd, + R2ILOp::PtrSub { .. } => OpTag::PtrSub, + R2ILOp::SegmentOp { .. } => OpTag::SegmentOp, + R2ILOp::New { .. } => OpTag::New, + R2ILOp::Cast { .. } => OpTag::Cast, + R2ILOp::Extract { .. } => OpTag::Extract, + R2ILOp::Insert { .. } => OpTag::Insert, + } + } + + /// The on-the-wire opcode string (stable snake_case; never reformat). + pub fn as_str(self) -> &'static str { + match self { + OpTag::Phi => "phi", + OpTag::Copy => "copy", + OpTag::Load => "load", + OpTag::Store => "store", + OpTag::Fence => "fence", + OpTag::LoadLinked => "load_linked", + OpTag::StoreConditional => "store_conditional", + OpTag::AtomicCAS => "atomic_cas", + OpTag::LoadGuarded => "load_guarded", + OpTag::StoreGuarded => "store_guarded", + OpTag::IntAdd => "int_add", + OpTag::IntSub => "int_sub", + OpTag::IntMult => "int_mult", + OpTag::IntDiv => "int_div", + OpTag::IntSDiv => "int_sdiv", + OpTag::IntRem => "int_rem", + OpTag::IntSRem => "int_srem", + OpTag::IntNegate => "int_negate", + OpTag::IntCarry => "int_carry", + OpTag::IntSCarry => "int_scarry", + OpTag::IntSBorrow => "int_sborrow", + OpTag::IntAnd => "int_and", + OpTag::IntOr => "int_or", + OpTag::IntXor => "int_xor", + OpTag::IntNot => "int_not", + OpTag::IntLeft => "int_left", + OpTag::IntRight => "int_right", + OpTag::IntSRight => "int_sright", + OpTag::IntEqual => "int_equal", + OpTag::IntNotEqual => "int_not_equal", + OpTag::IntLess => "int_less", + OpTag::IntSLess => "int_sless", + OpTag::IntLessEqual => "int_less_equal", + OpTag::IntSLessEqual => "int_sless_equal", + OpTag::IntZExt => "int_zext", + OpTag::IntSExt => "int_sext", + OpTag::BoolNot => "bool_not", + OpTag::BoolAnd => "bool_and", + OpTag::BoolOr => "bool_or", + OpTag::BoolXor => "bool_xor", + OpTag::Piece => "piece", + OpTag::Subpiece => "subpiece", + OpTag::PopCount => "pop_count", + OpTag::Lzcount => "lzcount", + OpTag::Branch => "branch", + OpTag::CBranch => "cbranch", + OpTag::BranchInd => "branch_ind", + OpTag::Call => "call", + OpTag::CallInd => "call_ind", + OpTag::CallDefine => "call_define", + OpTag::Return => "return", + OpTag::FloatAdd => "float_add", + OpTag::FloatSub => "float_sub", + OpTag::FloatMult => "float_mult", + OpTag::FloatDiv => "float_div", + OpTag::FloatNeg => "float_neg", + OpTag::FloatAbs => "float_abs", + OpTag::FloatSqrt => "float_sqrt", + OpTag::FloatCeil => "float_ceil", + OpTag::FloatFloor => "float_floor", + OpTag::FloatRound => "float_round", + OpTag::FloatNaN => "float_nan", + OpTag::FloatEqual => "float_equal", + OpTag::FloatNotEqual => "float_not_equal", + OpTag::FloatLess => "float_less", + OpTag::FloatLessEqual => "float_less_equal", + OpTag::Int2Float => "int2float", + OpTag::Float2Int => "float2int", + OpTag::FloatFloat => "float_float", + OpTag::Trunc => "trunc", + OpTag::CallOther => "call_other", + OpTag::Nop => "nop", + OpTag::Unimplemented => "unimplemented", + OpTag::CpuId => "cpu_id", + OpTag::Breakpoint => "breakpoint", + OpTag::PtrAdd => "ptr_add", + OpTag::PtrSub => "ptr_sub", + OpTag::SegmentOp => "segment_op", + OpTag::New => "new", + OpTag::Cast => "cast", + OpTag::Extract => "extract", + OpTag::Insert => "insert", + } + } +} + +// ============================================================================================ +// EdgeTag / CompareTag / OperandPos +// ============================================================================================ + +/// How a CFG edge relates to its source block's terminator. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum EdgeTag { + /// Neither branch of a conditional, nor closing a cycle (fallthrough, unconditional + /// branch, call continuation, switch case, …). + Normal, + /// The true target of a `BlockTerminator::ConditionalBranch`. + True, + /// The false target of a `BlockTerminator::ConditionalBranch`. + False, + /// Closes a cycle: the successor's `BlockId` does not come strictly after the source's in + /// `block_addrs()` (reverse-postorder) order. `JUDGMENT`: a conditional edge that also + /// closes a cycle (a `while`-loop's back branch) is classified `True`/`False` — the more + /// specific, locally available fact — never `Back`; this classification applies only to + /// non-conditional successors. + Back, +} + +/// One `SSAOp::Predicate`/`R2ILOp`-family comparison kind — mirrors `r2ssa::CompareKind` +/// one-for-one so `ore.rs` never has to `format!("{:?}")` an upstream enum. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum CompareTag { + Equal, + NotEqual, + Less, + SignedLess, + LessEqual, + SignedLessEqual, +} + +fn compare_tag(kind: CompareKind) -> CompareTag { + match kind { + CompareKind::Equal => CompareTag::Equal, + CompareKind::NotEqual => CompareTag::NotEqual, + CompareKind::Less => CompareTag::Less, + CompareKind::SignedLess => CompareTag::SignedLess, + CompareKind::LessEqual => CompareTag::LessEqual, + CompareKind::SignedLessEqual => CompareTag::SignedLessEqual, + } +} + +/// Which slot of an op an [`OreFact::Operand`] row describes. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub enum OperandPos { + /// The `n`th entry of `R2ILOp::inputs()`, in that `Vec`'s order. + Input(usize), + /// `R2ILOp::output()`. + Output, +} + +// ============================================================================================ +// FactProvenance +// ============================================================================================ + +/// Where a fact came from — see the module docs' "`FactProvenance.value` convention" section +/// for the per-`OreFact`-variant population rule. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] +pub struct FactProvenance { + pub inst: Option, + pub block: Option, + pub op_site: Option<(u64, usize)>, + pub value: Option, +} + +// ============================================================================================ +// OreFact +// ============================================================================================ + +/// One typed ore row. Deterministic, total, lossless w.r.t. the SSA surface — see the module +/// docs for the fixed enumeration order [`enumerate`] produces these in. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum OreFact { + Op { + prov: FactProvenance, + opcode: OpTag, + ordinal: usize, + input_arity: usize, + has_output: bool, + }, + /// The TYPED coordinate components, taken from the source `Varnode` via the verified + /// op-site join — never from an `SSAVar` name. + Operand { + prov: FactProvenance, + position: OperandPos, + value: Option, + space: SpaceId, + offset: u64, + size: u32, + }, + Edge { + from: BlockId, + to: BlockId, + kind: EdgeTag, + }, + PhiInput { + prov: FactProvenance, + index: usize, + pred: BlockId, + value: ValueId, + }, + MemoryUse { + prov: FactProvenance, + object: ObjectId, + version: u32, + size: u32, + }, + MemoryDef { + prov: FactProvenance, + object: ObjectId, + previous: u32, + next: u32, + size: u32, + }, + Predicate { + prov: FactProvenance, + id: PredicateId, + condition: ValueId, + comparison: Option, + true_target: u64, + false_target: u64, + }, + CallSite { + prov: FactProvenance, + id: CallSiteId, + target: ValueId, + direct_target: Option, + }, + /// The join failed at this site — emitted so the failure is enumerated, not skipped. + JoinFailure { + prov: FactProvenance, + expected: OpTag, + found: OpTag, + }, +} + +// ============================================================================================ +// enumerate +// ============================================================================================ + +/// Enumerate the ore. Deterministic, total, lossless w.r.t. the SSA surface. +/// +/// `blocks` is the SOURCE r2il, required for the typed operand coordinates (see the module +/// docs' "load-bearing consequence" section) — it MUST be the same block list `behavior` was +/// ingested from, or the op-site join has nothing correct to compare against. +pub fn enumerate(behavior: &FunctionBehavior, blocks: &[R2ILBlock]) -> Vec { + let mut facts = Vec::new(); + let graph = behavior.values(); + + for &block_addr in behavior.control().block_addrs() { + let Some(block_id) = graph.block_id_for_addr(block_addr) else { + continue; + }; + let Some(graph_block) = graph.block(block_id) else { + continue; + }; + + // Phis then ops, by GraphInst::ordinal — already the construction order upstream. + for &inst_id in &graph_block.insts { + let Some(inst) = graph.inst(inst_id) else { + continue; + }; + + match &inst.payload { + InstPayload::Phi { .. } => { + let prov = FactProvenance { + inst: Some(inst_id), + block: Some(block_id), + op_site: None, + value: inst.output, + }; + facts.push(OreFact::Op { + prov, + opcode: OpTag::Phi, + ordinal: inst.ordinal, + input_arity: inst.inputs.len(), + has_output: inst.output.is_some(), + }); + } + InstPayload::Op(ssa_op) => { + let opcode = OpTag::from_op(ssa_op); + let op_site = behavior.op_site(inst_id); + let base_prov = FactProvenance { + inst: Some(inst_id), + block: Some(block_id), + op_site, + value: inst.output, + }; + facts.push(OreFact::Op { + prov: base_prov, + opcode, + ordinal: inst.ordinal, + input_arity: inst.inputs.len(), + has_output: inst.output.is_some(), + }); + + // `CallDefine` is synthetic — no source varnode exists at all, so no join + // is even attempted (it becomes the furnace's `NoFacetCoordinate` residual). + if matches!(ssa_op, SSAOp::CallDefine { .. }) { + continue; + } + + let Some((site_addr, site_idx)) = op_site else { + continue; + }; + let Some(source_op) = find_source_op(blocks, site_addr, site_idx) else { + // No r2il op survives at this site (an index shift landed past the end + // of the source block). No comparison is possible, so — unlike a real + // tag mismatch — neither an Operand row nor a JoinFailure is fabricated; + // the Op row already emitted stands for this instruction. + continue; + }; + + let found = OpTag::from_r2il(source_op); + if found != opcode { + facts.push(OreFact::JoinFailure { + prov: base_prov, + expected: opcode, + found, + }); + continue; + } + + for (index, varnode) in source_op.inputs().into_iter().enumerate() { + let value = inst.inputs.get(index).copied(); + facts.push(OreFact::Operand { + prov: FactProvenance { value, ..base_prov }, + position: OperandPos::Input(index), + value, + space: varnode.space, + offset: varnode.offset, + size: varnode.size, + }); + } + if let Some(varnode) = source_op.output() { + let value = inst.output; + facts.push(OreFact::Operand { + prov: FactProvenance { value, ..base_prov }, + position: OperandPos::Output, + value, + space: varnode.space, + offset: varnode.offset, + size: varnode.size, + }); + } + } + } + } + + // Edge rows, in `GraphBlock::successors` order. + let terminator = behavior + .control() + .cfg() + .get_block(block_addr) + .map(|basic_block| &basic_block.terminator); + for &succ_id in &graph_block.successors { + let Some(succ_block) = graph.block(succ_id) else { + continue; + }; + let kind = classify_edge(terminator, succ_block.addr, block_id, succ_id); + facts.push(OreFact::Edge { + from: block_id, + to: succ_id, + kind, + }); + } + + // PhiInput rows: every phi in this block, in phi-then-source order. + for &inst_id in &graph_block.insts { + let Some(inst) = graph.inst(inst_id) else { + continue; + }; + if let InstPayload::Phi { predecessors } = &inst.payload { + let base_prov = FactProvenance { + inst: Some(inst_id), + block: Some(block_id), + op_site: None, + value: inst.output, + }; + for (index, (&pred, &value)) in + predecessors.iter().zip(inst.inputs.iter()).enumerate() + { + facts.push(OreFact::PhiInput { + prov: FactProvenance { + value: Some(value), + ..base_prov + }, + index, + pred, + value, + }); + } + } + } + } + + // Function-wide facts — every container is a BTreeMap, so iteration order is already the + // required ascending-id order; nothing is re-sorted here. + for (&inst_id, uses) in &behavior.memory().uses_by_inst { + let block = graph.inst(inst_id).map(|inst| inst.block); + let op_site = behavior.op_site(inst_id); + for use_fact in uses { + facts.push(OreFact::MemoryUse { + prov: FactProvenance { + inst: Some(inst_id), + block, + op_site, + value: None, + }, + object: use_fact.location.object, + version: use_fact.version.version, + size: use_fact.location.size, + }); + } + } + for (&inst_id, defs) in &behavior.memory().defs_by_inst { + let block = graph.inst(inst_id).map(|inst| inst.block); + let op_site = behavior.op_site(inst_id); + for def_fact in defs { + facts.push(OreFact::MemoryDef { + prov: FactProvenance { + inst: Some(inst_id), + block, + op_site, + value: None, + }, + object: def_fact.location.object, + previous: def_fact.previous_version.version, + next: def_fact.next_version.version, + size: def_fact.location.size, + }); + } + } + + for (&predicate_id, predicate_fact) in &behavior.predicates().predicates { + let block = graph.block_id_for_addr(predicate_fact.block_addr); + facts.push(OreFact::Predicate { + prov: FactProvenance { + inst: None, + block, + op_site: None, + value: Some(predicate_fact.condition), + }, + id: predicate_id, + condition: predicate_fact.condition, + comparison: predicate_fact + .comparison + .as_ref() + .map(|provenance| compare_tag(provenance.kind)), + true_target: predicate_fact.true_target, + false_target: predicate_fact.false_target, + }); + } + + for (&call_site_id, call_site_fact) in &behavior.calls().by_id { + let block = graph.inst(call_site_fact.at).map(|inst| inst.block); + let op_site = behavior.op_site(call_site_fact.at); + facts.push(OreFact::CallSite { + prov: FactProvenance { + inst: Some(call_site_fact.at), + block, + op_site, + value: Some(call_site_fact.target), + }, + id: call_site_id, + target: call_site_fact.target, + direct_target: call_site_fact.direct_target, + }); + } + + facts +} + +/// Look up the r2il op at a `(block_addr, op_idx)` site in the SOURCE block list — the typed +/// side of the op-site join. `None` when either the block or the index does not exist in +/// `blocks` (an index-shift landing past the end of a source block, or a site whose block was +/// not part of the ingested list). +fn find_source_op(blocks: &[R2ILBlock], addr: u64, idx: usize) -> Option<&R2ILOp> { + blocks + .iter() + .find(|block| block.addr == addr) + .and_then(|block| block.ops.get(idx)) +} + +/// The native instruction address an ore fact came from, when the lifter recorded one. +/// +/// SSA does not carry it — `SSAOp` has no address field at all — so this is the +/// `(block_addr, op_idx)` sidecar rejoin against `R2ILBlock::op_metadata`, the SAME key +/// `SsaGraph::op_inst_by_site` uses (and the same key [`FactProvenance::op_site`] already +/// carries). `None` when `prov.op_site` is `None` (a phi, a `CallDefine`, or a function-wide +/// fact with no single op site), when the site's block is not in `blocks`, or when the lifter +/// recorded no per-op metadata at that index (e.g. a single-instruction lift, where the block +/// address itself already IS the instruction address). +pub fn instruction_addr(prov: &FactProvenance, blocks: &[R2ILBlock]) -> Option { + let (addr, idx) = prov.op_site?; + blocks + .iter() + .find(|block| block.addr == addr)? + .op_metadata + .get(&idx)? + .instruction_addr +} + +/// Classify one CFG edge. `JUDGMENT` (undocumented upstream, no dedicated edge-kind type +/// exists): a conditional edge is classified by its role in the terminator FIRST (the more +/// specific, locally available fact); only a non-conditional successor is then checked for +/// closing a cycle, via the reverse-postorder `BlockId` ordering `SsaGraph::from_function` +/// assigns (`block_addrs()` is reverse postorder, and `BlockId`s are handed out in that same +/// order, so a successor whose id does not come strictly after the source's closes a cycle). +fn classify_edge( + terminator: Option<&BlockTerminator>, + succ_addr: u64, + from: BlockId, + to: BlockId, +) -> EdgeTag { + if let Some(BlockTerminator::ConditionalBranch { + true_target, + false_target, + }) = terminator + { + if succ_addr == *true_target { + return EdgeTag::True; + } + if succ_addr == *false_target { + return EdgeTag::False; + } + } + if to.0 <= from.0 { + return EdgeTag::Back; + } + EdgeTag::Normal +} + +// ============================================================================================ +// tests +// ============================================================================================ + +#[cfg(test)] +mod tests { + use super::*; + use r2il::Varnode; + + fn reg(offset: u64, size: u32) -> Varnode { + Varnode::register(offset, size) + } + + fn con(value: u64, size: u32) -> Varnode { + Varnode::constant(value, size) + } + + /// A minimal 2-block, no-merge, no-phi, no-`CallDefine` fixture: `0x1000` computes and + /// unconditionally branches to `0x1004`, which returns. Every SSA op-site join on this + /// fixture is clean (source op index == SSA `.ops` index throughout). + fn linear_fixture() -> Vec { + let mut b0 = R2ILBlock::new(0x1000, 4); + b0.push(R2ILOp::IntAdd { + dst: reg(0x00, 8), + a: reg(0x00, 8), + b: con(1, 8), + }); + b0.push(R2ILOp::Branch { + target: con(0x1004, 8), + }); + + let mut b1 = R2ILBlock::new(0x1004, 4); + b1.push(R2ILOp::Return { + target: reg(0x00, 8), + }); + + vec![b0, b1] + } + + /// A single-block fixture carrying a `Custom(7)`-space operand as an op's own output + /// varnode, plus a masked 64-bit register offset as its input — the two anti-vacuity facts + /// §6 test 2 asks for (a `Custom` space operand is unreachable from the SSA side without + /// parsing a display string). + fn custom_space_fixture() -> Vec { + let masked = 0x1234_5678_9ABC_DEF0_u64 & 0xFFFF; + let mut block = R2ILBlock::new(0x3000, 4); + block.push(R2ILOp::Copy { + dst: Varnode::new(SpaceId::Custom(7), 0x40, 8), + src: reg(masked, 8), + }); + block.push(R2ILOp::Return { + target: reg(masked, 8), + }); + vec![block] + } + + /// Own 2-block fixture (mirrors `tests/lossless_fixtures.rs` §10 test 8's own fixture): a + /// single-predecessor block whose r2il source is `Multiequal` then `Return`. The phi + /// placeholder that `Multiequal` renames into is extracted out of `SSABlock::ops` into + /// `SSABlock::phis` (`SSAFunction::from_blocks_raw`'s partition), so the surviving + /// `.ops[0]` is `Return` while the SOURCE r2il `.ops[0]` is `Multiequal` — a genuine + /// op-site index shift, not a contrived one. + fn multiequal_fixture() -> Vec { + let mut b0 = R2ILBlock::new(0x2000, 4); + b0.push(R2ILOp::Branch { + target: con(0x2004, 8), + }); + + let mut b1 = R2ILBlock::new(0x2004, 4); + b1.push(R2ILOp::Multiequal { + dst: reg(0x00, 8), + inputs: vec![reg(0x08, 8), reg(0x10, 8), reg(0x18, 8)], + }); + b1.push(R2ILOp::Return { + target: reg(0x00, 8), + }); + + vec![b0, b1] + } + + #[test] + fn enumeration_is_deterministic() { + let blocks_a = linear_fixture(); + let blocks_b = linear_fixture(); + + let behavior_a = + FunctionBehavior::from_blocks_raw(&blocks_a, None).expect("linear fixture ingests"); + let behavior_b = + FunctionBehavior::from_blocks_raw(&blocks_b, None).expect("linear fixture ingests"); + + let facts_a = enumerate(&behavior_a, &blocks_a); + let facts_b = enumerate(&behavior_b, &blocks_b); + assert_eq!( + facts_a, facts_b, + "enumerate over two freshly-ingested, value-identical block lists must agree \ + element by element" + ); + + // Re-running over the SAME behavior must also be stable (no interior mutability, no + // hidden HashMap iteration leaking non-determinism between calls). + let facts_a_again = enumerate(&behavior_a, &blocks_a); + assert_eq!(facts_a, facts_a_again); + assert!(!facts_a.is_empty()); + } + + #[test] + fn operand_coordinates_come_from_the_typed_source_not_from_names() { + let blocks = custom_space_fixture(); + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("custom-space fixture ingests"); + + let facts = enumerate(&behavior, &blocks); + + let masked = 0x1234_5678_9ABC_DEF0_u64 & 0xFFFF; + let register_operand_seen = facts.iter().any(|fact| { + matches!( + fact, + OreFact::Operand { + space: SpaceId::Register, + offset, + size: 8, + .. + } if *offset == masked + ) + }); + assert!( + register_operand_seen, + "the masked register offset must survive into an Operand row exactly, not via a \ + re-derived SSAVar name" + ); + + // Anti-vacuity (§6 test 2): a Custom(7) space operand is unreachable from the SSA side + // without parsing a display string — this can only come from the typed r2il source. + let custom_operand_seen = facts.iter().any(|fact| { + matches!( + fact, + OreFact::Operand { + space: SpaceId::Custom(7), + offset: 0x40, + size: 8, + .. + } + ) + }); + assert!( + custom_operand_seen, + "expected an Operand row carrying SpaceId::Custom(7)" + ); + } + + #[test] + fn an_op_site_join_mismatch_is_enumerated_not_skipped() { + let blocks = multiequal_fixture(); + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("multiequal fixture ingests"); + + let facts = enumerate(&behavior, &blocks); + + let join_failures: Vec<_> = facts + .iter() + .filter_map(|fact| match fact { + OreFact::JoinFailure { + expected, found, .. + } => Some((*expected, *found)), + _ => None, + }) + .collect(); + assert_eq!( + join_failures, + vec![(OpTag::Return, OpTag::Phi)], + "the shifted site must be reported as expected=Return (what SSA's `.ops[0]` is, \ + the Return the Multiequal-derived phi was extracted around) found=Phi (what the \ + source r2il `.ops[0]`, the Multiequal, actually is)" + ); + + // Independently-derived expected total (see this module's doc comment for the + // enumeration order this counts against): + // B0 (0x2000): Op(Branch) + Operand(Input(0), target) = 2 + // Edge B0 -> B1 (Normal) = 1 + // B1 (0x2004): Op(Phi) [no operand rows: phi payload, no join target] = 1 + // Op(Return) + JoinFailure (no Operand rows: mismatch) = 2 + // (Return has no successors -> 0 Edge rows) + // PhiInput (1 phi, fan-in truncated to the single CFG predecessor) = 1 + // memory / predicates / call sites: none in this fixture = 0 + // total = 7 + assert_eq!( + facts.len(), + 7, + "independently-derived expectation: 2 (B0 op+operand) + 1 (edge) + 3 (B1 phi-op, \ + return-op, join-failure) + 1 (phi input) = 7; got {facts:#?}" + ); + } + + #[test] + fn instruction_addr_rejoins_op_metadata_by_op_site_and_is_none_without_it() { + let mut blocks = linear_fixture(); + // op index 1 of block 0x1000 is the `Branch` — attach a real instruction address to it. + blocks[0].set_op_metadata( + 1, + r2il::OpMetadata { + instruction_addr: Some(0xDEAD_BEEF), + ..Default::default() + }, + ); + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("linear fixture ingests"); + let facts = enumerate(&behavior, &blocks); + + let branch_prov = facts + .iter() + .find_map(|fact| match fact { + OreFact::Op { + prov, + opcode: OpTag::Branch, + .. + } => Some(*prov), + _ => None, + }) + .expect("the fixture emits exactly one Branch Op row"); + assert_eq!( + instruction_addr(&branch_prov, &blocks), + Some(0xDEAD_BEEF), + "op index 1 carries metadata and must rejoin to its instruction address" + ); + + // Two-sided: op index 0 (the IntAdd) has NO metadata attached — a stub that always + // returns the same address, or that ignores op_idx, fails this half. + let int_add_prov = facts + .iter() + .find_map(|fact| match fact { + OreFact::Op { + prov, + opcode: OpTag::IntAdd, + .. + } => Some(*prov), + _ => None, + }) + .expect("the fixture emits exactly one IntAdd Op row"); + assert_eq!( + instruction_addr(&int_add_prov, &blocks), + None, + "op index 0 carries no metadata and must rejoin to None, not the sibling's address" + ); + } +} diff --git a/crates/ruff_r2il/src/slag.rs b/crates/ruff_r2il/src/slag.rs new file mode 100644 index 0000000000000..a4481376c0edf --- /dev/null +++ b/crates/ruff_r2il/src/slag.rs @@ -0,0 +1,524 @@ +//! Stage 3b — the ADDRESSED residual ledger ("slag"). +//! +//! [`crate::furnace`] melts the ore (`crate::ore::OreFact`, itself read off +//! [`crate::behavior::FunctionBehavior`]'s object graph) into flat, +//! facet-addressed rows under the CURRENT [`crate::convention::R2ilConvention`]. +//! Not every fact melts. This module is the ledger of what didn't, and WHY — +//! addressed at the [`crate::facet::VarnodeFacet`] coordinate where the melt +//! failed, so a proposer can attach a new [`crate::convention::ConventionRow`] +//! at that exact address and re-run the drill. +//! +//! # The slag doctrine +//! +//! See `ruff_spo_triplet::concept_split`'s "The slag doctrine" — the same +//! discipline applies here verbatim, one layer down the stack: **the +//! residual is not waste, it is the empirical boundary of the current +//! convention.** A recurring [`ResidualReason`] across many addresses names +//! the next convention row to add, not a defect to hide. +//! +//! **`residual` is NOT to be driven to 0 by widening a match arm.** It falls +//! only when the *convention* (`R2ilConvention`) gains a row — never by +//! [`ResidualReason`] growing a catch-all that reclassifies a shape as +//! "handled" without a corresponding convention change. Every such widening +//! of the convention lands with its measured before/after residual counts in +//! the harvest ledger (see `crate::furnace::HarvestReport`), never as a +//! silent code edit here. +//! +//! # HARD RULE — no catch-all, ever +//! +//! [`ResidualReason`] has **no** `Other`, `Opaque`, `Unknown`, or `_ =>` arm +//! that manufactures a reason for a shape that fits no named variant. A +//! shape that doesn't fit an existing variant means a variant is ADDED (with +//! its before/after counts recorded), never that an existing variant is +//! widened to swallow it. [`ResidualReason::ALL`] and the +//! `there_is_no_catch_all_reason` test below exist specifically to catch a +//! future violation of this rule. +//! +//! # Addressing, not naming +//! +//! [`ResidualFact::shape_id`] is computed from the reason's discriminant and +//! its own typed payload ONLY — never from [`ResidualFact::provenance`] or +//! [`ResidualFact::at`]. Two residuals with the identical reason shape at two +//! different addresses therefore share a `shape_id` and group together +//! ([`ResidualLedger::grouped`]); the address is carried alongside so the +//! proposer still knows *where* to attach the fix ([`ResidualLedger::by_address`]). + +use std::collections::BTreeMap; + +use crate::facet::{FacetPrefix, VarnodeFacet}; +use crate::ore::{FactProvenance, OpTag}; + +// FNV-1a 64, implemented inline — no hashing dependency. The exact constants +// from the FNV specification. +const FNV_OFFSET_BASIS_64: u64 = 0xcbf2_9ce4_8422_2325; +const FNV_PRIME_64: u64 = 0x0100_0000_01b3; + +fn fnv1a64(bytes: &[u8]) -> u64 { + let mut hash = FNV_OFFSET_BASIS_64; + for &byte in bytes { + hash ^= u64::from(byte); + hash = hash.wrapping_mul(FNV_PRIME_64); + } + hash +} + +/// FNV-1a 64 over a residual's SHAPE — [`ResidualReason`]'s discriminant and +/// its own typed payload only. **Never** the [`FactProvenance`] and **never** +/// the [`VarnodeFacet`] address. Identical shapes at different sites +/// therefore group under the same id, exactly as MedCare-rs's `fnv1a:` class +/// fingerprints group identical DAL-method shapes across files. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct ShapeId(pub u64); + +/// Why the CURRENT convention could not melt a fact. +/// +/// See the module docs' § "The slag doctrine" and § "HARD RULE". There is +/// **no** catch-all variant. A shape that fits no variant here means a +/// variant is added — with its measured before/after residual counts — never +/// that an existing variant absorbs it. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ResidualReason { + /// The op's [`OpTag`] is not in [`crate::convention::R2ilConvention`]'s + /// `classified_opcodes` set. + OpcodeNotInConvention { opcode: OpTag }, + /// `R2ilConvention::resolve` returned `None` at every prefix depth for + /// this facet. + NoConventionRowAtAddress, + /// The op references a `CallOther` userop index the convention's userop + /// table doesn't name. + UserOpNotInConvention { userop: u32 }, + /// The varnode's `SpaceId::Custom(raw)` is not in the convention's + /// [`crate::facet::CustomSpaceTable`]. + CustomSpaceNotInConvention { raw: u32 }, + /// The `Custom(raw)` space exists in the arch data but exceeds the + /// interned-ordinal budget at CONFIG-KEY construction time (§4's + /// promoted falsifier) — a typed overflow, never a silent truncation. + FacetOverflowAtKey { raw: u32 }, + /// An op's input arity exceeds what the current convention/furnace pass + /// is prepared to emit as fixed rows. + VariadicArity { arity: usize }, + /// A `Phi`'s input count exceeds its block's predecessor count — the + /// fan-in truncation `SSAFunction` construction performs by zipping + /// sources with `cfg.predecessors(addr)`. + PhiFanInExceedsPredecessors { inputs: usize, predecessors: usize }, + /// The memory fact's `ObjectKind` is `EscapedUnknown` — not a + /// classifiable object for this pass. + MemoryObjectEscaped, + /// A branch/call target has no typed `Const`/`Ram` value — the target is + /// indirect and this pass does not resolve indirect control flow. + IndirectTarget, + /// No source `Varnode` exists for this fact (a phi input or a + /// `CallDefine` insertion) — the **only** `ResidualReason` whose + /// [`ResidualFact::at`] is legitimately `None`. + NoFacetCoordinate, + /// The verified op-site join (§2's ⚠ note) found the R2IL op's tag did + /// not match the SSA op's tag at the same `(block_addr, op_idx)` site — + /// e.g. the `Multiequal` index shift or a `CallDefine` insertion. + OpSiteJoinMismatch { expected: OpTag, found: OpTag }, +} + +impl ResidualReason { + /// Every variant's stable `as_str()` name, for the no-catch-all test. + /// Adding a variant to [`ResidualReason`] without adding its name here + /// is exactly what `there_is_no_catch_all_reason` exists to catch. + pub const ALL: &'static [&'static str] = &[ + "opcode_not_in_convention", + "no_convention_row_at_address", + "userop_not_in_convention", + "custom_space_not_in_convention", + "facet_overflow_at_key", + "variadic_arity", + "phi_fan_in_exceeds_predecessors", + "memory_object_escaped", + "indirect_target", + "no_facet_coordinate", + "op_site_join_mismatch", + ]; + + /// Stable `snake_case` name of the variant. Independent of payload. + #[must_use] + pub fn as_str(&self) -> &'static str { + match self { + ResidualReason::OpcodeNotInConvention { .. } => "opcode_not_in_convention", + ResidualReason::NoConventionRowAtAddress => "no_convention_row_at_address", + ResidualReason::UserOpNotInConvention { .. } => "userop_not_in_convention", + ResidualReason::CustomSpaceNotInConvention { .. } => "custom_space_not_in_convention", + ResidualReason::FacetOverflowAtKey { .. } => "facet_overflow_at_key", + ResidualReason::VariadicArity { .. } => "variadic_arity", + ResidualReason::PhiFanInExceedsPredecessors { .. } => "phi_fan_in_exceeds_predecessors", + ResidualReason::MemoryObjectEscaped => "memory_object_escaped", + ResidualReason::IndirectTarget => "indirect_target", + ResidualReason::NoFacetCoordinate => "no_facet_coordinate", + ResidualReason::OpSiteJoinMismatch { .. } => "op_site_join_mismatch", + } + } + + /// FNV-1a 64 over the variant's discriminant name plus its own typed + /// payload — **never** provenance, **never** address. See the module + /// docs' § "Addressing, not naming". + #[must_use] + pub fn shape_id(&self) -> ShapeId { + let mut bytes: Vec = Vec::new(); + bytes.extend_from_slice(self.as_str().as_bytes()); + bytes.push(0xFF); // separator: variant name vs. payload + match self { + ResidualReason::OpcodeNotInConvention { opcode } => { + bytes.extend_from_slice(opcode.as_str().as_bytes()); + } + ResidualReason::NoConventionRowAtAddress => {} + ResidualReason::UserOpNotInConvention { userop } => { + bytes.extend_from_slice(&userop.to_le_bytes()); + } + ResidualReason::CustomSpaceNotInConvention { raw } => { + bytes.extend_from_slice(&raw.to_le_bytes()); + } + ResidualReason::FacetOverflowAtKey { raw } => { + bytes.extend_from_slice(&raw.to_le_bytes()); + } + ResidualReason::VariadicArity { arity } => { + bytes.extend_from_slice(&(*arity as u64).to_le_bytes()); + } + ResidualReason::PhiFanInExceedsPredecessors { + inputs, + predecessors, + } => { + bytes.extend_from_slice(&(*inputs as u64).to_le_bytes()); + bytes.push(0xFE); // separator between the two payload fields + bytes.extend_from_slice(&(*predecessors as u64).to_le_bytes()); + } + ResidualReason::MemoryObjectEscaped => {} + ResidualReason::IndirectTarget => {} + ResidualReason::NoFacetCoordinate => {} + ResidualReason::OpSiteJoinMismatch { expected, found } => { + bytes.extend_from_slice(expected.as_str().as_bytes()); + bytes.push(0xFE); + bytes.extend_from_slice(found.as_str().as_bytes()); + } + } + ShapeId(fnv1a64(&bytes)) + } +} + +/// One ADDRESSED residual row: it records WHERE in varnode identity space +/// the melt failed, so a proposer can emit a proposed +/// [`crate::convention::ConventionRow`] at that address and re-run the drill. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct ResidualFact { + /// Precomputed `reason.shape_id()` — carried on the row so + /// [`ResidualLedger::grouped`]/[`ResidualLedger::by_address`] never + /// recompute a hash per query. + pub shape_id: ShapeId, + pub reason: ResidualReason, + /// The facet coordinate where the melt failed. `None` only for + /// [`ResidualReason::NoFacetCoordinate`] — every other reason carries an + /// address (`residuals_carry_the_address_they_failed_at` pins this). + pub at: Option, + /// The longest [`crate::convention::R2ilConvention`] prefix that DID + /// resolve for this facet — where a proposer attaches the next, finer + /// row. `None` means nothing resolved at all, so a proposal attaches at + /// the coarsest `Space` prefix. + pub at_prefix: Option, + pub provenance: FactProvenance, +} + +/// The addressed residual ledger for one melt pass. +#[derive(Debug, Clone, Default)] +pub struct ResidualLedger { + rows: Vec, +} + +impl ResidualLedger { + #[must_use] + pub fn new() -> Self { + Self::default() + } + + pub fn push(&mut self, fact: ResidualFact) { + self.rows.push(fact); + } + + #[must_use] + pub fn len(&self) -> usize { + self.rows.len() + } + + #[must_use] + pub fn is_empty(&self) -> bool { + self.rows.is_empty() + } + + #[must_use] + pub fn rows(&self) -> &[ResidualFact] { + &self.rows + } + + /// Grouped and counted by [`ShapeId`], sorted by count DESC then + /// `ShapeId` ASC — deterministic artifact order across runs. Each group + /// reports one example address so a proposal has a coordinate to attach + /// at. + #[must_use] + pub fn grouped(&self) -> Vec<(ShapeId, &'static str, usize, Option)> { + let mut groups: BTreeMap)> = + BTreeMap::new(); + for row in &self.rows { + let entry = groups + .entry(row.shape_id) + .or_insert_with(|| (row.reason.as_str(), 0, row.at)); + entry.1 += 1; + } + let mut out: Vec<(ShapeId, &'static str, usize, Option)> = groups + .into_iter() + .map(|(id, (name, count, at))| (id, name, count, at)) + .collect(); + out.sort_by(|a, b| b.2.cmp(&a.2).then_with(|| a.0.cmp(&b.0))); + out + } + + /// Grouped by `(resolved prefix, shape)` — the proposer's actual work + /// queue: for each address a fix would attach at, which shapes recur + /// there and how often. + #[must_use] + pub fn by_address(&self) -> Vec<(Option, ShapeId, usize)> { + let mut groups: BTreeMap<(Option, ShapeId), usize> = BTreeMap::new(); + for row in &self.rows { + *groups.entry((row.at_prefix, row.shape_id)).or_insert(0) += 1; + } + groups + .into_iter() + .map(|((prefix, id), count)| (prefix, id, count)) + .collect() + } + + /// The largest [`ShapeId`] group's share of the whole ledger, in + /// `[0.0, 1.0]`. `0.0` for an empty ledger. A ledger where one shape + /// absorbs (nearly) everything is a catch-all wearing a reason's + /// clothes, and this is the number the pre-registered bar tests. + #[must_use] + pub fn dominant_share(&self) -> f64 { + let total = self.rows.len(); + if total == 0 { + return 0.0; + } + let dominant = self + .grouped() + .into_iter() + .map(|(_, _, count, _)| count) + .max() + .unwrap_or(0); + dominant as f64 / total as f64 + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn prov_at_site(offset: u64, op_idx: usize) -> FactProvenance { + FactProvenance { + inst: None, + block: None, + op_site: Some((offset, op_idx)), + value: None, + } + } + + fn prov_none() -> FactProvenance { + FactProvenance { + inst: None, + block: None, + op_site: None, + value: None, + } + } + + #[test] + fn shape_id_groups_identical_shapes_across_addresses() { + let reason_a = ResidualReason::OpcodeNotInConvention { + opcode: OpTag::AtomicCAS, + }; + let reason_a_again = ResidualReason::OpcodeNotInConvention { + opcode: OpTag::AtomicCAS, + }; + let reason_b = ResidualReason::OpcodeNotInConvention { + opcode: OpTag::CallOther, + }; + + // Same reason payload, different address/provenance -> same shape. + let fact1 = ResidualFact { + shape_id: reason_a.shape_id(), + reason: reason_a, + at: Some(VarnodeFacet([1; 16])), + at_prefix: Some(FacetPrefix::Space { discriminant: 1 }), + provenance: prov_at_site(0x1000, 0), + }; + let fact2 = ResidualFact { + shape_id: reason_a_again.shape_id(), + reason: reason_a_again, + at: Some(VarnodeFacet([2; 16])), + at_prefix: Some(FacetPrefix::Space { discriminant: 9 }), + provenance: prov_at_site(0x2000, 3), + }; + assert_eq!(fact1.shape_id, fact2.shape_id); + + // A different reason payload (different opcode) must differ. + assert_ne!(fact1.shape_id, reason_b.shape_id()); + } + + #[test] + fn residuals_carry_the_address_they_failed_at() { + let mut ledger = ResidualLedger::new(); + + let reason = ResidualReason::NoConventionRowAtAddress; + let prefix_a = FacetPrefix::Space { discriminant: 1 }; + let prefix_b = FacetPrefix::Space { discriminant: 2 }; + + let addressed = |facet_byte: u8, prefix: FacetPrefix| ResidualFact { + shape_id: reason.shape_id(), + reason, + at: Some(VarnodeFacet([facet_byte; 16])), + at_prefix: Some(prefix), + provenance: prov_none(), + }; + + // Two residuals of the same shape at the SAME prefix -> must merge + // to one `by_address` entry with count 2. + ledger.push(addressed(1, prefix_a)); + ledger.push(addressed(2, prefix_a)); + // One residual of the same shape at a DIFFERENT prefix -> its own + // `by_address` entry. + ledger.push(addressed(3, prefix_b)); + + // The sanctioned exception: NoFacetCoordinate carries no address. + let no_coord_reason = ResidualReason::NoFacetCoordinate; + ledger.push(ResidualFact { + shape_id: no_coord_reason.shape_id(), + reason: no_coord_reason, + at: None, + at_prefix: None, + provenance: prov_none(), + }); + + // Every row except NoFacetCoordinate carries an address. + for row in ledger.rows() { + if row.reason == ResidualReason::NoFacetCoordinate { + assert!(row.at.is_none()); + } else { + assert!(row.at.is_some()); + } + } + + let by_addr = ledger.by_address(); + assert!(by_addr.contains(&(Some(prefix_a), reason.shape_id(), 2))); + assert!(by_addr.contains(&(Some(prefix_b), reason.shape_id(), 1))); + assert!(by_addr.contains(&(None, no_coord_reason.shape_id(), 1))); + assert_eq!(by_addr.len(), 3); + } + + #[test] + fn grouping_is_exact_and_deterministically_ordered() { + let mut ledger = ResidualLedger::new(); + + let reason_three = ResidualReason::MemoryObjectEscaped; + let reason_one = ResidualReason::IndirectTarget; + let reason_two = ResidualReason::NoConventionRowAtAddress; + + let push_n = |ledger: &mut ResidualLedger, reason: ResidualReason, n: u8| { + for i in 0..n { + ledger.push(ResidualFact { + shape_id: reason.shape_id(), + reason, + at: Some(VarnodeFacet([i; 16])), + at_prefix: None, + provenance: prov_none(), + }); + } + }; + + push_n(&mut ledger, reason_three, 3); + push_n(&mut ledger, reason_one, 1); + push_n(&mut ledger, reason_two, 2); + + let counts: Vec = ledger + .grouped() + .into_iter() + .map(|(_, _, count, _)| count) + .collect(); + assert_eq!(counts, vec![3, 2, 1]); + + // Deterministic: running the grouping again gives the identical order. + let counts_again: Vec = ledger + .grouped() + .into_iter() + .map(|(_, _, count, _)| count) + .collect(); + assert_eq!(counts_again, vec![3, 2, 1]); + + let total: usize = counts.iter().sum(); + assert_eq!(total, ledger.len()); + } + + #[test] + fn there_is_no_catch_all_reason() { + // Exhaustive match with NO `_ =>` arm: if a variant is ever added to + // `ResidualReason`, this fails to COMPILE until this test (and + // `ResidualReason::ALL`) are updated to match. That non-compiling + // failure mode is the falsifier for "no catch-all reason". + let sample: [ResidualReason; 11] = [ + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::Copy, + }, + ResidualReason::NoConventionRowAtAddress, + ResidualReason::UserOpNotInConvention { userop: 0 }, + ResidualReason::CustomSpaceNotInConvention { raw: 0 }, + ResidualReason::FacetOverflowAtKey { raw: 0 }, + ResidualReason::VariadicArity { arity: 0 }, + ResidualReason::PhiFanInExceedsPredecessors { + inputs: 0, + predecessors: 0, + }, + ResidualReason::MemoryObjectEscaped, + ResidualReason::IndirectTarget, + ResidualReason::NoFacetCoordinate, + ResidualReason::OpSiteJoinMismatch { + expected: OpTag::Copy, + found: OpTag::Copy, + }, + ]; + + let mut variant_count = 0; + for reason in sample { + variant_count += match reason { + ResidualReason::OpcodeNotInConvention { .. } => 1, + ResidualReason::NoConventionRowAtAddress => 1, + ResidualReason::UserOpNotInConvention { .. } => 1, + ResidualReason::CustomSpaceNotInConvention { .. } => 1, + ResidualReason::FacetOverflowAtKey { .. } => 1, + ResidualReason::VariadicArity { .. } => 1, + ResidualReason::PhiFanInExceedsPredecessors { .. } => 1, + ResidualReason::MemoryObjectEscaped => 1, + ResidualReason::IndirectTarget => 1, + ResidualReason::NoFacetCoordinate => 1, + ResidualReason::OpSiteJoinMismatch { .. } => 1, + // Deliberately NO `_ =>` arm. + }; + } + assert_eq!(variant_count, 11); + assert_eq!(ResidualReason::ALL.len(), variant_count); + + let mut sorted = ResidualReason::ALL.to_vec(); + sorted.sort_unstable(); + sorted.dedup(); + assert_eq!( + sorted.len(), + ResidualReason::ALL.len(), + "ResidualReason::ALL has duplicate names" + ); + + for name in ResidualReason::ALL { + assert!( + !matches!(*name, "other" | "opaque" | "unknown" | "misc"), + "catch-all-shaped reason name found: {name}" + ); + } + } +} diff --git a/crates/ruff_r2il/src/vocab.rs b/crates/ruff_r2il/src/vocab.rs new file mode 100644 index 0000000000000..d13748daeb496 --- /dev/null +++ b/crates/ruff_r2il/src/vocab.rs @@ -0,0 +1,471 @@ +//! Stage 4 — DTO / codebook factoring. +//! +//! Feeds lance-graph's `ogar_codebook` — **read-only**. This module never constructs a parallel +//! codebook of its own; it produces a LOCAL, deterministic interning table plus a measurement of +//! what that table would save, so PR 2/3 can wire the real mint against +//! `lance_graph_contract::ogar_codebook` (the `NETWORK_LAYER = 0x0804` analog) with numbers in +//! hand instead of a guess. +//! +//! Three concerns, kept separate rather than folded into one bag of strings: +//! +//! - **Names that need interning** ([`VocabTable`], via [`VocabHarvest::ssa_names`] / +//! [`VocabHarvest::op_spaces`] / [`VocabHarvest::object_spaces`]) — deterministic, +//! order-independent, built from a `BTreeSet` so the table is a pure function of the name +//! *set*. +//! - **Numbers that are already dense and need only counting** ([`VocabHarvest::userops`]) — +//! `CallOther`'s `userop: u32` is already an integer id; wrapping it in a [`VocabTable`] would +//! spend a second id space on a value that already has one, so it is **counted, not +//! interned**. +//! - **The typed-vs-string measurement** ([`custom_space_ids_from_blocks`] vs +//! [`parse_custom_space_id`] / [`custom_space_id_from_var_name`]) — `SpaceId::Custom(u32)` is +//! fully recoverable from the **typed** R2IL side; the plan forbids parsing display strings as +//! a data path (§2), so the string-side functions exist solely to put a NUMBER on how much of +//! that typed truth a naive string-based harvest would have recovered anyway. `None` from +//! either string function is not a defect — it is the measurement working as intended. + +use std::collections::{BTreeMap, BTreeSet}; + +use r2il::{R2ILBlock, R2ILOp, SpaceId}; +use r2ssa::{InstPayload, ObjectKind, SSAOp}; + +use crate::behavior::FunctionBehavior; + +/// A deterministic, dense id into a [`VocabTable`]. 4 bytes — the whole point of interning. +#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] +pub struct VocabId(pub u32); + +/// Deterministic, **order-independent** interning of a name set. +/// +/// Built from a `BTreeSet`, ids assigned in sorted order — the table is a pure function of the +/// name *set*, never of insertion order or of how many times a name was seen. +/// +/// `JUDGMENT`: there is deliberately **no** public incremental `intern(&mut self)` in PR 1. Two +/// build paths — one incremental, one batch — would let the same name end up with a different id +/// depending on which one ran and in what order, which is exactly the nondeterminism this type +/// exists to rule out. A caller who needs to grow a table re-derives it via [`VocabTable::from_names`] +/// over the union of names, not by mutating one in place. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct VocabTable { + by_name: BTreeMap, + names: Vec, +} + +impl VocabTable { + /// Build from any iterable of names — owned or borrowed, any order, duplicates welcome. See + /// the struct docs for why none of that can affect the result. + pub fn from_names(names: I) -> Self + where + I: IntoIterator, + S: Into, + { + let unique: BTreeSet = names.into_iter().map(Into::into).collect(); + let names: Vec = unique.into_iter().collect(); + let by_name = names + .iter() + .enumerate() + .map(|(idx, name)| (name.clone(), VocabId(idx as u32))) + .collect(); + Self { by_name, names } + } + + /// The id for a name, if it was interned. + pub fn id_of(&self, name: &str) -> Option { + self.by_name.get(name).copied() + } + + /// The name for an id, if it is in range. + pub fn name_of(&self, id: VocabId) -> Option<&str> { + self.names.get(id.0 as usize).map(String::as_str) + } + + /// Number of distinct interned names. + pub fn len(&self) -> usize { + self.names.len() + } + + /// `true` iff nothing was interned. + pub fn is_empty(&self) -> bool { + self.names.is_empty() + } + + /// Every `(name, id)` pair, in sorted-name order — the same order ids were assigned in. + pub fn iter(&self) -> impl Iterator + '_ { + self.by_name.iter().map(|(name, id)| (name.as_str(), *id)) + } + + /// Total bytes of every **unique** interned name — the dictionary's own one-time size, not + /// the cost of referencing it from N occurrences. See [`VocabStats::ssa_name_bytes`] / + /// [`VocabStats::interned_id_bytes`] for the occurrence-level comparison. + pub fn name_bytes(&self) -> usize { + self.names.iter().map(String::len).sum() + } +} + +/// One function's vocabulary harvest, factored by concern rather than dumped into one bag of +/// strings. +/// +/// Feeds lance-graph's `ogar_codebook` — **read-only**; see the module docs. PR 1 produces this +/// local table and the [`VocabStats`] measurement; PR 2/3 wire the real mint. +#[derive(Debug, Clone, Default)] +pub struct VocabHarvest { + /// `SSAVar::name` over every entry of `SsaGraph::values` — every SSA value in the function + /// (phi outputs included), deduplicated **by name**, not by `(name, version)`. Two versions + /// of the same physical register (e.g. two writes to `reg:38`) intern to the same + /// [`VocabId`]. + pub ssa_names: VocabTable, + /// `SSAOp`'s own `space: String` field, present on the seven memory ops (`Load`, `Store`, + /// `LoadLinked`, `StoreConditional`, `AtomicCAS`, `LoadGuarded`, `StoreGuarded`). + /// Debug-formatted upstream (`"Ram"`, `"Custom(7)"`, …) — see [`parse_custom_space_id`]'s doc + /// comment for the exact upstream call site this mirrors. + pub op_spaces: VocabTable, + /// `ObjectKind::Global { space, .. }`'s `space` string, over every object in the function's + /// `ObjectModel`. + pub object_spaces: VocabTable, + /// `CallOther`'s `userop: u32` — **counted, not interned** (see the module docs). Key = + /// userop index, value = number of `CallOther` sites mentioning it. + pub userops: BTreeMap, + /// [`parse_custom_space_id`] applied to every observed `op_spaces` string, kept only where it + /// parsed. The STRING-recovered half of the [`custom_space_ids_from_blocks`] measurement — + /// compare against that typed oracle, never trust this set alone. + pub custom_spaces_from_strings: BTreeSet, + + // Private bookkeeping so `stats()` can report the interning-savings measurement without + // re-walking the source `FunctionBehavior` a second time. `JUDGMENT`: kept out of the public, + // concern-factored field list above because these two are pure occurrence counters, not a + // concern in their own right. + total_ssa_values: usize, + ssa_name_bytes_raw: usize, +} + +impl VocabHarvest { + /// Harvest one function's vocabulary. Deterministic: every table is built via + /// [`VocabTable::from_names`] over a `BTreeSet`, and every upstream container walked here + /// (`SsaGraph::insts`, `SsaGraph::values`, `ObjectModel::objects`) is itself ordered + /// (`Vec` / `BTreeMap`) — no `HashMap` iteration leaks into the result. + pub fn from_behavior(behavior: &FunctionBehavior) -> Self { + let graph = behavior.values(); + + let mut total_ssa_values = 0usize; + let mut ssa_name_bytes_raw = 0usize; + let mut ssa_name_list: Vec = Vec::with_capacity(graph.values.len()); + for value in &graph.values { + total_ssa_values += 1; + ssa_name_bytes_raw += value.var.name.len(); + ssa_name_list.push(value.var.name.clone()); + } + let ssa_names = VocabTable::from_names(ssa_name_list); + + let mut op_space_names: BTreeSet = BTreeSet::new(); + let mut userops: BTreeMap = BTreeMap::new(); + let mut custom_spaces_from_strings: BTreeSet = BTreeSet::new(); + + for inst in &graph.insts { + let InstPayload::Op(op) = &inst.payload else { + continue; + }; + if let Some(space) = ssa_op_memory_space(op) { + op_space_names.insert(space.to_string()); + if let Some(id) = parse_custom_space_id(space) { + custom_spaces_from_strings.insert(id); + } + } + if let SSAOp::CallOther { userop, .. } = op { + *userops.entry(*userop).or_insert(0) += 1; + } + } + let op_spaces = VocabTable::from_names(op_space_names); + + let mut object_space_names: BTreeSet = BTreeSet::new(); + for fact in behavior.objects().objects.values() { + if let ObjectKind::Global { space, .. } = &fact.kind { + object_space_names.insert(space.clone()); + } + } + let object_spaces = VocabTable::from_names(object_space_names); + + Self { + ssa_names, + op_spaces, + object_spaces, + userops, + custom_spaces_from_strings, + total_ssa_values, + ssa_name_bytes_raw, + } + } + + /// The measurement: vocabulary size per concern, plus the byte-cost-of-interning comparison + /// for `ssa_names` (by far the largest of the three tables in practice). + pub fn stats(&self) -> VocabStats { + VocabStats { + unique_ssa_names: self.ssa_names.len(), + unique_op_spaces: self.op_spaces.len(), + unique_object_spaces: self.object_spaces.len(), + unique_userops: self.userops.len(), + userop_mentions: self.userops.values().sum(), + unique_custom_spaces_from_strings: self.custom_spaces_from_strings.len(), + total_values: self.total_ssa_values, + ssa_name_bytes: self.ssa_name_bytes_raw, + interned_id_bytes: self.total_ssa_values * std::mem::size_of::(), + } + } +} + +/// The measurement PR 1 promised: how big each vocabulary concern is, and what interning +/// `ssa_names` would cost versus save. +/// +/// `total_values` / `ssa_name_bytes` / `interned_id_bytes` are `JUDGMENT` (the spec names the +/// fields but not their exact formula): `total_values` is the **raw**, pre-name-dedup count of +/// `SsaGraph::values` entries (so a register written twice contributes 2, even though it +/// contributes only 1 to `unique_ssa_names`); `ssa_name_bytes` is what it would cost to store +/// every one of those `total_values` occurrences as its own inline string, uninterned; +/// `interned_id_bytes` is the alternative — one 4-byte [`VocabId`] per occurrence, referencing +/// the deduplicated `ssa_names` table. Add `ssa_names.name_bytes()` (the one-time dictionary +/// cost) to `interned_id_bytes` for the full interned total, and compare against `ssa_name_bytes` +/// alone for the naive total. +#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)] +pub struct VocabStats { + pub unique_ssa_names: usize, + pub unique_op_spaces: usize, + pub unique_object_spaces: usize, + pub unique_userops: usize, + pub userop_mentions: usize, + pub unique_custom_spaces_from_strings: usize, + pub total_values: usize, + pub ssa_name_bytes: usize, + pub interned_id_bytes: usize, +} + +/// The address-space field carried directly by the seven memory [`SSAOp`] variants (distinct from +/// the space of any operand — an op's `addr` var is typically register/const-space and holds an +/// address value; this field says which memory space that address refers *into*). +fn ssa_op_memory_space(op: &SSAOp) -> Option<&str> { + match op { + SSAOp::Load { space, .. } + | SSAOp::Store { space, .. } + | SSAOp::LoadLinked { space, .. } + | SSAOp::StoreConditional { space, .. } + | SSAOp::AtomicCAS { space, .. } + | SSAOp::LoadGuarded { space, .. } + | SSAOp::StoreGuarded { space, .. } => Some(space.as_str()), + _ => None, + } +} + +/// The address-space field carried directly by the seven memory [`R2ILOp`] variants — the typed, +/// pre-rename counterpart of [`ssa_op_memory_space`], used by [`custom_space_ids_from_blocks`]. +fn r2il_op_memory_space(op: &R2ILOp) -> Option { + match op { + R2ILOp::Load { space, .. } + | R2ILOp::Store { space, .. } + | R2ILOp::LoadLinked { space, .. } + | R2ILOp::StoreConditional { space, .. } + | R2ILOp::AtomicCAS { space, .. } + | R2ILOp::LoadGuarded { space, .. } + | R2ILOp::StoreGuarded { space, .. } => Some(*space), + _ => None, + } +} + +/// The TYPED oracle for custom space ids: walks `Varnode::space` (every operand `inputs()` and, +/// if present, `output()`) and, for the seven memory ops that also carry an address-space field +/// of their own, that field too — on the **R2IL** side, before any SSA rename touches it. No +/// parsing, no strings. [`VocabStats::unique_custom_spaces_from_strings`] is measured against +/// this, never the other way around. +pub fn custom_space_ids_from_blocks(blocks: &[R2ILBlock]) -> BTreeSet { + fn note(space: SpaceId, ids: &mut BTreeSet) { + if let SpaceId::Custom(raw) = space { + ids.insert(raw); + } + } + + let mut ids = BTreeSet::new(); + for block in blocks { + for op in &block.ops { + if let Some(space) = r2il_op_memory_space(op) { + note(space, &mut ids); + } + for varnode in op.inputs() { + note(varnode.space, &mut ids); + } + if let Some(varnode) = op.output() { + note(varnode.space, &mut ids); + } + } + } + ids +} + +/// Recover a possible custom-space ordinal from an upstream **space string** — the kind of string +/// that ends up in an [`SSAOp`]'s own `space: String` field. +/// +/// Two known upstream shapes, both verified against `r2sleigh` source this session: +/// - `"Custom(7)"` — the function-level rename's `format!("{:?}", space)` +/// (`r2ssa/src/rename.rs:456`), which is what actually reaches `SSAOp`'s `space` fields via +/// `SSAFunction::from_blocks_raw` (and therefore [`FunctionBehavior::from_blocks_raw`]). +/// - `"space_7"` — `r2ssa::block::space_name` (`r2ssa/src/block.rs:145-152`), a **separate** +/// conversion path this crate's own ingest does not currently exercise, handled here anyway +/// because nothing prevents a caller from handing this function a string produced by it. +/// +/// ⚠ **MEASUREMENT ONLY — never a data path.** The plan forbids parsing display strings for +/// behavioral truth (§2's honesty notes); this function exists solely so [`VocabStats`] can put a +/// NUMBER on the loss by comparing its output against the TYPED oracle, +/// [`custom_space_ids_from_blocks`]. A `None` here means only that this particular string did not +/// match either known shape — it must never gate behavior, and no other module in this crate may +/// call it as part of ore/furnace/slag processing. +/// +/// [`FunctionBehavior::from_blocks_raw`]: crate::behavior::FunctionBehavior::from_blocks_raw +pub fn parse_custom_space_id(space: &str) -> Option { + if let Some(digits) = space + .strip_prefix("Custom(") + .and_then(|rest| rest.strip_suffix(')')) + { + return digits.parse().ok(); + } + space + .strip_prefix("space_") + .and_then(|digits| digits.parse().ok()) +} + +/// Recover a possible custom-space ordinal from an upstream **`SSAVar::name`**. +/// +/// Named variables for a `SpaceId::Custom(n)` varnode are built by `r2ssa::naming::varnode_to_name` +/// (`r2ssa/src/naming.rs:133`) as `format!("space{}:{:x}", id, offset)` — e.g. `"space7:20"` for +/// `Custom(7)` at offset `0x20`: no underscore between `space` and the id, offset in hex after the +/// colon, no version suffix (that is `SSAVar::display_name`, a different string this function does +/// not parse). This is a genuinely different shape from [`parse_custom_space_id`]'s inputs, which +/// is why the two are separate functions rather than one shared parser guessing at a shape — each +/// names its one real upstream source. +/// +/// ⚠ **MEASUREMENT ONLY — never a data path.** Same caveat as [`parse_custom_space_id`]: this +/// exists to measure loss against the typed oracle, [`custom_space_ids_from_blocks`], never to +/// drive behavior. +pub fn custom_space_id_from_var_name(name: &str) -> Option { + let rest = name.strip_prefix("space")?; + let digits = rest.split(':').next()?; + digits.parse().ok() +} + +#[cfg(test)] +mod tests { + use super::*; + use r2il::{MemoryOrdering, Varnode}; + + fn reg(offset: u64, size: u32) -> Varnode { + Varnode::register(offset, size) + } + + /// 1. `vocab_table_is_order_independent` — the same 5 names in two orders (one with + /// duplicates) build equal tables; anti-vacuity: `len() == 5` and at least two ids differ. + #[test] + fn vocab_table_is_order_independent() { + let ordered = VocabTable::from_names(["alpha", "beta", "gamma", "delta", "epsilon"]); + let scrambled_with_dupes = VocabTable::from_names([ + "epsilon", "delta", "delta", "gamma", "beta", "alpha", "alpha", + ]); + assert_eq!(ordered, scrambled_with_dupes); + + // Anti-vacuity: a `from_names` that mapped every name to the same id (or dropped + // duplicates into one bucket by accident) would still satisfy the equality above. + assert_eq!(ordered.len(), 5); + let alpha = ordered.id_of("alpha").expect("alpha was interned"); + let beta = ordered.id_of("beta").expect("beta was interned"); + assert_ne!(alpha, beta); + } + + /// 2. `harvest_counts_every_userop_mention_but_interns_names_once` — two `CallOther`s sharing + /// `userop: 7` plus one with `9` → `unique_userops == 2`, `userop_mentions == 3` (both + /// exact), reused name interned once. + /// + /// The fixture: a single `Return`-terminated block (proven ingestible in isolation by + /// `behavior::tests::empty_block_list_is_none_not_a_panic`) whose first two `CallOther`s both + /// *write* `reg:38` (two distinct SSA versions, same physical register — the "reused name"), + /// and all three `CallOther`s *read* `reg:30` (one SSA version, read three times). + #[test] + fn harvest_counts_every_userop_mention_but_interns_names_once() { + let mut block = R2ILBlock::new(0x2000, 4); + block.push(R2ILOp::CallOther { + output: Some(reg(0x38, 8)), + userop: 7, + inputs: vec![reg(0x30, 8)], + }); + block.push(R2ILOp::CallOther { + output: Some(reg(0x38, 8)), + userop: 7, + inputs: vec![reg(0x30, 8)], + }); + block.push(R2ILOp::CallOther { + output: None, + userop: 9, + inputs: vec![reg(0x30, 8)], + }); + block.push(R2ILOp::Return { + target: reg(0x38, 8), + }); + + let behavior = FunctionBehavior::from_blocks_raw(&[block], None) + .expect("a single Return-terminated block ingests"); + let harvest = VocabHarvest::from_behavior(&behavior); + let stats = harvest.stats(); + + assert_eq!(stats.unique_userops, 2, "userop ids 7 and 9, no more"); + assert_eq!(stats.userop_mentions, 3, "two CallOthers on 7, one on 9"); + + // "reused name interned once": reg:38 is DEFINED twice (two SSA versions, so it + // contributes 2 to the raw `total_values` count) yet only two distinct NAME strings + // ("reg:30", "reg:38") ever existed across the whole function. + assert_eq!(harvest.ssa_names.len(), 2); + assert!(harvest.ssa_names.id_of("reg:38").is_some()); + assert!(harvest.ssa_names.id_of("reg:30").is_some()); + assert_eq!( + stats.total_values, 3, + "reg:30 v0 (1 occurrence) + reg:38 v0,v1 (2 occurrences) = 3 raw values over 2 names" + ); + } + + /// 3. (recommended) `typed_custom_space_ids_are_the_oracle_for_the_string_set` — + /// `custom_space_ids_from_blocks == {7}`; the string-recovered set equals it on this + /// fixture, with a doc comment recording that the equality is a measurement, not a + /// guarantee. + #[test] + fn typed_custom_space_ids_are_the_oracle_for_the_string_set() { + let mut block = R2ILBlock::new(0x1010, 4); + block.push(R2ILOp::LoadGuarded { + dst: reg(0x30, 8), + space: SpaceId::Custom(7), + addr: reg(0x20, 8), + guard: reg(0x28, 1), + ordering: MemoryOrdering::Acquire, + }); + block.push(R2ILOp::Return { + target: reg(0x30, 8), + }); + let blocks = vec![block]; + + let typed = custom_space_ids_from_blocks(&blocks); + assert_eq!(typed, BTreeSet::from([7])); + + // The string-recovered set, built from the EXACT upstream string shapes this session + // verified by reading source (see `parse_custom_space_id` / `custom_space_id_from_var_name` + // doc comments for the cited call sites) — NOT by running the rename pass here: + // - rename.rs:456 stamps an `SSAOp::LoadGuarded`'s `space: String` with + // `format!("{:?}", SpaceId::Custom(7))` -> `"Custom(7)"`. + // - naming.rs:133 names the `Custom(7)` varnode at offset `0x20` as + // `format!("space{}:{:x}", 7, 0x20u64)` -> `"space7:20"`. + // + // Equality below is a MEASUREMENT on this fixture's two known shapes, never a guarantee + // that every upstream space string is recoverable this way. + let mut recovered = BTreeSet::new(); + if let Some(id) = parse_custom_space_id("Custom(7)") { + recovered.insert(id); + } + if let Some(id) = custom_space_id_from_var_name("space7:20") { + recovered.insert(id); + } + assert_eq!(recovered, typed); + + // Anti-vacuity for the string parsers themselves: an unrelated string must not resolve. + assert_eq!(parse_custom_space_id("Ram"), None); + assert_eq!(custom_space_id_from_var_name("reg:38"), None); + } +} diff --git a/crates/ruff_r2il/tests/lossless_fixtures.rs b/crates/ruff_r2il/tests/lossless_fixtures.rs new file mode 100644 index 0000000000000..2c422103527ec --- /dev/null +++ b/crates/ruff_r2il/tests/lossless_fixtures.rs @@ -0,0 +1,981 @@ +//! Losslessness fixtures for the R2IL behavioral IR, PR 1. +//! +//! Every test here ingests a hand-built `R2ILBlock` list through +//! `FunctionBehavior::from_blocks_raw` (never the generic/SCCP-applying path — a +//! losslessness claim must not go through an optimizer) and checks that the typed +//! upstream decomposition (`r2ssa::SSAFunction` / `SsaGraph` / `PreparedFunctionFacts`) +//! survives ingest unchanged. Tests 10-12 additionally run the fixture through the +//! furnace (`furnace::smelt`) to prove the ore -> furnace -> slag loop: stressor ops +//! land in a *named, addressed* residual ledger under a deliberately narrow pass-1 +//! convention, and widening the convention (in data, never in a match arm) moves +//! them out again. + +use std::collections::BTreeSet; + +use r2il::{ + ArchSpec, AtomicKind, MemoryOrdering, OpMetadata, PointerHint, R2ILBlock, R2ILOp, RegisterDef, + ScalarKind, SpaceId, Varnode, VarnodeMetadata, +}; +use r2ssa::{CompareKind, InstPayload, SSAOp}; + +use ruff_r2il::behavior::FunctionBehavior; +use ruff_r2il::convention::R2ilConvention; +use ruff_r2il::facet; +use ruff_r2il::furnace; +use ruff_r2il::ore::OpTag; +use ruff_r2il::slag::ResidualReason; + +// --------------------------------------------------------------------------- +// Fixture helpers +// --------------------------------------------------------------------------- + +fn reg(off: u64, size: u32) -> Varnode { + Varnode::register(off, size) +} + +fn con(v: u64, size: u32) -> Varnode { + Varnode::constant(v, size) +} + +/// Unwraps a `Result` without requiring `E: Debug`. `FacetOverflow`'s derive set +/// isn't pinned by the spec text, so the fixtures that construct a +/// `R2ilConvention`/`VarnodeFacet` via a fallible path stay independent of it. +fn must_ok(result: Result, msg: &str) -> T { + match result { + Ok(value) => value, + Err(_) => panic!("{msg}"), + } +} + +/// A stable, human-readable key for a `MemoryOrdering` value. `MemoryOrdering` +/// derives `Copy`/`Eq` but not `Ord`/`Hash`, so set membership is compared through +/// this discriminant string instead. +fn ordering_key(ordering: MemoryOrdering) -> &'static str { + match ordering { + MemoryOrdering::Relaxed => "relaxed", + MemoryOrdering::Acquire => "acquire", + MemoryOrdering::Release => "release", + MemoryOrdering::AcqRel => "acq_rel", + MemoryOrdering::SeqCst => "seq_cst", + MemoryOrdering::Unknown => "unknown", + } +} + +/// A stable, human-readable key for a `CompareKind` value, for the same reason. +fn compare_key(kind: CompareKind) -> &'static str { + match kind { + CompareKind::Equal => "equal", + CompareKind::NotEqual => "not_equal", + CompareKind::Less => "less", + CompareKind::SignedLess => "signed_less", + CompareKind::LessEqual => "less_equal", + CompareKind::SignedLessEqual => "signed_less_equal", + } +} + +/// 4 blocks, 3-way merge, deliberately loaded with every mandated op shape plus a +/// stressor block (B2) that pass-1 conventions cannot classify. See +/// `.claude/plans/r2il-behavioral-ir-v1-impl-spec.md` §10 for the exact op list this +/// transcribes verbatim. +fn fixture_function() -> Vec { + // B0 @ 0x1000 size 8 -> ConditionalBranch{true: 0x1018, false: 0x1008} + let mut b0 = R2ILBlock::new(0x1000, 8); + b0.push(R2ILOp::IntAdd { + dst: reg(0x00, 8), + a: reg(0x00, 8), + b: con(0x10, 8), + }); + b0.push(R2ILOp::IntSub { + dst: reg(0x58, 8), + a: reg(0x00, 8), + b: con(4, 8), + }); + b0.push(R2ILOp::IntLess { + dst: reg(0x40, 1), + a: reg(0x00, 8), + b: con(0x100, 8), + }); + b0.push(R2ILOp::CBranch { + target: con(0x1018, 8), + cond: reg(0x40, 1), + }); + + // B1 @ 0x1008 size 8 -> ConditionalBranch{true: 0x1018, false: 0x1010} + let mut b1 = R2ILBlock::new(0x1008, 8); + b1.push(R2ILOp::Load { + dst: reg(0x08, 8), + space: SpaceId::Ram, + addr: reg(0x00, 8), + }); + b1.push(R2ILOp::Store { + space: SpaceId::Ram, + addr: reg(0x00, 8), + val: reg(0x08, 8), + }); + b1.push(R2ILOp::Copy { + dst: reg(0x00, 8), + src: reg(0x08, 8), + }); + b1.push(R2ILOp::IntSLess { + dst: reg(0x44, 1), + a: reg(0x08, 8), + b: con(0, 8), + }); + b1.push(R2ILOp::CBranch { + target: con(0x1018, 8), + cond: reg(0x44, 1), + }); + + // B2 @ 0x1010 size 8 -> Fallthrough{next: 0x1018} -- the stressor block. + let mut b2 = R2ILBlock::new(0x1010, 8); + b2.push(R2ILOp::AtomicCAS { + dst: reg(0x00, 8), + space: SpaceId::Ram, + addr: reg(0x20, 8), + expected: con(0, 8), + replacement: con(1, 8), + ordering: MemoryOrdering::SeqCst, + }); + b2.push(R2ILOp::StoreConditional { + result: Some(reg(0x28, 1)), + space: SpaceId::Ram, + addr: reg(0x20, 8), + val: reg(0x00, 8), + ordering: MemoryOrdering::Release, + }); + b2.push(R2ILOp::StoreConditional { + result: None, + space: SpaceId::Ram, + addr: reg(0x20, 8), + val: reg(0x00, 8), + ordering: MemoryOrdering::Relaxed, + }); + b2.push(R2ILOp::LoadGuarded { + dst: reg(0x30, 8), + space: SpaceId::Custom(7), + // The ADDRESS varnode genuinely lives in Custom(7), not merely the op's `space:` field. + // MEASURED reason: an op's own `space:` is not a varnode and never surfaces through + // `R2ILOp::inputs()/output()`, so a Custom space that exists ONLY there is invisible to + // operand enumeration and could never reach `facet::project` -- the config-key falsifier + // would silently test nothing. A varnode in the space exercises the real path. + // (That op-level-space harvest gap is real and is recorded as a named plan item; it is a + // missing ore fact kind, not a conservation violation.) + addr: Varnode::new(SpaceId::Custom(7), 0x20, 8), + guard: reg(0x28, 1), + ordering: MemoryOrdering::Acquire, + }); + b2.push(R2ILOp::StoreGuarded { + space: SpaceId::Custom(7), + addr: reg(0x20, 8), + val: reg(0x30, 8), + guard: reg(0x28, 1), + ordering: MemoryOrdering::AcqRel, + }); + b2.push(R2ILOp::CallOther { + output: Some(reg(0x38, 8)), + userop: 42, + inputs: vec![reg(0x30, 8), reg(0x28, 1), con(1, 4), con(2, 4)], + }); + b2.push(R2ILOp::CallOther { + output: None, + userop: 42, + inputs: vec![reg(0x30, 8)], + }); + b2.push(R2ILOp::Insert { + dst: reg(0x48, 8), + src: reg(0x30, 8), + value: reg(0x38, 8), + position: con(3, 4), + }); + b2.push(R2ILOp::Load { + dst: reg(0x50, 8), + space: SpaceId::Ram, + addr: Varnode::ram(0x1234_5678_9ABC_DEF0, 8), + }); + b2.push(R2ILOp::Fence { + ordering: MemoryOrdering::Unknown, + }); + b2.push(R2ILOp::Copy { + dst: reg(0x00, 8), + src: reg(0x48, 8), + }); + + // Op metadata at index 0 (the AtomicCAS). + b2.set_op_metadata( + 0, + OpMetadata { + memory_ordering: Some(MemoryOrdering::SeqCst), + atomic_kind: Some(AtomicKind::CompareExchange), + ..Default::default() + }, + ); + + // Varnode metadata on op 3's dst (the LoadGuarded). + if let R2ILOp::LoadGuarded { dst, .. } = &mut b2.ops[3] { + dst.set_meta(VarnodeMetadata { + scalar_kind: Some(ScalarKind::UnsignedInt), + pointer_hint: Some(PointerHint::PointerLike), + ..Default::default() + }); + } else { + unreachable!("op index 3 must be the LoadGuarded pushed above"); + } + + // B3 @ 0x1018 size 8 -- merge, 3 predecessors, terminator Return (the reverse + // scan hits Return before Call; correct and intended). + let mut b3 = R2ILBlock::new(0x1018, 8); + b3.push(R2ILOp::Call { + target: con(0x2000, 8), + }); + b3.push(R2ILOp::Return { + target: reg(0x00, 8), + }); + + vec![b0, b1, b2, b3] +} + +/// Own 2-block fixture: B0 branches unconditionally into B1, but B1's `Multiequal` +/// declares 3 inputs even though the CFG gives it only 1 real predecessor. This is +/// the phi-zip stressor test 8 probes. +fn multiequal_fixture() -> Vec { + let mut b0 = R2ILBlock::new(0x2000, 4); + b0.push(R2ILOp::Branch { + target: con(0x2004, 8), + }); + + let mut b1 = R2ILBlock::new(0x2004, 4); + b1.push(R2ILOp::Multiequal { + dst: reg(0x00, 8), + inputs: vec![reg(0x08, 8), reg(0x10, 8), reg(0x18, 8)], + }); + b1.push(R2ILOp::Return { + target: reg(0x00, 8), + }); + + vec![b0, b1] +} + +// --------------------------------------------------------------------------- +// 1. every_mandated_op_shape_survives_ingest_as_a_typed_ssa_op +// --------------------------------------------------------------------------- + +#[test] +fn every_mandated_op_shape_survives_ingest_as_a_typed_ssa_op() { + let blocks = fixture_function(); + let behavior = + FunctionBehavior::from_blocks_raw(&blocks, None).expect("fixture_function must ingest"); + + let mut observed: BTreeSet<&'static str> = BTreeSet::new(); + for inst in &behavior.values().insts { + let tag = match &inst.payload { + InstPayload::Phi { .. } => OpTag::Phi, + InstPayload::Op(op) => OpTag::from_op(op), + }; + observed.insert(tag.as_str()); + } + + let expected: BTreeSet<&'static str> = [ + OpTag::Phi, + OpTag::Copy, + OpTag::Load, + OpTag::Store, + OpTag::Fence, + OpTag::StoreConditional, + OpTag::AtomicCAS, + OpTag::LoadGuarded, + OpTag::StoreGuarded, + OpTag::IntAdd, + OpTag::IntSub, + OpTag::IntLess, + OpTag::IntSLess, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + OpTag::CallOther, + OpTag::Insert, + ] + .into_iter() + .map(OpTag::as_str) + .collect(); + + // Set equality: a missing op fails, and an unexpected extra fails too. + assert_eq!(observed, expected); +} + +// --------------------------------------------------------------------------- +// 2. op_order_within_a_block_is_preserved_in_ordinal_order +// --------------------------------------------------------------------------- + +#[test] +fn op_order_within_a_block_is_preserved_in_ordinal_order() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + let graph = behavior.values(); + + // B2 carries no Multiequal, so the 1:1 rename rule holds and its op order must + // survive untouched. + let block = graph + .blocks + .iter() + .find(|b| b.addr == 0x1010) + .expect("block 0x1010 must exist"); + + assert_eq!(block.insts.len(), 11, "B2 has no phis, 11 ops 1:1"); + + let mut last_ordinal: Option = None; + let mut observed_tags: Vec<&'static str> = Vec::new(); + for &inst_id in &block.insts { + let inst = graph.inst(inst_id).expect("inst must resolve"); + if let Some(last) = last_ordinal { + assert!(inst.ordinal > last, "ordinal must strictly increase"); + } + last_ordinal = Some(inst.ordinal); + + let InstPayload::Op(op) = &inst.payload else { + panic!("B2 has no merge point, so no inst there may be phi-shaped"); + }; + observed_tags.push(OpTag::from_op(op).as_str()); + } + + let expected_tags: Vec<&'static str> = [ + OpTag::AtomicCAS, + OpTag::StoreConditional, + OpTag::StoreConditional, + OpTag::LoadGuarded, + OpTag::StoreGuarded, + OpTag::CallOther, + OpTag::CallOther, + OpTag::Insert, + OpTag::Load, + OpTag::Fence, + OpTag::Copy, + ] + .into_iter() + .map(OpTag::as_str) + .collect(); + + assert_eq!(observed_tags.len(), 11); + assert_eq!(observed_tags, expected_tags); +} + +// --------------------------------------------------------------------------- +// 3. facts_are_populated_and_counted_against_upstreams_own_classification +// --------------------------------------------------------------------------- + +#[test] +fn facts_are_populated_and_counted_against_upstreams_own_classification() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + // Calls: exactly the one Call in B3, resolving to a direct target. + let calls = behavior.calls(); + assert_eq!(calls.by_id.len(), 1); + let call = calls.by_id.values().next().expect("exactly one call site"); + assert_eq!(call.direct_target, Some(0x2000)); + + // Predicates: exactly the two CBranch-terminated blocks (B0, B1), and their + // comparison kinds are exactly the two compare ops that feed them. + let predicates = behavior.predicates(); + assert_eq!(predicates.predicates.len(), 2); + let kinds: BTreeSet<&'static str> = predicates + .predicates + .values() + .filter_map(|p| p.comparison.as_ref()) + .map(|c| compare_key(c.kind)) + .collect(); + assert_eq!( + kinds, + BTreeSet::from(["less", "signed_less"]), + "B0 compares via IntLess, B1 via IntSLess" + ); + + // Memory counts, computed from upstream's OWN classification rule (never + // hardcoded): uses <- Load/LoadLinked/LoadGuarded/AtomicCAS/StoreConditional; + // defs <- Store/StoreGuarded/StoreConditional/AtomicCAS; both <- Call/CallInd. + let graph = behavior.values(); + let mut expected_uses = 0usize; + let mut expected_defs = 0usize; + for inst in &graph.insts { + let InstPayload::Op(op) = &inst.payload else { + continue; + }; + match op { + SSAOp::Load { .. } | SSAOp::LoadLinked { .. } | SSAOp::LoadGuarded { .. } => { + expected_uses += 1; + } + SSAOp::AtomicCAS { .. } | SSAOp::StoreConditional { .. } => { + expected_uses += 1; + expected_defs += 1; + } + SSAOp::Store { .. } | SSAOp::StoreGuarded { .. } => { + expected_defs += 1; + } + SSAOp::Call { .. } | SSAOp::CallInd { .. } => { + expected_uses += 1; + expected_defs += 1; + } + _ => {} + } + } + assert!( + expected_uses >= 5, + "anti-vacuity: the fixture must exercise enough memory uses" + ); + + let memory = behavior.memory(); + let observed_uses: usize = memory.uses_by_inst.values().map(Vec::len).sum(); + let observed_defs: usize = memory.defs_by_inst.values().map(Vec::len).sum(); + assert_eq!(observed_uses, expected_uses); + assert_eq!(observed_defs, expected_defs); + + // Objects: the 64-bit RAM literal in B2 must resolve to a global object. + // + // NOTE (measurement, not an assertion): the resolved `GlobalObjectKey.space` + // is upstream's own Debug-formatted `SpaceId::Ram` ("Ram"), produced by + // `r2ssa::rename`'s function-level renaming. That formatting is upstream's, + // not ours, so it is recorded here rather than asserted on. + let objects = behavior.objects(); + assert!( + objects + .global_objects + .keys() + .any(|key| key.address == 0x1234_5678_9ABC_DEF0), + "the exact 64-bit RAM literal must resolve to a global object" + ); +} + +// --------------------------------------------------------------------------- +// 4. phi_fan_in_equals_the_predecessor_count_at_the_three_way_merge +// --------------------------------------------------------------------------- + +#[test] +fn phi_fan_in_equals_the_predecessor_count_at_the_three_way_merge() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + let preds = behavior.control().predecessors(0x1018); + assert_eq!(preds.len(), 3, "B3 merges three blocks"); + + let block = behavior + .control() + .get_block(0x1018) + .expect("the merge block must exist"); + assert!( + !block.phis.is_empty(), + "reg:0 is live-into B3 with three distinct definitions, so at least one phi must exist" + ); + for phi in &block.phis { + assert_eq!(phi.sources.len(), 3); + assert_eq!(phi.sources.len(), preds.len()); + } +} + +// --------------------------------------------------------------------------- +// 5. custom_space_and_every_memory_ordering_survive_into_typed_ssa_ops +// --------------------------------------------------------------------------- + +#[test] +fn custom_space_and_every_memory_ordering_survive_into_typed_ssa_ops() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + let mut load_guarded_space: Option = None; + let mut store_guarded_space: Option = None; + let mut load_guarded_ordering: Option = None; + let mut store_guarded_ordering: Option = None; + let mut orderings: BTreeSet<&'static str> = BTreeSet::new(); + + for inst in &behavior.values().insts { + let InstPayload::Op(op) = &inst.payload else { + continue; + }; + match op { + SSAOp::LoadGuarded { + space, ordering, .. + } => { + load_guarded_space = Some(space.clone()); + load_guarded_ordering = Some(*ordering); + orderings.insert(ordering_key(*ordering)); + } + SSAOp::StoreGuarded { + space, ordering, .. + } => { + store_guarded_space = Some(space.clone()); + store_guarded_ordering = Some(*ordering); + orderings.insert(ordering_key(*ordering)); + } + SSAOp::AtomicCAS { ordering, .. } => { + orderings.insert(ordering_key(*ordering)); + } + SSAOp::StoreConditional { ordering, .. } => { + orderings.insert(ordering_key(*ordering)); + } + SSAOp::Fence { ordering } => { + orderings.insert(ordering_key(*ordering)); + } + _ => {} + } + } + + let lg_space = load_guarded_space.expect("LoadGuarded must survive ingest"); + let sg_space = store_guarded_space.expect("StoreGuarded must survive ingest"); + assert!(!lg_space.is_empty(), "the custom space must carry a name"); + assert_eq!( + lg_space, sg_space, + "LoadGuarded and StoreGuarded share the same custom space" + ); + + assert_eq!(load_guarded_ordering, Some(MemoryOrdering::Acquire)); + assert_eq!(store_guarded_ordering, Some(MemoryOrdering::AcqRel)); + + let expected_orderings: BTreeSet<&'static str> = [ + MemoryOrdering::Relaxed, + MemoryOrdering::Acquire, + MemoryOrdering::Release, + MemoryOrdering::AcqRel, + MemoryOrdering::SeqCst, + MemoryOrdering::Unknown, + ] + .into_iter() + .map(ordering_key) + .collect(); + assert_eq!(orderings, expected_orderings, "exact set equality"); +} + +// --------------------------------------------------------------------------- +// 6. op_metadata_rejoins_by_op_site_even_though_ssa_does_not_carry_it +// --------------------------------------------------------------------------- + +#[test] +fn op_metadata_rejoins_by_op_site_even_though_ssa_does_not_carry_it() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + let inst_id = behavior + .inst_at(0x1010, 0) + .expect("op site (0x1010, 0) must resolve to an inst"); + + let round_trip = behavior + .op_site(inst_id) + .expect("op_site must round-trip the inst id"); + assert_eq!(round_trip, (0x1010, 0)); + + let inst = behavior.values().inst(inst_id).expect("inst must resolve"); + let InstPayload::Op(SSAOp::AtomicCAS { .. }) = &inst.payload else { + panic!("op site (0x1010, 0) must be the AtomicCAS the fixture placed there"); + }; + + // The source R2IL block still carries the metadata; SSA never does, so this + // rejoin can only happen through the (block_addr, op_idx) key. + let source_meta = blocks + .iter() + .find(|b| b.addr == 0x1010) + .and_then(|b| b.op_metadata(0)) + .expect("source op metadata must be present at index 0"); + assert_eq!(source_meta.memory_ordering, Some(MemoryOrdering::SeqCst)); + assert_eq!(source_meta.atomic_kind, Some(AtomicKind::CompareExchange)); +} + +// --------------------------------------------------------------------------- +// 7. varnode_metadata_is_advisory_and_does_not_change_ingest +// --------------------------------------------------------------------------- + +#[test] +fn varnode_metadata_is_advisory_and_does_not_change_ingest() { + fn one_op_block(with_meta: bool) -> Vec { + let mut dst = reg(0x30, 8); + if with_meta { + dst.set_meta(VarnodeMetadata { + scalar_kind: Some(ScalarKind::UnsignedInt), + pointer_hint: Some(PointerHint::PointerLike), + ..Default::default() + }); + } + let mut block = R2ILBlock::new(0x3000, 4); + block.push(R2ILOp::IntAdd { + dst, + a: reg(0x00, 8), + b: con(1, 8), + }); + block.push(R2ILOp::Return { + target: reg(0x30, 8), + }); + vec![block] + } + + let plain = FunctionBehavior::from_blocks_raw(&one_op_block(false), None) + .expect("plain fixture must ingest"); + let annotated = FunctionBehavior::from_blocks_raw(&one_op_block(true), None) + .expect("annotated fixture must ingest"); + + let value_triples = |behavior: &FunctionBehavior| -> Vec<(String, u32, u32)> { + behavior + .values() + .values + .iter() + .map(|v| (v.var.name.clone(), v.var.version, v.var.size)) + .collect() + }; + + assert_eq!(value_triples(&plain), value_triples(&annotated)); + assert_eq!(plain.values().insts.len(), annotated.values().insts.len()); +} + +// --------------------------------------------------------------------------- +// 8. multiequal_ingest_becomes_a_phi_zipped_to_the_predecessor_count +// --------------------------------------------------------------------------- + +#[test] +fn multiequal_ingest_becomes_a_phi_zipped_to_the_predecessor_count() { + let blocks = multiequal_fixture(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + let preds = behavior.control().predecessors(0x2004); + let block = behavior + .control() + .get_block(0x2004) + .expect("the multiequal block must exist"); + + assert_eq!( + block.phis.len(), + 1, + "exactly one phi at the multiequal site" + ); + let phi = &block.phis[0]; + + // State the rule, not the number: fan-in is truncated to zip against the + // CFG's own predecessor count, even though the source Multiequal declared 3 + // inputs against a block with only 1 real predecessor. + assert_eq!(phi.sources.len(), preds.len()); + + for op in &block.ops { + assert!( + !matches!(op, SSAOp::Phi { .. }), + "phi-shaped ops must never remain in block.ops" + ); + } +} + +// --------------------------------------------------------------------------- +// 9. sixty_four_bit_offsets_are_not_truncated_on_ingest +// --------------------------------------------------------------------------- + +#[test] +fn sixty_four_bit_offsets_are_not_truncated_on_ingest() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + let objects = behavior.objects(); + assert!( + objects + .global_objects + .keys() + .any(|key| key.address == 0x1234_5678_9ABC_DEF0), + "the exact 64-bit address must survive" + ); + + // Two-sided anti-truncation: the low-32-bit-truncated form must never appear. + let truncated_name = "ram:9abcdef0"; + assert!( + behavior + .values() + .values + .iter() + .all(|v| v.var.name != truncated_name), + "no SSAVar name may equal the 32-bit-truncated form" + ); +} + +// --------------------------------------------------------------------------- +// 10. stressors_land_in_slag_under_pass_one_and_are_named_and_addressed +// --------------------------------------------------------------------------- + +#[test] +fn stressors_land_in_slag_under_pass_one_and_are_named_and_addressed() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + let conv = R2ilConvention::minimal_pass_one(); + + let (_flat, ledger, report) = furnace::smelt(&behavior, &blocks, &conv); + + assert_eq!(report.dropped, 0); + assert!(report.is_conserved()); + assert!(report.residual > 0, "pass one must leave residue"); + assert!( + !ledger.rows().is_empty(), + "the ledger itself must be non-empty" + ); + + let mut found_atomic_cas = false; + let mut found_call_other = false; + let mut found_store_guarded = false; + let mut found_insert = false; + let mut found_custom_space = false; + let mut found_variadic = false; + + for row in ledger.rows() { + match &row.reason { + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::AtomicCAS, + } => found_atomic_cas = true, + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::CallOther, + } => found_call_other = true, + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::StoreGuarded, + } => found_store_guarded = true, + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::Insert, + } => found_insert = true, + ResidualReason::CustomSpaceNotInConvention { raw: 7 } => found_custom_space = true, + ResidualReason::VariadicArity { arity: 4 } => found_variadic = true, + _ => {} + } + + // The addressed-slag rule: every residual except NoFacetCoordinate carries + // its facet coordinate. (Not printing `row.reason` here: `ResidualReason`'s + // derive set isn't pinned by the spec text, so this assertion does not lean + // on it implementing `Debug`.) + if !matches!(row.reason, ResidualReason::NoFacetCoordinate) { + assert!( + row.at.is_some(), + "every addressed residual must carry a facet coordinate" + ); + } + } + + // Each asserted individually so no single absorbing group can satisfy the test. + assert!( + found_atomic_cas, + "AtomicCAS must land in slag under pass one" + ); + assert!( + found_call_other, + "CallOther must land in slag under pass one" + ); + assert!( + found_store_guarded, + "StoreGuarded must land in slag under pass one" + ); + assert!(found_insert, "Insert must land in slag under pass one"); + assert!( + found_custom_space, + "the Custom(7) space must land in slag under pass one" + ); + assert!( + found_variadic, + "the 4-input CallOther must land in slag under pass one" + ); +} + +// --------------------------------------------------------------------------- +// 11. widening_the_convention_moves_a_stressor_out_of_slag +// --------------------------------------------------------------------------- + +#[test] +fn widening_the_convention_moves_a_stressor_out_of_slag() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + // Both conventions are built from the SAME (empty) ArchSpec, so their row sets + // are identical -- the ONLY difference between them is whether AtomicCAS is in + // the classified-opcode set. That isolates the measured delta to exactly what + // widening the ladder changes, with nothing else free to move. + let empty_arch = ArchSpec::new("widen-test"); + + let base_conv = must_ok( + R2ilConvention::from_arch( + &empty_arch, + [ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + ], + ), + "the base convention must build within budget", + ); + + let widened_conv = must_ok( + R2ilConvention::from_arch( + &empty_arch, + [ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + OpTag::AtomicCAS, + ], + ), + "the widened convention must build within budget", + ); + + let (_flat_before, ledger_before, report_before) = + furnace::smelt(&behavior, &blocks, &base_conv); + let (_flat_after, ledger_after, report_after) = + furnace::smelt(&behavior, &blocks, &widened_conv); + + let atomic_cas_op_residual_before = ledger_before + .rows() + .iter() + .filter(|r| { + matches!( + r.reason, + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::AtomicCAS + } + ) + }) + .count(); + // MEASURED, not assumed: the fixture holds ONE AtomicCAS op, but an unclassified op blocks + // every ore fact that depends on it — its own `Op` row plus its four operand rows and the + // memory use/def rows it generates, all tagged with the PARENT's opcode. Conservation demands + // exactly that (every ore fact is classified or residual, never dropped), so the count is the + // op's whole dependent fan-out, not 1. An earlier `== 1` here encoded the wrong model and is + // the reason this comment exists. + assert!( + atomic_cas_op_residual_before > 1, + "AtomicCAS must block its dependent fan-out, not merely its own Op row" + ); + + let atomic_cas_op_residual_after = ledger_after + .rows() + .iter() + .filter(|r| { + matches!( + r.reason, + ResidualReason::OpcodeNotInConvention { + opcode: OpTag::AtomicCAS + } + ) + }) + .count(); + assert_eq!( + atomic_cas_op_residual_after, 0, + "AtomicCAS's residual row must disappear once it is classified" + ); + + // Its residual rows disappear, `classified` rises by exactly that count and + // `residual` falls by the same -- proving the ledger tracks the convention, + // not noise. Since the two conventions differ ONLY in whether AtomicCAS + // classifies, this delta is attributable entirely to that one change. + let delta_classified = report_after.classified - report_before.classified; + let delta_residual = report_before.residual - report_after.residual; + assert!( + delta_classified > 0, + "widening must move something out of slag" + ); + // MEASURED: widening reclassifies only the subset that has nothing else blocking it — the op + // row and its memory rows melt, while its OPERAND rows stay residual under a DIFFERENT named + // reason (`NoConventionRowAtAddress`: `minimal_pass_one()` carries no address rows, so an + // operand's facet resolves to nothing). So the blocked fan-out is an upper bound on the + // delta, never an equality — asserting equality here was wrong and this bound is the true + // claim. + assert!( + delta_classified <= atomic_cas_op_residual_before, + "widening cannot reclassify more rows than the opcode was blocking" + ); + // Nothing evaporates in the transition: the same ore is harvested both times, so every row + // that stopped being an AtomicCAS residual is either classified now or carries another named + // reason — it cannot have been dropped. + assert_eq!( + report_before.harvested, report_after.harvested, + "the same ore is harvested regardless of convention; only its fate changes" + ); + assert_eq!( + delta_classified, delta_residual, + "every newly classified fact must vacate residual in lockstep" + ); + + assert_eq!(report_before.dropped, 0); + assert_eq!(report_after.dropped, 0); + assert!(report_before.is_conserved()); + assert!(report_after.is_conserved()); +} + +// --------------------------------------------------------------------------- +// 12. a_bootstrapped_convention_resolves_register_operands_that_pass_one_cannot +// --------------------------------------------------------------------------- + +#[test] +fn a_bootstrapped_convention_resolves_register_operands_that_pass_one_cannot() { + let blocks = fixture_function(); + let behavior = FunctionBehavior::from_blocks_raw(&blocks, None).unwrap(); + + // Pass one: no rows at all, so register operands resolve nowhere. + let base_conv = R2ilConvention::minimal_pass_one(); + let (_flat_a, ledger_a, report_a) = furnace::smelt(&behavior, &blocks, &base_conv); + + let unresolved_before = ledger_a + .rows() + .iter() + .filter(|r| matches!(r.reason, ResidualReason::NoConventionRowAtAddress)) + .count(); + assert!( + unresolved_before > 0, + "pass one must leave register operands unresolved" + ); + + // Bootstrap from an ArchSpec naming reg 0x00/8 and reg 0x08/8. + let mut spec = ArchSpec::new("bootstrap-test"); + spec.registers.push(RegisterDef::new("named0", 0x00, 8)); + spec.registers.push(RegisterDef::new("named8", 0x08, 8)); + + let bootstrapped = must_ok( + R2ilConvention::from_arch( + &spec, + [ + OpTag::Copy, + OpTag::IntAdd, + OpTag::Load, + OpTag::Store, + OpTag::CBranch, + OpTag::Call, + OpTag::Return, + ], + ), + "two named registers must fit within the interning budget", + ); + + let (_flat_b, ledger_b, report_b) = furnace::smelt(&behavior, &blocks, &bootstrapped); + + let unresolved_after = ledger_b + .rows() + .iter() + .filter(|r| matches!(r.reason, ResidualReason::NoConventionRowAtAddress)) + .count(); + + assert!( + unresolved_after < unresolved_before, + "named register operands must move from residual to classified" + ); + assert!( + report_b.classified > report_a.classified, + "classified must rise once register operands resolve" + ); + + // An UNNAMED offset (0x58, only touched by the un-classified IntSub in B0) + // still resolves through the coarse Space fallback row, at depth() == 1 -- + // ties §5's bootstrap rule directly to the melt. + let unnamed_facet = must_ok( + facet::project(®(0x58, 8), bootstrapped.spaces()), + "the register space is always within the interning budget", + ); + bootstrapped + .resolve(&unnamed_facet) + .expect("the coarse register-space row must catch an unnamed offset"); + let prefix = bootstrapped + .resolved_prefix(&unnamed_facet) + .expect("resolved_prefix must report the matched prefix"); + assert_eq!( + prefix.depth(), + 1, + "an unnamed register offset resolves at the coarse Space depth" + ); +} diff --git a/crates/ruff_ruby_spo/src/actions.rs b/crates/ruff_ruby_spo/src/actions.rs index cb1f0b261e6d5..422d2241711fc 100644 --- a/crates/ruff_ruby_spo/src/actions.rs +++ b/crates/ruff_ruby_spo/src/actions.rs @@ -52,7 +52,7 @@ use std::path::{Path, PathBuf}; use ruff_spo_triplet::{Predicate, Provenance, Triple}; /// The mutating HTTP verb an action affordance carries. GET is deliberately -/// absent — a GET affordance is navigation ([`crate::navigation`]), not an +/// absent — a GET affordance is navigation (`crate::navigation`), not an /// action. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)] pub enum ActionVerb { diff --git a/crates/ruff_ruby_spo/src/lib.rs b/crates/ruff_ruby_spo/src/lib.rs index d59ece623b1c5..3daf819fb3ed1 100644 --- a/crates/ruff_ruby_spo/src/lib.rs +++ b/crates/ruff_ruby_spo/src/lib.rs @@ -7,17 +7,17 @@ //! //! # Architecture //! -//! - [`mod@parse`] walks the directory, parses each `*.rb` with +//! - `parse` walks the directory, parses each `*.rb` with //! `lib-ruby-parser`, and finds class definitions (recursing into //! `module ... end` namespaces). One AST pass per file. -//! - [`mod@walk`] takes a class body and dispatches each top-level +//! - `walk` takes a class body and dispatches each top-level //! `Send` call (`belongs_to :project`, `validates :x`, `acts_as_list`, //! …) to the right [`Declaration`] variant by method-name match. //! - [`extract`] unpacks each class's `declarations: Vec` //! into the typed `Model::{associations, validations, callbacks, …}` //! sibling slots the shared IR consumes. //! -//! Method-body extraction ([`extract_fields`] / [`extract_functions`]) +//! Method-body extraction (`extract_fields` / `extract_functions`) //! is intentionally minimal in D-AR-3: the 100 % coverage gate (D-AR-4) //! measures *declarations*, not field/function depth. The two body //! extractors return empty vecs; the follow-up D-AR-3.5 implements them @@ -99,7 +99,7 @@ pub struct RubyClass { /// sibling fields the shared IR consumes. pub declarations: Vec, /// Method-body extraction (D-AR-3.5): one [`Function`] per `def` - /// in the class body. Populated by [`parse::parse_models`] alongside + /// in the class body. Populated by `parse::parse_models` alongside /// `declarations`, then flowed straight onto `Model::functions` by /// [`extract`] below. pub functions: Vec, diff --git a/crates/ruff_ruby_spo/src/menu_regions.rs b/crates/ruff_ruby_spo/src/menu_regions.rs index 82be23dd72cd2..6f034b448e6e0 100644 --- a/crates/ruff_ruby_spo/src/menu_regions.rs +++ b/crates/ruff_ruby_spo/src/menu_regions.rs @@ -110,7 +110,7 @@ pub struct RegionEntry { /// The raw declared position directive. pub position: Position, /// The resolved 0-based sibling ordinal (§3), assigned by the single- - /// pass Rails `TreeNode` replay in [`resolve_group`]. Always `Some` + /// pass Rails `TreeNode` replay in `resolve_group`. Always `Some` /// under that model (declaration-order resolution always terminates); /// the `Option` is retained so any future regression that fails to /// assign an ordinal surfaces as `None` + a non-zero @@ -126,7 +126,7 @@ pub struct RegionEntry { /// The `controller:` kwarg's VALUE (e.g. `"/work_packages"`, /// `"/admin/settings"`), when STATICALLY resolvable (a `Sym`/`Str` /// literal) — the identity-binding arm's raw signal (see - /// [`derive_model_from_controller`]). `None` for an absent `controller:` + /// `derive_model_from_controller`). `None` for an absent `controller:` /// (mirrors `has_controller == false`) OR a dynamic value (e.g. /// `options[:controller]` in the each-loop expansion — `has_controller` /// stays `true` there, but there is nothing static to derive from). @@ -896,7 +896,7 @@ fn positional_args(args: &[Node]) -> Vec<&Node> { // (`Setting.*` / `admin?` / `logged?`). Both operands of a disjunction are // GUARDED-BY the item's visibility, so BOTH are emitted — the honest // weaker claim the OQ-GUARD-1 probe established (a flat "requires" would -// mis-encode the one real disjunction in the corpus). Dynamic permission +// misencode the one real disjunction in the corpus). Dynamic permission // arguments (a `Hash`/method-call, not a `Sym` literal) yield no symbol — // nothing is fabricated. // ───────────────────────────────────────────────────────────────────────── @@ -1078,7 +1078,7 @@ impl RegionEntry { /// within-screen layout plane). `part_of` is Authoritative (Rails declares /// the parent via `parent:`); `purpose` classifies the target `action:` /// (a bare `controller:` defaults to Rails' `index`) through the shared - /// [`classify_purpose`] engine + the [`RAILS_PURPOSE`] config. + /// [`classify_purpose`] engine + the `RAILS_PURPOSE` config. #[must_use] pub fn to_quad(&self, namespace: &str) -> MenuQuad { let token = match &self.action { @@ -1208,7 +1208,7 @@ fn bind_identities(entries: &[RegionEntry], roster: &HashSet) -> Vec beta absent -> plain add -> [alpha] - /// beta after:alpha -> alpha at 0 -> add_at 1 -> [alpha, beta] + /// beta after:alpha -> alpha at 0 -> `add_at` 1 -> [alpha, beta] #[test] fn mutual_after_reference_resolves_deterministically_single_pass() { let root = scratch_dir("mutual_after"); @@ -1827,8 +1827,8 @@ mod tests { /// `last_count`), so the trailing `last:` band is still {a}; the plain /// `c` inserts just before it. A phase-separated model that applied Last /// after Before/After would push `b` to the very end — the confirmed bug. - /// a last -> push, last_count=1 -> [a] - /// b after:a -> a at 0 -> add_at 1 -> [a, b] (last_count still 1) + /// a last -> push, `last_count=1` -> [a] + /// b after:a -> a at 0 -> `add_at` 1 -> [a, b] (`last_count` still 1) /// c plain -> insert at 2-1=1 -> [a, c, b] #[test] fn plain_push_after_splice_onto_last_respects_live_boundary() { @@ -1989,7 +1989,7 @@ mod tests { /// probe below for the same finding on real data). #[test] fn menu_quad_round_trip_lowers_bare_name_chain_without_classid() { - let quads = vec![ + let quads = [ MenuQuad { node: "app:root_item".to_string(), parent: None, @@ -2145,6 +2145,10 @@ mod tests { // Independent radix-walk mirroring `nav_digest::menu_address`'s // fallback path (bare node name per ancestor — Rails MenuQuads never // bind `identity_concept`, so classid resolution never fires here). + #[expect( + clippy::items_after_statements, + reason = "helper is scoped to this one probe test and reads clearest right where it's used" + )] fn expected_address(node: &str, parent_of: &BTreeMap) -> String { let mut chain = Vec::new(); let mut seen = BTreeSet::new(); @@ -2286,7 +2290,7 @@ mod tests { /// A disjunction (`allowed_globally?(:add_project) || /// allowed_in_project?(:add_subprojects, project)`) → BOTH symbols /// emitted. This is the honest `guarded_by` semantics: both permissions - /// appear in the visibility guard, so a flat "requires" would mis-encode + /// appear in the visibility guard, so a flat "requires" would misencode /// it. (The `allowed_in_project?(:sym, project)` shape keeps `:sym` FIRST /// — the `project` receiver-context arg trails it, so this is normal-form, /// not receiver-style.) diff --git a/crates/ruff_ruby_spo/src/navigation.rs b/crates/ruff_ruby_spo/src/navigation.rs index 43be76d98e77c..9af70d285b01d 100644 --- a/crates/ruff_ruby_spo/src/navigation.rs +++ b/crates/ruff_ruby_spo/src/navigation.rs @@ -68,7 +68,7 @@ pub enum NavShape { ErbClick, /// Shape B — a controller *redirect* edge (`redirect_to`/`redirect_back`). ControllerRedirect, - /// Shape C — a [`crate::menu`] side-nav registration edge + /// Shape C — a `crate::menu` side-nav registration edge /// (`Redmine::MenuManager`-style `.push :label, …`), source fixed /// to the synthetic `"menu"` root. The Rails twin of the Odoo `` /// root (ruff #66) and op-nexgen's hand-authored `nav::MENU_NAV_EDGES`. diff --git a/crates/ruff_ruby_spo/src/representers.rs b/crates/ruff_ruby_spo/src/representers.rs index f58dfed19e921..7c6860b50d08b 100644 --- a/crates/ruff_ruby_spo/src/representers.rs +++ b/crates/ruff_ruby_spo/src/representers.rs @@ -121,7 +121,7 @@ const KEYWORDS: &[&str] = &[ /// Scan `root` recursively for `**/*_representer.rb` files and extract /// their declared fields. Files that produce zero declarations are -/// omitted (presence-only, same discipline as [`crate::views`]'s +/// omitted (presence-only, same discipline as `crate::views`'s /// `views_with_hits` gate). Deterministic: files sorted by path, decls /// in file (source) order. #[must_use] diff --git a/crates/ruff_ruby_spo/src/routes.rs b/crates/ruff_ruby_spo/src/routes.rs index 8c16baa2ca304..0860f8625ed26 100644 --- a/crates/ruff_ruby_spo/src/routes.rs +++ b/crates/ruff_ruby_spo/src/routes.rs @@ -57,7 +57,7 @@ use ruff_spo_triplet::{Predicate, Provenance, Triple}; use crate::actions::ActionVerb; /// The mutating HTTP verb an action affordance carries. GET is deliberately -/// absent — a GET affordance is navigation ([`crate::navigation`]), not an +/// absent — a GET affordance is navigation (`crate::navigation`), not an /// action. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum RouteVerb { @@ -149,7 +149,7 @@ impl RouteScopeKind { pub struct RouteEntry { /// The full Rails helper stem this route generates (the literal token /// Rails' route-helper generator produces — the same join contract - /// [`crate::actions::action_target`] uses, no `new_`/`edit_` + /// `crate::actions::action_target` uses, no `new_`/`edit_` /// stripping). `None` when no helper is derivable (a bare /// dynamic-segment string path with no `as:`). pub stem: Option, diff --git a/crates/ruff_ruby_spo/src/schema.rs b/crates/ruff_ruby_spo/src/schema.rs index fa0317e6b6467..c62dfff25ac35 100644 --- a/crates/ruff_ruby_spo/src/schema.rs +++ b/crates/ruff_ruby_spo/src/schema.rs @@ -202,12 +202,12 @@ const COLUMN_TYPES: &[&str] = &[ /// model are recorded in the report — the join-table population is real /// and expected (`changesets_work_packages` et al. have no AR class). /// -/// After the column merge, a **compute-linkage pass** ([`link_computed_fields`]) +/// After the column merge, a **compute-linkage pass** (`link_computed_fields`) /// runs over every model: a `def compute_` whose class ALSO has a /// (schema-merged) field named `` gets `field.emitted_by = Some("compute_")` /// — the Rails-side equivalent of Odoo's declared `compute=`. This only ever /// runs here (the schema-aware path), because the model-only stratum -/// ([`crate::extract_fields`]) never populates `fields` at all — the pass +/// (`crate::extract_fields`) never populates `fields` at all — the pass /// would be a no-op there. It never synthesizes a `Field` from a method name /// alone: linkage requires the field to already exist. /// @@ -215,11 +215,11 @@ const COLUMN_TYPES: &[&str] = &[ /// /// `/db/migrate/tables/` is checked first: if it contains any `.rb` /// file, this is the `OpenProject`-style squashed baseline and the -/// [`parse_tables_dir`] path runs, followed by -/// [`replay_post_baseline_migrations`] (module doc: "Post-baseline +/// `parse_tables_dir` path runs, followed by +/// `replay_post_baseline_migrations` (module doc: "Post-baseline /// replay"). Otherwise, if `/db/migrate/` itself contains any `.rb` /// file, this is a classic Rails app (Redmine and similar — no baseline -/// squash, only the full migration history) and [`parse_migrations_dir`] +/// squash, only the full migration history) and `parse_migrations_dir` /// runs instead — untouched by this pass. Neither directory existing /// leaves [`SchemaReport::tables_seen`] at zero, same as before the classic /// fallback was added. diff --git a/crates/ruff_server/tests/e2e/main.rs b/crates/ruff_server/tests/e2e/main.rs index 65db9e9d450f7..782ededd59b37 100644 --- a/crates/ruff_server/tests/e2e/main.rs +++ b/crates/ruff_server/tests/e2e/main.rs @@ -516,7 +516,7 @@ impl TestServer { { panic!( "Received multiple publish diagnostic notifications for {url}: ({existing:#?})", - url = ¬ification.uri + url = notification.uri ); } } diff --git a/crates/ruff_spo_address/README.md b/crates/ruff_spo_address/README.md index 38387f47027f3..6dd271fed0c1d 100644 --- a/crates/ruff_spo_address/README.md +++ b/crates/ruff_spo_address/README.md @@ -28,18 +28,18 @@ f.is_a_chain(); // == FacetCascade::lo_chain (taxonomy / typeHierarchy) ``` - **Prefix-routable both ways.** Members of the same class share a leading - `part_of_chain` prefix; subtypes of the same base share a leading - `is_a_chain` prefix — so an LSP `documentSymbol` / `typeHierarchy` query is a - longest-common-prefix over the cache-resident key column. + `part_of_chain` prefix; subtypes of the same base share a leading + `is_a_chain` prefix — so an LSP `documentSymbol` / `typeHierarchy` query is a + longest-common-prefix over the cache-resident key column. - **Exact below the per-tier cap, not a PQ approximation.** Ranks are a - deterministic assignment (sorted sibling order) — roundtrip-lossless and - injective *as long as every sibling set is ≤ 255 and depth ≤ 6*. Iron-rule - clean per `I-VSA-IDENTITIES` (encodes identity positions, never bundles - content). The cap is real and measured — see the fence below. + deterministic assignment (sorted sibling order) — roundtrip-lossless and + injective *as long as every sibling set is ≤ 255 and depth ≤ 6*. Iron-rule + clean per `I-VSA-IDENTITIES` (encodes identity positions, never bundles + content). The cap is real and measured — see the fence below. - **`facet_classid`.** `mint` leaves it `0`; `mint_with_classid` takes a - resolver so a caller holding the OGAR codebook can stamp the canonical - class-id (e.g. `lance_graph_contract::canonical_concept_id`) **BBB-safely** — - this crate stays pure `std` + `ruff_spo_triplet`, never linking the codebook. + resolver so a caller holding the OGAR codebook can stamp the canonical + class-id (e.g. `lance_graph_contract::canonical_concept_id`) **BBB-safely** — + this crate stays pure `std` + `ruff_spo_triplet`, never linking the codebook. ## Honest fence — MEASURED on a real corpus, not assumed @@ -50,7 +50,7 @@ not a lossless address (deeper levels are the registry/ref-escape's job). Earlier this doc claimed "for class graphs (depth ≈ 3–4) nothing truncates." **That was falsified against a real multi-thousand-node corpus** (a Roslyn harvest of a production C# codebase, run downstream via `ruff_csharp_spo`): -the naive [`mint`] produced real collisions and truncations once two +the naive \[`mint`\] produced real collisions and truncations once two structures crossed the 255-sibling cap — a **God-class** (a single class with hundreds of fields, the part_of axis) and a **flat is_a root** (a kind- discriminator type with thousands of direct children, the is_a axis). Coarse @@ -61,7 +61,7 @@ the *coarse* prefix, which never saturates; only fine-grained identity The fix is not a bigger int — it is that a member's **kind** (e.g. Property/Function) belongs in its `facet_classid`, not in a 6-tier sibling rank under a mega-root. That is the same move the classid-gate proposes. -[`mint_factored`] is the corrected minter: it builds is_a from +\[`mint_factored`\] is the corrected minter: it builds is_a from `inherits_from` only (the kind-discriminator mega-root never enters the sibling rank) and gives part_of a base-255 positional path that cascades deeper instead of saturating — both failure modes go to zero on the same diff --git a/crates/ruff_spo_address/src/lib.rs b/crates/ruff_spo_address/src/lib.rs index f15b497524274..80e13632cc4cc 100644 --- a/crates/ruff_spo_address/src/lib.rs +++ b/crates/ruff_spo_address/src/lib.rs @@ -1,16 +1,16 @@ //! `ruff_spo_address` — the deterministic `(part_of:is_a)` rank-minter. //! //! This is the one genuinely-new brick between the `ruff_*_spo` SPO harvest and -//! the lance-graph `(part_of:is_a)` GUID SoA (see lance-graph +//! the lance-graph `(part_of:is_a)` GUID `SoA` (see lance-graph //! `.claude/knowledge/ast-as-partof-isa-address.md` — "The missing brick"). The -//! carrier ([`lance_graph_contract::facet::FacetCascade`], shipped #613/#614) is +//! carrier (`lance_graph_contract::facet::FacetCascade`, shipped #613/#614) is //! already there; this crate fills the *mint*. //! //! Given a corpus's two structural relations: //! -//! - **part_of** (mereology / membership) — harvested as `has_field` / -//! `has_function` (`class → member`, so the member is *part_of* the class); -//! - **is_a** (taxonomy / typing) — harvested as `inherits_from` (`class → +//! - **`part_of`** (mereology / membership) — harvested as `has_field` / +//! `has_function` (`class → member`, so the member is *`part_of`* the class); +//! - **`is_a`** (taxonomy / typing) — harvested as `inherits_from` (`class → //! base`) and, for leaves, `rdf:type` (`member → kind`); //! //! it assigns every node a deterministic `(part_of_rank, is_a_rank)` at each of @@ -26,7 +26,7 @@ //! //! So [`Facet::part_of_chain`] == `FacetCascade::hi_chain` and //! [`Facet::is_a_chain`] == `FacetCascade::lo_chain`. Both chains are -//! **prefix-routable**: two nodes in the same part_of subtree share a leading +//! **prefix-routable**: two nodes in the same `part_of` subtree share a leading //! `part_of_chain` prefix (a `documentSymbol` / containment query is a longest- //! common-prefix), and two nodes under the same supertype share a leading //! `is_a_chain` prefix (a `typeHierarchy` walk). @@ -44,15 +44,15 @@ //! saturated rank at every tier their facets collide. Verified against a real //! multi-thousand-node corpus, the failure has two distinct causes: **God- //! classes** (a single class with hundreds of fields, e.g. a large UI form) -//! overflowing the part_of axis, and **flat is_a roots** (a kind-discriminator +//! overflowing the `part_of` axis, and **flat `is_a` roots** (a kind-discriminator //! type with thousands of direct children, e.g. every "Property" or "Function" -//! node parented straight under one root) overflowing the is_a axis. So +//! node parented straight under one root) overflowing the `is_a` axis. So //! "exact" holds for a class graph whose every sibling set is ≤ 255 and depth //! ≤ 6 — NOT for arbitrary real corpora. The flat-is_a-root case is the //! dominant one and is addressable: a member's *kind* (e.g. Property/Function) //! belongs in its `facet_classid`, not in a 6-tier sibling rank under a //! mega-root. [`mint_factored`] is the corrected minter that fixes both -//! failure modes (base-255 positional part_of paths + is_a built from +//! failure modes (base-255 positional `part_of` paths + `is_a` built from //! `inherits_from` only, kind moved to a bounded leaf enum). //! //! # `facet_classid` @@ -114,14 +114,14 @@ impl Facet { u32::from_le_bytes([self.bytes[0], self.bytes[1], self.bytes[2], self.bytes[3]]) } - /// The **part_of** chain (coarse→fine) — the `FacetCascade` `hi_chain`. + /// The **`part_of`** chain (coarse→fine) — the `FacetCascade` `hi_chain`. #[must_use] pub const fn part_of_chain(self) -> [u8; TIERS] { let b = &self.bytes; [b[5], b[7], b[9], b[11], b[13], b[15]] } - /// The **is_a** chain (coarse→fine) — the `FacetCascade` `lo_chain`. + /// The **`is_a`** chain (coarse→fine) — the `FacetCascade` `lo_chain`. #[must_use] pub const fn is_a_chain(self) -> [u8; TIERS] { let b = &self.bytes; @@ -130,7 +130,7 @@ impl Facet { } /// The result of minting a corpus: each node's [`Facet`], plus the nodes whose -/// part_of/is_a depth exceeded the 6 tiers (or whose sibling count exceeded the +/// `part_of/is_a` depth exceeded the 6 tiers (or whose sibling count exceeded the /// 255-per-tier byte) and were therefore truncated. #[derive(Clone, Debug, Default)] pub struct Mint { @@ -162,7 +162,7 @@ impl Mint { self.facets.iter().map(|(k, &v)| (k.as_str(), v)) } - /// Nodes whose address was truncated (part_of/is_a depth > 6 tiers, or a + /// Nodes whose address was truncated (`part_of/is_a` depth > 6 tiers, or a /// sibling set larger than 255). Empty for a corpus that fits — the honest /// fence on "exact": beyond the cap the facet is a routing prefix, not a /// lossless address (deeper levels are the registry/ref-escape's job). @@ -183,12 +183,12 @@ impl Mint { } /// Which chain a [`RadixCodebook`] is ordered by — the two prefix-routable axes -/// of a [`Facet`] (`5+2t` bytes = part_of / hi; `4+2t` bytes = is_a / lo). +/// of a [`Facet`] (`5+2t` bytes = `part_of` / hi; `4+2t` bytes = `is_a` / lo). #[derive(Clone, Copy, PartialEq, Eq, Debug)] pub enum Axis { - /// part_of (mereology / containment) — the `documentSymbol` axis (`hi_chain`). + /// `part_of` (mereology / containment) — the `documentSymbol` axis (`hi_chain`). PartOf, - /// is_a (taxonomy / inheritance) — the `typeHierarchy` axis (`lo_chain`). + /// `is_a` (taxonomy / inheritance) — the `typeHierarchy` axis (`lo_chain`). IsA, } @@ -202,7 +202,7 @@ pub enum Axis { /// /// # When to build it (PROS) — modest-cardinality, readable, prefix-query work /// -/// The Odoo / Redmine / MedCare *app-concept* scale (hundreds–thousands of nodes): +/// The Odoo / Redmine / `MedCare` *app-concept* scale (hundreds–thousands of nodes): /// /// - **O(log n + k) prefix queries.** `documentSymbol(class)` / /// `typeHierarchy(base)` become a range scan; no graph traversal. @@ -210,7 +210,7 @@ pub enum Axis { /// radix order; built from facets you already hold, dropped when the workload /// ends. Nothing is stored in the layout — the gate is "call the method", not /// "feature-flag the address". -/// - **Readable.** Each key is the legible part_of/is_a address, not a hash — you +/// - **Readable.** Each key is the legible `part_of/is_a` address, not a hash — you /// route, group, and reason on it without decoding a value. /// /// # When NOT to build it (CONS) @@ -222,7 +222,7 @@ pub enum Axis { /// *positionally* (the chain IS the coordinate — compute, don't look up). The /// codebook is for the regime where a table is affordable AND the structure is /// worth keeping readable; past that it is the wrong tool. -/// - **One axis per codebook.** part_of and is_a sort differently; build both +/// - **One axis per codebook.** `part_of` and `is_a` sort differently; build both /// only if you need both query directions. pub struct RadixCodebook { axis: Axis, @@ -250,8 +250,8 @@ impl RadixCodebook { } /// Every `(chain, node)` whose chain shares `prefix`'s first `depth` tiers — - /// the contiguous range under that ancestor. `documentSymbol` (PartOf) / - /// `typeHierarchy` (IsA) in `O(log n + k)`, no graph walk. `depth = 0` + /// the contiguous range under that ancestor. `documentSymbol` (`PartOf`) / + /// `typeHierarchy` (`IsA`) in `O(log n + k)`, no graph walk. `depth = 0` /// returns the whole codebook. #[must_use] pub fn under_prefix(&self, prefix: [u8; TIERS], depth: usize) -> &[([u8; TIERS], String)] { @@ -374,7 +374,7 @@ pub fn mint_with_classid(triples: &[Triple], classid_of: impl Fn(&str) -> u32) - /// Structural mints are unaffected (both maps are always non-empty there). /// /// Determinism + the 6-tier / 255-sibling caps are inherited unchanged from -/// [`ranks`] (a node exceeding either is flagged in [`Mint`]'s `truncated`). +/// `ranks` (a node exceeding either is flagged in [`Mint`]'s `truncated`). #[must_use] pub fn mint_from_parents<'a>( nodes: &BTreeSet<&'a str>, @@ -429,11 +429,11 @@ pub fn mint_from_parents<'a>( /// names: /// /// - **duplication** (a kind-discriminator mega-root, e.g. every `Property` / -/// `Function` node parented straight under one root): is_a is built from +/// `Function` node parented straight under one root): `is_a` is built from /// `inherits_from` **only** (the real inheritance fan-out stays far below /// 255, so it never explodes); a member's *kind* becomes a bounded leaf enum /// (`field` = 1, `fn` = 2), not a ranked child of a many-thousand-wide root. -/// - **conflation** (a class with > 255 members): the part_of address is a +/// - **conflation** (a class with > 255 members): the `part_of` address is a /// **base-255 positional path** — each generation consumes /// `ceil(log255(sibling_count))` tiers, so no tier exceeds 255 and the address /// stays injective and prefix-routable (a child's chain extends its parent's). @@ -548,6 +548,10 @@ const fn b255_width(n: usize) -> usize { /// Write the base-255 digits of 0-based sibling index `idx` into /// `out[at..at+w]`, big-endian, each digit `1..=255`. +#[expect( + clippy::cast_possible_truncation, + reason = "x % 255 is always in 0..255, so the cast to u8 never truncates" +)] fn b255_write(idx: usize, w: usize, out: &mut [u8; TIERS], at: usize) { let mut x = idx; for k in (0..w).rev() { @@ -612,6 +616,10 @@ fn forest<'a>( /// The coarse→fine rank chain for `node` in one forest. Rank at tier `t` is the /// 1-based index of the ancestor at depth `t` among its siblings (1..=255; 0 /// means "tier below this node's depth"). Returns `(chain, truncated)`. +#[expect( + clippy::cast_possible_truncation, + reason = "clamped via `.min(u8::MAX as usize)` immediately before the cast, so it never truncates" +)] fn ranks( node: &str, parent: &BTreeMap<&str, &str>, @@ -656,7 +664,7 @@ mod tests { use super::*; use ruff_spo_triplet::from_ndjson; - /// The exact shape `ruff_csharp_spo`'s harvester emits for one MedCare model. + /// The exact shape `ruff_csharp_spo`'s harvester emits for one `MedCare` model. fn medcare_patient() -> Vec { let ndjson = concat!( r#"{"s":"medcare:Patient","p":"rdf:type","o":"ogit:ObjectType","f":1.0,"c":0.9}"#, diff --git a/crates/ruff_spo_address/src/soc.rs b/crates/ruff_spo_address/src/soc.rs index 61cb0ba1230ed..dd800e33e2ba5 100644 --- a/crates/ruff_spo_address/src/soc.rs +++ b/crates/ruff_spo_address/src/soc.rs @@ -35,7 +35,7 @@ pub const MAX_SIBLINGS_PER_TIER: usize = u8::MAX as usize; /// canonical GUID → `3×4` (all `G·D = 12`, 8-bit tiers; the per-group depth /// `D ∈ {2,3,4}` is a *per-class* constant, picked from the class condition). /// This module only bounds the god-object cardinality: `< 256` is maskable -/// (clean), `≥ 256` is the SoC split signal. (operator 2026-06-29: the shape is +/// (clean), `≥ 256` is the `SoC` split signal. (operator 2026-06-29: the shape is /// inherited — don't lock a `[u64; 4]` "quadruplet"; `D` is class-conditioned.) pub const FIELD_MASK_CAP: usize = MAX_SIBLINGS_PER_TIER; diff --git a/crates/ruff_spo_triplet/SPO_TRIPLET_EXTRACTION.md b/crates/ruff_spo_triplet/SPO_TRIPLET_EXTRACTION.md index 2649f16f54031..526728c460a25 100644 --- a/crates/ruff_spo_triplet/SPO_TRIPLET_EXTRACTION.md +++ b/crates/ruff_spo_triplet/SPO_TRIPLET_EXTRACTION.md @@ -7,7 +7,7 @@ > write ndjson. The triple vocabulary, truth calibration, and IRI shape > are fixed in `ruff_spo_triplet`; you only write the AST→IR step. ---- +______________________________________________________________________ ## 1. What this is and why it exists @@ -25,7 +25,7 @@ single downstream consumer (`lance_graph`'s SPO store, the Foundry-shape `action_emitter`, the `link_chain` splitter) works on either without modification. -``` +```text Python AST ─┐ ├─► ModelGraph (IR) ─► expand() ─► Vec ─► ndjson ─► SPO store Ruby AST ──┘ ▲ ▲ ▲ @@ -36,7 +36,7 @@ modification. The reuse seam is the `ModelGraph` IR. Everything below the IR is shared; everything above it is the per-language frontend. ---- +______________________________________________________________________ ## 2. The triple schema (closed vocabulary) @@ -47,19 +47,19 @@ extensions), and is the source of truth for the complete vocabulary. `ns` is the namespace prefix you choose for the source app (`odoo`, `openproject`, …). -| predicate | subject | object | provenance | meaning | -| --- | --- | --- | --- | --- | -| `rdf:type` | `ns:model` | `ogit:ObjectType` | Structural | this name is an entity | -| `rdf:type` | `ns:model.field` | `ogit:Property` | Structural | this name is a field | -| `rdf:type` | `ns:model.fn` | `ogit:Function` | Structural | this name is a method | -| `has_function` | `ns:model` | `ns:model.fn` | Structural | entity owns method | -| `emitted_by` | `ns:model.field` | `ns:model.fn` | Authoritative | method writes field | -| `depends_on` | `ns:model.field` | `ns:model.` | Authoritative | field's declared compute deps | -| `reads_field` | `ns:model.fn` | `ns:model.field` | Inferred | method body reads field | -| `raises` | `ns:model.fn` | `exc:` | Authoritative | method raises error | -| `traverses_relation` | `ns:model.fn` | `ns:model.` | Inferred | method walks relation | -| `inherits_from` | `ns:model` | `ns:parent` | CppExtracted (C++ base) / OpenProjectExtracted (Rails STI) | model inherits from a base/parent | -| `column_not_null` | `ns:model.field` | `"true"` | Authoritative | schema-declared `NOT NULL` constraint on the column (D-AR-3.5 schema stratum) | +| predicate | subject | object | provenance | meaning | +| -------------------- | ---------------- | ----------------- | ---------------------------------------------------------- | ----------------------------------------------------------------------------- | +| `rdf:type` | `ns:model` | `ogit:ObjectType` | Structural | this name is an entity | +| `rdf:type` | `ns:model.field` | `ogit:Property` | Structural | this name is a field | +| `rdf:type` | `ns:model.fn` | `ogit:Function` | Structural | this name is a method | +| `has_function` | `ns:model` | `ns:model.fn` | Structural | entity owns method | +| `emitted_by` | `ns:model.field` | `ns:model.fn` | Authoritative | method writes field | +| `depends_on` | `ns:model.field` | `ns:model.` | Authoritative | field's declared compute deps | +| `reads_field` | `ns:model.fn` | `ns:model.field` | Inferred | method body reads field | +| `raises` | `ns:model.fn` | `exc:` | Authoritative | method raises error | +| `traverses_relation` | `ns:model.fn` | `ns:model.` | Inferred | method walks relation | +| `inherits_from` | `ns:model` | `ns:parent` | CppExtracted (C++ base) / OpenProjectExtracted (Rails STI) | model inherits from a base/parent | +| `column_not_null` | `ns:model.field` | `"true"` | Authoritative | schema-declared `NOT NULL` constraint on the column (D-AR-3.5 schema stratum) | **IRI shape.** Subjects and objects are `":."`. The single dot separates model from member; dotted *dependency paths* @@ -73,11 +73,11 @@ not a project-local namespace. Don't invent `https://…/ObjectType`. ### Provenance → truth (the NARS calibration) -| tier | `(f, c)` | when | -| --- | --- | --- | -| `Structural` | `(1.0, 1.0)` | true by construction (a name *is* a model/field/method; ownership) | -| `Authoritative` | `(0.95, 0.90)`| declared or directly observed in body (`@api.depends`, a `raise`, the field a compute assigns) | -| `Inferred` | `(0.85, 0.75)`| heuristic from body shape (an attribute read, a loop-target relation) | +| tier | `(f, c)` | when | +| --------------- | -------------- | ---------------------------------------------------------------------------------------------- | +| `Structural` | `(1.0, 1.0)` | true by construction (a name *is* a model/field/method; ownership) | +| `Authoritative` | `(0.95, 0.90)` | declared or directly observed in body (`@api.depends`, a `raise`, the field a compute assigns) | +| `Inferred` | `(0.85, 0.75)` | heuristic from body shape (an attribute read, a loop-target relation) | The downstream store gates queries by NARS *expectation*, so a strict query can drop `Inferred` edges and keep only declared facts. The tier is @@ -86,7 +86,7 @@ gives the calibrated default; override per-edge only when your frontend can *prove* a stronger tier (e.g. a Rails frontend that statically resolves a read can promote `reads_field` to `Authoritative`). ---- +______________________________________________________________________ ## 3. The IR you fill (`ModelGraph`) @@ -116,7 +116,7 @@ underscores (`account_move`) so the IRI dot is unambiguously the model↔member separator. Rails class names (`WorkPackage`) have no dots — use them as-is. ---- +______________________________________________________________________ ## 4. The query this enables ("a + b → c through d?") @@ -135,7 +135,7 @@ lookup. This is what makes the extracted ontology a *compute graph* `ActionSpec { effects, inputs, raises, reads, traverses }` records straight off these edges. ---- +______________________________________________________________________ ## 5. Writing a new frontend — the Ruby/Rails (OpenProject) guide @@ -146,11 +146,11 @@ Five steps. Only step 2 is real work. Options, cheapest first: - **`lib-ruby-parser`** (Rust crate, pure Rust, no Ruby runtime) — best - fit for a Rust frontend; gives you a typed AST. *Recommended.* + fit for a Rust frontend; gives you a typed AST. *Recommended.* - **tree-sitter-ruby** (via the `tree-sitter` crate) — robust, lossy on - some semantics but great for structural sweeps. + some semantics but great for structural sweeps. - Shell out to Ruby's own `ripper`/`parser` gem and read s-expressions — - only if you already have a Ruby toolchain in the loop. + only if you already have a Ruby toolchain in the loop. A scaffold crate (`ruff_ruby_spo`, see §6) is provided wired for `lib-ruby-parser` with `todo!()` markers at each extraction point. @@ -160,34 +160,34 @@ A scaffold crate (`ruff_ruby_spo`, see §6) is provided wired for This is the whole job. The mapping (mirror of the Odoo column in the cheat-sheet in `src/ir.rs`): -| IR target | Rails / ActiveRecord source | -| --- | --- | -| `Model::name` | `class WorkPackage < ApplicationRecord` → `WorkPackage` | -| `Field::name` | DB columns (from `db/schema.rb`), `attribute :x`, `attr_accessor`, `store_accessor` | -| `Field::depends_on` | association chains a derived attribute reads (`time_entries.hours`); if you parse `schema.rb` you can also seed column→column deps | -| `Field::emitted_by` | a memoized/derived method that assigns the attribute (`def total_hours; @total_hours ||= …; end`) | -| `Function::name` | instance methods (`def compute_total_hours`) | -| `Function::reads` | `self.x` reads and bare attribute reads in the method body | -| `Function::raises` | `raise X`, `errors.add(...)`, and `validates`/`validate` callbacks (treat the validation as a guard that raises `ActiveRecord::RecordInvalid`) | -| `Function::traverses` | association walks in the body (`children.each`, `time_entries.map`, `project.members`) — the association name is the relation | +| IR target | Rails / ActiveRecord source | +| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `Model::name` | `class WorkPackage < ApplicationRecord` → `WorkPackage` | +| `Field::name` | DB columns (from `db/schema.rb`), `attribute :x`, `attr_accessor`, `store_accessor` | +| `Field::depends_on` | association chains a derived attribute reads (`time_entries.hours`); if you parse `schema.rb` you can also seed column→column deps | +| `Field::emitted_by` | a memoized/derived method that assigns the attribute (\`def total_hours; @total_hours | +| `Function::name` | instance methods (`def compute_total_hours`) | +| `Function::reads` | `self.x` reads and bare attribute reads in the method body | +| `Function::raises` | `raise X`, `errors.add(...)`, and `validates`/`validate` callbacks (treat the validation as a guard that raises `ActiveRecord::RecordInvalid`) | +| `Function::traverses` | association walks in the body (`children.each`, `time_entries.map`, `project.members`) — the association name is the relation | Notes specific to Rails: - **Associations are your relations.** `belongs_to :project`, - `has_many :time_entries` declare the traversable relations. A method - body that calls `time_entries` is traversing `time_entries`. Seed the - set of valid relation names from the association declarations so you can - distinguish a relation walk from an ordinary method call. + `has_many :time_entries` declare the traversable relations. A method + body that calls `time_entries` is traversing `time_entries`. Seed the + set of valid relation names from the association declarations so you can + distinguish a relation walk from an ordinary method call. - **Validations are guards.** `validates :subject, presence: true` and - `validate :custom_check` are the Rails analogue of Odoo's - `@api.constrains` + `raise`. Emit them as `raises exc:ActiveRecord::RecordInvalid` - (Authoritative) on the validating method, or on a synthetic - `_validate` function for declarative `validates`. + `validate :custom_check` are the Rails analogue of Odoo's + `@api.constrains` + `raise`. Emit them as `raises exc:ActiveRecord::RecordInvalid` + (Authoritative) on the validating method, or on a synthetic + `_validate` function for declarative `validates`. - **`exc:` namespace is shared.** Ruby exception class names keep their - `::` (`exc:ActiveRecord::RecordInvalid`) — the `exc:` prefix is the same - one Odoo uses (`exc:UserError`). Don't translate; just prefix. + `::` (`exc:ActiveRecord::RecordInvalid`) — the `exc:` prefix is the same + one Odoo uses (`exc:UserError`). Don't translate; just prefix. - **Callbacks (`before_save`, `after_create`) → functions** whose - `traverses`/`reads`/`raises` you extract from the referenced method. + `traverses`/`reads`/`raises` you extract from the referenced method. ### Step 3 — build the `ModelGraph` @@ -216,7 +216,7 @@ The ndjson loads directly into `lance_graph::graph::spo::odoo_ontology::load_ont `action_emitter::emit_actions` and `link_chain::split_all_depends_on` then work on the OpenProject graph exactly as they do on Odoo's. ---- +______________________________________________________________________ ## 6. The scaffold crate (`ruff_ruby_spo`) @@ -225,16 +225,16 @@ then work on the OpenProject graph exactly as they do on Odoo's. - depends on `ruff_spo_triplet`, - exposes `extract(source_tree: &Path) -> ModelGraph`, - has `todo!()` bodies at each of the step-2 extraction points with a - doc-comment naming the exact Rails construct to read, + doc-comment naming the exact Rails construct to read, - ships a unit test that builds a hand-written `ModelGraph` and asserts - the `expand()` output — so the *target shape* is locked even before the - parser is wired. + the `expand()` output — so the *target shape* is locked even before the + parser is wired. Start there: replace the `todo!()`s one predicate at a time, running the locked-shape test after each. When all are filled, point it at the OpenProject `app/models/` tree. ---- +______________________________________________________________________ ## 7. Verifying parity with the Odoo extraction @@ -245,14 +245,14 @@ crate's own tests pin this: - `triple::tests::provenance_truth_tiers_match_odoo_calibration` - `expand::tests::truth_tiers_are_assigned_per_predicate` - `integration_tests::two_model_graph_round_trips_through_ndjson` - (uses a Rails-shaped `ModelGraph`) + (uses a Rails-shaped `ModelGraph`) When you wire the Ruby frontend, add a fixture test that runs a small real OpenProject model through `extract()` + `expand()` and asserts the expected `ActionSpec` shape downstream. That closes the loop: same IR contract → same triples → same Foundry-shape actions. ---- +______________________________________________________________________ ## 8. Pointers @@ -261,6 +261,6 @@ contract → same triples → same Foundry-shape actions. - `src/expand.rs` — the deterministic IR→triples projection. - `src/ndjson.rs` — the on-disk format (matches the `lance_graph` loader). - Downstream consumers (in the `lance-graph` repo): - `crates/lance-graph/src/graph/spo/odoo_ontology.rs` (loader), - `…/action_emitter.rs` (Foundry `ActionSpec` composer), - `…/link_chain.rs` (dotted-path splitter). + `crates/lance-graph/src/graph/spo/odoo_ontology.rs` (loader), + `…/action_emitter.rs` (Foundry `ActionSpec` composer), + `…/link_chain.rs` (dotted-path splitter). diff --git a/crates/ruff_spo_triplet/examples/nav_digest.rs b/crates/ruff_spo_triplet/examples/nav_digest.rs index 3876b7c96c518..426e4c238331e 100644 --- a/crates/ruff_spo_triplet/examples/nav_digest.rs +++ b/crates/ruff_spo_triplet/examples/nav_digest.rs @@ -1,4 +1,4 @@ -//! The Klickwege structure-parity oracle (transcode doctrine: "MySQL = value +//! The Klickwege structure-parity oracle (transcode doctrine: "`MySQL` = value //! parity, Klickwege = structure parity"). Prints a deterministic digest of //! the UI-navigation plane harvested from a corpus — screens, navigation //! edges (`navigates_to`), tab/view selections (`selects_view`), concept @@ -33,7 +33,8 @@ #![expect( clippy::print_stdout, - reason = "the whole point of this example is to print the nav digest" + clippy::print_stderr, + reason = "the whole point of this example is to print the nav digest (and a usage error to stderr on bad args)" )] use ruff_spo_triplet::{build_nav_digest, from_ndjson, parse}; diff --git a/crates/ruff_spo_triplet/examples/rekey_exam.rs b/crates/ruff_spo_triplet/examples/rekey_exam.rs index 7e474e669a9f0..e4f6461411bef 100644 --- a/crates/ruff_spo_triplet/examples/rekey_exam.rs +++ b/crates/ruff_spo_triplet/examples/rekey_exam.rs @@ -50,7 +50,8 @@ #![expect( clippy::print_stdout, - reason = "the whole point of this example is to print the exam report" + clippy::print_stderr, + reason = "the whole point of this example is to print the exam report (and a usage error to stderr on bad args)" )] use std::collections::BTreeMap; diff --git a/crates/ruff_spo_triplet/src/concept_split.rs b/crates/ruff_spo_triplet/src/concept_split.rs index 24fcde12774bf..23fba181fc5e9 100644 --- a/crates/ruff_spo_triplet/src/concept_split.rs +++ b/crates/ruff_spo_triplet/src/concept_split.rs @@ -280,7 +280,7 @@ mod tests { use crate::ir::Function; /// Boundary-4 on the ALIAS branch: a convention row whose VALUE is - /// PascalCase cannot smuggle a case-divergent concept past the seam — + /// `PascalCase` cannot smuggle a case-divergent concept past the seam — /// `resolve_concept` folds the alias value through `concept_key`, so /// `SplitName::concept` is a fixed point on BOTH branches. #[test] diff --git a/crates/ruff_spo_triplet/src/corpus.rs b/crates/ruff_spo_triplet/src/corpus.rs index 93bd5647024fb..e137bc059121d 100644 --- a/crates/ruff_spo_triplet/src/corpus.rs +++ b/crates/ruff_spo_triplet/src/corpus.rs @@ -105,7 +105,7 @@ fn push_unique(v: &mut Vec, item: &str) { /// Reassemble a flat SPO triple stream into a [`ModelGraph`] on the /// **core-7 plane** (`Model::fields` / `Model::functions`) — the shape a /// "class + typed fields + methods" frontend (C#/Roslyn, Java, …) emits, -/// and the shape [`crate::mint`]-adjacent consumers +/// and the shape `crate::mint`-adjacent consumers /// (`ogar-from-ruff::compile_graph_csharp`) walk. /// /// Distinct from [`crate::reassemble::reassemble`], which recovers the diff --git a/crates/ruff_spo_triplet/src/exam_config.rs b/crates/ruff_spo_triplet/src/exam_config.rs index 4cf087c364e05..fef886f698d90 100644 --- a/crates/ruff_spo_triplet/src/exam_config.rs +++ b/crates/ruff_spo_triplet/src/exam_config.rs @@ -32,7 +32,7 @@ pub struct ExamConfig { pub codebook: Vec<(String, u16)>, /// Concepts that MUST bind for the exam to pass. pub expect: Vec, - /// `(dock token, region name)` rows binding a WinForms `Dock` value to + /// `(dock token, region name)` rows binding a `WinForms` `Dock` value to /// a canonical region name (the six-region layout frame: `top_bar` / /// `left_nav` / `right_panel` / `bottom_bar` / `center` / `popup`, /// canonically — but region names are free strings supplied by config; diff --git a/crates/ruff_spo_triplet/src/expand.rs b/crates/ruff_spo_triplet/src/expand.rs index dd2ece234e6aa..5368bdda75dea 100644 --- a/crates/ruff_spo_triplet/src/expand.rs +++ b/crates/ruff_spo_triplet/src/expand.rs @@ -104,7 +104,7 @@ use crate::triple::{EntityKind, Predicate, Provenance, Triple}; /// `(s, p, o)` is produced twice with different provenance, the /// first-in-sort-order (which, after sort, is deterministic but provenance- /// arbitrary) wins. Frontends should not emit conflicting provenance for -/// one identity; [`crate::ndjson`] round-trips assume a clean IR. +/// one identity; `crate::ndjson` round-trips assume a clean IR. #[must_use] pub fn expand(graph: &ModelGraph) -> Vec { let mut exp = Expander::new(); diff --git a/crates/ruff_spo_triplet/src/ir.rs b/crates/ruff_spo_triplet/src/ir.rs index e61d2fce035c4..9cff29b5b5ee4 100644 --- a/crates/ruff_spo_triplet/src/ir.rs +++ b/crates/ruff_spo_triplet/src/ir.rs @@ -99,7 +99,7 @@ pub struct Model { /// language's plain "extends ``"). Names are already /// frontend-normalised (dot→underscore); the expander emits /// `(ns:model, inherits_from, ns:parent)` per entry with - /// [`Provenance::Authoritative`]. Distinct from `bases` (C++ base + /// `Provenance::Authoritative`. Distinct from `bases` (C++ base /// classes, `CppExtracted`) and `sti` (single-parent Rails STI): a /// multi-parent list carrying no per-parent metadata. Self-references /// (an Odoo reopen where the sole `_inherit` == the model name) are diff --git a/crates/ruff_spo_triplet/src/nav_digest.rs b/crates/ruff_spo_triplet/src/nav_digest.rs index c4f07176ed971..3b969bc88d80a 100644 --- a/crates/ruff_spo_triplet/src/nav_digest.rs +++ b/crates/ruff_spo_triplet/src/nav_digest.rs @@ -1,4 +1,4 @@ -//! The Klickwege structure-parity oracle (transcode doctrine: "MySQL = value +//! The Klickwege structure-parity oracle (transcode doctrine: "`MySQL` = value //! parity, Klickwege = structure parity"). //! //! [`build_nav_digest`] folds the UI-navigation plane of a harvest — the @@ -20,6 +20,10 @@ use crate::exam_config::ExamConfig; use crate::region::RegionSubject; use crate::triple::Triple; +/// `(screen, region) -> [(dock/popup token, dock order)]` groupings used +/// while assembling the `[regions]` digest section. +type RegionEntries = BTreeMap<(String, String), Vec<(String, Option)>>; + /// Strip a triple's namespace prefix (`"ns:"`), returning the local part. /// An IRI with no `:` passes through unchanged. fn strip_ns(iri: &str) -> &str { @@ -55,7 +59,7 @@ fn control_of(iri: &str) -> &str { /// (`0x`) when it resolves, else its screen name (fallback). There is no /// stored ordinal — the address IS the walked rail (V3 LE-contract §3: the /// existing concept ontology is the radix trie; menu location is a path -/// through it). Cycle-guarded and depth-bounded so a mis-declared `part_of` +/// through it). Cycle-guarded and depth-bounded so a misdeclared `part_of` /// cycle terminates instead of looping. fn menu_address( node: &str, @@ -140,10 +144,10 @@ fn resolve_token(token: &str, config: &ExamConfig) -> Option { /// (both ends) union every `selects_view` subject. /// - `[klickwege]` / `[views]` strip the namespace prefix from both sides. /// - `[concepts]` resolves each `surfaces_concept` object token via -/// [`resolve_token`]; the screen is the namespace-stripped subject. +/// `resolve_token`; the screen is the namespace-stripped subject. /// - `[screen surface]` only lists screens with `controls + handlers > 0`; /// the screen is the namespace-stripped subject segment up to the first -/// `.` (see [`screen_of`]). +/// `.` (see `screen_of`). /// - `regions` / `[regions]` fold `docked_at` facts into the region frame: /// the dock token is resolved through /// [`crate::exam_config::ExamConfig::regions`] to a region name, falling @@ -164,7 +168,7 @@ fn resolve_token(token: &str, config: &ExamConfig) -> Option { /// Indentation is exactly two spaces. /// - `[menu-quad]` lowers the `(location, purpose, identity, action)` quad per /// menu node into the existing classid ontology. `location` is the `part_of` -/// rail walked as a radix-trie address ([`menu_address`]): the ancestor +/// rail walked as a radix-trie address (`menu_address`): the ancestor /// classid path, root-first, `0x` per resolved ancestor and the bare /// screen name as fallback — **no stored ordinal**, the address IS the walk /// (V3 LE-contract §3). `identity` is the node's own classid (`-` when @@ -297,8 +301,7 @@ pub fn build_nav_digest(triples: &[Triple], config: &ExamConfig) -> String { } // Group docked controls (+ popup targets) by resolved region. - let mut region_entries: BTreeMap<(String, String), Vec<(String, Option)>> = - BTreeMap::new(); + let mut region_entries: RegionEntries = BTreeMap::new(); for ((screen, control), token) in &dock_of { let region = config .regions @@ -665,7 +668,7 @@ Invoice loc=Invoice purpose=list id=- action=root // dot is NOT mistaken for the control boundary). assert!( digest.contains("widget_views.xml#view_form / center: partner_id(0)"), - "Odoo dotted-screen fact missing/mis-split in [regions]:\n{digest}" + "Odoo dotted-screen fact missing or wrongly split in [regions]:\n{digest}" ); } @@ -771,7 +774,7 @@ Invoice loc=Invoice purpose=list id=- action=root Triple::new( "app:Leaf", Predicate::PartOf, - "app:Aparent", + "app:Apparent", Provenance::Inferred, ), ]; @@ -782,14 +785,14 @@ Invoice loc=Invoice purpose=list id=- action=root forward, reversed, "conflicting part_of must be order-independent" ); - // Aparent (lexicographically smallest) is the canonical parent. + // Apparent (lexicographically smallest) is the canonical parent. assert!( - forward.contains("Leaf loc=Aparent/Leaf"), + forward.contains("Leaf loc=Apparent/Leaf"), "smallest parent must win deterministically:\n{forward}" ); } - /// A mis-declared `part_of` cycle must terminate (cycle-guarded walk), not + /// A misdeclared `part_of` cycle must terminate (cycle-guarded walk), not /// loop forever — the address is the bounded chain up to the first repeat. #[test] fn menu_quad_part_of_cycle_terminates() { diff --git a/crates/ruff_spo_triplet/src/reassemble.rs b/crates/ruff_spo_triplet/src/reassemble.rs index ca2b3daec0cf9..ebed7dfee4286 100644 --- a/crates/ruff_spo_triplet/src/reassemble.rs +++ b/crates/ruff_spo_triplet/src/reassemble.rs @@ -499,8 +499,8 @@ mod tests { } /// The core falsifier: a full round-trip recovers the emitted projection - /// of the source graph exactly. Would fail on any mis-attribution, - /// overload collapse, parameter loss, or name mis-parse. + /// of the source graph exactly. Would fail on any misattribution, + /// overload collapse, parameter loss, or name misparse. #[test] fn round_trip_recovers_cpp_emitted_projection() { let g = cpp_fixture(); diff --git a/crates/ruff_spo_triplet/src/structured_names.rs b/crates/ruff_spo_triplet/src/structured_names.rs index 35798a00fc13d..242dcd5262a67 100644 --- a/crates/ruff_spo_triplet/src/structured_names.rs +++ b/crates/ruff_spo_triplet/src/structured_names.rs @@ -12,7 +12,7 @@ //! caller-supplied [`NameGrammar`]. The grammar is deliberately //! **data-as-config**: no corpus tokens live in this file, only the //! tokenizer and the tier-walk. The parsed [`StructuredName`] feeds -//! [`part_of_edges`], which is the natural input to the FacetCascade +//! [`part_of_edges`], which is the natural input to the `FacetCascade` //! `part_of` forest — each tier becomes a parent node, and the subject //! is `part_of` the innermost one. @@ -25,8 +25,8 @@ pub struct NameGrammar { /// Empty string = no marker required, the path starts at the first /// numeric token. pub marker: String, - /// Names for each numeric tier, outermost first (e.g. ["form","section"]). - /// Extra numeric tokens beyond this list get tier name "level". + /// Names for each numeric tier, outermost first (e.g. `["form","section"]`). + /// Extra numeric tokens beyond this list get tier name "`level`". pub tier_names: Vec, } @@ -39,7 +39,7 @@ pub struct StructuredName { /// The numbered path, outermost first: [("form",2),("section",3)]. pub tiers: Vec, /// The remaining non-numeric tokens after the path (lowercased) — - /// concept/qualifier residue for the concept_split pass to resolve. + /// concept/qualifier residue for the `concept_split` pass to resolve. pub residue: Vec, } @@ -164,7 +164,10 @@ pub fn part_of_edges(subject: &str, ns: &str, parsed: &StructuredName) -> Vec<(S if !path.is_empty() { path.push('/'); } - path.push_str(&format!("{tier_name}_{n}")); + { + use std::fmt::Write as _; + let _ = write!(path, "{tier_name}_{n}"); + } nodes.push(format!("{ns}:{path}")); } diff --git a/crates/ruff_spo_triplet/src/surface_schema.rs b/crates/ruff_spo_triplet/src/surface_schema.rs index d6cee663f8cf9..dab0c7faa266f 100644 --- a/crates/ruff_spo_triplet/src/surface_schema.rs +++ b/crates/ruff_spo_triplet/src/surface_schema.rs @@ -16,7 +16,7 @@ //! the DO-arm candidate pool *before* recipe classification runs, or the //! enum/template plumbing pollutes the capability surface with fake //! "actions" that are really config reads. This ties directly into -//! param-enum fidelity — the walk_enums harvest (`ruff_cpp_spo`, +//! param-enum fidelity — the `walk_enums` harvest (`ruff_cpp_spo`, //! "Future-synergy-2") is what makes the *typed* enum/template space this //! module only classifies the *surface* of; concept and facet residue //! resolution (`["cipher", "typ"]` → the concrete `Cipher::typ` enum arm) @@ -178,7 +178,7 @@ mod tests { } } - /// `get_combo_cipher_typ` — verb `get`, surface `combo` (EnumSource), + /// `get_combo_cipher_typ` — verb `get`, surface `combo` (`EnumSource`), /// residue `["cipher", "typ"]`. #[test] fn get_combo_names_enum_source_with_concept_facet_residue() { @@ -191,7 +191,7 @@ mod tests { /// `add_option_widget_template` — BOTH `option` and `template` are /// present as surface tokens. The LAST match in name order wins: the /// trailing table-kind token names the actual storage ("a TEMPLATE - /// of options"), so `add_option_*_template` is a TemplateSource — + /// of options"), so `add_option_*_template` is a `TemplateSource` — /// never an enum source (codex P2 on PR #72). Every matched surface /// token leaves the residue: only the concept material remains. #[test] diff --git a/crates/ruff_spo_triplet/src/triple.rs b/crates/ruff_spo_triplet/src/triple.rs index dd9b2edc8ed31..53e48495807b5 100644 --- a/crates/ruff_spo_triplet/src/triple.rs +++ b/crates/ruff_spo_triplet/src/triple.rs @@ -1302,7 +1302,7 @@ mod tests { // visibility-honestly (`guarded_by`, NOT `requires`): the OQ-GUARD-1 // probe measured that guards combine permissions via `&&`/`||` plus // non-permission conditions, so a flat "requires" claim would - // mis-encode the disjunctions. Inferred — a proc-body heuristic) = 79. + // misencode the disjunctions. Inferred — a proc-body heuristic) = 79. // NB (#77 lesson): this count lives HERE only. No other arm may // re-assert it — cross-crate count duplication is the "monitor N pins" // anti-pattern; a predicate added here must never trip a routes.rs test. diff --git a/crates/ruff_sqlalchemy_spo/src/lib.rs b/crates/ruff_sqlalchemy_spo/src/lib.rs index 4d846ec344542..f88d830c8acd8 100644 --- a/crates/ruff_sqlalchemy_spo/src/lib.rs +++ b/crates/ruff_sqlalchemy_spo/src/lib.rs @@ -12,16 +12,16 @@ //! `WoA`'s convention, unlike Odoo's dotted `_name`; the `__tablename__` //! value is recorded but not used as the model identity). //! - `col = db.Column(db.TYPE(...), nullable=, primary_key=, default=, -//! db.ForeignKey('table.col'))` → a [`Field`] (see [`columns`] for the -//! type-mapping table in [`types`]). +//! db.ForeignKey('table.col'))` → a [`Field`] (see `columns` for the +//! type-mapping table in `types`). //! - `rel = db.relationship('Target', backref=…, uselist=…)` → an -//! [`ruff_spo_triplet::AssocDecl`] (see [`relationships`]), opportunistically +//! [`ruff_spo_triplet::AssocDecl`] (see `relationships`), opportunistically //! paired with a sibling FK column by the `_id` convention — the raw //! FK column is ALSO kept as a plain typed `Field` (mirrors the Rails //! design: physical column + declared association, deduped downstream by //! OGAR's `project_sqlalchemy_fields`, not here). //! - `def foo(self, …):` → a name-only [`Function`] (schema-only v0; body -//! facts are a follow-up — see [`functions`]). +//! facts are a follow-up — see `functions`). //! //! # Module layout (`SoC` mandate — Ruling a) //! @@ -29,15 +29,15 @@ //! dialect (Django, Sequel) can reuse the type-mapper without inheriting the //! Flask-SQLAlchemy-specific column/relationship recognition. //! -//! - [`parse`] — source → raw classes (mirrors `ruff_python_spo::parse`). -//! - [`walk`] — one class body → a [`RawClass`]. -//! - [`columns`] — the column-walker (`db.Column(...)` → [`columns::RawColumn`]). -//! - [`relationships`] — the relationship-walker (`db.relationship(...)` → -//! [`relationships::RawRelationship`] + the to-one/to-many heuristic + +//! - `parse` — source → raw classes (mirrors `ruff_python_spo::parse`). +//! - `walk` — one class body → a `RawClass`. +//! - `columns` — the column-walker (`db.Column(...)` → `columns::RawColumn`). +//! - `relationships` — the relationship-walker (`db.relationship(...)` → +//! `relationships::RawRelationship` + the to-one/to-many heuristic + //! FK pairing into [`ruff_spo_triplet::AssocDecl`]). -//! - [`types`] — the SQLAlchemy DSL → `field_type` mapping table, standalone +//! - `types` — the SQLAlchemy DSL → `field_type` mapping table, standalone //! so it's reusable by a future dialect frontend. -//! - [`functions`] — method harvest (name-only in v0). +//! - `functions` — method harvest (name-only in v0). use std::fs; use std::path::Path; diff --git a/crates/ty/src/lib.rs b/crates/ty/src/lib.rs index be073baa154c0..21f214316b01b 100644 --- a/crates/ty/src/lib.rs +++ b/crates/ty/src/lib.rs @@ -79,7 +79,7 @@ pub(crate) fn version(output_format: HelpFormat) -> Result<()> { match output_format { HelpFormat::Text => { - writeln!(stdout, "ty {}", &version_info)?; + writeln!(stdout, "ty {version_info}")?; } HelpFormat::Json => { serde_json::to_writer_pretty(&mut stdout, &version_info)?; diff --git a/crates/ty_module_resolver/src/resolve.rs b/crates/ty_module_resolver/src/resolve.rs index 90c04fba624c2..9ac9eefc0eed8 100644 --- a/crates/ty_module_resolver/src/resolve.rs +++ b/crates/ty_module_resolver/src/resolve.rs @@ -2237,7 +2237,7 @@ mod tests { Some(foo_real), path_to_module(&db, &FilePath::System(src.join("foo.py"))) ); - assert!(foo_real != foo); + assert_ne!(foo_real, foo); } #[test] diff --git a/crates/ty_project/src/glob.rs b/crates/ty_project/src/glob.rs index 9c023103c0a92..31dd69fbe3c4f 100644 --- a/crates/ty_project/src/glob.rs +++ b/crates/ty_project/src/glob.rs @@ -107,7 +107,7 @@ impl Default for IncludeExcludeFilter { impl std::fmt::Display for IncludeExcludeFilter { fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!(f, "include={}, exclude={}", &self.include, &self.exclude) + write!(f, "include={}, exclude={}", self.include, self.exclude) } } diff --git a/crates/ty_python_semantic/src/suppression.rs b/crates/ty_python_semantic/src/suppression.rs index 21dba30d6cbec..001a5f03b9c69 100644 --- a/crates/ty_python_semantic/src/suppression.rs +++ b/crates/ty_python_semantic/src/suppression.rs @@ -227,7 +227,7 @@ fn check_invalid_suppression(context: &mut CheckSuppressionsContext) { diag.into_diagnostic(format_args!( "Invalid `{kind}` comment: {reason}", kind = invalid.kind, - reason = &invalid.error + reason = invalid.error )); } } diff --git a/crates/ty_python_semantic/src/types/infer/builder.rs b/crates/ty_python_semantic/src/types/infer/builder.rs index 670eac3d90a19..66ae2490f701d 100644 --- a/crates/ty_python_semantic/src/types/infer/builder.rs +++ b/crates/ty_python_semantic/src/types/infer/builder.rs @@ -1482,7 +1482,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { { builder.into_diagnostic(format_args!( "Cyclic definition of `{}`", - &type_alias.name.as_name_expr().unwrap().id, + type_alias.name.as_name_expr().unwrap().id, )); } // Replace with `Divergent`. @@ -4432,7 +4432,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { { let mut diag = builder.into_diagnostic(format_args!( "Type annotation on enum member `{}` is not allowed", - &name_expr.id + name_expr.id )); diag.info( "See: https://typing.python.org/en/latest/spec/enums.html#enum-members", diff --git a/crates/ty_python_semantic/src/types/infer/builder/function.rs b/crates/ty_python_semantic/src/types/infer/builder/function.rs index 3ea5622c22475..69a1dc14ba801 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/function.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/function.rs @@ -475,7 +475,7 @@ impl<'db, 'ast> TypeInferenceBuilder<'db, 'ast> { }; let mut diagnostic = builder.into_diagnostic(format_args!( "Useless body for `@overload`-decorated function `{}`", - &function.name + function.name )); diagnostic.set_primary_message("This statement will never be executed"); diagnostic.info( diff --git a/crates/ty_python_semantic/src/types/infer/builder/post_inference/overloaded_function.rs b/crates/ty_python_semantic/src/types/infer/builder/post_inference/overloaded_function.rs index f48c2f37ee672..434283c51e963 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/post_inference/overloaded_function.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/post_inference/overloaded_function.rs @@ -110,7 +110,7 @@ pub(crate) fn check_overloaded_function<'db>( if let Some(builder) = context.report_lint(&INVALID_OVERLOAD, &function_node.name) { let mut diagnostic = builder.into_diagnostic(format_args!( "Overloaded function `{}` requires at least two overloads", - &function_node.name + function_node.name )); diagnostic.set_primary_message("Only one overload defined here"); if let Some(decorator) = @@ -158,11 +158,11 @@ pub(crate) fn check_overloaded_function<'db>( let mut diagnostic = builder.into_diagnostic(format_args!( "Overloads for function `{}` must be followed by a \ non-`@overload`-decorated implementation function", - &function_node.name + function_node.name )); diagnostic.info(format_args!( "Attempting to call `{}` will raise `TypeError` at runtime", - &function_node.name + function_node.name )); diagnostic.info("Overloaded functions without implementations are only permitted:"); diagnostic.info(" - in stub files"); @@ -183,7 +183,7 @@ pub(crate) fn check_overloaded_function<'db>( let mut diagnostic = builder.into_diagnostic(format_args!( "Overloaded function `{}` does not use the `@{}` decorator \ consistently", - &function_node.name, inconsistency.decorator_name + function_node.name, inconsistency.decorator_name )); for function in inconsistency.missing { diagnostic.annotate( diff --git a/crates/ty_python_semantic/src/types/infer/builder/post_inference/type_param_validation.rs b/crates/ty_python_semantic/src/types/infer/builder/post_inference/type_param_validation.rs index 8ed101693a20b..4a55cd33b926f 100644 --- a/crates/ty_python_semantic/src/types/infer/builder/post_inference/type_param_validation.rs +++ b/crates/ty_python_semantic/src/types/infer/builder/post_inference/type_param_validation.rs @@ -48,7 +48,7 @@ pub(crate) fn check_no_default_after_typevar_tuple_pep695( diagnostic.set_concise_message(format_args!( "Type parameter `{single_name}` with a default follows TypeVarTuple `{}`", - &typevar_tuple.name + typevar_tuple.name )); diagnostic.set_primary_message(format_args!("`{single_name}` has a default")); @@ -57,7 +57,7 @@ pub(crate) fn check_no_default_after_typevar_tuple_pep695( diagnostic.set_concise_message(format_args!( "Type parameters {names} with defaults follow TypeVarTuple `{}`", - &typevar_tuple.name + typevar_tuple.name )); diagnostic.set_primary_message(format_args!( @@ -77,7 +77,7 @@ pub(crate) fn check_no_default_after_typevar_tuple_pep695( diagnostic.annotate( context .secondary(typevar_tuple) - .message(format_args!("`{}` is a TypeVarTuple", &typevar_tuple.name)), + .message(format_args!("`{}` is a TypeVarTuple", typevar_tuple.name)), ); diagnostic.info("See https://typing.python.org/en/latest/spec/generics.html#defaults-following-typevartuple"); diff --git a/crates/ty_python_semantic/src/types/narrow.rs b/crates/ty_python_semantic/src/types/narrow.rs index a1c0794328e37..767e4e49d5808 100644 --- a/crates/ty_python_semantic/src/types/narrow.rs +++ b/crates/ty_python_semantic/src/types/narrow.rs @@ -2060,11 +2060,8 @@ impl<'db, 'ast> NarrowingConstraintsBuilder<'db, 'ast> { if let Some(ref mut first) = first { for rest_constraint in rest { - if let Some(rest_constraint) = rest_constraint { - merge_constraints_or(first, rest_constraint); - } else { - return None; - } + let rest_constraint = rest_constraint?; + merge_constraints_or(first, rest_constraint); } } first diff --git a/crates/ty_python_semantic/src/types/overrides.rs b/crates/ty_python_semantic/src/types/overrides.rs index 21929e71b7238..be38a2076544c 100644 --- a/crates/ty_python_semantic/src/types/overrides.rs +++ b/crates/ty_python_semantic/src/types/overrides.rs @@ -177,7 +177,7 @@ fn check_class_declaration<'db>( { let mut diagnostic = builder.into_diagnostic(format_args!( "Cannot overwrite NamedTuple attribute `{}`", - &member.name + member.name )); diagnostic.info("This will cause the class creation to fail at runtime"); } @@ -287,7 +287,7 @@ fn check_class_declaration<'db>( ) { let mut diagnostic = builder.into_diagnostic(format_args!( "Enum member `{}` value is not assignable to expected type", - &member.name + member.name )); diagnostic.info(format_args!( "Expected `{}`, got `{}`", @@ -1061,7 +1061,7 @@ fn check_explicit_overrides<'db>( } diagnostic.info(format_args!( "No `{member}` definitions were found on any superclasses of `{class}`", - member = &member.name, + member = member.name, class = class.name(db) )); } diff --git a/crates/ty_python_semantic/src/types/protocol_class.rs b/crates/ty_python_semantic/src/types/protocol_class.rs index 24c9fbd3fedbb..df59f0dba59e3 100644 --- a/crates/ty_python_semantic/src/types/protocol_class.rs +++ b/crates/ty_python_semantic/src/types/protocol_class.rs @@ -504,10 +504,10 @@ impl<'db> ProtocolMemberData<'db> { ProtocolMemberKind::Property(property) => { let mut d = f.debug_struct("PropertyMember"); if let Some(getter) = property.getter(self.db) { - d.field("getter", &format_args!("`{}`", &getter.display(self.db))); + d.field("getter", &format_args!("`{}`", getter.display(self.db))); } if let Some(setter) = property.setter(self.db) { - d.field("setter", &format_args!("`{}`", &setter.display(self.db))); + d.field("setter", &format_args!("`{}`", setter.display(self.db))); } d.finish() } diff --git a/crates/ty_server/tests/e2e/main.rs b/crates/ty_server/tests/e2e/main.rs index b43d9cd809cff..1a5443abe27a5 100644 --- a/crates/ty_server/tests/e2e/main.rs +++ b/crates/ty_server/tests/e2e/main.rs @@ -579,7 +579,7 @@ impl TestServer { { panic!( "Received multiple publish diagnostic notifications for {url}: ({existing:#?})", - url = ¬ification.uri + url = notification.uri ); } } diff --git a/docs/OGAR-POLYGLOT-AST-INTEGRATION.md b/docs/OGAR-POLYGLOT-AST-INTEGRATION.md index bdb969ea9bfd2..3a1c7596bb778 100644 --- a/docs/OGAR-POLYGLOT-AST-INTEGRATION.md +++ b/docs/OGAR-POLYGLOT-AST-INTEGRATION.md @@ -9,13 +9,13 @@ typed follow-up. polyglot AST substrate** — source (Python / C++ / C#) → OGAR IR → re-emitted source (Python / Rust / C#) — with the IR as a content-addressed interlingua. ---- +______________________________________________________________________ ## 0 · Why this is mostly "fix an asymmetry," not "build a transpiler" The interlingua already exists and is already bidirectional: -``` +```text SOURCE frontend IR (interlingua) address ────── ──────── ──────────────── ─────── Python ─ ruff_python_dto_check ─▶ (JSON bundles) ─┐ @@ -33,23 +33,24 @@ The interlingua already exists and is already bidirectional: ``` Three load-bearing facts (verified against `main`): + 1. **`ruff_spo_triplet::ir::ModelGraph` is the interlingua.** `expand` - (ModelGraph→triples) is general, but **`reassemble` (triples→ModelGraph) - today recovers only the C++ machine-plane projection** — it does *not* - reconstruct the core-7 `fields`/`functions` or the OpenProject collections - (`ruff_spo_triplet/src/reassemble.rs:16-20`). So a **general reassembler is - the first build item (Phase 0)**, not a given. "A new language is a new - frontend, not a new ontology" (crate doc) — the *intent* is bidirectional; - the *reassembler* is not yet general. -2. **The asymmetry is 4 frontends vs 1 backend.** `ruff_cpp_codegen` already - proves `ModelGraph → Rust source` (it renders `MethodSig` manifests - targeting `lance_graph_contract::codegen_manifest`). The superpower is - *generalizing the back door*. -3. **The frontends do not enter uniformly:** C++/Ruby produce `ModelGraph` - directly; Python produces JSON `ModuleHarvest` bundles; C# runs an - out-of-process Roslyn tool → ndjson → `load() -> Vec`. - ---- + (ModelGraph→triples) is general, but **`reassemble` (triples→ModelGraph) + today recovers only the C++ machine-plane projection** — it does *not* + reconstruct the core-7 `fields`/`functions` or the OpenProject collections + (`ruff_spo_triplet/src/reassemble.rs:16-20`). So a **general reassembler is + the first build item (Phase 0)**, not a given. "A new language is a new + frontend, not a new ontology" (crate doc) — the *intent* is bidirectional; + the *reassembler* is not yet general. +1. **The asymmetry is 4 frontends vs 1 backend.** `ruff_cpp_codegen` already + proves `ModelGraph → Rust source` (it renders `MethodSig` manifests + targeting `lance_graph_contract::codegen_manifest`). The superpower is + *generalizing the back door*. +1. **The frontends do not enter uniformly:** C++/Ruby produce `ModelGraph` + directly; Python produces JSON `ModuleHarvest` bundles; C# runs an + out-of-process Roslyn tool → ndjson → `load() -> Vec`. + +______________________________________________________________________ ## 1 · The IR record (Phase-0 contract — lock this first) @@ -69,11 +70,11 @@ enum FacetMode { // carved from facet_classid (mirrors TailVariant) ``` - **Cascade** = *position*: 6 tiers deep, predicate implied (`part_of`+`is_a`). - Subsumption / containment as bit-ops. The address/index view. + Subsumption / containment as bit-ops. The address/index view. - **Triplet** = *local edges*: 4 SPO edges, predicate explicit (256-way). The - raw-graph view. **A `ruff_spo_triplet::Triple` IS a triplet-mode facet** - (interned via the same `ruff_spo_address` mint) — this is what unifies the - SPO corpus with the facet primitive (today they are separate substrates). + raw-graph view. **A `ruff_spo_triplet::Triple` IS a triplet-mode facet** + (interned via the same `ruff_spo_address` mint) — this is what unifies the + SPO corpus with the facet primitive (today they are separate substrates). A cascade tier `(part_of:is_a)` is a degenerate triplet with its predicate implied; triplet mode is the generalization (spend 2 tiers → buy an explicit @@ -81,7 +82,7 @@ predicate). ### 1.2 The 512-byte record as 32 tenants — canon `key(16) + value(496)` -``` +```text NodeRow 512B = key(16) | value(496) (OGAR canon, CLAUDE.md:51-52) ≡ 32 × 16B slots ≡ 32 tenants × [GUID; N] ("tenant" = one GUID column) slot 0 = Self GUID (the 16B key; addressable with zero value decode) @@ -100,6 +101,7 @@ label-only). So this is an expansion of the existing `lance_graph_contract::Clas composition DAG with dedup-by-content. ### 1.3 Capacity == the separation-of-concerns lint + Every cap is a structural-quality signal, on both axes: `>64 fields` (FieldMask) · `>256 per tier` / `>6 deep` (cascade) · `>4 edges` (triplet) · `>31 value tenants`. Overflow is **the signal**; the fix is always @@ -110,71 +112,79 @@ reusable check (`ruff_spo_address::soc`, "the 256-cap-is-a-lint law", classifying overflow as DUPLICATION and/or CONFLATION) but is **not** wired as a diagnostic — see Phase 2. ---- +______________________________________________________________________ ## 2 · Phases ### Phase 0 — Lock the IR contract + - Freeze `ModelGraph` + a **versioned closed `Predicate` registry**: a core - agnostic set (`part_of`, `is_a`, `has_field`, `has_function`, - `inherits_from`, `rdf:type` — the six the mint needs) + per-plane extension - predicates, all under one registry with a conformance test. (Current totals: - **18 C++ machine-plane variants**, 57 total in `Predicate::ALL` — see - `triple.rs:595-613,810-821`.) + agnostic set (`part_of`, `is_a`, `has_field`, `has_function`, + `inherits_from`, `rdf:type` — the six the mint needs) + per-plane extension + predicates, all under one registry with a conformance test. (Current totals: + **18 C++ machine-plane variants**, 57 total in `Predicate::ALL` — see + `triple.rs:595-613,810-821`.) - **Build a general reassembler** that recovers the core-7 `fields`/`functions` - + per-plane collections — today `reassemble` is C++-projection-only - (`reassemble.rs:16-20`), which is the prerequisite for the Phase-1 round-trip - gate to hold for Python/Ruby. + - per-plane collections — today `reassemble` is C++-projection-only + (`reassemble.rs:16-20`), which is the prerequisite for the Phase-1 round-trip + gate to hold for Python/Ruby. - Lock the dual-mode `FacetMode` + the `[Facet; 32]` / tenant layout + - `tenant_schema`. + `tenant_schema`. - Fix the predicate-count doc-drift (comments say 34/53; code/test carry 57). ### Phase 1 — Normalize the three frontends to one `extract() -> ModelGraph` -| Lang | Today | Action | -|---|---|---| -| **C++** | working entry points are `ruff_cpp_spo::extract_dir` / `extract_tree` (libclang, caller-supplied args); the convenience `extract()` is a `todo!()` panic stub (`lib.rs:151-156`) | fill `extract()`; register its **18** C++ predicates (incl. `returns_type`, `has_param_type`, `is_const`, `is_static`, `has_visibility`); widen corpus | -| **Python** | `ruff_python_dto_check` → JSON `ModuleHarvest`, not ModelGraph | add `bundle → ModelGraph` adapter (reuse extractors/matcher); the parse is done, only the IR shape is missing | -| **C#** | Roslyn `.NET` tool → ndjson → `load() -> Vec`, `NAMESPACE="medcare"` hardcoded | generalize the harvester past MedCare; wrap `load → reassemble → ModelGraph` as a one-call `extract()`; document the out-of-proc Roslyn seam | + +| Lang | Today | Action | +| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | +| **C++** | working entry points are `ruff_cpp_spo::extract_dir` / `extract_tree` (libclang, caller-supplied args); the convenience `extract()` is a `todo!()` panic stub (`lib.rs:151-156`) | fill `extract()`; register its **18** C++ predicates (incl. `returns_type`, `has_param_type`, `is_const`, `is_static`, `has_visibility`); widen corpus | +| **Python** | `ruff_python_dto_check` → JSON `ModuleHarvest`, not ModelGraph | add `bundle → ModelGraph` adapter (reuse extractors/matcher); the parse is done, only the IR shape is missing | +| **C#** | Roslyn `.NET` tool → ndjson → `load() -> Vec`, `NAMESPACE="medcare"` hardcoded | generalize the harvester past MedCare; wrap `load → reassemble → ModelGraph` as a one-call `extract()`; document the out-of-proc Roslyn seam | + - **Gate:** each frontend round-trips `ModelGraph → expand → ndjson → reassemble` - losslessly (`#[cfg(test)]` per crate). **Prerequisite:** the Phase-0 general - reassembler — today this gate only holds for the C++ projection. + losslessly (`#[cfg(test)]` per crate). **Prerequisite:** the Phase-0 general + reassembler — today this gate only holds for the C++ projection. ### Phase 2 — Convergence + the SoC lint (guardrails) + - **Cross-language convergence:** the same construct in Python/C++/C# mints the - **same `Facet`** — a CI test (the ruff analogue of - `bridge_codebook_convergence`). This is the proof the IR is agnostic. + **same `Facet`** — a CI test (the ruff analogue of + `bridge_codebook_convergence`). This is the proof the IR is agnostic. - **Promote §[G] → a real `ruff` diagnostic** (`OGAR-SOC`): on - 64-field / 256-tier / 6-deep / 4-edge / 30-slot overflow, emit the two-way - verdict (`DUPLICATION → masked ClassView`; `CONFLATION → split data⊥behaviour, - hoist constructor to compute_dag`). ruff is the linter; this is its home. + 64-field / 256-tier / 6-deep / 4-edge / 30-slot overflow, emit the two-way + verdict (`DUPLICATION → masked ClassView`; `CONFLATION → split data⊥behaviour, hoist constructor to compute_dag`). ruff is the linter; this is its home. ### Phase 3 — The backend / adapter family (the superpower) + Lift the one existing backend into a trait, mirroring OGAR's adapter pattern (SurrealQL / ClickHouse / TTL) but targeting *source code*: + ```rust pub trait LangBackend { fn render(&self, g: &ModelGraph) -> String; // ModelGraph → target source } ``` + - **Rust** ◀ `ruff_cpp_codegen` (exists) — generalize beyond C++ method manifests. - **Python** ◀ extend the existing `ruff_python_codegen` (the formatter's - generator) to render *from* ModelGraph, not just re-format ASTs — big leverage. + generator) to render *from* ModelGraph, not just re-format ASTs — big leverage. - **C#** ◀ new `ruff_csharp_codegen` (text emit, Roslyn-free). - **Gate (round-trip conformance — what makes it a compiler substrate):** - `source(L1) → ModelGraph → source(L2)`. `L1==L2` → defined normal form; - `L1≠L2` → structural arm transpiles, behavioural arm is **flagged, not - silently dropped**. + `source(L1) → ModelGraph → source(L2)`. `L1==L2` → defined normal form; + `L1≠L2` → structural arm transpiles, behavioural arm is **flagged, not + silently dropped**. ### Phase 4 — Land in the OGAR substrate + ndjson → `lance_graph` SPO store. **Firewall invariant:** the IR triples stay the canonical artifact (in-memory / compile-time); only a *derived ANN index* goes to Lance — code is never lowered to Lance rows. Guard with a conformance test (today this holds partly by omission — the production Lance `SpoStore` doesn't exist yet). ---- +______________________________________________________________________ ## 3 · The honest scope boundary + **Structure transpiles; behaviour does not.** `OGAR-AS-IR.md`: "the behavioural arm cannot survive lowering and stays in the IR" — and tellingly the existing backend renders `MethodSig` *signatures*, not method bodies. So Phase 3 is a @@ -184,8 +194,8 @@ transpilation (method bodies → executable target logic) is a separate research arm via `ActionDef` / `KausalSpec`, **not** Phase 3. ## 4 · Critical path -`Phase 0 (incl. general reassembler) → Phase 1 (Python + C# normalization) → -Phase 3 (LangBackend + Python backend)`. C++ is the end-to-end smoke test + +`Phase 0 (incl. general reassembler) → Phase 1 (Python + C# normalization) → Phase 3 (LangBackend + Python backend)`. C++ is the end-to-end smoke test (frontend via `extract_dir`/`extract_tree` + backend via `ruff_cpp_codegen`), because `reassemble` already covers the C++ projection — but the smoke test must call `extract_dir`/`extract_tree` directly (or fill the `todo!()` diff --git a/mkdocs.template.yml b/mkdocs.template.yml index cdbc32c983be9..92ed767da1d41 100644 --- a/mkdocs.template.yml +++ b/mkdocs.template.yml @@ -106,6 +106,7 @@ extra_javascript: not_in_nav: | /rules/* /formatter/* + /OGAR-POLYGLOT-AST-INTEGRATION.md extra: analytics: provider: fathom diff --git a/vendor/lsp-types/release.sh b/vendor/lsp-types/release.sh index 68e4844557de9..5cf8dce7510a0 100755 --- a/vendor/lsp-types/release.sh +++ b/vendor/lsp-types/release.sh @@ -7,9 +7,9 @@ if [ -z "$LEVEL" ]; then exit 1 fi -clog --$LEVEL +clog --"$LEVEL" git add CHANGELOG.md git commit -m "Update changelog" -cargo release $LEVEL --execute +cargo release "$LEVEL" --execute