From c5d5e6fbd83d07a720852c3a2f996a9f1b61a1fd Mon Sep 17 00:00:00 2001 From: NERVOSYS Date: Tue, 11 Aug 2026 21:36:56 -0700 Subject: [PATCH 1/2] effects: the system can now discharge an effect, not only track one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Effects could be declared, annotated, inferred, and enforced, but never *eliminated*: `/ audit` propagated outward forever and the only way to satisfy it was to keep declaring it. `effect` blocks named operations that nothing read — an `effect` declaration was decoration. Three parts, because none is useful alone. INTRODUCTION. `Audit.record(x)` performs the operation: it is checked against the signature in the `effect` block and puts `audit` in the calling function's effect set. It previously typechecked as an unknown method returning a fresh variable, accepted any arguments at all, and then died at run time with `unknown function \`record\`` — the same typechecks-then-does-not-evaluate shape as five earlier bugs this week. ELIMINATION. `handle { body } with Audit { record(e) => … }` removes the effect from the block it wraps, so a function can be pure despite calling something effectful. The subtraction is per *block*, not per function: the calls inside a handled block are collected into their own bucket, resolved, and discharged separately, so an unhandled call sitting beside a handled one still reports. Deleting the effect from the whole function would have been simpler and unsound. What the arm itself does is attributed honestly — handling `audit` by writing a file makes the handling function `/ fs`. A handler exchanges one effect for the effects of handling it, and says so. DECLARATION. An effect annotation naming nothing is now an error. `/ nte` used to be accepted as a *different effect* from `/ net` — enforced consistently and matching nothing, so a typo invented an effect instead of failing. Built-in kinds still need no declaration; the rule is about names that mean nothing. Handlers do not resume: an operation dispatches to its arm and returns like an ordinary call, which is what a tree-walking evaluator can do without capturing continuations. `resume` is the natural next step and is not here. Handlers are found dynamically (innermost wins) and evaluated lexically (the arm sees the scope the handler was written in); both are tested, as is the stack discipline that stops a handler outliving its block. `effects-showcase` demonstrates all three and drops two claims that are no longer true. It gains `effect Db {}`, the one migration the stricter rule required — found by running the checker over every `.mg` in the repository rather than by guessing at the blast radius. Counts move with the suite: prototype 1,094 → 1,106, total 2,802 → 2,814. Co-Authored-By: Claude Opus 5 --- ARCHITECTURE.md | 4 +- HANDOFF.md | 36 +++- MEASUREMENTS.md | 6 +- ROADMAP.md | 4 +- examples/effects-showcase/src/main.mg | 62 +++++-- prototype/src/ast.rs | 31 ++++ prototype/src/effects.rs | 233 +++++++++++++++++++++++++- prototype/src/elision.rs | 12 ++ prototype/src/eval.rs | 153 ++++++++++++++++- prototype/src/fmt.rs | 21 +++ prototype/src/mlir.rs | 3 + prototype/src/parser.rs | 53 ++++++ prototype/src/resolve.rs | 15 ++ prototype/src/token_budget.rs | 15 ++ prototype/src/types.rs | 103 +++++++++++- scripts/check-examples.sh | 2 +- scripts/test-all.ps1 | 4 +- scripts/test-all.sh | 4 +- 18 files changed, 728 insertions(+), 33 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 70bd8ed97..66619afd8 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -7,7 +7,7 @@ text. It is the leverage the text-token floor denies the language track (see [IDEAL_AGENTIC_LANGUAGE.md](IDEAL_AGENTIC_LANGUAGE.md) for that analysis). > **Scope.** Everything below is implemented and test-covered in `prototype/` -> (**1,094 tests** green) and scored in the sibling `agentic-eval` crate (80 +> (**1,106 tests** green) and scored in the sibling `agentic-eval` crate (80 > tests, in the AetherShell repository and not verifiable from here). The one > deliberate non-feature is agent/swarm *execution* — see > [Honest boundaries](#honest-boundaries). @@ -175,7 +175,7 @@ are **five independent Cargo workspaces**: | Path | Crate | Tests | Notes | |---|---|--:|---| | `RecursiveMachineIntelligence/` | `rmi` | 1,380 | The low-level neurosymbolic framework. Feature-gated (`cpu` / `gpu` / `cuda`); build with `--no-default-features --features cpu` for the portable set | -| `prototype/` | `mage-prototype` | 1,094 | Compiler, evaluator, ABL, RAP server. Path-depends on `rmi` | +| `prototype/` | `mage-prototype` | 1,106 | Compiler, evaluator, ABL, RAP server. Path-depends on `rmi` | | `ribosome/` | `ribosome` | 164 | The distributed build engine. Depends on nothing in this repository — see below | | `germline/` | `germline` | 112 | Model succession, handoff, fallback — the RSI control plane. Path-depends on `ribosome` | | `forge/` | `forge` | 52 | The package registry, and only that | diff --git a/HANDOFF.md b/HANDOFF.md index 27e0f1752..6579a766e 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -15,7 +15,7 @@ run, and are pinned to their output. See "The example rewrite" below. | | | |---|---| -| Tests | **2,802** — rmi 1,380 · prototype 1,094 · ribosome 164 · germline 112 · forge 52 | +| Tests | **2,814** — rmi 1,380 · prototype 1,106 · ribosome 164 · germline 112 · forge 52 | | CUDA | **1,071 passing** on dual RTX 3090 Ti, driver 610.88 | | Warnings | 0 compiler, 0 clippy in the four owned crates (`rmi` keeps 2 — vendored) | | Vulnerabilities | 0 Rust across five lockfiles, 0 npm | @@ -109,7 +109,7 @@ Six of the eleven — #5, #6, #7, #8, #9, #11 — are the same class: a bug that **typechecks and then does not evaluate**. `--check` cannot find these. Only `--eval` can, which is why the pin now runs it. -Prototype tests **1,066 → 1,094**, all green. +Prototype tests **1,066 → 1,106**, all green. ### And a twelfth, from this list itself @@ -125,6 +125,35 @@ Surveyed first rather than assumed: every `guard` in the repository — two examples, four `prototype/examples/*.mg`, two parser tests — already returns explicitly, so nothing had come to depend on the fall-through. +### And the effect system got its elimination rule + +`handle { … } with E { … }`. The gap this closes was the one left in the +"found and left alone" list: effects could be declared, annotated, inferred, +and enforced, but never *discharged*, so `/ audit` propagated outward forever. + +Three things landed together, because none of them is useful alone: + +- **Introduction.** `Audit.record(x)` performs the operation and puts `audit` + in the calling function's effect set. Before this an `effect` block declared + operations that no analysis attributed to anyone and that the evaluator + rejected with `unknown function` — a thirteenth bug of the familiar + typechecks-then-does-not-evaluate kind. +- **Elimination.** `handle` removes the effect from the block it wraps, so a + function can be pure despite calling something effectful. The subtraction is + **per block, not per function**: an unhandled call sitting beside a handled + one still reports. Whatever the arm itself does is attributed honestly, so + handling `audit` by writing a file makes the handling function `/ fs`. +- **Declaration.** An effect annotation naming nothing is now an error. + `/ nte` used to be accepted as a *different effect* from `/ net`, enforced + consistently and matching nothing — a typo invented an effect instead of + failing. + +Handlers do not resume. An operation call dispatches to its arm and returns +like an ordinary call, which is what a tree-walking evaluator can do without +capturing continuations. Handlers are found **dynamically** (innermost wins) +and evaluated **lexically** (the arm sees the scope the handler was written +in), and both are tested. + ### The examples are now pinned to their output, not to their exit status `check-examples.sh` used to record which examples typechecked. That bar was too @@ -142,9 +171,6 @@ the answer. `--print` regenerates the block after an intentional change. ### Found and left alone -- **There is no `handle` form.** `--check` reports `unresolved name: handle`. - Effects can be declared, annotated, inferred, and enforced, but never - *discharged* — the effect system has no elimination rule. - **`agent` and `unsafe` cannot be written as effect names**, because both lex as keywords: `/ agent` is a parse error. `rand` is not built in either — the built-in kind is `rng`, and anything else silently becomes `Effect::Custom`. diff --git a/MEASUREMENTS.md b/MEASUREMENTS.md index 877571853..954c350dc 100644 --- a/MEASUREMENTS.md +++ b/MEASUREMENTS.md @@ -6,8 +6,8 @@ numbers are machine-dependent; the shapes (throughput, scaling) are not. Date: 2026-06-10. Build: `release` for perf, `cargo test` for functionality. -> **Re-verified 2026-08-11** — all five crates tested: prototype **1,094**, rmi -> **1,380**, ribosome **164**, germline **112**, forge **52** = **2,802 passing, +> **Re-verified 2026-08-11** — all five crates tested: prototype **1,106**, rmi +> **1,380**, ribosome **164**, germline **112**, forge **52** = **2,814 passing, > 0 failing, 0 warnings**. No figure below has regressed. > > *A measurement that was wrong.* `BuildReport::cache_hit_ratio` was @@ -65,7 +65,7 @@ Date: 2026-06-10. Build: `release` for perf, `cargo test` for functionality. ### Test suites (all green) | Suite | Tests | Cmd | |---|---|---| -| MAGE prototype | **1094 pass** (+2 ignored perf harnesses) | `cargo test` | +| MAGE prototype | **1106 pass** (+2 ignored perf harnesses) | `cargo test` | | rmi (`cpu`) | **1380 pass** | `cargo test --no-default-features --features cpu` | | ribosome (build engine) | **164 pass** | `cargo test --manifest-path ribosome/Cargo.toml` | | germline (RSI control plane) | **112 pass** | `cargo test --manifest-path germline/Cargo.toml` | diff --git a/ROADMAP.md b/ROADMAP.md index 58b09c03f..f1f9ffa73 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,8 +4,8 @@ > Each step is a concrete, testable increment. > > **Last verified: 2026-08-11** — all five crates built and tested: prototype -> **1,094**, rmi **1,380**, ribosome **164**, germline **112**, forge **52** — -> **2,802 tests, 0 failures, 0 warnings**. The crate count went from three to +> **1,106**, rmi **1,380**, ribosome **164**, germline **112**, forge **52** — +> **2,814 tests, 0 failures, 0 warnings**. The crate count went from three to > five when the build engine (step 148) and the RSI control plane (step 149) > were extracted from `forge`; the total is unchanged by those moves, and > `forge`'s 52 is what the registry alone measured before they were parked in diff --git a/examples/effects-showcase/src/main.mg b/examples/effects-showcase/src/main.mg index 199c9a551..f6288cdb5 100644 --- a/examples/effects-showcase/src/main.mg +++ b/examples/effects-showcase/src/main.mg @@ -8,8 +8,8 @@ // - where that is *enforced*: private functions infer their effects, public // ones must declare them. The check is at the module boundary. // - composition, both `/ io, net` and `/ io + net` (the parser takes either) -// - a custom effect: any name that is not built in becomes one -// - `effect` declarations +// - a custom effect, declared by an `effect` block +// - performing an operation, and `handle … with` to *discharge* the effect // - `guard` for early exit, and `defer` // // The rule the checker enforces is *under*-declaration: a published function @@ -18,9 +18,10 @@ // description. Every claim in this file was checked by running // `mage-parse --check` on an edited copy, not by reading the compiler. // -// Not shown, because it does not exist yet: effect *handlers*. There is no -// `handle` form — `mage-parse --check` reports `unresolved name: handle`. An -// effect can be declared, annotated, inferred, and enforced, but not discharged. +// The elimination rule is `handle { … } with E { … }`: it removes an effect +// from the block it wraps, so a function can be *pure* despite calling +// something effectful. Handlers do not resume — an operation call dispatches to +// its arm and returns like an ordinary call. // // Run: forge run (or: mage-parse --eval src/main.mg main) @@ -28,10 +29,24 @@ // An `effect` block names an effect and the operations that belong to it. The // trailing semicolon on each signature is required. +// +// A declaration is not optional: an effect annotation naming nothing is an +// error. It used to be accepted, which meant `/ nte` was not a misspelling of +// `/ net` but a silently different effect that matched nothing. +// +// The declaration is `Audit`, the annotation is `/ audit`. The two spellings +// are matched case-insensitively. effect Audit { - fn record(entry: String); + fn record(entry: String) -> usize; } +// `db` is a custom effect too — it is not one of the built-in kinds (`io`, +// `net`, `fs`, `async`, `alloc`, `panic`, `ffi`, `env`, `time`, `gpu`, `npu`, +// `llm`, `evolve`, `learn`, `rng` — see `Effect::from_name` in `hir.rs`). It +// carries no operations, which is allowed: a bare `effect` block is how you +// name an effect you only want to track, not perform through. +effect Db {} + // ── Pure core ──────────────────────────────────────────────────────── // No annotation means no effects, and the checker holds this to it: calling @@ -77,10 +92,8 @@ fn jitter() -> i32 / rng { 17 } -// `db` is not one of the built-in effect kinds (`io`, `net`, `fs`, `async`, -// `alloc`, `panic`, `ffi`, `env`, `time`, `gpu`, `npu`, `llm`, `evolve`, -// `learn`, `rng` — see `Effect::from_name` in `hir.rs`). Any other name becomes -// a custom effect and is tracked the same way, so the system is open, not fixed. +// Custom effects propagate and are enforced exactly like built-in ones, so the +// system is open rather than fixed. fn persist(record: String) -> usize / db { len(chars(record)) } @@ -128,6 +141,33 @@ fn audit(entry: String) -> String / db { f"audited {len(chars(entry))} chars" } +// ── Performing an effect, and discharging it ───────────────────────── + +// `Audit.record(...)` *performs* the operation. That is what puts `audit` in +// this function's effect set — the annotation is checked against it, not the +// source of it. +fn transcribe(entry: String) -> usize / audit { + Audit.record(entry) +} + +// And here it is discharged. `handle { … } with Audit { … }` removes `audit` +// from the block, so this function is **pure** even though `transcribe` is not: +// `--check` reports `f summarize_audit: pure`. +// +// The subtraction is per-block, not per-function. A second, unhandled call to +// `transcribe` outside this `handle` would still be reported — handling one +// call does not launder the rest. +// +// Whatever the arm itself does is honestly attributed: make `record` call +// `persist` and this function becomes `/ db`, because that is what it now +// performs. A handler exchanges one effect for the effects of handling it. +fn summarize_audit(entry: String) -> String { + val n = handle { transcribe(entry) } with Audit { + record(e) => len(chars(e)) + } + f"recorded {n} chars" +} + // ── Entry point ────────────────────────────────────────────────────── // The union of everything reachable: `fs` and `net` from `check_host`, `time` @@ -140,6 +180,7 @@ pub fn main() -> String / fs, net, time, rng, db { val health = check_host("up.example", "app.toml") val stamped = stamp("up.example") val logged = audit(health) + val transcribed = summarize_audit(health) val codes = [200, 404, 503] val report = map(codes, fn(code) => severity(code)) @@ -150,6 +191,7 @@ pub fn main() -> String / fs, net, time, rng, db { health, stamped, logged, + transcribed, join(report, "/"), summarize(codes), ], diff --git a/prototype/src/ast.rs b/prototype/src/ast.rs index a216f9f9f..8c7420a09 100644 --- a/prototype/src/ast.rs +++ b/prototype/src/ast.rs @@ -284,6 +284,25 @@ pub enum Expr { scrutinee: Option>, arms: Vec, }, + /// `handle { body } with Audit { record(e) => … }` — the effect system's + /// elimination rule. + /// + /// Effects could be declared, annotated, inferred, and enforced, but never + /// *discharged*: there was no way to satisfy `/ audit` other than to keep + /// propagating it outward forever. This is what removes it. The handled + /// effect leaves the body's effect set; whatever the arms themselves do + /// takes its place, so handling `audit` by writing to a file is honestly + /// reported as `/ fs`. + /// + /// Handlers do not resume. An operation call dispatches to the matching arm + /// and returns its value like an ordinary call, which is what a + /// tree-walking evaluator can implement without capturing continuations. + Handle { + body: Block, + /// The declared `effect` block being discharged. + effect: String, + arms: Vec, + }, Loop { body: Block, }, @@ -366,6 +385,18 @@ pub struct MatchArm { pub body: Expr, } +/// One operation of a handled effect: `record(entry) => persist(entry)`. +/// +/// The parameters are bare names rather than patterns. Their types come from +/// the `effect` block's declaration of the operation, so writing them again +/// here would be a second place for them to disagree. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct HandlerArm { + pub op: String, + pub params: Vec, + pub body: Expr, +} + #[derive(Debug, Clone, Serialize, Deserialize)] #[serde(tag = "type")] pub enum Pattern { diff --git a/prototype/src/effects.rs b/prototype/src/effects.rs index 359d1fbee..adb6f9496 100644 --- a/prototype/src/effects.rs +++ b/prototype/src/effects.rs @@ -25,9 +25,39 @@ pub struct EffectInfer { call_graph: HashMap>, /// Currently being inferred (for cycle detection). in_progress: Vec, + /// Per function, the `handle` regions found in its body. + handled: HashMap>, + /// Regions collected while walking the function currently being collected. + /// Drained by `collect_function`; never meaningful between functions. + pending_regions: Vec, + /// Declared `effect` block names, lowercased. + /// + /// Lowercased because the two spellings of an effect are the declaration + /// (`effect Audit`) and the annotation (`/ audit`), and they have to name + /// the same thing. Keeping the fold here rather than at each use means the + /// convention is stated once. + effect_names: std::collections::HashSet, pub diagnostics: Vec, } +/// One `handle { … } with E { … }` occurrence. +/// +/// The calls inside the handled block are kept *out* of the function's ordinary +/// callee list and recorded here instead, because their effects have to be +/// resolved and then have `effect` removed before they join the enclosing +/// function's set. Discharging by simply deleting the effect from the whole +/// function would be unsound — an unhandled call to the same operation +/// elsewhere in the body would be silenced along with it. +#[derive(Debug, Clone)] +struct HandledRegion { + /// The effect this region discharges. + effect: Effect, + /// Functions called inside the handled block. + callees: Vec, + /// Effects performed directly inside the handled block. + local: EffectSet, +} + impl Default for EffectInfer { fn default() -> Self { Self::new() @@ -41,6 +71,9 @@ impl EffectInfer { inferred: HashMap::new(), call_graph: HashMap::new(), in_progress: Vec::new(), + handled: HashMap::new(), + pending_regions: Vec::new(), + effect_names: std::collections::HashSet::new(), diagnostics: Vec::new(), } } @@ -52,6 +85,16 @@ impl EffectInfer { // declarations are required here; private functions infer silently — // their effects still propagate to any public caller (see Pass 3). let mut boundary: std::collections::HashSet = std::collections::HashSet::new(); + + // Pass 0: the declared effects. Needed before anything else, because an + // operation call introduces its effect and an annotation naming no + // declared effect is an error — both need the full set up front. + for item in &module.items { + if let ast::ItemKind::Effect(ed) = &item.kind { + self.effect_names.insert(ed.name.to_lowercase()); + } + } + // Pass 1: collect function declarations and their call graphs. for item in &module.items { if let ast::ItemKind::Function(fd) = &item.kind { @@ -62,6 +105,34 @@ impl EffectInfer { } } + // Pass 1.5: every effect named in an annotation must exist — a built-in + // kind, or an `effect` block in this module. + // + // Anything else used to become `Effect::Custom` silently, so `/ nte` + // was not a misspelling of `/ net` but a *different effect*, enforced + // perfectly consistently and matching nothing. A typo invented an + // effect rather than failing, which is the most expensive way for a + // capability system to be wrong. + let mut named: Vec<(&String, &EffectSet)> = self.declared.iter().collect(); + named.sort_by_key(|(n, _)| n.as_str()); + for (func, effects) in named { + for effect in effects { + let Effect::Custom(name) = effect else { continue }; + if self.effect_names.contains(&name.to_lowercase()) { + continue; + } + self.diagnostics.push(Diagnostic::categorized( + crate::hir::Severity::Error, + format!( + "function `{func}` declares unknown effect `{name}` — it is not \ + a built-in kind, and no `effect {name} {{ … }}` declares it" + ), + DiagnosticCategory::UndeclaredEffect, + None, + )); + } + } + // Pass 2: infer effects bottom-up. let fn_names: Vec = self.call_graph.keys().cloned().collect(); for name in &fn_names { @@ -123,7 +194,12 @@ impl EffectInfer { // Build call graph for this function by walking its body. let mut callees = Vec::new(); let mut local_effects = EffectSet::new(); + self.pending_regions.clear(); self.collect_calls_in_block(&fd.body, &mut callees, &mut local_effects); + let regions = std::mem::take(&mut self.pending_regions); + if !regions.is_empty() { + self.handled.insert(fd.name.clone(), regions); + } self.call_graph.insert(fd.name.clone(), callees); @@ -134,7 +210,7 @@ impl EffectInfer { } fn collect_calls_in_block( - &self, + &mut self, block: &ast::Block, callees: &mut Vec, local_effects: &mut EffectSet, @@ -148,7 +224,7 @@ impl EffectInfer { } fn collect_calls_in_stmt( - &self, + &mut self, stmt: &ast::Stmt, callees: &mut Vec, local_effects: &mut EffectSet, @@ -178,7 +254,7 @@ impl EffectInfer { } fn collect_calls_in_expr( - &self, + &mut self, expr: &ast::Expr, callees: &mut Vec, local_effects: &mut EffectSet, @@ -205,7 +281,18 @@ impl EffectInfer { self.collect_calls_in_expr(arg, callees, local_effects); } } - ast::Expr::MethodCall { receiver, args, .. } => { + ast::Expr::MethodCall { receiver, method, args, .. } => { + // `Audit.record(x)` performs `audit`. This is the introduction + // rule: without it an `effect` block declared operations that + // no analysis attributed to anyone, so a function calling one + // inferred `pure` while claiming `/ audit`. + if let ast::Expr::Ident { name } = receiver.as_ref() { + let lowered = name.to_lowercase(); + if self.effect_names.contains(&lowered) { + let _ = method; + local_effects.insert(Effect::from_name(&lowered)); + } + } self.collect_calls_in_expr(receiver, callees, local_effects); for arg in args { self.collect_calls_in_expr(arg, callees, local_effects); @@ -230,6 +317,24 @@ impl EffectInfer { self.collect_calls_in_expr(&arm.body, callees, local_effects); } } + // The handled block's calls go into a *separate* bucket, so the + // effect can be removed from them alone. The arms are different: + // they really do run in the enclosing function, so whatever they + // perform belongs to it — handling `audit` by writing a file is + // honestly reported as `/ fs`. + ast::Expr::Handle { body, effect, arms } => { + let mut inner_callees = Vec::new(); + let mut inner_local = EffectSet::new(); + self.collect_calls_in_block(body, &mut inner_callees, &mut inner_local); + self.pending_regions.push(HandledRegion { + effect: Effect::from_name(&effect.to_lowercase()), + callees: inner_callees, + local: inner_local, + }); + for arm in arms { + self.collect_calls_in_expr(&arm.body, callees, local_effects); + } + } ast::Expr::Loop { body } => { self.collect_calls_in_block(body, callees, local_effects); } @@ -399,6 +504,23 @@ impl EffectInfer { effects.extend(callee_effects); } + // Handled regions, resolved the same way and then discharged. The + // subtraction happens per region rather than over the whole function, + // so a second, unhandled call to the same effect elsewhere in the body + // still surfaces. + for region in self.handled.get(name).cloned().unwrap_or_default() { + let mut region_effects = region.local.clone(); + for callee in ®ion.callees { + let mut callee_effects = self.infer_function(callee); + if let Some(declared) = self.declared.get(callee) { + callee_effects.extend(declared.iter().cloned()); + } + region_effects.extend(callee_effects); + } + region_effects.remove(®ion.effect); + effects.extend(region_effects); + } + self.in_progress.retain(|n| n != name); self.inferred.insert(name.to_string(), effects.clone()); effects @@ -674,3 +796,106 @@ mod tests { assert!(ei.effects_of("with_io").is_empty()); } } + +#[cfg(test)] +mod handler_tests { + use super::*; + use crate::lexer; + use crate::parser; + + fn infer_source(src: &str) -> EffectInfer { + let tokens = lexer::lex(src); + let module = parser::parse(&tokens).expect("parse failed"); + infer_effects(&module) + } + + fn errors(src: &str) -> Vec { + infer_source(src) + .diagnostics + .iter() + .map(|d| d.message.clone()) + .collect() + } + + const DECL: &str = "effect Audit { f record(e: str) -> usize; }\n"; + + /// The introduction rule. An `effect` block used to declare operations that + /// no analysis attributed to anyone, so a function calling one inferred + /// `pure` while claiming `/ audit`. + #[test] + fn performing_an_operation_introduces_its_effect() { + let src = format!("{DECL}+f w(e: str) -> usize {{ Audit.record(e) }}"); + let msgs = errors(&src); + assert!( + msgs.iter().any(|m| m.contains("undeclared effects") && m.contains("audit")), + "expected `audit` to be attributed to `w`, got {msgs:?}" + ); + } + + /// The elimination rule: the whole point. `main` is pure despite calling + /// something that performs `audit`. + #[test] + fn handle_discharges_the_effect_it_names() { + let src = format!( + "{DECL}f w(e: str) -> usize / audit {{ Audit.record(e) }}\n\ + +f main() -> usize {{ handle {{ w(\"x\") }} with Audit {{ record(e) => len(chars(e)) }} }}" + ); + assert!(errors(&src).is_empty(), "errors: {:?}", errors(&src)); + } + + /// The soundness property. Discharging by deleting the effect from the + /// whole function would silence this second, *unhandled* call — so the + /// subtraction is per handled block instead. + #[test] + fn an_unhandled_call_beside_a_handled_one_still_reports() { + let src = format!( + "{DECL}f w(e: str) -> usize / audit {{ Audit.record(e) }}\n\ + +f main() -> usize {{ v a = handle {{ w(\"x\") }} with Audit {{ record(e) => 1 }}\n\ + v b = w(\"y\")\n a + b }}" + ); + let msgs = errors(&src); + assert!( + msgs.iter().any(|m| m.contains("audit")), + "the unhandled call must still surface, got {msgs:?}" + ); + } + + /// A handler is not free: handling `audit` by touching the filesystem makes + /// the handling function perform `fs`. + #[test] + fn a_handlers_own_effects_are_attributed_to_the_handling_function() { + let src = format!( + "{DECL}f w(e: str) -> usize / audit {{ Audit.record(e) }}\n\ + f to_disk(e: str) -> usize / fs {{ 1 }}\n\ + +f main() -> usize {{ handle {{ w(\"x\") }} with Audit {{ record(e) => to_disk(e) }} }}" + ); + let msgs = errors(&src); + assert!( + msgs.iter().any(|m| m.contains("FS")), + "expected the handler's `fs` to surface, got {msgs:?}" + ); + } + + /// An effect annotation naming nothing used to be accepted, so `/ nte` was + /// a different effect rather than a misspelling of `/ net`. + #[test] + fn an_effect_annotation_naming_nothing_is_an_error() { + let msgs = errors("+f a() -> i32 / nte { 1 }"); + assert!( + msgs.iter().any(|m| m.contains("unknown effect")), + "expected an unknown-effect diagnostic, got {msgs:?}" + ); + } + + #[test] + fn a_declared_custom_effect_is_accepted() { + assert!(errors("effect Db {}\n+f a() -> i32 / db { 1 }").is_empty()); + } + + /// Built-in kinds need no declaration — the rule is about names that mean + /// nothing, not about forcing boilerplate for `fs`. + #[test] + fn builtin_effect_kinds_need_no_declaration() { + assert!(errors("+f a() -> i32 / fs, net, rng { 1 }").is_empty()); + } +} diff --git a/prototype/src/elision.rs b/prototype/src/elision.rs index b7a993f5e..a8d40b2f6 100644 --- a/prototype/src/elision.rs +++ b/prototype/src/elision.rs @@ -445,6 +445,18 @@ fn elide_expr(expr: &Expr) -> Expr { | Expr::Continue | Expr::Error { .. } => expr.clone(), + Expr::Handle { body, effect, arms } => Expr::Handle { + body: elide_block(body), + effect: effect.clone(), + arms: arms + .iter() + .map(|a| crate::ast::HandlerArm { + op: a.op.clone(), + params: a.params.clone(), + body: elide_expr(&a.body), + }) + .collect(), + }, Expr::Binary { op, left, right } => Expr::Binary { op: op.clone(), left: Box::new(elide_expr(left)), diff --git a/prototype/src/eval.rs b/prototype/src/eval.rs index 1f139f445..28e94883a 100644 --- a/prototype/src/eval.rs +++ b/prototype/src/eval.rs @@ -129,6 +129,7 @@ fn err(m: impl Into) -> Result { } /// Lexical environment: a stack of scopes. +#[derive(Clone)] struct Env { scopes: Vec>, } @@ -187,6 +188,26 @@ pub struct Interp { /// same (`Left { X }` and `Right { X }`), and keying by variant alone let the /// second registration evict the first, so one of the two stopped resolving. enum_variants: HashMap<(String, String), usize>, + /// Effect handlers currently installed, innermost last. + /// + /// A dynamic stack, because that is what an effect handler is: the call + /// that performs the operation may be many frames below the `handle` that + /// discharges it, and it does not know which handler it will reach. The + /// *arm bodies*, though, close over the environment where the handler was + /// written, so a handler is dynamically found and lexically evaluated. + /// + /// `RefCell` because `eval` takes `&self`; the alternative is threading a + /// handler stack through every expression form. + handlers: RefCell>, +} + +/// One installed `handle … with E { … }`. +struct HandlerFrame { + effect: String, + /// op name → (parameter names, body). + arms: HashMap, Expr)>, + /// The environment the handler was written in, captured at installation. + env: Env, } impl Interp { @@ -219,7 +240,24 @@ impl Interp { _ => {} } } - Interp { funcs, methods, enum_variants } + Interp { funcs, methods, enum_variants, handlers: RefCell::new(Vec::new()) } + } + + /// The innermost installed handler for `effect.op`, if any. + /// + /// Innermost-first, so a nested `handle` shadows an outer one for the same + /// effect — the ordinary scoping rule, and the reason the stack is searched + /// backwards rather than forwards. + fn handler_for(&self, effect: &str, op: &str) -> Option<(Vec, Expr, Env)> { + self.handlers + .borrow() + .iter() + .rev() + .find(|f| f.effect == effect && f.arms.contains_key(op)) + .map(|f| { + let (params, body) = &f.arms[op]; + (params.clone(), body.clone(), f.env.clone()) + }) } /// The type name a value dispatches on, for values that can carry methods. @@ -434,6 +472,26 @@ impl Interp { } } Expr::Block { block } => self.eval_block(block, env), + Expr::Handle { body, effect, arms } => { + self.handlers.borrow_mut().push(HandlerFrame { + effect: effect.clone(), + arms: arms + .iter() + .map(|a| (a.op.clone(), (a.params.clone(), a.body.clone()))) + .collect(), + // The environment the handler was *written* in. An arm + // must not see the locals of whatever frame happened to + // perform the operation. + env: env.clone(), + }); + // Bound rather than `?`-propagated: the frame has to come off + // the stack even when the body returns or errors, or a `return` + // out of a handled block would leave the handler installed for + // everything that ran afterwards. + let out = self.eval_block(body, env); + self.handlers.borrow_mut().pop(); + out + } Expr::Match { scrutinee, arms } => { let v = match scrutinee { Some(e) => self.eval(e, env)?, @@ -597,6 +655,33 @@ impl Interp { // shape, and is a constructor rather than a call: check it // first, since there is no receiver value to evaluate. if let Expr::Ident { name } = receiver.as_ref() { + // An effect operation: `Audit.record(x)`. Dispatched to the + // innermost installed handler. Before this, an operation + // declared in an `effect` block typechecked and then died + // with `unknown function`, because nothing ever read the + // block's operations. + if let Some((params, body, captured)) = self.handler_for(name, method) { + let mut av = Vec::with_capacity(args.len()); + for a in args { + av.push(self.eval(a, env)?); + } + if params.len() != av.len() { + return err(format!( + "effect operation `{name}.{method}` takes {} argument(s), \ + given {}", + params.len(), + av.len() + )); + } + // Evaluated in the handler's own environment, with the + // operation's arguments bound on top. + let mut henv = captured; + henv.push(); + for (p, v) in params.iter().zip(av) { + henv.define(p.clone(), v); + } + return self.eval(&body, &mut henv); + } if let Some((enum_name, arity)) = self.as_enum_variant(name, method) { if arity != args.len() { return err(format!( @@ -2023,3 +2108,69 @@ mod tests { assert_eq!(run(src, "s", &[]), Value::Int(25)); } } + +#[cfg(test)] +mod handler_eval_tests { + use super::*; + + fn run(src: &str, f: &str) -> Value { + run_source(src, f, &[]).expect("run failed") + } + + const DECL: &str = "effect Audit { f record(e: i32) -> i32; }\n"; + + /// An operation declared in an `effect` block used to typecheck and then + /// die with `unknown function \`record\``, because nothing read the block's + /// operations at run time. + #[test] + fn an_operation_dispatches_to_its_handler() { + let src = format!( + "{DECL}f w() {{ Audit.record(7) }}\n\ + f s() {{ handle {{ w() }} with Audit {{ record(e) => e * 3 }} }}" + ); + assert_eq!(run(&src, "s"), Value::Int(21)); + } + + /// A handler is found dynamically — `w` does not mention the handler and + /// sits a call frame below it. + #[test] + fn the_innermost_handler_wins() { + let src = format!( + "{DECL}f w() {{ Audit.record(1) }}\n\ + f s() {{ handle {{ handle {{ w() }} with Audit {{ record(e) => 20 }} }} \ + with Audit {{ record(e) => 10 }} }}" + ); + assert_eq!(run(&src, "s"), Value::Int(20)); + } + + /// …but evaluated lexically: the arm sees the environment the *handler* was + /// written in, not the frame that performed the operation. + #[test] + fn an_arm_body_sees_the_handlers_own_scope() { + let src = format!( + "{DECL}f w() {{ v hidden = 999\n Audit.record(1) }}\n\ + f s() {{ v scale = 5\n handle {{ w() }} with Audit {{ record(e) => e * scale }} }}" + ); + assert_eq!(run(&src, "s"), Value::Int(5)); + } + + /// The frame comes off the stack when the block ends, or everything after a + /// handled call would keep dispatching to a handler that is out of scope. + #[test] + fn a_handler_does_not_outlive_its_block() { + let src = format!( + "{DECL}f w() {{ Audit.record(1) }}\n\ + f s() {{ v a = handle {{ w() }} with Audit {{ record(e) => 4 }}\n a + w() }}" + ); + let out = run_source(&src, "s", &[]); + assert!(out.is_err(), "the second, unhandled call must fail: {out:?}"); + } + + #[test] + fn the_handled_block_still_produces_its_own_value() { + let src = format!( + "{DECL}f s() {{ handle {{ 6 * 7 }} with Audit {{ record(e) => 0 }} }}" + ); + assert_eq!(run(&src, "s"), Value::Int(42)); + } +} diff --git a/prototype/src/fmt.rs b/prototype/src/fmt.rs index b3a92ee2c..00d1ef055 100644 --- a/prototype/src/fmt.rs +++ b/prototype/src/fmt.rs @@ -1264,6 +1264,27 @@ fn emit_expr(buf: &mut String, expr: &Expr, mode: Mode) { buf.push('}'); } } + Expr::Handle { body, effect, arms } => { + match mode { + Mode::Agent => buf.push_str("hx "), + Mode::Human => buf.push_str("handle "), + } + buf.push_str("{\n"); + emit_block_body(buf, body, mode, 1); + buf.push_str("} with "); + buf.push_str(effect); + buf.push_str(" {\n"); + for arm in arms { + buf.push_str(" "); + buf.push_str(&arm.op); + buf.push('('); + buf.push_str(&arm.params.join(", ")); + buf.push_str(") => "); + emit_expr(buf, &arm.body, mode); + buf.push_str(",\n"); + } + buf.push('}'); + } Expr::Match { scrutinee, arms } => { match mode { Mode::Agent => buf.push_str("?= "), diff --git a/prototype/src/mlir.rs b/prototype/src/mlir.rs index bad680366..43b16f52f 100644 --- a/prototype/src/mlir.rs +++ b/prototype/src/mlir.rs @@ -630,6 +630,9 @@ impl<'a> EmitCtx<'a> { ast::Expr::Match { arms, .. } => { format!("MAGE.match({} arms)", arms.len()) } + ast::Expr::Handle { effect, arms, .. } => { + format!("MAGE.handle({effect}, {} ops)", arms.len()) + } ast::Expr::Loop { .. } => "MAGE.loop { ... }".to_string(), ast::Expr::While { .. } => "MAGE.while { ... }".to_string(), ast::Expr::For { .. } => "MAGE.for { ... }".to_string(), diff --git a/prototype/src/parser.rs b/prototype/src/parser.rs index 53cc34e86..3a8e9320b 100644 --- a/prototype/src/parser.rs +++ b/prototype/src/parser.rs @@ -1088,6 +1088,49 @@ impl<'a> Parser<'a> { // ── Effect ────────────────────────────────────────────── + /// `handle { body } with Audit { record(e) => …, flush() => … }` + /// + /// The effect name is required rather than inferred from the arms: an + /// operation name alone does not say which effect it belongs to, and two + /// effects are allowed to declare an operation with the same name. + fn parse_handle_expr(&mut self) -> Result { + self.advance(); // `handle` / `hx` + let body = self.parse_block()?; + + // `with` is not a keyword — the lexer leaves it a plain identifier — + // so it is matched on text, the same way `as` is for casts. + if self.peek() != TokenKind::Ident || self.current().text != "with" { + return Err(self.error("expected `with ` after `handle { … }`")); + } + self.advance(); + + let effect = self.expect_ident()?; + self.expect(TokenKind::LBrace)?; + + let mut arms = Vec::new(); + while self.peek() != TokenKind::RBrace && self.peek() != TokenKind::Eof { + let op = self.expect_ident()?; + self.expect(TokenKind::LParen)?; + let mut params = Vec::new(); + while self.peek() != TokenKind::RParen && self.peek() != TokenKind::Eof { + params.push(self.expect_ident()?); + if self.peek() == TokenKind::Comma { + self.advance(); + } + } + self.expect(TokenKind::RParen)?; + self.expect(TokenKind::FatArrow)?; + let arm_body = self.parse_expr()?; + arms.push(HandlerArm { op, params, body: arm_body }); + if self.peek() == TokenKind::Comma { + self.advance(); + } + } + self.expect(TokenKind::RBrace)?; + + Ok(Expr::Handle { body, effect, arms }) + } + fn parse_effect_def(&mut self) -> Result { self.expect(TokenKind::KwEffect)?; let name = self.expect_ident()?; @@ -3721,6 +3764,16 @@ impl<'a> Parser<'a> { // Same shape as the sum-type-constructor arm below: parse // as a plain Ident, with an optional call-args suffix so // `guard(x)` works too. + // `handle {` is the effect elimination form; anything else + // beginning with `handle` is the corpus using it as a plain name + // (`handle.spawn(...)`), which the arm below still allows. The + // brace is what separates them, and no identifier use can be + // followed by one — a `{` after a bare name would be a block, not + // a continuation of the expression. + TokenKind::KwHandle if self.peek_n(1) == TokenKind::LBrace => { + self.parse_handle_expr() + } + TokenKind::KwGuard | TokenKind::KwHandle | TokenKind::KwNet diff --git a/prototype/src/resolve.rs b/prototype/src/resolve.rs index 456aa6cbe..59f87449e 100644 --- a/prototype/src/resolve.rs +++ b/prototype/src/resolve.rs @@ -848,6 +848,21 @@ impl Resolver { self.pop_scope(); } } + // The handled body resolves in the enclosing scope; each arm gets + // its own, with the operation's parameters bound as plain names. + // Their types come from the `effect` declaration, so the resolver + // only has to make them visible. + ast::Expr::Handle { body, arms, .. } => { + self.resolve_block(body); + for arm in arms { + self.push_scope(); + for p in &arm.params { + self.define_value(p, SymbolKind::Variable { mutable: false }); + } + self.resolve_expr(&arm.body); + self.pop_scope(); + } + } ast::Expr::Loop { body } => { self.resolve_block(body); } diff --git a/prototype/src/token_budget.rs b/prototype/src/token_budget.rs index cbfe426fa..5504543bf 100644 --- a/prototype/src/token_budget.rs +++ b/prototype/src/token_budget.rs @@ -474,6 +474,14 @@ fn count_stmt_agent(stmt: &Stmt) -> u32 { fn count_expr_agent(expr: &Expr) -> u32 { match expr { + // `hx { … } with E { op(p) => … }` + Expr::Handle { body, arms, .. } => { + let mut n = 2 + count_block_agent(body); // `hx` + effect name + for arm in arms { + n += 3 + arm.params.len() as u32 + count_expr_agent(&arm.body); + } + n + } Expr::Literal { .. } => 1, Expr::Ident { .. } => 1, Expr::Binary { left, right, .. } => { @@ -920,6 +928,13 @@ fn count_stmt_human(stmt: &Stmt) -> u32 { fn count_expr_human(expr: &Expr) -> u32 { match expr { + Expr::Handle { body, arms, .. } => { + let mut n = 3 + count_block_human(body); // `handle` + `with` + name + for arm in arms { + n += 3 + arm.params.len() as u32 + count_expr_human(&arm.body); + } + n + } Expr::Literal { .. } => 1, Expr::Ident { .. } => 1, Expr::Binary { left, right, .. } => 1 + count_expr_human(left) + count_expr_human(right), diff --git a/prototype/src/types.rs b/prototype/src/types.rs index ba2e4f0e9..5191ff238 100644 --- a/prototype/src/types.rs +++ b/prototype/src/types.rs @@ -307,6 +307,16 @@ pub struct TypeChecker { /// Enum definitions: enum name → its variant names. Used for match /// exhaustiveness checking. enum_defs: HashMap>, + /// Operations of each declared `effect` block: `(effect, op)` → parameter + /// types and return type. + /// + /// Before this the operations in an `effect` block were parsed, stored, and + /// read by nothing — an `effect` declaration was decoration. They are the + /// signature an operation call is checked against and the types a handler + /// arm's parameters are bound at, so they exist in exactly one place. + effect_ops: HashMap<(String, String), (Vec, Ty)>, + /// Names of declared `effect` blocks, in declaration order. + effect_defs: Vec, /// Interned names of user-defined types: name → id, and id → name. /// /// Gives `Ty::Named` real identity. Two vectors rather than one bimap @@ -344,6 +354,8 @@ impl TypeChecker { struct_defs: HashMap::new(), fn_sigs: HashMap::new(), enum_defs: HashMap::new(), + effect_ops: HashMap::new(), + effect_defs: Vec::new(), named_ids: HashMap::new(), named_names: Vec::new(), ret_stack: Vec::new(), @@ -520,6 +532,20 @@ impl TypeChecker { fn collect_item_sig(&mut self, item: &ast::Item) { match &item.kind { + ast::ItemKind::Effect(ed) => { + self.effect_defs.push(ed.name.clone()); + for op in &ed.operations { + let params: Vec = + op.params.iter().map(|p| self.lower_type(&p.ty)).collect(); + // An operation with no `->` returns unit, like a function. + let ret = match &op.return_type { + Some(t) => self.lower_type(t), + None => Ty::Unit, + }; + self.effect_ops + .insert((ed.name.clone(), op.name.clone()), (params, ret)); + } + } ast::ItemKind::Function(fd) => { let params: Vec = fd.params.iter().map(|p| self.lower_type(&p.ty)).collect(); // No return annotation → a fresh inference var, resolved from the @@ -1324,7 +1350,34 @@ impl TypeChecker { self.subst.apply(&ret) } - ast::Expr::MethodCall { receiver, args, .. } => { + ast::Expr::MethodCall { receiver, method, args, .. } => { + // `Audit.record(x)` — an effect operation, not a method. The + // receiver is the effect's name, so it is checked before the + // receiver is inferred as a value; there is no value there to + // infer, and treating it as one is what made an operation call + // return a fresh variable and accept any arguments at all. + if let ast::Expr::Ident { name } = receiver.as_ref() + && let Some((params, ret)) = + self.effect_ops.get(&(name.clone(), method.clone())).cloned() + { + if params.len() != args.len() { + self.emit_error(format!( + "effect operation `{name}.{method}` takes {} argument(s), \ + given {}", + params.len(), + args.len() + )); + } + for (arg, want) in args.iter().zip(params.iter()) { + let got = self.infer_expr(arg); + if let Err(e) = unify(&mut self.subst, &got, want) { + self.emit_error(format!( + "effect operation `{name}.{method}`: {e}" + )); + } + } + return ret; + } self.infer_expr(receiver); for arg in args { self.infer_expr(arg); @@ -1560,6 +1613,54 @@ impl TypeChecker { result } + // `handle { body } with E { op(p) => … }`. The value is the body's, + // exactly as if the handler were not there — a handler discharges + // an effect, it does not change what the computation produces. + ast::Expr::Handle { body, effect, arms } => { + if !self.effect_defs.contains(effect) { + self.emit_error(format!( + "unknown effect `{effect}`: `handle … with` needs an \ + `effect {effect} {{ … }}` declaration" + )); + } + for arm in arms { + let Some((params, ret)) = + self.effect_ops.get(&(effect.clone(), arm.op.clone())).cloned() + else { + self.emit_error(format!( + "effect `{effect}` declares no operation `{}`", + arm.op + )); + continue; + }; + if params.len() != arm.params.len() { + self.emit_error(format!( + "handler for `{effect}.{}` binds {} parameter(s), the \ + operation declares {}", + arm.op, + arm.params.len(), + params.len() + )); + } + // The arm's parameters take their types from the effect + // declaration rather than from annotations on the arm, so + // there is only one place for them to be written. + self.env.push(); + for (name, ty) in arm.params.iter().zip(params.iter()) { + self.env.insert(name.clone(), ty.clone()); + } + let body_ty = self.infer_expr(&arm.body); + self.env.pop(); + if let Err(e) = unify(&mut self.subst, &body_ty, &ret) { + self.emit_error(format!( + "handler for `{effect}.{}` must produce what the \ + operation returns: {e}", + arm.op + )); + } + } + self.infer_block(body) + } ast::Expr::Loop { body } => { self.infer_block(body); // Loop type is determined by break expressions. diff --git a/scripts/check-examples.sh b/scripts/check-examples.sh index 7e4215a9a..d2b66ff5b 100755 --- a/scripts/check-examples.sh +++ b/scripts/check-examples.sh @@ -78,7 +78,7 @@ declare -A EXPECTED=( [cli-tool]='"-i alpha -> alpha beta | ALPHA delta; -c beta -> 2; no pattern -> usage error"' [cost-aware-optimizer]='"3 benchmark samples; x86-64: size | aarch64: balanced | riscv64: balanced | wasm32: balanced; x86-64: size | aarch64: size | riscv64: no candidate within budget | wasm32: balanced"' [data-structures]='"points=3, total_distance=15, closest=3"' - [effects-showcase]='"statuses=3 missing=0; 3 configured, live ok; up.example@1700000017; audited 21 chars; ok/warn/error; 3 samples, worst 503"' + [effects-showcase]='"statuses=3 missing=0; 3 configured, live ok; up.example@1700000017; audited 21 chars; recorded 21 chars; ok/warn/error; 3 samples, worst 503"' [hello-world]='"Hello, MAGE! (your name has 4 letters)"' [http-client]='"1: user 1 Ada (active=true); 2: rate limited, retry in 30s; 3: not found; 4: decode: expected 3 fields, got 1"' [live-compiler]='"live r2 passing 4; rollbacks 1; after explicit rollback r1 passing 3; type: handler:9 unresolved placeholder; revert placeholder@70; no repair proposed"' diff --git a/scripts/test-all.ps1 b/scripts/test-all.ps1 index 64813680d..9b486378d 100644 --- a/scripts/test-all.ps1 +++ b/scripts/test-all.ps1 @@ -8,12 +8,12 @@ This is the single entry point that covers everything CI covers: rmi (cpu) 1,380 tests - prototype 1,094 tests + prototype 1,106 tests ribosome 164 tests germline 112 tests forge 52 tests ------------------------- - total 2,802 tests, 0 warnings + total 2,814 tests, 0 warnings .PARAMETER Release Build and test in release mode (slower to build, much faster to run). diff --git a/scripts/test-all.sh b/scripts/test-all.sh index 1f1a58d2d..3b6c1b563 100755 --- a/scripts/test-all.sh +++ b/scripts/test-all.sh @@ -6,12 +6,12 @@ # This is the single entry point that covers everything CI covers: # # rmi (cpu) 1,380 tests -# prototype 1,094 tests +# prototype 1,106 tests # ribosome 164 tests # germline 112 tests # forge 52 tests # ------------------------- -# total 2,802 tests, 0 warnings +# total 2,814 tests, 0 warnings # # Usage: # scripts/test-all.sh # debug From 011b046751b191e633ee5bf830335218a973c7d7 Mon Sep 17 00:00:00 2001 From: NERVOSYS Date: Wed, 12 Aug 2026 09:15:07 -0700 Subject: [PATCH 2/2] effects: a misspelled operation was the same bug, one level down MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Audit.recrod(x)` typechecked clean and died at run time with `unknown function`. The effect analysis attributes an effect from the *receiver* alone, so the misspelling was counted as genuinely performing `audit`, satisfied the annotation, and passed every check — and then there was nothing to dispatch to. This is the exact bug class the feature it sits inside was built to eliminate, written into the first version of that feature. Fixing a category of mistake confers no immunity to committing another instance of it; the only thing that caught this one was running a probe with a typo in it, which is also how the other twelve were found. An operation call on a declared effect is now checked against the effect's declarations, and the diagnostic lists what the effect actually declares: error: effect `Audit` declares no operation `recrod` (it declares: record) Counts move with the suite: prototype 1,106 → 1,107, total 2,814 → 2,815. Co-Authored-By: Claude Opus 5 --- ARCHITECTURE.md | 4 ++-- HANDOFF.md | 13 +++++++++---- MEASUREMENTS.md | 6 +++--- ROADMAP.md | 4 ++-- prototype/src/effects.rs | 16 ++++++++++++++++ prototype/src/types.rs | 22 ++++++++++++++++++++++ scripts/test-all.ps1 | 4 ++-- scripts/test-all.sh | 4 ++-- 8 files changed, 58 insertions(+), 15 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 66619afd8..5582bda7c 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -7,7 +7,7 @@ text. It is the leverage the text-token floor denies the language track (see [IDEAL_AGENTIC_LANGUAGE.md](IDEAL_AGENTIC_LANGUAGE.md) for that analysis). > **Scope.** Everything below is implemented and test-covered in `prototype/` -> (**1,106 tests** green) and scored in the sibling `agentic-eval` crate (80 +> (**1,107 tests** green) and scored in the sibling `agentic-eval` crate (80 > tests, in the AetherShell repository and not verifiable from here). The one > deliberate non-feature is agent/swarm *execution* — see > [Honest boundaries](#honest-boundaries). @@ -175,7 +175,7 @@ are **five independent Cargo workspaces**: | Path | Crate | Tests | Notes | |---|---|--:|---| | `RecursiveMachineIntelligence/` | `rmi` | 1,380 | The low-level neurosymbolic framework. Feature-gated (`cpu` / `gpu` / `cuda`); build with `--no-default-features --features cpu` for the portable set | -| `prototype/` | `mage-prototype` | 1,106 | Compiler, evaluator, ABL, RAP server. Path-depends on `rmi` | +| `prototype/` | `mage-prototype` | 1,107 | Compiler, evaluator, ABL, RAP server. Path-depends on `rmi` | | `ribosome/` | `ribosome` | 164 | The distributed build engine. Depends on nothing in this repository — see below | | `germline/` | `germline` | 112 | Model succession, handoff, fallback — the RSI control plane. Path-depends on `ribosome` | | `forge/` | `forge` | 52 | The package registry, and only that | diff --git a/HANDOFF.md b/HANDOFF.md index 6579a766e..54e61f8cb 100644 --- a/HANDOFF.md +++ b/HANDOFF.md @@ -15,7 +15,7 @@ run, and are pinned to their output. See "The example rewrite" below. | | | |---|---| -| Tests | **2,814** — rmi 1,380 · prototype 1,106 · ribosome 164 · germline 112 · forge 52 | +| Tests | **2,815** — rmi 1,380 · prototype 1,107 · ribosome 164 · germline 112 · forge 52 | | CUDA | **1,071 passing** on dual RTX 3090 Ti, driver 610.88 | | Warnings | 0 compiler, 0 clippy in the four owned crates (`rmi` keeps 2 — vendored) | | Vulnerabilities | 0 Rust across five lockfiles, 0 npm | @@ -109,7 +109,7 @@ Six of the eleven — #5, #6, #7, #8, #9, #11 — are the same class: a bug that **typechecks and then does not evaluate**. `--check` cannot find these. Only `--eval` can, which is why the pin now runs it. -Prototype tests **1,066 → 1,106**, all green. +Prototype tests **1,066 → 1,107**, all green. ### And a twelfth, from this list itself @@ -143,10 +143,15 @@ Three things landed together, because none of them is useful alone: **per block, not per function**: an unhandled call sitting beside a handled one still reports. Whatever the arm itself does is attributed honestly, so handling `audit` by writing a file makes the handling function `/ fs`. -- **Declaration.** An effect annotation naming nothing is now an error. +- **Declaration.** An operation the effect does not declare is an error, and + so is an effect annotation naming nothing. `/ nte` used to be accepted as a *different effect* from `/ net`, enforced consistently and matching nothing — a typo invented an effect instead of - failing. + failing. The operation check was added after the first version of this + feature shipped it broken: the analysis attributes the effect from the + *receiver* alone, so `Audit.recrod(x)` counted as performing `audit`, + checked clean, and died at run time. Building the fix for a bug class is no + protection against writing another instance of it one level down. Handlers do not resume. An operation call dispatches to its arm and returns like an ordinary call, which is what a tree-walking evaluator can do without diff --git a/MEASUREMENTS.md b/MEASUREMENTS.md index 954c350dc..90ccd9d38 100644 --- a/MEASUREMENTS.md +++ b/MEASUREMENTS.md @@ -6,8 +6,8 @@ numbers are machine-dependent; the shapes (throughput, scaling) are not. Date: 2026-06-10. Build: `release` for perf, `cargo test` for functionality. -> **Re-verified 2026-08-11** — all five crates tested: prototype **1,106**, rmi -> **1,380**, ribosome **164**, germline **112**, forge **52** = **2,814 passing, +> **Re-verified 2026-08-11** — all five crates tested: prototype **1,107**, rmi +> **1,380**, ribosome **164**, germline **112**, forge **52** = **2,815 passing, > 0 failing, 0 warnings**. No figure below has regressed. > > *A measurement that was wrong.* `BuildReport::cache_hit_ratio` was @@ -65,7 +65,7 @@ Date: 2026-06-10. Build: `release` for perf, `cargo test` for functionality. ### Test suites (all green) | Suite | Tests | Cmd | |---|---|---| -| MAGE prototype | **1106 pass** (+2 ignored perf harnesses) | `cargo test` | +| MAGE prototype | **1107 pass** (+2 ignored perf harnesses) | `cargo test` | | rmi (`cpu`) | **1380 pass** | `cargo test --no-default-features --features cpu` | | ribosome (build engine) | **164 pass** | `cargo test --manifest-path ribosome/Cargo.toml` | | germline (RSI control plane) | **112 pass** | `cargo test --manifest-path germline/Cargo.toml` | diff --git a/ROADMAP.md b/ROADMAP.md index f1f9ffa73..ab5aec3ec 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -4,8 +4,8 @@ > Each step is a concrete, testable increment. > > **Last verified: 2026-08-11** — all five crates built and tested: prototype -> **1,106**, rmi **1,380**, ribosome **164**, germline **112**, forge **52** — -> **2,814 tests, 0 failures, 0 warnings**. The crate count went from three to +> **1,107**, rmi **1,380**, ribosome **164**, germline **112**, forge **52** — +> **2,815 tests, 0 failures, 0 warnings**. The crate count went from three to > five when the build engine (step 148) and the RSI control plane (step 149) > were extracted from `forge`; the total is unchanged by those moves, and > `forge`'s 52 is what the registry alone measured before they were parked in diff --git a/prototype/src/effects.rs b/prototype/src/effects.rs index adb6f9496..fabfe33e0 100644 --- a/prototype/src/effects.rs +++ b/prototype/src/effects.rs @@ -887,6 +887,22 @@ mod handler_tests { ); } + /// A misspelled operation. The effect analysis attributes the effect from + /// the *receiver* alone, so this was counted as performing `audit`, checked + /// clean, and then died at run time with `unknown function` — the same bug + /// one level down from the one this feature exists to fix. + #[test] + fn a_misspelled_operation_on_a_declared_effect_is_rejected() { + let tokens = crate::lexer::lex(&format!("{DECL}+f a() -> i32 / audit {{ Audit.recrod(1) }}")); + let module = crate::parser::parse(&tokens).expect("parse failed"); + let tc = crate::types::check(&module); + assert!( + tc.diagnostics.iter().any(|d| d.message.contains("declares no operation")), + "expected a misspelled-operation diagnostic, got {:?}", + tc.diagnostics + ); + } + #[test] fn a_declared_custom_effect_is_accepted() { assert!(errors("effect Db {}\n+f a() -> i32 / db { 1 }").is_empty()); diff --git a/prototype/src/types.rs b/prototype/src/types.rs index 5191ff238..86d7982ce 100644 --- a/prototype/src/types.rs +++ b/prototype/src/types.rs @@ -1378,6 +1378,28 @@ impl TypeChecker { } return ret; } + // The receiver names a declared effect, but the operation is + // not one of its declarations. That is a misspelling, and it + // has to be an error here: the effect analysis attributes the + // effect on the *receiver* alone, so `Audit.recrod(x)` was + // accepted, counted as performing `audit`, and then died at run + // time with `unknown function`. Exactly the shape of bug this + // whole feature exists to stop being possible. + if let ast::Expr::Ident { name } = receiver.as_ref() + && self.effect_defs.contains(name) + { + let mut ops: Vec<&str> = self + .effect_ops + .keys() + .filter(|(e, _)| e == name) + .map(|(_, op)| op.as_str()) + .collect(); + ops.sort_unstable(); + self.emit_error(format!( + "effect `{name}` declares no operation `{method}` (it declares: {})", + if ops.is_empty() { "none".to_string() } else { ops.join(", ") } + )); + } self.infer_expr(receiver); for arg in args { self.infer_expr(arg); diff --git a/scripts/test-all.ps1 b/scripts/test-all.ps1 index 9b486378d..70abcf692 100644 --- a/scripts/test-all.ps1 +++ b/scripts/test-all.ps1 @@ -8,12 +8,12 @@ This is the single entry point that covers everything CI covers: rmi (cpu) 1,380 tests - prototype 1,106 tests + prototype 1,107 tests ribosome 164 tests germline 112 tests forge 52 tests ------------------------- - total 2,814 tests, 0 warnings + total 2,815 tests, 0 warnings .PARAMETER Release Build and test in release mode (slower to build, much faster to run). diff --git a/scripts/test-all.sh b/scripts/test-all.sh index 3b6c1b563..b409d876b 100755 --- a/scripts/test-all.sh +++ b/scripts/test-all.sh @@ -6,12 +6,12 @@ # This is the single entry point that covers everything CI covers: # # rmi (cpu) 1,380 tests -# prototype 1,106 tests +# prototype 1,107 tests # ribosome 164 tests # germline 112 tests # forge 52 tests # ------------------------- -# total 2,814 tests, 0 warnings +# total 2,815 tests, 0 warnings # # Usage: # scripts/test-all.sh # debug