feat(vm): real call frames and callables

README.md

@@ -44,6 +44,7 @@ cargo build --workspace --release
   - [Bytecode and VMBC](#bytecode-and-vmbc)
   - [JIT](#jit)
   - [Regex Cache](#regex-cache)
+  - [Script Call Depth](#script-call-depth)
   - [Fuel Metering](#fuel-metering)
   - [Epoch Interruption](#epoch-interruption)
   - [Wasm Lint](#wasm-lint)
@@ -198,6 +199,21 @@ Reducing the capacity evicts least-recently-used entries immediately. Cache
 statistics are available through `regex_cache_entry_count()`,
 `regex_cache_compile_count()`, and `regex_cache_hit_count()`.
 
+### Script Call Depth
+
+Desktop and `no_std` VMs default to 1024 simultaneously active script call
+frames. Embedders can inspect or change the positive limit per VM:
+
+```rust
+let mut vm = Vm::new(program);
+assert_eq!(vm.max_script_call_depth(), 1024);
+vm.set_max_script_call_depth(256)?;
+```
+
+`pd-vm-run --max-call-depth 256 script.rss` applies the same limit from the
+CLI. A value of zero is rejected. Exceeding the configured limit returns
+`VmError::CallStackOverflow`.
+
 ### Fuel Metering
 
 `pd-vm` provides Wasmtime-style fuel controls on both `Vm` and `Store<T>`:
@@ -341,10 +357,10 @@ Browser playground wasm runtime is provided by sibling crate `pd-vm-wasm` built
 
 ### `no_std` Embedded Runtime
 
-The sibling crate [`pd-vm-nostd`](pd-vm-nostd) provides the VMBC v8 decoder and compact interpreter
-using only `core` and `alloc`. It supports direct bytecode execution, synchronous host callbacks,
-and instruction fuel while leaving source compilation, CLI, debugger, JIT/AOT, async host
-operations, and operating-system integrations in `pd-vm`.
+The sibling crate [`pd-vm-nostd`](pd-vm-nostd) provides the VMBC v9 decoder and compact interpreter
+using only `core` and `alloc`. It supports direct bytecode execution, script call frames and callable
+values, synchronous host callbacks, and instruction fuel while leaving source compilation, CLI,
+debugger, JIT/AOT, async host operations, and operating-system integrations in `pd-vm`.
 
 RP2040 compile check:
 
@@ -654,38 +670,39 @@ Directives:
 - `.local NAME [INDEX]` defines a named local
 
 
-#### Builtins and Bridged `call` Opcode
+#### Host Calls and Callable Values
 
-The compiler uses one call shape (`Expr::Call` -> `OpCode::Call`) and distinguishes targets by call
-index.
+The compiler keeps direct host dispatch separate from script callable dispatch.
 
 1. Builtin calls (fixed reserved indices)
-   - Builtins use `BuiltinFunction::call_index()`
-   - parser lowering emits these for helpers such as `len`, `get`, `set`, `slice`, `count`,
-     `type_of`, `assert`, and `io::*`/`re::*`/`json::*`/`jit::*`
+   - Builtins use `BuiltinFunction::call_index()`.
+   - Parser lowering emits these for helpers such as `len`, `get`, `set`, `slice`, `count`,
+     `type_of`, `assert`, and `io::*`/`re::*`/`json::*`/`jit::*`.
 2. Runtime host imports (per-program remapped indices)
-   - non-inlined runtime imports are remapped to dense import slots (`call_index_remap`)
-   - emitted as `call <slot>, <argc>`
-   - exposed as `Program.imports` and bound via `HostFunctionRegistry`
-   - `runtime::sleep(ms)` is available as a default host import; native runtimes block for the requested duration and wasm runtimes return `true` immediately
-3. Inlined RustScript function bodies
-   - calls to targets with `FunctionImpl` are inlined (no emitted `call`)
-
-At runtime, `call` is bridged through `Vm::execute_host_call`:
-
-- builtin call indices dispatch to `vm/builtin_runtime.rs`
-- non-builtin indices dispatch to bound host imports
-- trace-JIT records supported hot paths into SSA and falls back to the interpreter for call-heavy
-  or otherwise unsupported traces, preserving interpreter semantics
+   - Runtime imports are remapped to dense import slots (`call_index_remap`).
+   - Direct calls are emitted as `call <slot>, <argc>`.
+   - Host functions used as values are Program-owned callable prototypes and execute through
+     `callvalue <argc>`.
+3. RustScript function items and closures
+   - Every script body is emitted once in a function region.
+   - Named functions and closure evaluations produce callable values; script invocation uses
+     `callvalue <argc>` and a real execution frame.
+   - `ret` completes the active typed continuation: root halt, caller resume, or host return.
+
+At runtime, direct `call` uses `Vm::execute_host_call`, while `callvalue` validates the callable's
+prototype, schema, arity, and frame layout before dispatch. VMBC v9 carries the
+script-function, prototype, function-region, and root-binding tables. See
+[`docs/callable-runtime.md`](docs/callable-runtime.md) for the bytecode, lifecycle, callback, and
+optimized-backend contracts.
 
 #### Current Compiler Subset Limitations
 
 Core compiler/IR:
 
-- callable locals can be passed and called, but callables are not runtime `Value`s
-- callable values cannot currently be stored in arrays/maps or returned from functions
+- callable locals can be passed, returned, called, and stored as array or map values
+- callable values cannot be used as map keys or serialized as constants
 - callable return-type inference propagates through direct named calls, callable locals, closures,
-  and callable parameters when the compiler can still identify the callee
+  and callable parameters when the compiler can identify a compatible signature
 - known `if`/`else` expression results and branch-local merges with incompatible concrete types are
   compile errors
 - RustScript uses explicit nullable schemas such as `int?` and `Profile?`; non-optional declared
@@ -695,8 +712,8 @@ Core compiler/IR:
   that binds `Some(name)`
 - after optional handling, the compiler and wasm lint keep the concrete inner type instead of
   degrading back to `unknown`
-- recursive RustScript function declarations are not supported by current inlining-based lowering
-- function declarations can be nested and implicitly capture outer locals (closure-like snapshot at declaration time)
+- direct and mutual recursion are supported with a 1,024-frame script recursion limit
+- function declarations can be nested and implicitly capture outer locals at declaration time
 - in RustScript move-semantics mode, implicit captures follow expression semantics (`x` may move, `x.copy()` copies, `&x`/`&mut x` capture borrowed views)
 - `match` patterns are limited to int/string/null literals, `None`, `Some(name)`, `_`, and type constructors (`Some(TypeName)`)
 - `break` and `continue` are only valid inside loops

build.rs

@@ -494,7 +494,7 @@ fn render_builtin_catalog(
         .and_then(|value| value.checked_add(1))
         .expect("builtin call base should fit in u16");
     assert!(
-        builtin_call_base >= 14,
+        builtin_call_base >= 15,
         "builtin call base must leave room for reserved special builtins"
     );
 
@@ -667,6 +667,16 @@ fn render_builtin_catalog(
         "    (BUILTIN_CALL_BASE - 13, BuiltinFunction::MapIterClose),"
     )
     .unwrap();
+    writeln!(
+        &mut out,
+        "    (BUILTIN_CALL_BASE - 14, BuiltinFunction::BindCallable),"
+    )
+    .unwrap();
+    writeln!(
+        &mut out,
+        "    (BUILTIN_CALL_BASE - 15, BuiltinFunction::DetachLocal),"
+    )
+    .unwrap();
     writeln!(&mut out, "];").unwrap();
     writeln!(&mut out).unwrap();
 
@@ -860,6 +870,16 @@ fn render_builtin_catalog(
         "            BuiltinFunction::MapIterClose => BUILTIN_CALL_BASE - 13,"
     )
     .unwrap();
+    writeln!(
+        &mut out,
+        "            BuiltinFunction::BindCallable => BUILTIN_CALL_BASE - 14,"
+    )
+    .unwrap();
+    writeln!(
+        &mut out,
+        "            BuiltinFunction::DetachLocal => BUILTIN_CALL_BASE - 15,"
+    )
+    .unwrap();
     writeln!(
         &mut out,
         "            _ => BUILTIN_CALL_BASE + self as u16,"
@@ -1492,6 +1512,8 @@ fn appended_builtin_order() -> &'static [&'static str] {
         "__map_iter_take_key",
         "__map_iter_take_value",
         "__map_iter_close",
+        "__bind_callable",
+        "__detach_local",
     ]
 }
 
@@ -1518,7 +1540,12 @@ fn required_language_builtin_stubs() -> &'static [&'static str] {
 }
 
 fn required_internal_builtin_stubs() -> &'static [&'static str] {
-    &["__format_template", "__to_string"]
+    &[
+        "__format_template",
+        "__to_string",
+        "__bind_callable",
+        "__detach_local",
+    ]
 }
 
 fn is_language_builtin_stub_name(name: &str) -> bool {
@@ -1662,6 +1689,8 @@ fn main_range_builtin_variants(builtin_variant_order: &[String]) -> Vec<String>
                     | "MapIterTakeKey"
                     | "MapIterTakeValue"
                     | "MapIterClose"
+                    | "BindCallable"
+                    | "DetachLocal"
             )
         })
         .cloned()

docs/callable-runtime.md

@@ -0,0 +1,43 @@
+# Script call frames and callable values
+
+RustScript bytecode format version 9 introduces runtime script call frames and first-class callable values.
+
+## Bytecode contract
+
+- `call <import:u16> <argc:u8>` remains the direct host/builtin operation.
+- `callvalue <argc:u8>` consumes a stack segment in `callee, arg0, ..., argN` order.
+- callable environments are bound through the internal builtin call path; callable creation adds no bytecode opcode.
+- `ret` completes the active script frame. A nested frame leaves exactly one result at the caller segment base, using `null` when the body produced no value. Root `ret` keeps the historical program-result stack behavior.
+
+VMBC v9 is a hard format boundary. Decoders reject older versions. The stream includes script-function entry ranges, callable prototypes, function regions, and root callable bindings. PDRC v4 recordings and AOT artifacts use their corresponding bumped format/ABI versions and include callable metadata in cache identity.
+
+## Runtime model
+
+Each script invocation owns:
+
+- a typed continuation (`ResumeBytecode`, `ReturnToHost`, or root `Halt`);
+- operand-stack and local-stack bases;
+- frame-local count;
+- active prototype and callable identity.
+
+Arguments, captures, named callable bindings, and the self binding are installed before control moves to the function entry. Recursive calls therefore allocate independent local storage and are limited to 1,024 script frames.
+
+Branches are restricted to the active function region. Validation rejects cross-region targets before execution, and the interpreter repeats the check at runtime.
+
+## Callable identity and lifetime
+
+A callable contains its prototype ID, kind, and optional environment. The Program/Store owns the callable lifetime. Capture-free function items compare by prototype identity inside that Program; closures compare by runtime environment identity. Callable constants are forbidden; functions are initialized from Program metadata and closures are materialized at their declaration site.
+
+Reset clears Program runtime values and rebinds root function items from Program metadata. Program replacement cancels and releases the old Store callback registry before dropping the old Program. Raw callable values are Program-local and are not portable across Program/Store boundaries.
+
+`Vm::invoke_callable` is the synchronous host-entry API for a callable retained while the current Program is active. For resumable work, `Vm::start_callable` returns `VmStatus`; after `Yielded` or `Waiting`, continue with `Vm::resume` and read the completed value with `Vm::take_callable_result`. `ScriptCallback::start` and `Store::take_callback_result` expose the same flow for typed callbacks. `Store::script_callback` validates Store ownership, arity, and the copied callable schema and returns a typed `ScriptCallback<Args, Ret>` with no stale identity field. A callback can invoke directly, create a `Send` queued invocation on another thread, unsubscribe all aliases, or enter the Store FIFO through `enqueue_callback`; queue errors propagate and no implicit coalescing occurs. `Vm::shutdown` clears queued work, runtime values and host resources before invalidating every exported callback through the Store registry.
+
+PDRC recordings preserve full execution-frame metadata. Callable environments use identity-table encoding, so aliases still share one environment after decode.
+
+## Optimized backends
+
+Whole-program AOT and Trace JIT use the same builtin call path for environment binding and native frame dispatch for `callvalue`. Script-frame entry and return preserve frame-relative locals and typed continuations.
+
+## Embedded runtime
+
+`pd-vm-nostd` decodes the same VMBC v9 callable metadata and executes callable binding, `callvalue`, recursive frames, captures, and direct host targets using `core` plus `alloc`.

examples/mini_bench.rs

@@ -18,6 +18,7 @@ const DEFAULT_RUN_TRIALS: usize = 7;
 const DEFAULT_RSS_VM_COUNT: usize = 256;
 const DEFAULT_HOT_LOOP_INNER: i64 = 40_000;
 const DEFAULT_HOT_LOOP_OUTER: i64 = 8;
+const DEFAULT_CALLBACK_ITERS: usize = 50_000;
 const LOAD_HOST_COUNTS: [usize; 6] = [0, 1, 10, 50, 100, 500];
 
 fn main() {
@@ -34,11 +35,16 @@ fn real_main() -> Result<(), String> {
         println!("{}", sample.to_child_line());
         return Ok(());
     }
+    if config.callback_only {
+        benchmark_retained_callbacks(&config)?;
+        return Ok(());
+    }
 
     print_banner(&config);
     benchmark_compile(&config)?;
     benchmark_load(&config)?;
     benchmark_runtime(&config)?;
+    benchmark_retained_callbacks(&config)?;
     benchmark_rss(&config)?;
     Ok(())
 }
@@ -91,6 +97,8 @@ struct BenchConfig {
     rss_vm_count: usize,
     hot_loop_inner: i64,
     hot_loop_outer: i64,
+    callback_iters: usize,
+    callback_only: bool,
     rss_child_mode: Option<RssMode>,
 }
 
@@ -105,6 +113,8 @@ impl Default for BenchConfig {
             rss_vm_count: DEFAULT_RSS_VM_COUNT,
             hot_loop_inner: DEFAULT_HOT_LOOP_INNER,
             hot_loop_outer: DEFAULT_HOT_LOOP_OUTER,
+            callback_iters: DEFAULT_CALLBACK_ITERS,
+            callback_only: false,
             rss_child_mode: None,
         }
     }
@@ -141,6 +151,10 @@ impl BenchConfig {
                 "--hot-loop-outer" => {
                     config.hot_loop_outer = parse_i64_flag("--hot-loop-outer", args.next())?;
                 }
+                "--callback-iters" => {
+                    config.callback_iters = parse_usize_flag("--callback-iters", args.next())?;
+                }
+                "--callback-only" => config.callback_only = true,
                 "--rss-child" => {
                     let value = args
                         .next()
@@ -164,6 +178,7 @@ impl BenchConfig {
             || config.load_iters == 0
             || config.run_trials == 0
             || config.rss_vm_count == 0
+            || config.callback_iters == 0
         {
             return Err("iteration counts must be >= 1".to_string());
         }
@@ -199,12 +214,14 @@ fn print_help() {
     println!("  --rss-vms <n>");
     println!("  --hot-loop-inner <n>");
     println!("  --hot-loop-outer <n>");
+    println!("  --callback-iters <n>");
+    println!("  --callback-only");
 }
 
 fn print_banner(config: &BenchConfig) {
     println!("pd-vm mini benchmark platform");
     println!(
-        "config: compile_iters={} compile_stress_lines={} load_iters={} load_locals={} run_trials={} rss_vms={} hot_loop_inner={} hot_loop_outer={} native_jit_supported={}",
+        "config: compile_iters={} compile_stress_lines={} load_iters={} load_locals={} run_trials={} rss_vms={} hot_loop_inner={} hot_loop_outer={} callback_iters={} native_jit_supported={}",
         config.compile_iters,
         config.compile_stress_lines,
         config.load_iters,
@@ -213,6 +230,7 @@ fn print_banner(config: &BenchConfig) {
         config.rss_vm_count,
         config.hot_loop_inner,
         config.hot_loop_outer,
+        config.callback_iters,
         native_jit_supported()
     );
     println!();
@@ -322,6 +340,50 @@ fn benchmark_runtime(config: &BenchConfig) -> Result<(), String> {
     Ok(())
 }
 
+fn benchmark_retained_callbacks(config: &BenchConfig) -> Result<(), String> {
+    println!("[callback]");
+    let compiled = compile_source_with_flavor(
+        r#"
+            fn add_one(value: int) -> int { value + 1 }
+            add_one;
+        "#,
+        SourceFlavor::RustScript,
+    )
+    .map_err(|err| format!("callback compile failed: {err}"))?;
+    let mut vm = Vm::new(compiled.program);
+    let status = vm
+        .run()
+        .map_err(|err| format!("callback root run failed: {err}"))?;
+    if status != VmStatus::Halted {
+        return Err(format!("callback root returned {status:?}"));
+    }
+    let callable = vm
+        .stack()
+        .last()
+        .cloned()
+        .ok_or_else(|| "callback root did not return a callable".to_string())?;
+
+    let start = Instant::now();
+    for value in 0..config.callback_iters {
+        let expected = i64::try_from(value).map_err(|_| "callback iteration overflow")? + 1;
+        let result = vm
+            .invoke_callable(callable.clone(), &[Value::Int(expected - 1)])
+            .map_err(|err| format!("callback invocation failed: {err}"))?;
+        if result != Value::Int(expected) {
+            return Err(format!("callback returned {result:?}, expected {expected}"));
+        }
+    }
+    let elapsed = start.elapsed();
+    println!(
+        "  retained         iters={:<10} total_ms={:<10} ns_per_call={:.1}",
+        config.callback_iters,
+        elapsed.as_millis(),
+        elapsed.as_nanos() as f64 / config.callback_iters as f64,
+    );
+    println!();
+    Ok(())
+}
+
 fn benchmark_runtime_workload(
     label: &str,
     program: &Program,

pd-vm-nostd/README.md

@@ -6,8 +6,8 @@ compiler, parser, CLI, debugger, JIT/AOT backends, filesystem support, and opera
 
 ## Runtime surface
 
-- VMBC v8 decoding with type/debug metadata skipped after validation
-- stack and local execution for direct bytecode opcodes
+- VMBC v9 decoding with script-call and callable metadata
+- stack, local, and recursive script-frame execution for direct bytecode opcodes
 - instruction fuel with pause/resume support
 - synchronous named host bindings and dynamic host dispatch
 - `Rc`-backed strings, bytes, arrays, and maps for single-threaded targets