Skip to content

fix(compile) #1110 + chore(ffi) #1112: re-export-only FFI manifest collection; perry-ffi to crates.io - #1118

Merged
proggeramlug merged 4 commits into
mainfrom
worktree-issues-1110-1112-1113-1114
May 19, 2026
Merged

fix(compile) #1110 + chore(ffi) #1112: re-export-only FFI manifest collection; perry-ffi to crates.io#1118
proggeramlug merged 4 commits into
mainfrom
worktree-issues-1110-1112-1113-1114

Conversation

@proggeramlug

Copy link
Copy Markdown
Contributor

Summary

v0.5.1010 — two related fixes that fall out of trying to ship @perryts/storekit and similar scoped npm wrappers without a sibling Perry checkout.

Closes #1110, partially closes #1112 (final cargo publish is maintainer's step — needs crates.io credentials).

#1113 and #1114 surveyed but not fixed in this PR — both need substantial follow-up work and the latter needs the user's actual shop-admin repro. Both documented in CHANGELOG with concrete next steps.

#1110 in one paragraph

#1085 skipped the wrapper-symbol registration in import_function_prefixes for direct imports of FFI functions declared in perry.nativeLibrary.functions. But the manifest is only added to ctx.native_libraries in collect_modules.rs's import-walk (line ~700 / ~805) — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (a wrapper module does export { js_foo } from "@perryts/storekit", the entry imports from the wrapper) ended up with an empty ctx.native_libraries, so ffi_signatures was empty in every codegen pass, lower_call.rs's ffi_signatures.contains_key(name) returned false, and the early js_* direct-call branch emitted a call to the FFI symbol from the consumer's IR without pushing a matching declare external. clang then rejected the IR with use of undefined value '@js_storekit_get_jws'.

Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Plus a defensive change in lower_call.rs — when ffi_signatures knows the name, force the FFI-manifest path even if some other code path registers an entry in import_function_prefixes. Future re-export shapes that leak entries past the per-specifier skip in #1085 are now routed through the FFI path anyway.

#1112 in one paragraph

perry-ffi's only direct reference to perry-runtime::* was a set of #[cfg(all(test, feature = "runtime-link"))] layout-assertion tests. Moving perry-runtime to a path-only [dev-dependencies] entry (no version specifier) lets cargo publish strip it from the published manifest entirely. Layout tests keep working in-tree via the relative path. External wrappers leave the runtime-link feature off and don't pull perry-runtime in — Perry's compiler driver already links libperry_runtime.a into the final binary at compile-time. Added keywords / categories / readme + version on the workspace dep lines for both perry-runtime and perry-ffi. cargo publish --dry-run -p perry-ffi --allow-dirty now reaches Uploading perry-ffi v0.5.1010 ... aborting upload due to dry run.

Open after this PR

Test plan

  • reproduce the IR-verifier failure end-to-end: export { ... } from "@perryts/storekit" in wrap.ts, entry imports from ./wrap, await js_storekit_get_jws() inside an async fn → use of undefined value '@js_storekit_get_jws'.
  • post-fix: same setup produces a clean .o for both modules; main.ts.o references the plain manifest symbol _js_storekit_get_jws; wrap.ts.o no longer emits the perry_fn_wrap_ts__js_storekit_* wrapper externs at all.
  • sanity-check the simple-direct-import path still works (no regression from the defensive force_ffi_path change).
  • cargo build --release -p perry-runtime -p perry-stdlib -p perry clean.
  • cargo test --release -p perry-ffi -p perry-codegen passes.
  • cargo publish --dry-run -p perry-ffi --allow-dirty reaches the upload step.

@proggeramlug
proggeramlug force-pushed the worktree-issues-1110-1112-1113-1114 branch from f5fa33a to 449476d Compare May 19, 2026 13:45
…llection; perry-ffi to crates.io

v0.5.1010 — two issues, same root: shipping `@perryts/storekit` and
similar scoped npm wrappers without a sibling Perry checkout.

collection. `#1085` skipped the wrapper-symbol registration for direct
imports of FFI functions, but the manifest was only added to
`ctx.native_libraries` in `collect_modules.rs`'s import-walk —
never in the re-export-walk at line ~954. Programs that reach the
FFI-bearing package only through a re-export chain
(`export { js_foo } from "@perryts/storekit"` in a wrapper module)
ended up with an empty `ffi_signatures` map in every codegen pass,
fell through to direct-calling the FFI symbol without emitting a
matching `declare external`, and clang rejected the IR with
`use of undefined value '@js_storekit_get_jws'`.

Fix: mirror the per-kind manifest collection inside the
re-export-walk loop. Also defensively force the FFI-manifest path
in `lower_call.rs` whenever `ffi_signatures` knows the name —
guards against future re-export shapes leaking entries past the
per-specifier skip in `#1085`.

moves from optional dep to path-only dev-dependency (its only
references in `crates/perry-ffi/src/` were `#[cfg(test)]` layout
assertions). `runtime-link` feature kept as empty stub for the
existing `[dev-dependencies] features = ["runtime-link"]` lines in
every in-tree `perry-ext-*`. Adds keywords/categories/readme +
crates.io-version metadata on the workspace `perry-runtime` and
`perry-ffi` dep lines. `cargo publish --dry-run -p perry-ffi`
now reaches the upload step cleanly.

Adds `crates/perry-ffi/README.md` for the crates.io rendering.

EventEmitter needs hyper accept-loop wiring (substantial); the
GC-thrash regression bisect surfaced fastify non-blocking listen
(`634e1f58`) as the most likely candidate but needs the user's
actual shop-admin repro to land cleanly. Both documented in
CHANGELOG.
… loaded

User's actual /tmp/repro1110_real reduction (direct `import { ... }
from "@perryts/storekit"`) compiled clean — `main_ts.o` referenced
`_js_storekit_get_jws` correctly — but the LINK then failed:

  Undefined symbols for architecture arm64:
    "_perry_ffi_promise_new", referenced from:
        perry_ffi::async_runtime::JsPromise::new::h... in libperry_storekit.a[7](...)
    "_perry_ffi_promise_resolve_bits", referenced from:
        perry_ffi::async_runtime::JsPromise::resolve_string::h... in libperry_storekit.a[7](...)

The link command was `Linking (runtime-only)...` — no
libperry_stdlib.a passed to ld. The user's TS source didn't touch
anything that triggered `ctx.needs_stdlib = true` (console.log alone
doesn't). But every `returns: "promise"` manifest entry in a
nativeLibrary package compiles to a perry-ffi JsPromise::new() call
in the wrapper crate; those C-ABI shims (`perry_ffi_promise_*`,
`perry_ffi_spawn_*`) are *declared* in perry-ffi and *defined* in
`perry-stdlib::perry_ffi_async`. Without stdlib linked, the
references stayed undefined.

Force `ctx.needs_stdlib = true` whenever any `perry.nativeLibrary`
manifest is loaded. One-line change in compile.rs right after
`ffi_functions` is materialized.

Verified end-to-end against /tmp/repro1110_real — both the direct
import form (user's filing) and the re-export-chain shape from the
codegen half of #1110 now reach `Wrote executable: repro` and the
binary runs.
@proggeramlug
proggeramlug force-pushed the worktree-issues-1110-1112-1113-1114 branch from 449476d to b14031f Compare May 19, 2026 14:24
…fix); publish script

FastifyApp handle pointer-tagged so `typeof app.server === "object"`,
and `.on(event, cb)` dispatches through a new `js_fastify_app_on`
extern that stores upgrade callbacks in
`FastifyApp::upgrade_handlers`. GC scanner pins the closures across
cycles. Hyper accept loop emits a 501 + diagnostic when an `Upgrade:`
request arrives against a registered handler. Bidirectional WS
upgrade through hyper (the second half of the user's pattern with
`wss.handleUpgrade(req, socket, head, …)`) is documented as the
tracked follow-up — needs `hyper::upgrade::on(req)` wiring plus a
Node-compatible (req, socket, head) triple back to TS.

Codegen path: `Call { callee: PropertyGet { object: NativeMethodCall
{ module, … }, property: P }, args }` chains are forwarded into the
NATIVE_MODULE_TABLE arm for `(module, P)` whenever the table
recognises `P` as a method of `module`. Pre-PR the property read
returned undefined and `(undefined)(…)` returned NaN — no exception,
no callback registered. Scoped narrowly; falls back to the generic
Call lowering on a miss.

Mirror changes landed in both perry-stdlib's `crate::fastify`
(bundled-fastify path) and perry-ext-fastify (auto-optimize-flipped
path) since either can be live depending on whether the
well-known flip routed `import 'fastify'`.

maintainer to run `cargo publish -p perry-ffi --really-publish`
once they `cargo login`. Dry-run cleanly reaches the Uploading step.

cleanly. Bisect candidates documented in CHANGELOG.
Built two synthetic repros for the user's setInterval+MySQL CPU
wedge: a timer-only fastify shape (no DB) and a real-MySQL variant
against a local server (createPool + FOR UPDATE SKIP LOCKED).
Neither triggered the wedge — CPU stayed at 0%, healthz responded
in milliseconds. shop-admin's ~68-server-file shape is required to
trigger it; the minimal reductions don't.

Surveyed the diff between v0.5.1008 and v0.5.1009. The CHANGELOG
only credited the misaligned-pointer Object.assign fix, but
`git log --first-parent 0a90839..c71c780` shows 9 other on-main
commits in that range, all collapsing into the one v0.5.1009 release.
The fastify-flavoured candidates ranked by suspicion: 634e1f5
(non-blocking listen + main-thread pump) > 3856caa (transform
async early-return rewrite) > 5284700 (jsruntime V8-fallback CJS
require cycles) > 7e3bd5a (codegen Effect.succeed narrowing).
634e1f5 is the leading hypothesis given the symptom shape (madvise
hot suggests per-tick allocations; the pump now walks every
FastifyServerHandle on every main-pump tick via
iter_handle_ids_of).

Ships scripts/bisect_1114.sh — walks first-parent commits in the
range, rebuilds runtime+stdlib+perry at each step, runs the user's
binary ($PERRY_REPRO_CMD), and reports the first commit where CPU
exceeds the configured limit. The user can run this against their
actual shop-admin binary to localise the regression in ~15 minutes.

Notes in CHANGELOG. Actual fix needs the bad commit to be confirmed
against the real repro first.
@proggeramlug
proggeramlug force-pushed the worktree-issues-1110-1112-1113-1114 branch from b14031f to d5615bf Compare May 19, 2026 14:31
@proggeramlug
proggeramlug merged commit eaa6d72 into main May 19, 2026
9 checks passed
@proggeramlug
proggeramlug deleted the worktree-issues-1110-1112-1113-1114 branch May 19, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant