fix(compile): PerryTS/storekit#1 — route FFI imports through manifest path - #1085
Merged
Conversation
… path Imports from a package declaring `perry.nativeLibrary` (e.g. `@perryts/ storekit`) carry ambient `export declare function` signatures only — the real implementation lives in the linked static library. The compile-side import loop was unconditionally registering every named/default specifier in `import_function_prefixes`, which made `lower_call.rs` emit a call against `perry_fn_<src>__<name>` for symbols whose source module never emits that wrapper (no body to compile). The result was a clean compile followed by a link failure (`Undefined symbols: _perry_fn_..._js_storekit_ start_listener`), or — on older Perry builds — an LLVM IR verifier error when the call was emitted without a matching declaration. Skip the `import_function_prefixes` registration when the specifier's exported name is listed in the source package's `perry.nativeLibrary. functions` manifest. `lower_call.rs` then falls through to the FFI- manifest path (`ctx.ffi_signatures`), which emits a direct call to the declared FFI symbol along with a matching `declare external` at the top of the caller IR. Pre-computes the native-library lookup once per import (outside the per-specifier loop) so the check is constant-time.
proggeramlug
added a commit
that referenced
this pull request
May 19, 2026
…llection; perry-ffi to crates.io v0.5.1010 — two issues, same root: shipping `@perryts/storekit` and similar scoped npm wrappers without a sibling Perry checkout. collection. `#1085` skipped the wrapper-symbol registration for direct imports of FFI functions, but the manifest was only added to `ctx.native_libraries` in `collect_modules.rs`'s import-walk — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (`export { js_foo } from "@perryts/storekit"` in a wrapper module) ended up with an empty `ffi_signatures` map in every codegen pass, fell through to direct-calling the FFI symbol without emitting a matching `declare external`, and clang rejected the IR with `use of undefined value '@js_storekit_get_jws'`. Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Also defensively force the FFI-manifest path in `lower_call.rs` whenever `ffi_signatures` knows the name — guards against future re-export shapes leaking entries past the per-specifier skip in `#1085`. moves from optional dep to path-only dev-dependency (its only references in `crates/perry-ffi/src/` were `#[cfg(test)]` layout assertions). `runtime-link` feature kept as empty stub for the existing `[dev-dependencies] features = ["runtime-link"]` lines in every in-tree `perry-ext-*`. Adds keywords/categories/readme + crates.io-version metadata on the workspace `perry-runtime` and `perry-ffi` dep lines. `cargo publish --dry-run -p perry-ffi` now reaches the upload step cleanly. Adds `crates/perry-ffi/README.md` for the crates.io rendering. EventEmitter needs hyper accept-loop wiring (substantial); the GC-thrash regression bisect surfaced fastify non-blocking listen (`634e1f58`) as the most likely candidate but needs the user's actual shop-admin repro to land cleanly. Both documented in CHANGELOG.
proggeramlug
added a commit
that referenced
this pull request
May 19, 2026
…llection; perry-ffi to crates.io v0.5.1010 — two issues, same root: shipping `@perryts/storekit` and similar scoped npm wrappers without a sibling Perry checkout. collection. `#1085` skipped the wrapper-symbol registration for direct imports of FFI functions, but the manifest was only added to `ctx.native_libraries` in `collect_modules.rs`'s import-walk — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (`export { js_foo } from "@perryts/storekit"` in a wrapper module) ended up with an empty `ffi_signatures` map in every codegen pass, fell through to direct-calling the FFI symbol without emitting a matching `declare external`, and clang rejected the IR with `use of undefined value '@js_storekit_get_jws'`. Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Also defensively force the FFI-manifest path in `lower_call.rs` whenever `ffi_signatures` knows the name — guards against future re-export shapes leaking entries past the per-specifier skip in `#1085`. moves from optional dep to path-only dev-dependency (its only references in `crates/perry-ffi/src/` were `#[cfg(test)]` layout assertions). `runtime-link` feature kept as empty stub for the existing `[dev-dependencies] features = ["runtime-link"]` lines in every in-tree `perry-ext-*`. Adds keywords/categories/readme + crates.io-version metadata on the workspace `perry-runtime` and `perry-ffi` dep lines. `cargo publish --dry-run -p perry-ffi` now reaches the upload step cleanly. Adds `crates/perry-ffi/README.md` for the crates.io rendering. EventEmitter needs hyper accept-loop wiring (substantial); the GC-thrash regression bisect surfaced fastify non-blocking listen (`634e1f58`) as the most likely candidate but needs the user's actual shop-admin repro to land cleanly. Both documented in CHANGELOG.
proggeramlug
added a commit
that referenced
this pull request
May 19, 2026
…llection; perry-ffi to crates.io (#1118) * fix(compile) #1110 + chore(ffi) #1112: re-export-only FFI manifest collection; perry-ffi to crates.io v0.5.1010 — two issues, same root: shipping `@perryts/storekit` and similar scoped npm wrappers without a sibling Perry checkout. collection. `#1085` skipped the wrapper-symbol registration for direct imports of FFI functions, but the manifest was only added to `ctx.native_libraries` in `collect_modules.rs`'s import-walk — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (`export { js_foo } from "@perryts/storekit"` in a wrapper module) ended up with an empty `ffi_signatures` map in every codegen pass, fell through to direct-calling the FFI symbol without emitting a matching `declare external`, and clang rejected the IR with `use of undefined value '@js_storekit_get_jws'`. Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Also defensively force the FFI-manifest path in `lower_call.rs` whenever `ffi_signatures` knows the name — guards against future re-export shapes leaking entries past the per-specifier skip in `#1085`. moves from optional dep to path-only dev-dependency (its only references in `crates/perry-ffi/src/` were `#[cfg(test)]` layout assertions). `runtime-link` feature kept as empty stub for the existing `[dev-dependencies] features = ["runtime-link"]` lines in every in-tree `perry-ext-*`. Adds keywords/categories/readme + crates.io-version metadata on the workspace `perry-runtime` and `perry-ffi` dep lines. `cargo publish --dry-run -p perry-ffi` now reaches the upload step cleanly. Adds `crates/perry-ffi/README.md` for the crates.io rendering. EventEmitter needs hyper accept-loop wiring (substantial); the GC-thrash regression bisect surfaced fastify non-blocking listen (`634e1f58`) as the most likely candidate but needs the user's actual shop-admin repro to land cleanly. Both documented in CHANGELOG. * fix(compile) #1110 (also): force needs_stdlib when a nativeLibrary is loaded User's actual /tmp/repro1110_real reduction (direct `import { ... } from "@perryts/storekit"`) compiled clean — `main_ts.o` referenced `_js_storekit_get_jws` correctly — but the LINK then failed: Undefined symbols for architecture arm64: "_perry_ffi_promise_new", referenced from: perry_ffi::async_runtime::JsPromise::new::h... in libperry_storekit.a[7](...) "_perry_ffi_promise_resolve_bits", referenced from: perry_ffi::async_runtime::JsPromise::resolve_string::h... in libperry_storekit.a[7](...) The link command was `Linking (runtime-only)...` — no libperry_stdlib.a passed to ld. The user's TS source didn't touch anything that triggered `ctx.needs_stdlib = true` (console.log alone doesn't). But every `returns: "promise"` manifest entry in a nativeLibrary package compiles to a perry-ffi JsPromise::new() call in the wrapper crate; those C-ABI shims (`perry_ffi_promise_*`, `perry_ffi_spawn_*`) are *declared* in perry-ffi and *defined* in `perry-stdlib::perry_ffi_async`. Without stdlib linked, the references stayed undefined. Force `ctx.needs_stdlib = true` whenever any `perry.nativeLibrary` manifest is loaded. One-line change in compile.rs right after `ffi_functions` is materialized. Verified end-to-end against /tmp/repro1110_real — both the direct import form (user's filing) and the re-export-chain shape from the codegen half of #1110 now reach `Wrote executable: repro` and the binary runs. * fix(fastify) #1113 + chore(ffi) #1112: app.server EventEmitter (boot fix); publish script FastifyApp handle pointer-tagged so `typeof app.server === "object"`, and `.on(event, cb)` dispatches through a new `js_fastify_app_on` extern that stores upgrade callbacks in `FastifyApp::upgrade_handlers`. GC scanner pins the closures across cycles. Hyper accept loop emits a 501 + diagnostic when an `Upgrade:` request arrives against a registered handler. Bidirectional WS upgrade through hyper (the second half of the user's pattern with `wss.handleUpgrade(req, socket, head, …)`) is documented as the tracked follow-up — needs `hyper::upgrade::on(req)` wiring plus a Node-compatible (req, socket, head) triple back to TS. Codegen path: `Call { callee: PropertyGet { object: NativeMethodCall { module, … }, property: P }, args }` chains are forwarded into the NATIVE_MODULE_TABLE arm for `(module, P)` whenever the table recognises `P` as a method of `module`. Pre-PR the property read returned undefined and `(undefined)(…)` returned NaN — no exception, no callback registered. Scoped narrowly; falls back to the generic Call lowering on a miss. Mirror changes landed in both perry-stdlib's `crate::fastify` (bundled-fastify path) and perry-ext-fastify (auto-optimize-flipped path) since either can be live depending on whether the well-known flip routed `import 'fastify'`. maintainer to run `cargo publish -p perry-ffi --really-publish` once they `cargo login`. Dry-run cleanly reaches the Uploading step. cleanly. Bisect candidates documented in CHANGELOG. * chore(repro) #1114: bisect harness + investigation notes Built two synthetic repros for the user's setInterval+MySQL CPU wedge: a timer-only fastify shape (no DB) and a real-MySQL variant against a local server (createPool + FOR UPDATE SKIP LOCKED). Neither triggered the wedge — CPU stayed at 0%, healthz responded in milliseconds. shop-admin's ~68-server-file shape is required to trigger it; the minimal reductions don't. Surveyed the diff between v0.5.1008 and v0.5.1009. The CHANGELOG only credited the misaligned-pointer Object.assign fix, but `git log --first-parent 0a90839..c71c780` shows 9 other on-main commits in that range, all collapsing into the one v0.5.1009 release. The fastify-flavoured candidates ranked by suspicion: 634e1f5 (non-blocking listen + main-thread pump) > 3856caa (transform async early-return rewrite) > 5284700 (jsruntime V8-fallback CJS require cycles) > 7e3bd5a (codegen Effect.succeed narrowing). 634e1f5 is the leading hypothesis given the symptom shape (madvise hot suggests per-tick allocations; the pump now walks every FastifyServerHandle on every main-pump tick via iter_handle_ids_of). Ships scripts/bisect_1114.sh — walks first-parent commits in the range, rebuilds runtime+stdlib+perry at each step, runs the user's binary ($PERRY_REPRO_CMD), and reports the first commit where CPU exceeds the configured limit. The user can run this against their actual shop-admin binary to localise the regression in ~15 minutes. Notes in CHANGELOG. Actual fix needs the bad commit to be confirmed against the real repro first.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes PerryTS/storekit#1. Imports from a package declaring
perry.nativeLibrary(e.g.@perryts/storekit) carry ambientexport declare functionsignatures only — the real implementation lives in the linked static library. The compile-side import loop was unconditionally registering every named/default specifier inimport_function_prefixes, which madelower_call.rsemit a call againstperry_fn_<src>__<name>for symbols whose source module never emits that wrapper (no body to compile). Result: link failure (Undefined symbols: _perry_fn_..._js_storekit_start_listener), or — on older Perry builds where the codegen path differed — an LLVM IR verifier error reportinguse of undefined value '@js_storekit_*'because the call was emitted with no matchingdeclare.Fix: when the import source matches a
perry.nativeLibrarypackage and the specifier's exported name is listed inperry.nativeLibrary.functions, skip theimport_function_prefixesregistration.lower_call.rsthen falls through to the FFI-manifest path (ctx.ffi_signatures), which emits a direct call to the declared FFI symbol along with a matchingdeclare externalat the top of the caller IR. The native-library lookup is pre-computed once per import (outside the per-specifier loop) so the check is constant-time.Before / after
main.ts:Before:
After (caller IR):
For the 6-symbol smoke test (all params/returns shapes in storekit's manifest):
All declares match the calls — types come from the FFI-manifest path in
lower_call.rs(which is whatffi_signatures-driven calls were always supposed to use).Test plan
cargo test --release -p perry --bin perry→ 332 passed, 0 failedimport { js_storekit_start_listener } from \"@perryts/storekit\"; js_storekit_start_listener();) — compiles and runs (exits 0) instead of failing at linkdeclare externalnext to itscallperry-searchbird-apple-auth) to confirm no regression on the path that already worked