fix(compile) #1110 + chore(ffi) #1112: re-export-only FFI manifest collection; perry-ffi to crates.io - #1118
Merged
Merged
Conversation
proggeramlug
force-pushed
the
worktree-issues-1110-1112-1113-1114
branch
from
May 19, 2026 13:45
f5fa33a to
449476d
Compare
…llection; perry-ffi to crates.io v0.5.1010 — two issues, same root: shipping `@perryts/storekit` and similar scoped npm wrappers without a sibling Perry checkout. collection. `#1085` skipped the wrapper-symbol registration for direct imports of FFI functions, but the manifest was only added to `ctx.native_libraries` in `collect_modules.rs`'s import-walk — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (`export { js_foo } from "@perryts/storekit"` in a wrapper module) ended up with an empty `ffi_signatures` map in every codegen pass, fell through to direct-calling the FFI symbol without emitting a matching `declare external`, and clang rejected the IR with `use of undefined value '@js_storekit_get_jws'`. Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Also defensively force the FFI-manifest path in `lower_call.rs` whenever `ffi_signatures` knows the name — guards against future re-export shapes leaking entries past the per-specifier skip in `#1085`. moves from optional dep to path-only dev-dependency (its only references in `crates/perry-ffi/src/` were `#[cfg(test)]` layout assertions). `runtime-link` feature kept as empty stub for the existing `[dev-dependencies] features = ["runtime-link"]` lines in every in-tree `perry-ext-*`. Adds keywords/categories/readme + crates.io-version metadata on the workspace `perry-runtime` and `perry-ffi` dep lines. `cargo publish --dry-run -p perry-ffi` now reaches the upload step cleanly. Adds `crates/perry-ffi/README.md` for the crates.io rendering. EventEmitter needs hyper accept-loop wiring (substantial); the GC-thrash regression bisect surfaced fastify non-blocking listen (`634e1f58`) as the most likely candidate but needs the user's actual shop-admin repro to land cleanly. Both documented in CHANGELOG.
… loaded
User's actual /tmp/repro1110_real reduction (direct `import { ... }
from "@perryts/storekit"`) compiled clean — `main_ts.o` referenced
`_js_storekit_get_jws` correctly — but the LINK then failed:
Undefined symbols for architecture arm64:
"_perry_ffi_promise_new", referenced from:
perry_ffi::async_runtime::JsPromise::new::h... in libperry_storekit.a[7](...)
"_perry_ffi_promise_resolve_bits", referenced from:
perry_ffi::async_runtime::JsPromise::resolve_string::h... in libperry_storekit.a[7](...)
The link command was `Linking (runtime-only)...` — no
libperry_stdlib.a passed to ld. The user's TS source didn't touch
anything that triggered `ctx.needs_stdlib = true` (console.log alone
doesn't). But every `returns: "promise"` manifest entry in a
nativeLibrary package compiles to a perry-ffi JsPromise::new() call
in the wrapper crate; those C-ABI shims (`perry_ffi_promise_*`,
`perry_ffi_spawn_*`) are *declared* in perry-ffi and *defined* in
`perry-stdlib::perry_ffi_async`. Without stdlib linked, the
references stayed undefined.
Force `ctx.needs_stdlib = true` whenever any `perry.nativeLibrary`
manifest is loaded. One-line change in compile.rs right after
`ffi_functions` is materialized.
Verified end-to-end against /tmp/repro1110_real — both the direct
import form (user's filing) and the re-export-chain shape from the
codegen half of #1110 now reach `Wrote executable: repro` and the
binary runs.
proggeramlug
force-pushed
the
worktree-issues-1110-1112-1113-1114
branch
from
May 19, 2026 14:24
449476d to
b14031f
Compare
…fix); publish script
FastifyApp handle pointer-tagged so `typeof app.server === "object"`,
and `.on(event, cb)` dispatches through a new `js_fastify_app_on`
extern that stores upgrade callbacks in
`FastifyApp::upgrade_handlers`. GC scanner pins the closures across
cycles. Hyper accept loop emits a 501 + diagnostic when an `Upgrade:`
request arrives against a registered handler. Bidirectional WS
upgrade through hyper (the second half of the user's pattern with
`wss.handleUpgrade(req, socket, head, …)`) is documented as the
tracked follow-up — needs `hyper::upgrade::on(req)` wiring plus a
Node-compatible (req, socket, head) triple back to TS.
Codegen path: `Call { callee: PropertyGet { object: NativeMethodCall
{ module, … }, property: P }, args }` chains are forwarded into the
NATIVE_MODULE_TABLE arm for `(module, P)` whenever the table
recognises `P` as a method of `module`. Pre-PR the property read
returned undefined and `(undefined)(…)` returned NaN — no exception,
no callback registered. Scoped narrowly; falls back to the generic
Call lowering on a miss.
Mirror changes landed in both perry-stdlib's `crate::fastify`
(bundled-fastify path) and perry-ext-fastify (auto-optimize-flipped
path) since either can be live depending on whether the
well-known flip routed `import 'fastify'`.
maintainer to run `cargo publish -p perry-ffi --really-publish`
once they `cargo login`. Dry-run cleanly reaches the Uploading step.
cleanly. Bisect candidates documented in CHANGELOG.
Built two synthetic repros for the user's setInterval+MySQL CPU wedge: a timer-only fastify shape (no DB) and a real-MySQL variant against a local server (createPool + FOR UPDATE SKIP LOCKED). Neither triggered the wedge — CPU stayed at 0%, healthz responded in milliseconds. shop-admin's ~68-server-file shape is required to trigger it; the minimal reductions don't. Surveyed the diff between v0.5.1008 and v0.5.1009. The CHANGELOG only credited the misaligned-pointer Object.assign fix, but `git log --first-parent 0a90839..c71c780` shows 9 other on-main commits in that range, all collapsing into the one v0.5.1009 release. The fastify-flavoured candidates ranked by suspicion: 634e1f5 (non-blocking listen + main-thread pump) > 3856caa (transform async early-return rewrite) > 5284700 (jsruntime V8-fallback CJS require cycles) > 7e3bd5a (codegen Effect.succeed narrowing). 634e1f5 is the leading hypothesis given the symptom shape (madvise hot suggests per-tick allocations; the pump now walks every FastifyServerHandle on every main-pump tick via iter_handle_ids_of). Ships scripts/bisect_1114.sh — walks first-parent commits in the range, rebuilds runtime+stdlib+perry at each step, runs the user's binary ($PERRY_REPRO_CMD), and reports the first commit where CPU exceeds the configured limit. The user can run this against their actual shop-admin binary to localise the regression in ~15 minutes. Notes in CHANGELOG. Actual fix needs the bad commit to be confirmed against the real repro first.
proggeramlug
force-pushed
the
worktree-issues-1110-1112-1113-1114
branch
from
May 19, 2026 14:31
b14031f to
d5615bf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v0.5.1010 — two related fixes that fall out of trying to ship
@perryts/storekitand similar scoped npm wrappers without a sibling Perry checkout.perry.nativeLibrarymanifest collection incollect_modules.rs, leaving every downstream module'sffi_signaturesempty and producinguse of undefined value '@js_storekit_*'LLVM-verifier failures.perry-ffiis now publishable to crates.io.cargo publish --dry-run -p perry-ffireaches the upload step cleanly.Closes #1110, partially closes #1112 (final
cargo publishis maintainer's step — needs crates.io credentials).#1113 and #1114 surveyed but not fixed in this PR — both need substantial follow-up work and the latter needs the user's actual shop-admin repro. Both documented in CHANGELOG with concrete next steps.
#1110 in one paragraph
#1085skipped the wrapper-symbol registration inimport_function_prefixesfor direct imports of FFI functions declared inperry.nativeLibrary.functions. But the manifest is only added toctx.native_librariesincollect_modules.rs's import-walk (line ~700 / ~805) — never in the re-export-walk at line ~954. Programs that reach the FFI-bearing package only through a re-export chain (a wrapper module doesexport { js_foo } from "@perryts/storekit", the entry imports from the wrapper) ended up with an emptyctx.native_libraries, soffi_signatureswas empty in every codegen pass,lower_call.rs'sffi_signatures.contains_key(name)returnedfalse, and the earlyjs_*direct-call branch emitted a call to the FFI symbol from the consumer's IR without pushing a matchingdeclare external. clang then rejected the IR withuse of undefined value '@js_storekit_get_jws'.Fix: mirror the per-kind manifest collection inside the re-export-walk loop. Plus a defensive change in
lower_call.rs— whenffi_signaturesknows the name, force the FFI-manifest path even if some other code path registers an entry inimport_function_prefixes. Future re-export shapes that leak entries past the per-specifier skip in#1085are now routed through the FFI path anyway.#1112 in one paragraph
perry-ffi's only direct reference toperry-runtime::*was a set of#[cfg(all(test, feature = "runtime-link"))]layout-assertion tests. Movingperry-runtimeto a path-only[dev-dependencies]entry (noversionspecifier) letscargo publishstrip it from the published manifest entirely. Layout tests keep working in-tree via the relative path. External wrappers leave theruntime-linkfeature off and don't pullperry-runtimein — Perry's compiler driver already linkslibperry_runtime.ainto the final binary at compile-time. Addedkeywords/categories/readme+versionon the workspace dep lines for bothperry-runtimeandperry-ffi.cargo publish --dry-run -p perry-ffi --allow-dirtynow reachesUploading perry-ffi v0.5.1010 ... aborting upload due to dry run.Open after this PR
FastifyInstance.serverreturns a number rather than anEventEmitterhandle, blocking the canonical fastify+ws upgrade-piggybacking pattern. Fix requires either wiringapp.serverto an EventEmitter-shaped handle (with.on("upgrade", cb)registering a callback that hyper's accept loop invokes when the upgrade headers match) or a new method-dispatch arm. Documented as a "Punted gaps — WebSocket upgrade" item incrates/perry-ext-fastify/src/lib.rsalready.setInterval+ async@perryts/mysqlquery loop wedges the runtime withmadvisehot in profile, regressed v0.5.1008 → v0.5.1009. The user's minimal repro didn't trigger it; shop-admin's ~68-server-file shape is too specific to recreate from scratch. Bisect candidates (most fastify-flavoured first):634e1f58(fastify non-blocking listen + main-thread pump),7e3bd5a4(codegen Tracking: Effect framework end-to-end compat (post-#309 / #310) #321 effect succeed via named-import-of-namespace-reexport),3856caad(transform async fn early-return followed by an unreached await infinite-loops #1047 async early return + unreached await),52847008(jsruntime compat: async lowering blocks self-fetch (express/fastify/nestjs/hono) #1021 V8-fallback CJS require cycles + process.exit shim). Needs the user's actual repro to land cleanly.Test plan
export { ... } from "@perryts/storekit"inwrap.ts, entry imports from./wrap,await js_storekit_get_jws()inside an async fn →use of undefined value '@js_storekit_get_jws'..ofor both modules;main.ts.oreferences the plain manifest symbol_js_storekit_get_jws;wrap.ts.ono longer emits theperry_fn_wrap_ts__js_storekit_*wrapper externs at all.force_ffi_pathchange).cargo build --release -p perry-runtime -p perry-stdlib -p perryclean.cargo test --release -p perry-ffi -p perry-codegenpasses.cargo publish --dry-run -p perry-ffi --allow-dirtyreaches the upload step.