diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 0de76d93b..4b672d17d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -295,6 +295,14 @@ jobs: # path under `npm test`. - name: webjs SQLite busy_timeout on Bun run: bun test/bun/sqlite-busy-timeout.mjs + # Vendor specifier scan on Bun (#1399): the importmap is now derived by + # walking the module graph from the browser-bound entries, so an app + # scaffolded with `--runtime bun` runs the whole analysis under Bun. A set + # that drifted between runtimes would serve a Bun author a different + # importmap from the same source. Also asserts the runtime set stays a + # subset of the pin set, the relation pinned/unpinned parity rests on. + - name: Vendor specifier scan on Bun + run: bun test/bun/vendor-scan.mjs # The Bun test MATRIX (#509): run the runtime-sensitive node:test suite # (core + server + cross-package test/) under Bun, file by file, classifying # each result. Documented Node-only files + Bun-test-runner-quirk files are diff --git a/framework-dev.md b/framework-dev.md index d4c503259..ebc0fcf35 100644 --- a/framework-dev.md +++ b/framework-dev.md @@ -179,7 +179,7 @@ The scaffold gate is one of a FAMILY of tier-2 coverage gates that keep the fram ### The e2e's elision-off server resolves vendors from the repo (#1228) -`test/e2e/e2e.test.mjs`'s `differential elision (#181)` block runs the blog twice and asserts the two builds render identically. The two builds do not have the same network footprint, and that asymmetry is created by elision itself. Elision ON drops `components/vendor-badge.ts`, the blog's only vendor consumer, so `scanBareImports` finds nothing, `api.jspm.io` is never called, `dayjs` never enters the importmap, and the browser never contacts a third party. Elision OFF ships that component, so the same page acquires two live jspm dependencies: a blocking `api.jspm.io/generate` POST on the server's cold first request, and a `https://ga.jspm.io/...` module fetch inside `app/page.ts`'s graph in the browser. +`test/e2e/e2e.test.mjs`'s `differential elision (#181)` block runs the blog twice and asserts the two builds render identically. The two builds do not have the same network footprint, and that asymmetry is created by elision itself. Elision ON drops `components/vendor-badge.ts`, the blog's only vendor consumer, so the dev server's vendor scan (`reachedBareImports`, which prunes the elision skip set out of the graph walk) finds nothing, `api.jspm.io` is never called, `dayjs` never enters the importmap, and the browser never contacts a third party. Elision OFF ships that component, so the same page acquires two live jspm dependencies: a blocking `api.jspm.io/generate` POST on the server's cold first request, and a `https://ga.jspm.io/...` module fetch inside `app/page.ts`'s graph in the browser. An ES module graph instantiates as a unit, so a failure at either point means `app/page.ts` never evaluates and nothing it imports registers, including components whose own modules fetched fine. The visible symptom is `customElements.define` never running, which reads as an elision defect and is not one. That is what redded this block on and off from 2026-08-02. diff --git a/packages/server/AGENTS.md b/packages/server/AGENTS.md index 7f423583c..cc656b287 100644 --- a/packages/server/AGENTS.md +++ b/packages/server/AGENTS.md @@ -31,7 +31,7 @@ with metadata, Suspense, streaming) for HTML, or `api.js` / | File | What it owns | |---|---| | `dev.js` | The request handler. File serving, TypeScript stripping (Node 24+ built-in `module.stripTypeScriptTypes`, backed by the `amaro` package; non-erasable syntax fails at strip time with a 500), **server-file guardrail**, live reload via SSE. Also the observability seam (#239): `handle()` mints / honors the per-request id (`X-Request-Id` + `setRequestId`), emits the one-line structured access log via `logger.info` after the response (suppressing `/__webjs/*` probe traffic), and routes unhandled errors to the app's `onError` sink (best-effort, threaded into the SSR error path, the action endpoint, middleware, metadata, and the top-level catch); applies conditional GET (#240, via `applyConditionalGet`) as the final funnel step so every cacheable response gets an ETag + honors If-None-Match -> 304; commits the server HTML cache (#241, via `commitHtmlCache`) just before conditional-GET so the store decision sees the final post-middleware response (threads `cspEnabled` into the page `ssrOpts` so a CSP page is never HTML-cached); `produce()` answers the `/__webjs/version` build-info probe. **Root middleware resolution:** `loadMiddleware` tries `middleware.ts` / `.js` / `.mts` / `.mjs` beside `appDir` in that order (`.ts` first, matching the dev supervisor's watch order). It used to look up the single literal `middleware.js`, so a root `middleware.ts`, what the scaffold writes and what the docs document, was silently never loaded: no error, no warning, indistinguishable from an app with no middleware. Tests: `test/dev/root-middleware-resolution.test.js` plus the cross-runtime `test/bun/root-middleware.mjs`. Dev error overlay (#264): `reportDevError(error, info)` builds a frame (via `dev-error.js`) and pushes it to the open tab over the SSE channel as a `webjs-error` event, fed by three sources (the SSR render catch via `ssrOpts.onDevError`, the `tsResponse` strip-failure, and the `rebuild` catch); a successful rebuild clears `state.lastDevError`; the page-render branch stamps its frame with `withBasePath(url.pathname, basePath()) + url.search` so the browser gate can compare it against `location` (the RAW pathname, and the base path put back, since the ingress strip removed it), SKIPS the hook entirely for a request carrying `x-webjs-prefetch: 1` so a speculative link prefetch never raises an overlay or becomes the retained error, and after a successful GET render clears `state.lastDevError` when it is still the same object AND names this url, so the replay cannot resurrect a superseded error while a good render of an unrelated page leaves a still-current one standing (#1047); `reloadClientJs` also calls `installDevOverlayNavSync()` so the overlay tracks the page on screen; `startServer`'s SSE replays the current frame to a freshly-connected tab; `reloadClientJs` renders the dev-only plain-DOM overlay (`textContent` only). **Graceful reload (#893):** a reload never paints into a half-restarted server (Node's `node --watch` briefly kills the process on an app edit), so the client gates every reload on a `/__webjs/version` readiness probe (probe-then-reload, instant under an in-process reload, waits out a restart), and a reconnect after a drop is itself treated as an edit signal so an app edit whose in-process reload frame was killed with the old process still reloads without a manual refresh (the SSE `hello` carries a short `retry: 300` so the reconnect is prompt). The reconnect-reload lives in the `dev-reload-worker.js` relay (shared connection) and the per-tab `__webjsDirectEvents` fallback. **Extra watch roots (#894):** `startServer`'s recursive `fs.watch` also follows the dirs in `webjs.dev.watch` (`readDevWatchPathsFromApp`), for content the app reads from OUTSIDE its appDir (blog markdown in a repo-root `blog/`). Dev-only: `reportDevError` early-returns in prod and `/__webjs/reload.js` 404s. **Listener shell (#511):** `startServer` builds a shared `SseHub` + `ListenerContext`, then selects a shell by `serverRuntime()`: `startNodeListener` (in this file, the node:http path: `toWebRequest` -> `app.handle` -> `sendWebResponse`, 103 Early Hints, node WS via `attachWebSocket`, node:http timeouts) on Node, or the dynamically-imported `startBunListener` (`listener-bun.js`) on Bun. The SSE registry/fanout, the live-reload predicate, the WS module loader, and the lifecycle wiring live in `listener-core.js` so the two shells share them. `isCompressible` (used by `sendWebResponse`) also moved there | -| `router.js` | Scans `app/` once, builds the route table, matches pages + APIs (`buildRouteTable`, `matchPage`, `matchApi`). Page order is set by `compareSpecificity` (#750): POSITIONAL specificity (per URL segment, static `0` < dynamic `1` < catch-all `2` via `segKind`, lexicographic on the kind arrays with shorter-prefix-first), so the catch-all kind is lowest AT ITS POSITION (a literal-prefixed catch-all like `docs/[[...slug]]` outranks an all-dynamic `[org]/[repo]`), NOT a global catch-all-last bucket, then a stable alphabetical `routeDir` tiebreak. This replaces the old coarse 3-bucket `dynScore` whose same-bucket ties resolved by fs-walk order (so `/[org]/[repo]` vs `/[user]/settings` could match the wrong page). `matchPage` returns the first pattern that matches in that deterministic order | +| `router.js` | Scans `app/` once, builds the route table, matches pages + APIs (`buildRouteTable`, `matchPage`, `matchApi`). Page order is set by `compareSpecificity` (#750): POSITIONAL specificity (per URL segment, static `0` < dynamic `1` < catch-all `2` via `segKind`, lexicographic on the kind arrays with shorter-prefix-first), so the catch-all kind is lowest AT ITS POSITION (a literal-prefixed catch-all like `docs/[[...slug]]` outranks an all-dynamic `[org]/[repo]`), NOT a global catch-all-last bucket, then a stable alphabetical `routeDir` tiebreak. This replaces the old coarse 3-bucket `dynScore` whose same-bucket ties resolved by fs-walk order (so `/[org]/[repo]` vs `/[user]/settings` could match the wrong page). `matchPage` returns the first pattern that matches in that deterministic order. The table also carries the app-ROOT convention files that are not router stems: `instrumentationClient` (#1399, resolved here rather than by each caller, because a consumer that built its own table and forgot to attach it lost a browser-bound entry and inverted the vendor pin-is-a-superset invariant) | | `route-types.js` | Route-types generator (#258). `generateRouteTypes(appDir)` reuses `buildRouteTable` to emit the `.d.ts` text that augments `@webjsdev/core` (the `WebjsRoutes` href union + `RouteParamMap` per-route params), backing `webjs types` and the dev-startup emit. Pages-only (a `route.{js,ts}` API path is not a navigable href); strips route groups, excludes `_private`; an optional catch-all `[[...x]]` emits both the without-segment and a normalized `[...x]` href key while keeping the doubled literal as the param-map key. Deterministic (sorted keys). Helpers `routeKeyFromDir` / `dynamicSegments` / `paramTypeForKey` / `webjsRoutesKeysForKey` are exported for unit tests | | `ssr.js` | SSR pipeline: nested layouts, metadata → ``, Suspense streaming, error boundaries. `ssrPage` accepts `actionData` (put on `ctx.actionData` for the page + layouts) and `status` (default 200; the form-action re-render passes 422). Server HTML cache (#241): on a plain GET render it loads the page module once to read `export const revalidate`, serves a cache HIT via `cachedHtmlResponse` (re-minting the build id), and on a miss stamps the `HTML_CACHE_MARKER` so the funnel writes the final body. Skipped for the form-action re-render and partial-nav (`X-Webjs-Have`) requests. **Partial responses are never shared-cacheable (#1140):** a reduced `X-Webjs-Have` body is sliced by a request header, so `privateFragment(res)` rewrites its `Cache-Control` to `private` (stripping `public` / `s-maxage` / `proxy-revalidate`, and any qualified `private="field"`, which per RFC 9111 would leave the response shared-storable). `Vary: X-Webjs-Have` is still sent, but as belt-and-braces: it is not the guarantee, because Cloudflare and others honour only `Accept-Encoding`. The helper is quote-aware (a comma inside `no-cache="Set-Cookie,X-Foo"` is not a separator), fails CLOSED on an absent header (no `Cache-Control` is heuristically shared-storable), and is exported for unit testing (not re-exported from the package index, so it is not public API). The fragment KEEPS its ETag, which is why `conditional-get.js` no longer excludes `private`. A non-200 (the 422 form-action re-render, which carries the submitter's own field values) never inherits the page's `cacheControl`. **Frame subtree render (#253):** after `renderChain`, when the request carries `x-webjs-frame: ` (a `` self-load or a click-driven frame nav) AND the render is non-streamed, it extracts the matching `` subtree from the rendered body via `frame-render.js` and returns ONLY that (byte-equivalent to the client's extraction from a full page, but far fewer bytes); an absent frame id falls through to the full page (the client's `webjs:frame-missing` handles it), and a request with no `x-webjs-frame` header is byte-identical to before. **Vendor modulepreload (#754):** `reachedVendorSpecifiers(graph, shippedEntryFiles, componentUrls, appDir, elidable, serverFiles)` collects the bare specifiers (`bareImports(graph)`) reached by the page's SHIPPED modules. Its walk ROOTS are the boot's actually-shipped set: the caller passes the absolute paths of `moduleUrls` (which already drops INERT page/layout modules and substitutes an IMPORT-ONLY page with its components) plus the rendered `componentUrls`, then it walks the non-elided transitive closure and collects each reached file's bare imports, skipping server files (the `.server.*` suffix AND the `serverFiles` action index). Because the roots are the shipped set, a vendor reached ONLY through a dropped module, a dropped page's SSR-only DIRECT vendor import OR its SSR-only RELATIVE HELPER's vendor, is never collected (pages/layouts are not importable, so nothing that ships reaches them), so there is no over-fetch. `vendorPreloadTargets` (importmap.js) maps that set to `{ href, integrity }`, and `wrapHead` emits one `` per target (no `fp()` rewrite, deduped against the app module/component preloads, with `crossorigin` + `integrity`), flattening the cross-origin CDN waterfall one level. Only reached + non-elided + pinned vendors are hinted. **App-module modulepreload shipped-roots (#780):** `deduplicatedPreloads` (the app-module + component preload walk) roots at the SAME shipped set as the vendor walk. `shippedRoots` (the absolute paths of `moduleUrls`) is computed once and passed to BOTH `deduplicatedPreloads` and `reachedVendorSpecifiers`, NOT the raw `[route.file, ...route.layouts]`. So a module reached ONLY through a dropped page/layout (its SSR-only DIRECT app import OR its SSR-only RELATIVE HELPER) gets no `modulepreload` hint, the app-module analog of the #754 vendor over-fetch. A module that also ships another way (a component shared with a live route, or one reached through an import-only page's substituted components) is still reached via a real shipped root, so its hint stays (no under-fetch). `seen = new Set(moduleUrls)` already excludes the shipped page/layout URLs themselves, so this closes the TRANSITIVE gap only | | `dev-error.js` | Dev error overlay frame builder (#264). `buildDevErrorFrame(error, { kind, appDir, file?, line?, hint?, url? })` returns a JSON-serializable frame (message, parsed `file`/`line`/`column`, a source `codeFrame`, an optional `hint`, and an optional `url`); `parseStackLocation(stack, appDir)` finds the first app frame (preferring non-`node_modules`, splitting off the dev loader's `?t=` cache-bust query); `readCodeFrame(file, line, column)` reads the source excerpt with a `>` line marker + a caret. `url` (#1047) is the request URL that produced the frame, `null` when absent: only a `render` frame carries one, and it is what lets the browser overlay refuse a frame for a page the tab is not viewing. A `ts-strip` / `rebuild` frame passes none and stays unscoped, because it describes a still-broken build rather than one page. PURE (the only side effect is a guarded source read) and DEV-ONLY by the caller's contract, so no path / source is built in prod | @@ -80,9 +80,10 @@ with metadata, Suspense, streaming) for HTML, or `api.js` / | `sitemap.js` | Sitemap helpers (#276). `sitemap(entries)` serializes an array of `{ url, lastModified?, changeFrequency?, priority? }` into spec-valid `` XML (XML-escaping each url, formatting `lastModified` as a W3C datetime, validating priority 0..1 + the changefreq enum, skipping a urlless entry); `sitemapIndex(sitemaps)` builds the `` for sharding a site past the 50k-URL limit. Both pure + dependency-free; the `app/sitemap.{js,ts}` default export returns the string, which `dev.js` serves as `application/xml`. Exported from `index.js` | | `stream.js` | Server-side stream-action builders (#248). `stream.append/prepend/before/after/replace/update/remove(target, content?)` compose the `` HTML (one `