From c725ac994c9c0124b683330068d2a6b6a0d6a682 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 18:37:34 +0530 Subject: [PATCH 01/13] fix: stop rapid dev edits leaving the page unstyled Editing several files a few seconds apart in dev left the browser showing unstyled HTML until a manual refresh. Two independent causes, both fixed here. Each save produces TWO reload signals, not one: the in-process rebuild frame, then a changed boot id when the browser reconnects to the process node --watch restarted (measured 429ms apart, with 1071ms between saves). Acting on every one reloads into a server that is about to be killed again. Both signals now route through one debounced emitter in the reload relay, which fires once the signals stop for 2 seconds or at the latest 5 seconds into a sustained burst, so the page never freezes on stale content either. The cap timer is armed once per batch and never re-armed, so it measures from the first signal instead of sliding with the burst. Error frames are not debounced, since an overlay has to appear at once. The per-tab fallback now runs the same relay over a shim port rather than a second copy of the boot-id rule. The stylesheet is the visible casualty because a failed stylesheet request is non-fatal to the browser and /public/tailwind.css is the slowest request on the page: it waited on ensureReady(), measured at 1907ms cold on the website app. In dev, /public/* is now served ahead of the analysis, since a static asset needs neither the module graph nor the vendor importmap. Dev only, because in prod /__webjs/ready already holds traffic off a cold instance and hoisting there would un-gate a file an app middleware protects. Both call sites share one function, so the traversal guard exists in exactly one place. Closes #1397 --- .github/workflows/ci.yml | 8 +- framework-dev.md | 2 + packages/server/AGENTS.md | 18 +- packages/server/src/dev-reload-worker.js | 65 +++++- packages/server/src/dev.js | 146 +++++++++----- .../test/dev/browser/reload-worker.test.js | 186 +++++++++++++++++- .../test/dev/public-before-analysis.test.js | 161 +++++++++++++++ .../test/dev/reload-shared-connection.test.js | 33 ++-- test/bun/dev-public-before-warm.mjs | 115 +++++++++++ test/bun/dev-public-before-warm.test.mjs | 13 ++ website/app/docs/middleware/page.ts | 3 +- 11 files changed, 673 insertions(+), 77 deletions(-) create mode 100644 packages/server/test/dev/public-before-analysis.test.js create mode 100644 test/bun/dev-public-before-warm.mjs create mode 100644 test/bun/dev-public-before-warm.test.mjs diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 4b672d17d..592ae561d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -198,11 +198,17 @@ jobs: # replays the retained dev error frame with its own code (node writes into # a node:http res, Bun enqueues into a ReadableStream controller), so the # frame's new url field has to be proven on both (#1047). - - name: webjs dev extra-watch + reload-retry + overlay-scope on Bun + # dev-public-before-warm rides here too: the #1397 hoist serves /public/* + # ahead of ensureReady() in dev, and the two listener shells reach + # handle() differently (the Bun shell may rebuild the Request and + # classifies the body through readBufferedOrStream), so the ordering has + # to be proven on both. + - name: webjs dev extra-watch + reload-retry + overlay-scope + public-before-warm on Bun run: | bun test/bun/dev-extra-watch.mjs bun test/bun/dev-reload-retry.mjs bun test/bun/dev-overlay-scope.mjs + bun test/bun/dev-public-before-warm.mjs # The app-source deploy signal (#899) is derived from an fs source walk + # a node:crypto digest, so it must be byte-identical on the Bun.serve path. - name: App-source deploy signal on Bun diff --git a/framework-dev.md b/framework-dev.md index ebc0fcf35..c143701f0 100644 --- a/framework-dev.md +++ b/framework-dev.md @@ -256,4 +256,6 @@ In development, three error sources push a structured error frame to the open ta **A render frame is scoped to the URL that produced it (#1047).** A `render` frame carries that url, and the browser half is the single gate deciding whether a frame belongs on the page currently being viewed, so an overlay comes down when the client router navigates away and never goes up for someone else's page. Three consequences worth knowing. A speculative link PREFETCH of a throwing page reports no frame at all, so hovering a link cannot break the page you are looking at (the reported symptom: the frame fans out to every open tab over the shared SSE channel, with no navigation anywhere). A render error in one tab raises nothing in a tab viewing a different page. And a successful render of a url supersedes a retained error for that SAME url, so the replay cannot hand a recovered error to a freshly-connected tab; a good render of an unrelated page deliberately leaves it standing. `ts-strip` and `rebuild` frames carry NO url and are never scoped, because they describe a still-broken build rather than one page, so navigation leaves them alone and only the next successful rebuild clears them. The gate is order-independent: the SSE frame is pushed during the render, before the navigation response is even sent, so a frame for the page being navigated TO is held and rendered once the URL advances. A held frame renders only for the navigation it belongs to, so one that arrived while the tab sat idle is dropped rather than painted on a later visit, when the page may well render fine. An idle-time frame comes from a render this tab did not navigate for (another tab's page, a background fetch of some other url), never from a link prefetch, which reports nothing at all. Mechanism: `renderDevOverlay` + `syncDevOverlayToLocation` + `installDevOverlayNavSync` in `packages/server/src/dev-overlay.js`, wired by the dev reload client to the client router's `webjs:navigate` and `popstate` (a navigation finished) plus `webjs:before-cache` (a navigation STARTED, since the router snapshots the page it is leaving first). That last one also detaches the overlay across the snapshot read and re-attaches it a microtask later, so the cached HTML never carries a copy the module does not own; what it must not do is strip the overlay for good, which would tear a `rebuild` overlay off the page on any link click. `packages/core` is untouched by any of it. +**A reload is coalesced (#1397), so after a burst of edits the page reloads once the edits settle** rather than once per saved file. Each save produces two reload signals (the in-process rebuild frame, then a changed boot id when the browser reconnects to the process `node --watch` restarted), and acting on every one reloads into a server about to be killed again, which is what leaves the page unstyled. The relay holds the reload until the signals stop for 2 seconds, or at most 5 seconds into a sustained burst. So if a reload looks "missing" right after you saved, wait two seconds before looking for a bug: that is the first thing to check. An error overlay is never held, since it is not a reload. + The overlay client uses `textContent` throughout (never `innerHTML`), so the error content cannot inject markup. It is **strictly dev-only**: `reportDevError` early-returns when `!dev`, `/__webjs/reload.js` 404s in prod, and the prod 500 stays terse (only `error.message`, never the stack or a file path), so no source leaks. An embedding host can observe the same frames via the `onDevError` option on `createRequestHandler` / `startServer`. Mechanism: `buildDevErrorFrame` in `packages/server/src/dev-error.js`, `reportDevError` + the SSE push in `packages/server/src/dev.js`, the SSR-catch hook in `packages/server/src/ssr.js`. diff --git a/packages/server/AGENTS.md b/packages/server/AGENTS.md index cc656b287..086ae9cf3 100644 --- a/packages/server/AGENTS.md +++ b/packages/server/AGENTS.md @@ -30,7 +30,7 @@ with metadata, Suspense, streaming) for HTML, or `api.js` / | File | What it owns | |---|---| -| `dev.js` | The request handler. File serving, TypeScript stripping (Node 24+ built-in `module.stripTypeScriptTypes`, backed by the `amaro` package; non-erasable syntax fails at strip time with a 500), **server-file guardrail**, live reload via SSE. Also the observability seam (#239): `handle()` mints / honors the per-request id (`X-Request-Id` + `setRequestId`), emits the one-line structured access log via `logger.info` after the response (suppressing `/__webjs/*` probe traffic), and routes unhandled errors to the app's `onError` sink (best-effort, threaded into the SSR error path, the action endpoint, middleware, metadata, and the top-level catch); applies conditional GET (#240, via `applyConditionalGet`) as the final funnel step so every cacheable response gets an ETag + honors If-None-Match -> 304; commits the server HTML cache (#241, via `commitHtmlCache`) just before conditional-GET so the store decision sees the final post-middleware response (threads `cspEnabled` into the page `ssrOpts` so a CSP page is never HTML-cached); `produce()` answers the `/__webjs/version` build-info probe. **Root middleware resolution:** `loadMiddleware` tries `middleware.ts` / `.js` / `.mts` / `.mjs` beside `appDir` in that order (`.ts` first, matching the dev supervisor's watch order). It used to look up the single literal `middleware.js`, so a root `middleware.ts`, what the scaffold writes and what the docs document, was silently never loaded: no error, no warning, indistinguishable from an app with no middleware. Tests: `test/dev/root-middleware-resolution.test.js` plus the cross-runtime `test/bun/root-middleware.mjs`. Dev error overlay (#264): `reportDevError(error, info)` builds a frame (via `dev-error.js`) and pushes it to the open tab over the SSE channel as a `webjs-error` event, fed by three sources (the SSR render catch via `ssrOpts.onDevError`, the `tsResponse` strip-failure, and the `rebuild` catch); a successful rebuild clears `state.lastDevError`; the page-render branch stamps its frame with `withBasePath(url.pathname, basePath()) + url.search` so the browser gate can compare it against `location` (the RAW pathname, and the base path put back, since the ingress strip removed it), SKIPS the hook entirely for a request carrying `x-webjs-prefetch: 1` so a speculative link prefetch never raises an overlay or becomes the retained error, and after a successful GET render clears `state.lastDevError` when it is still the same object AND names this url, so the replay cannot resurrect a superseded error while a good render of an unrelated page leaves a still-current one standing (#1047); `reloadClientJs` also calls `installDevOverlayNavSync()` so the overlay tracks the page on screen; `startServer`'s SSE replays the current frame to a freshly-connected tab; `reloadClientJs` renders the dev-only plain-DOM overlay (`textContent` only). **Graceful reload (#893):** a reload never paints into a half-restarted server (Node's `node --watch` briefly kills the process on an app edit), so the client gates every reload on a `/__webjs/version` readiness probe (probe-then-reload, instant under an in-process reload, waits out a restart), and a reconnect after a drop is itself treated as an edit signal so an app edit whose in-process reload frame was killed with the old process still reloads without a manual refresh (the SSE `hello` carries a short `retry: 300` so the reconnect is prompt). The reconnect-reload lives in the `dev-reload-worker.js` relay (shared connection) and the per-tab `__webjsDirectEvents` fallback. **Extra watch roots (#894):** `startServer`'s recursive `fs.watch` also follows the dirs in `webjs.dev.watch` (`readDevWatchPathsFromApp`), for content the app reads from OUTSIDE its appDir (blog markdown in a repo-root `blog/`). Dev-only: `reportDevError` early-returns in prod and `/__webjs/reload.js` 404s. **Listener shell (#511):** `startServer` builds a shared `SseHub` + `ListenerContext`, then selects a shell by `serverRuntime()`: `startNodeListener` (in this file, the node:http path: `toWebRequest` -> `app.handle` -> `sendWebResponse`, 103 Early Hints, node WS via `attachWebSocket`, node:http timeouts) on Node, or the dynamically-imported `startBunListener` (`listener-bun.js`) on Bun. The SSE registry/fanout, the live-reload predicate, the WS module loader, and the lifecycle wiring live in `listener-core.js` so the two shells share them. `isCompressible` (used by `sendWebResponse`) also moved there | +| `dev.js` | The request handler. File serving, TypeScript stripping (Node 24+ built-in `module.stripTypeScriptTypes`, backed by the `amaro` package; non-erasable syntax fails at strip time with a 500), **server-file guardrail**, live reload via SSE. Also the observability seam (#239): `handle()` mints / honors the per-request id (`X-Request-Id` + `setRequestId`), emits the one-line structured access log via `logger.info` after the response (suppressing `/__webjs/*` probe traffic), and routes unhandled errors to the app's `onError` sink (best-effort, threaded into the SSR error path, the action endpoint, middleware, metadata, and the top-level catch); applies conditional GET (#240, via `applyConditionalGet`) as the final funnel step so every cacheable response gets an ETag + honors If-None-Match -> 304; commits the server HTML cache (#241, via `commitHtmlCache`) just before conditional-GET so the store decision sees the final post-middleware response (threads `cspEnabled` into the page `ssrOpts` so a CSP page is never HTML-cached); `produce()` answers the `/__webjs/version` build-info probe. **Root middleware resolution:** `loadMiddleware` tries `middleware.ts` / `.js` / `.mts` / `.mjs` beside `appDir` in that order (`.ts` first, matching the dev supervisor's watch order). It used to look up the single literal `middleware.js`, so a root `middleware.ts`, what the scaffold writes and what the docs document, was silently never loaded: no error, no warning, indistinguishable from an app with no middleware. Tests: `test/dev/root-middleware-resolution.test.js` plus the cross-runtime `test/bun/root-middleware.mjs`. Dev error overlay (#264): `reportDevError(error, info)` builds a frame (via `dev-error.js`) and pushes it to the open tab over the SSE channel as a `webjs-error` event, fed by three sources (the SSR render catch via `ssrOpts.onDevError`, the `tsResponse` strip-failure, and the `rebuild` catch); a successful rebuild clears `state.lastDevError`; the page-render branch stamps its frame with `withBasePath(url.pathname, basePath()) + url.search` so the browser gate can compare it against `location` (the RAW pathname, and the base path put back, since the ingress strip removed it), SKIPS the hook entirely for a request carrying `x-webjs-prefetch: 1` so a speculative link prefetch never raises an overlay or becomes the retained error, and after a successful GET render clears `state.lastDevError` when it is still the same object AND names this url, so the replay cannot resurrect a superseded error while a good render of an unrelated page leaves a still-current one standing (#1047); `reloadClientJs` also calls `installDevOverlayNavSync()` so the overlay tracks the page on screen; `startServer`'s SSE replays the current frame to a freshly-connected tab; `reloadClientJs` renders the dev-only plain-DOM overlay (`textContent` only). **Graceful reload (#893):** a reload never paints into a half-restarted server (Node's `node --watch` briefly kills the process on an app edit), so the client gates every reload on a `/__webjs/version` readiness probe (probe-then-reload, instant under an in-process reload, waits out a restart), and a reconnect after a drop is itself treated as an edit signal so an app edit whose in-process reload frame was killed with the old process still reloads without a manual refresh (the SSE `hello` carries a short `retry: 300` so the reconnect is prompt). The reconnect-reload lives in the `dev-reload-worker.js` relay (shared connection) and the per-tab `__webjsDirectEvents` fallback. **Reload coalescing (#1397):** an agent saves several files a second or two apart, and EACH save produces TWO reload signals (the in-process `reload` frame from the fs.watch rebuild, then a changed boot id when the browser reconnects to the process `node --watch` restarted, measured 429ms apart with 1071ms between saves), so acting on every one reloads into a server about to be killed again and the page ends up unstyled. Both signals now route through ONE debounced emitter in the relay, which reloads once the signals stop for `RELOAD_QUIET_MS` (2000ms, above the measured inter-save gap so a realistic burst collapses, and under the 1900ms analysis warm so the wait overlaps work the reload would have blocked on anyway) or at the latest `RELOAD_MAX_HOLD_MS` (5000ms) after the FIRST signal of the batch, so a sustained burst still repaints rather than freezing on stale content. The cap timer is armed once per batch and never re-armed, which is what makes it measure from the first signal instead of sliding with the burst. `webjs-error` is NOT debounced (an overlay has to appear at once and it is not a reload) and the cached error is cleared on the signal rather than on the debounced emit, so a tab connecting mid-window is not replayed an overlay for an error the rebuild already fixed. The debounce must live browser-side because the server process dies on every edit, so no server-side timer can span a burst; the SharedWorker survives, being keyed by script URL. The per-tab `__webjsDirectEvents` fallback now RUNS THE SAME RELAY over a shim port rather than re-implementing the boot-id rule, so the boot-id rule, the error cache, and the debounce are one implementation with nothing to drift. Each tab still gates its reload on the `/__webjs/version` readiness probe, and `isRegenerateOutputPath` still suppresses a regenerate-output write upstream of the emitter. **Extra watch roots (#894):** `startServer`'s recursive `fs.watch` also follows the dirs in `webjs.dev.watch` (`readDevWatchPathsFromApp`), for content the app reads from OUTSIDE its appDir (blog markdown in a repo-root `blog/`). Dev-only: `reportDevError` early-returns in prod and `/__webjs/reload.js` 404s. **Listener shell (#511):** `startServer` builds a shared `SseHub` + `ListenerContext`, then selects a shell by `serverRuntime()`: `startNodeListener` (in this file, the node:http path: `toWebRequest` -> `app.handle` -> `sendWebResponse`, 103 Early Hints, node WS via `attachWebSocket`, node:http timeouts) on Node, or the dynamically-imported `startBunListener` (`listener-bun.js`) on Bun. The SSE registry/fanout, the live-reload predicate, the WS module loader, and the lifecycle wiring live in `listener-core.js` so the two shells share them. `isCompressible` (used by `sendWebResponse`) also moved there | | `router.js` | Scans `app/` once, builds the route table, matches pages + APIs (`buildRouteTable`, `matchPage`, `matchApi`). Page order is set by `compareSpecificity` (#750): POSITIONAL specificity (per URL segment, static `0` < dynamic `1` < catch-all `2` via `segKind`, lexicographic on the kind arrays with shorter-prefix-first), so the catch-all kind is lowest AT ITS POSITION (a literal-prefixed catch-all like `docs/[[...slug]]` outranks an all-dynamic `[org]/[repo]`), NOT a global catch-all-last bucket, then a stable alphabetical `routeDir` tiebreak. This replaces the old coarse 3-bucket `dynScore` whose same-bucket ties resolved by fs-walk order (so `/[org]/[repo]` vs `/[user]/settings` could match the wrong page). `matchPage` returns the first pattern that matches in that deterministic order. The table also carries the app-ROOT convention files that are not router stems: `instrumentationClient` (#1399, resolved here rather than by each caller, because a consumer that built its own table and forgot to attach it lost a browser-bound entry and inverted the vendor pin-is-a-superset invariant) | | `route-types.js` | Route-types generator (#258). `generateRouteTypes(appDir)` reuses `buildRouteTable` to emit the `.d.ts` text that augments `@webjsdev/core` (the `WebjsRoutes` href union + `RouteParamMap` per-route params), backing `webjs types` and the dev-startup emit. Pages-only (a `route.{js,ts}` API path is not a navigable href); strips route groups, excludes `_private`; an optional catch-all `[[...x]]` emits both the without-segment and a normalized `[...x]` href key while keeping the doubled literal as the param-map key. Deterministic (sorted keys). Helpers `routeKeyFromDir` / `dynamicSegments` / `paramTypeForKey` / `webjsRoutesKeysForKey` are exported for unit tests | | `ssr.js` | SSR pipeline: nested layouts, metadata → ``, Suspense streaming, error boundaries. `ssrPage` accepts `actionData` (put on `ctx.actionData` for the page + layouts) and `status` (default 200; the form-action re-render passes 422). Server HTML cache (#241): on a plain GET render it loads the page module once to read `export const revalidate`, serves a cache HIT via `cachedHtmlResponse` (re-minting the build id), and on a miss stamps the `HTML_CACHE_MARKER` so the funnel writes the final body. Skipped for the form-action re-render and partial-nav (`X-Webjs-Have`) requests. **Partial responses are never shared-cacheable (#1140):** a reduced `X-Webjs-Have` body is sliced by a request header, so `privateFragment(res)` rewrites its `Cache-Control` to `private` (stripping `public` / `s-maxage` / `proxy-revalidate`, and any qualified `private="field"`, which per RFC 9111 would leave the response shared-storable). `Vary: X-Webjs-Have` is still sent, but as belt-and-braces: it is not the guarantee, because Cloudflare and others honour only `Accept-Encoding`. The helper is quote-aware (a comma inside `no-cache="Set-Cookie,X-Foo"` is not a separator), fails CLOSED on an absent header (no `Cache-Control` is heuristically shared-storable), and is exported for unit testing (not re-exported from the package index, so it is not public API). The fragment KEEPS its ETag, which is why `conditional-get.js` no longer excludes `private`. A non-200 (the 422 form-action re-render, which carries the submitter's own field values) never inherits the page's `cacheControl`. **Frame subtree render (#253):** after `renderChain`, when the request carries `x-webjs-frame: ` (a `` self-load or a click-driven frame nav) AND the render is non-streamed, it extracts the matching `` subtree from the rendered body via `frame-render.js` and returns ONLY that (byte-equivalent to the client's extraction from a full page, but far fewer bytes); an absent frame id falls through to the full page (the client's `webjs:frame-missing` handles it), and a request with no `x-webjs-frame` header is byte-identical to before. **Vendor modulepreload (#754):** `reachedVendorSpecifiers(graph, shippedEntryFiles, componentUrls, appDir, elidable, serverFiles)` collects the bare specifiers (`bareImports(graph)`) reached by the page's SHIPPED modules. Its walk ROOTS are the boot's actually-shipped set: the caller passes the absolute paths of `moduleUrls` (which already drops INERT page/layout modules and substitutes an IMPORT-ONLY page with its components) plus the rendered `componentUrls`, then it walks the non-elided transitive closure and collects each reached file's bare imports, skipping server files (the `.server.*` suffix AND the `serverFiles` action index). Because the roots are the shipped set, a vendor reached ONLY through a dropped module, a dropped page's SSR-only DIRECT vendor import OR its SSR-only RELATIVE HELPER's vendor, is never collected (pages/layouts are not importable, so nothing that ships reaches them), so there is no over-fetch. `vendorPreloadTargets` (importmap.js) maps that set to `{ href, integrity }`, and `wrapHead` emits one `` per target (no `fp()` rewrite, deduped against the app module/component preloads, with `crossorigin` + `integrity`), flattening the cross-origin CDN waterfall one level. Only reached + non-elided + pinned vendors are hinted. **App-module modulepreload shipped-roots (#780):** `deduplicatedPreloads` (the app-module + component preload walk) roots at the SAME shipped set as the vendor walk. `shippedRoots` (the absolute paths of `moduleUrls`) is computed once and passed to BOTH `deduplicatedPreloads` and `reachedVendorSpecifiers`, NOT the raw `[route.file, ...route.layouts]`. So a module reached ONLY through a dropped page/layout (its SSR-only DIRECT app import OR its SSR-only RELATIVE HELPER) gets no `modulepreload` hint, the app-module analog of the #754 vendor over-fetch. A module that also ships another way (a component shared with a live route, or one reached through an import-only page's substituted components) is still reached via a real shipped root, so its hint stays (no under-fetch). `seen = new Set(moduleUrls)` already excludes the shipped page/layout URLs themselves, so this closes the TRANSITIVE gap only | @@ -349,6 +349,22 @@ and the reader key set never diverge (a counterfactual unknown key proves `ensureReady` completes); that is correct, since they are framework infrastructure the app needs to function, not app routes. `handleCore` keeps a fallback call so it stays correct if entered directly. + **In DEV, `/public/*` is served there too** (#1397), along with the `/sw.js` + + `/offline.html` root remaps and `/favicon.ico`, through + `tryServePublicAsset` called right after `tryServeFrameworkStatic`. A static + asset needs neither the module graph nor the vendor importmap, and on the + website app a cold `/public/tailwind.css` measured 1907ms of which 1900ms was + `ensureReady()`, which is why the stylesheet is the request most exposed to + the next `node --watch` restart and why a burst of edits leaves the page + unstyled. This is DEV ONLY on purpose: in prod `/__webjs/ready` already holds + traffic off a cold instance until the analysis and the first vendor attempt + have both completed, and hoisting there would silently un-gate a `/public/*` + file an app middleware protects. The consequence to know about is that in dev + root middleware does NOT run for those paths, the same trade the framework + statics above already make in both modes. The `handleCore` branch is the LIVE + path in prod and the fallback in dev, and both call sites share the one + `tryServePublicAsset`, so the directory-traversal containment guard exists in + exactly one place and cannot drift between two copies. 4. **One pluggable cache store, four built-in consumers.** `cache.js` is shared by `cache-fn.js`, `cache-tags.js`, `session.js` (store-backed), and `rate-limit.js`. A single `setStore(redisStore({…}))` call at diff --git a/packages/server/src/dev-reload-worker.js b/packages/server/src/dev-reload-worker.js index b6a5c5c95..61128ce1c 100644 --- a/packages/server/src/dev-reload-worker.js +++ b/packages/server/src/dev-reload-worker.js @@ -16,6 +16,32 @@ * @param {new (url: string) => any} EventSourceCtor the `EventSource` constructor * @param {string} eventsUrl the base-path-aware `/__webjs/events` URL */ + +/** + * Reload coalescing (#1397). An agent saves several files a second or two + * apart, and EACH save produces TWO reload signals: the in-process `reload` + * frame from the fs.watch rebuild, then a changed boot id when the browser + * reconnects to the process `node --watch` restarted (measured 429ms apart, and + * 1071ms between one save's reconnect and the next save's in-process frame). + * Acting on every one reloads into a server that is about to be killed again, + * which is how the page ends up unstyled. + * + * 2000ms is above the measured 1071ms inter-save gap so a realistic burst + * collapses to one reload, and deliberately not NEAR it, since a window close + * to that gap fires just as the next restart begins. It is also under the + * 1900ms analysis warm measured on the website app, so on a real app the wait + * overlaps work the reload request would have blocked on anyway. + */ +export const RELOAD_QUIET_MS = 2000; + +/** + * The longest a reload is ever held, measured from the FIRST signal of a batch + * (#1397). A quiet window alone would freeze the page on stale content for a + * whole agent burst, so a sustained burst still repaints at least this often. + * 2.5x the quiet window, so it never fires for an ordinary pair of edits. + */ +export const RELOAD_MAX_HOLD_MS = 5000; + export function startReloadWorker(scope, EventSourceCtor, eventsUrl) { /** @type {Set} */ const ports = new Set(); @@ -24,6 +50,13 @@ export function startReloadWorker(scope, EventSourceCtor, eventsUrl) { /** @type {string | null} the last-seen per-process boot id (#893) */ let lastBoot = null; + // Timers come from the worker global when it has them (the real SharedWorker + // scope does, and so does a tab running the per-tab fallback), else the + // ambient globals. Reading them off `scope` is what lets the browser test + // drive a fake clock and keep its assertions synchronous, and calling them as + // `timers.setTimeout(...)` keeps `this` the global in a real browser. + const timers = scope && typeof scope.setTimeout === 'function' ? scope : globalThis; + // A MessagePort has no reliable close event, so prune a port when a post to it // throws (a closed tab). Some browsers silently no-op instead of throwing, // leaving a dead port in the set, but that is a harmless dev-only no-op and @@ -34,6 +67,26 @@ export function startReloadWorker(scope, EventSourceCtor, eventsUrl) { } } + // The ONE debounced reload emitter (#1397). Two timers rather than a clock: + // the quiet timer is re-armed by every signal, the cap timer is armed once + // per batch and NEVER re-armed, so the cap measures from the first signal of + // the batch instead of sliding with the burst. Whichever fires first emits + // and cancels the other, which also starts a fresh batch. + /** @type {any} */ let quietTimer = null; + /** @type {any} */ let capTimer = null; + + function emitReload() { + if (quietTimer !== null) { timers.clearTimeout(quietTimer); quietTimer = null; } + if (capTimer !== null) { timers.clearTimeout(capTimer); capTimer = null; } + fanout({ type: 'reload' }); + } + + function requestReload() { + if (quietTimer !== null) timers.clearTimeout(quietTimer); + quietTimer = timers.setTimeout(emitReload, RELOAD_QUIET_MS); + if (capTimer === null) capTimer = timers.setTimeout(emitReload, RELOAD_MAX_HOLD_MS); + } + const es = new EventSourceCtor(eventsUrl); // The `hello` frame fires on every (re)connect and carries the server's @@ -45,13 +98,19 @@ export function startReloadWorker(scope, EventSourceCtor, eventsUrl) { // tab still gates it on a readiness probe). A transient reconnect (sleep/wake, // a network blip, a tab evicted at the HTTP/1.1 cap) reconnects to the SAME // process with the SAME id, so it never reloads. The first `hello` only - // records the baseline. + // records the baseline. Both reload paths route through the ONE debounced + // emitter above (#1397), since a burst of edits produces one signal of each + // kind per save. es.addEventListener('hello', (e) => { - if (lastBoot !== null && e.data !== lastBoot) fanout({ type: 'reload' }); + if (lastBoot !== null && e.data !== lastBoot) requestReload(); lastBoot = e.data; }); - es.addEventListener('reload', () => { lastError = null; fanout({ type: 'reload' }); }); + // The cached error is cleared IMMEDIATELY, not on the debounced emit: a tab + // connecting during the pending window must not be replayed an overlay for an + // error the rebuild already fixed. `webjs-error` itself stays undebounced, + // since an overlay has to appear at once and it is not a reload. + es.addEventListener('reload', () => { lastError = null; requestReload(); }); es.addEventListener('webjs-error', (e) => { lastError = e.data; fanout({ type: 'webjs-error', data: e.data }); }); scope.onconnect = (e) => { diff --git a/packages/server/src/dev.js b/packages/server/src/dev.js index 954488bc3..d65b60683 100644 --- a/packages/server/src/dev.js +++ b/packages/server/src/dev.js @@ -1567,6 +1567,25 @@ export async function createRequestHandler(opts) { try { assetVersioned = new URL(req.url).searchParams.has('v'); } catch { /* none */ } const staticResp = await tryServeFrameworkStatic(assetPath, req.method.toUpperCase(), { coreDir, appDir, dev, versioned: assetVersioned }); if (staticResp) return staticResp; + // `/public/*` needs neither the module graph nor the vendor importmap, so + // in DEV serve it here rather than behind the analysis (#1397). A cold + // `/public/tailwind.css` measured 1907ms at 4a335549, of which 1900ms was + // `ensureReady()`, which is why the stylesheet is the request most exposed + // to the next `node --watch` restart. DEV ONLY on purpose: in prod + // `/__webjs/ready` already holds traffic off a cold instance until the + // analysis and the first vendor attempt have both completed, and hoisting + // there would silently un-gate a `/public/*` file that an app middleware + // protects. The consequence in dev is that root middleware does not run + // for these paths, the same trade the framework statics above already make + // in both modes. `state.regenerateRules` is loaded at boot inside + // createRequestHandler, so the #967 on-request rebuild is available here, + // ahead of ensureReady. + if (dev) { + const publicResp = await tryServePublicAsset(assetPath, { + appDir, dev, versioned: assetVersioned, regenerateRules: state.regenerateRules, + }); + if (publicResp) return publicResp; + } // Build all whole-app analysis on the first request (memoized), before // any SSR, module serve, gate check, action dispatch, or middleware runs. await ensureReady(); @@ -2066,6 +2085,64 @@ async function tryServeFrameworkStatic(path, method, ctx) { return null; } +/** + * Serve `/public/*`, plus a small set of ROOT assets that must serve at the + * site root even though they live under public/. A service worker registered at + * /sw.js scopes to the origin root, so it MUST serve at / (not /public/sw.js), + * and so must its offline fallback. Same remap shape as the /favicon.ico + * special-case. (#830) + * + * Extracted (#1397) so the dev pre-`ensureReady()` call site and the + * `handleCore` one share ONE implementation, and the containment guard below + * cannot drift between two copies. + * + * @param {string} path decoded pathname + * @param {{ appDir: string, dev: boolean, versioned?: boolean, regenerateRules: any[] }} ctx + * @returns {Promise} a Response, or null when the path is not a + * public asset OR the file does not exist, in both of which cases the caller + * must fall through to the rest of the pipeline (a missing `/public/x.png` + * 404s through normal routing today, and that must not change). A containment + * rejection returns a 404 Response and short-circuits. + */ +async function tryServePublicAsset(path, ctx) { + const { appDir, dev, versioned, regenerateRules } = ctx; + const ROOT_ASSETS = { '/sw.js': '/public/sw.js', '/offline.html': '/public/offline.html' }; + if (!(path.startsWith('/public/') || path === '/favicon.ico' || path in ROOT_ASSETS)) return null; + const p = path === '/favicon.ico' ? '/public/favicon.ico' : (ROOT_ASSETS[path] || path); + const abs = join(appDir, p); + // Containment check. `join` normalises `..` segments, so a path + // like `/public/%2E%2E/secret/x.svg` decodes (after URL parsing, + // which doesn't touch `%2E`) to `/public/../secret/x.svg` and + // `join(appDir, ...)` resolves it to `appDir/secret/x.svg`. The + // resulting `abs` could be inside `appDir` but OUTSIDE `appDir/ + // public/`, exposing files the user reasonably thought were + // private under their non-public directories. Reject anything + // that doesn't stay under `appDir/public/` (and the favicon + // exception, which is already validated above). + const publicRoot = join(appDir, 'public') + sep; + if (!abs.startsWith(publicRoot)) { + return new Response(null, { status: 404 }); + } + // On-request regeneration (#967): in dev, if a `webjs.dev.regenerate` rule + // matches this output and it is stale (a source is newer, or it is missing), + // rebuild it to completion BEFORE serving, so a newly added utility class is + // never served stale. No-op when no rule matches or the output is fresh, and + // never runs in prod (rules are empty there). This replaces the fragile + // `tailwindcss --watch` that could die mid-session and serve stale CSS. + if (dev && regenerateRules.length) { + await maybeRegenerate(appDir, p.replace(/^\/+/, ''), regenerateRules); + } + // A `?v=` public asset is content-addressed -> immutable (#243). + if (await exists(abs)) { + const res = await fileResponse(abs, { dev, immutable: versioned }); + // A worker served below its registration path only controls that subtree + // unless the response opts it up to the root scope. (#830) + if (path === '/sw.js') res.headers.set('Service-Worker-Allowed', '/'); + return res; + } + return null; +} + async function handleCore(req, ctx) { const { state, appDir, coreDir, dev, reportError, reportDevError, hasOnError, logger, cspEnabled, allowedOrigins } = ctx; const url = new URL(req.url); @@ -2109,46 +2186,11 @@ async function handleCore(req, ctx) { return invokeAction(state.actionIndex, actMatch[1], actMatch[2], req, onActionError, allowedOrigins); } - // Static: /public/*, plus a small set of ROOT assets that must serve at the - // site root even though they live under public/. A service worker registered - // at /sw.js scopes to the origin root, so it MUST serve at / (not - // /public/sw.js), and so must its offline fallback. Same remap shape as the - // /favicon.ico special-case below. (#830) - const ROOT_ASSETS = { '/sw.js': '/public/sw.js', '/offline.html': '/public/offline.html' }; - if (path.startsWith('/public/') || path === '/favicon.ico' || path in ROOT_ASSETS) { - const p = path === '/favicon.ico' ? '/public/favicon.ico' : (ROOT_ASSETS[path] || path); - const abs = join(appDir, p); - // Containment check. `join` normalises `..` segments, so a path - // like `/public/%2E%2E/secret/x.svg` decodes (after URL parsing, - // which doesn't touch `%2E`) to `/public/../secret/x.svg` and - // `join(appDir, ...)` resolves it to `appDir/secret/x.svg`. The - // resulting `abs` could be inside `appDir` but OUTSIDE `appDir/ - // public/`, exposing files the user reasonably thought were - // private under their non-public directories. Reject anything - // that doesn't stay under `appDir/public/` (and the favicon - // exception, which is already validated above). - const publicRoot = join(appDir, 'public') + sep; - if (!abs.startsWith(publicRoot)) { - return new Response(null, { status: 404 }); - } - // On-request regeneration (#967): in dev, if a `webjs.dev.regenerate` rule - // matches this output and it is stale (a source is newer, or it is missing), - // rebuild it to completion BEFORE serving, so a newly added utility class is - // never served stale. No-op when no rule matches or the output is fresh, and - // never runs in prod (rules are empty there). This replaces the fragile - // `tailwindcss --watch` that could die mid-session and serve stale CSS. - if (dev && state.regenerateRules.length) { - await maybeRegenerate(appDir, p.replace(/^\/+/, ''), state.regenerateRules); - } - // A `?v=` public asset is content-addressed -> immutable (#243). - if (await exists(abs)) { - const res = await fileResponse(abs, { dev, immutable: versioned }); - // A worker served below its registration path only controls that subtree - // unless the response opts it up to the root scope. (#830) - if (path === '/sw.js') res.headers.set('Service-Worker-Allowed', '/'); - return res; - } - } + // Static: /public/*, the #830 root remaps, and /favicon.ico. In dev this + // already ran before ensureReady() (#1397); this stays the LIVE path in prod + // and the fallback in dev, and both call sites share one implementation. + const publicResp = await tryServePublicAsset(path, { appDir, dev, versioned, regenerateRules: state.regenerateRules }); + if (publicResp) return publicResp; // User source modules (served as ES modules, with action-file rewriting). // @@ -2944,6 +2986,7 @@ function reloadClientJs(bp) { const versionUrl = JSON.stringify(withBasePath('/__webjs/version', bp)); return `// webjs dev reload client ${DEV_OVERLAY_SRC} +${RELOAD_WORKER_SRC} function __webjsApplyError(data) { let f; try { f = JSON.parse(data); } catch (_) { return; } renderDevOverlay(f); @@ -2972,18 +3015,17 @@ function __webjsReloadWhenReady() { attempt(); } function __webjsDirectEvents() { - // Same restart-reload as the SharedWorker relay (#893), for the per-tab - // fallback: the hello frame carries the server's per-process boot id, so a - // CHANGED id on reconnect means a real restart (reload), while a transient - // reconnect to the same process keeps the id (no spurious reload). - var lastBoot = null; - const es = new EventSource(${eventsUrl}); - es.addEventListener('hello', (e) => { - if (lastBoot !== null && e.data !== lastBoot) __webjsReloadWhenReady(); - lastBoot = e.data; - }); - es.addEventListener('reload', () => __webjsReloadWhenReady()); - es.addEventListener('webjs-error', (e) => __webjsApplyError(e.data)); + // No SharedWorker (Chrome for Android has none) or its construction threw (a + // strict dev CSP with no worker-src). Run the SAME relay here in the tab over + // a shim port instead of a second copy of the boot-id rule (#887, #893) and + // the reload debounce (#1397). The shim scope has no setTimeout, so the relay + // picks up the tab's own timers. + const scope = {}; + startReloadWorker(scope, EventSource, ${eventsUrl}); + scope.onconnect({ ports: [{ start() {}, postMessage(m) { + if (m.type === 'reload') __webjsReloadWhenReady(); + else if (m.type === 'webjs-error') __webjsApplyError(m.data); + } }] }); } try { if (typeof SharedWorker !== 'undefined') { diff --git a/packages/server/test/dev/browser/reload-worker.test.js b/packages/server/test/dev/browser/reload-worker.test.js index f47219dac..b642f50bf 100644 --- a/packages/server/test/dev/browser/reload-worker.test.js +++ b/packages/server/test/dev/browser/reload-worker.test.js @@ -10,7 +10,7 @@ * runs in a real browser. The relay is driven with a fake EventSource + fake * MessagePorts so it needs no live SSE server. */ -import { startReloadWorker } from '../../../src/dev-reload-worker.js'; +import { startReloadWorker, RELOAD_QUIET_MS, RELOAD_MAX_HOLD_MS } from '../../../src/dev-reload-worker.js'; import { assert } from '../../../../../test/browser-assert.js'; @@ -25,21 +25,61 @@ function fakePort() { return { received, port: { start() {}, postMessage(m) { received.push(m); } } }; } +/** + * A fake clock handed to the relay as its `scope` (#1397). The relay reads + * `setTimeout` / `clearTimeout` off the scope precisely so a test can drive the + * reload debounce deterministically and keep its assertions synchronous, with + * no real waiting for a 2 to 5 second window. + */ +function fakeClock() { + let now = 0; + let id = 0; + const jobs = new Map(); + const scope = { + setTimeout(fn, ms) { jobs.set(++id, { at: now + ms, fn }); return id; }, + clearTimeout(t) { jobs.delete(t); }, + }; + return { + scope, + // Fire due jobs strictly in time order, re-scanning after each one. The + // re-scan is what makes this faithful: the emitter CANCELS its sibling + // timer as it fires, so a snapshot taken up front would run a job that no + // longer exists and report two reloads where the relay emits one. + tick(ms) { + const target = now + ms; + for (;;) { + let nextId = null; + let nextAt = Infinity; + for (const [t, j] of jobs) { + if (j.at <= target && j.at < nextAt) { nextAt = j.at; nextId = t; } + } + if (nextId === null) break; + now = nextAt; + const j = jobs.get(nextId); + jobs.delete(nextId); + j.fn(); + } + now = target; + }, + }; +} + suite('dev reload SharedWorker relay (#887)', () => { test('fans a reload out to every connected tab (one connection, many tabs)', () => { - const scope = {}; + const { scope, tick } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); const a = fakePort(); const b = fakePort(); scope.onconnect({ ports: [a.port] }); scope.onconnect({ ports: [b.port] }); FakeEventSource.last.fire('reload'); + tick(RELOAD_QUIET_MS); assert.deepEqual(a.received, [{ type: 'reload' }], 'tab A reloaded'); assert.deepEqual(b.received, [{ type: 'reload' }], 'tab B reloaded from the same worker'); }); test('relays an error frame to every connected tab', () => { - const scope = {}; + const { scope } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); const a = fakePort(); scope.onconnect({ ports: [a.port] }); @@ -48,7 +88,7 @@ suite('dev reload SharedWorker relay (#887)', () => { }); test('caches the error and replays it to a tab that connects later', () => { - const scope = {}; + const { scope } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); FakeEventSource.last.fire('webjs-error', 'FRAME_JSON'); // error before the tab opens const late = fakePort(); @@ -57,17 +97,18 @@ suite('dev reload SharedWorker relay (#887)', () => { }); test('clears the cached error on reload so a later tab does not see a stale overlay', () => { - const scope = {}; + const { scope, tick } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); FakeEventSource.last.fire('webjs-error', 'FRAME_JSON'); FakeEventSource.last.fire('reload'); // the fix landed + tick(RELOAD_QUIET_MS); const late = fakePort(); scope.onconnect({ ports: [late.port] }); assert.equal(late.received.length, 0, 'no stale error replayed after a reload'); }); test('connects the single EventSource at the given events URL', () => { - const scope = {}; + const { scope } = fakeClock(); const { es } = startReloadWorker(scope, FakeEventSource, '/base/__webjs/events'); assert.equal(es.url, '/base/__webjs/events', 'the one connection uses the base-path-aware URL'); }); @@ -77,23 +118,152 @@ suite('dev reload SharedWorker relay (#887)', () => { // the edit would need a manual refresh. The `hello` frame carries a // per-process boot id, so a CHANGED id on reconnect is the reload signal. test('a reconnect to a NEW process (changed boot id) fans a reload', () => { - const scope = {}; + const { scope, tick } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); const a = fakePort(); scope.onconnect({ ports: [a.port] }); FakeEventSource.last.fire('hello', 'BOOT_A'); // initial connect: baseline only + tick(RELOAD_QUIET_MS); assert.deepEqual(a.received, [], 'the first hello does not reload'); FakeEventSource.last.fire('hello', 'BOOT_B'); // reconnected to a fresh process + tick(RELOAD_QUIET_MS); assert.deepEqual(a.received, [{ type: 'reload' }], 'a new boot id reloads the tab'); }); test('a transient reconnect to the SAME process (same boot id) never reloads', () => { - const scope = {}; + const { scope, tick } = fakeClock(); startReloadWorker(scope, FakeEventSource, '/__webjs/events'); const a = fakePort(); scope.onconnect({ ports: [a.port] }); FakeEventSource.last.fire('hello', 'BOOT_A'); // first connect FakeEventSource.last.fire('hello', 'BOOT_A'); // sleep/wake or blip: same process + tick(RELOAD_QUIET_MS); assert.deepEqual(a.received, [], 'a same-process reconnect is not an edit (no state loss)'); }); }); + +// #1397: an agent saves several files a second or two apart, and EACH save +// produces TWO reload signals (the in-process `reload` frame, then a changed +// boot id when the browser reconnects to the restarted process). Acting on +// every one reloads into a server that is about to be killed again, which is +// how the page ends up unstyled. Both signals route through one debounced +// emitter instead. +suite('dev reload coalescing (#1397)', () => { + test('a burst of signals inside the quiet window fans exactly ONE reload', () => { + const { scope, tick } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + const es = FakeEventSource.last; + es.fire('hello', 'BOOT_A'); // baseline, no signal + // The measured shape of one agent burst: an in-process frame, then a + // reconnect with a new boot id, twice over. The gaps are inside the quiet + // window and the whole burst plus its window is inside the cap, so this is + // the case the debounce is meant to collapse completely. + const gap = 700; + es.fire('reload'); + tick(gap); + es.fire('hello', 'BOOT_B'); + tick(gap); + es.fire('reload'); + tick(gap); + es.fire('hello', 'BOOT_C'); + assert.deepEqual(a.received, [], 'nothing fires while the edits are still landing'); + tick(RELOAD_QUIET_MS); + assert.deepEqual(a.received, [{ type: 'reload' }], 'four signals coalesce into one reload'); + }); + + test('a single signal in a quiet session reloads after exactly the quiet window', () => { + const { scope, tick } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + FakeEventSource.last.fire('reload'); + tick(RELOAD_QUIET_MS - 1); + assert.deepEqual(a.received, [], 'not yet, the window has not elapsed'); + tick(1); + assert.deepEqual(a.received, [{ type: 'reload' }], 'and never later than the window'); + }); + + test('a sustained burst still reloads at the cap, measured from the first signal', () => { + const { scope, tick } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + const es = FakeEventSource.last; + const step = RELOAD_QUIET_MS - 100; // close enough that the quiet timer never expires + es.fire('reload'); // t = 0, the first signal of the batch + let elapsed = 0; + while (elapsed + step < RELOAD_MAX_HOLD_MS) { + tick(step); + elapsed += step; + es.fire('reload'); + assert.deepEqual(a.received, [], 'the quiet window keeps being pushed out by the burst'); + } + tick(RELOAD_MAX_HOLD_MS - elapsed); + assert.deepEqual(a.received, [{ type: 'reload' }], 'the cap fires at RELOAD_MAX_HOLD_MS from the FIRST signal'); + }); + + // COUNTERFACTUAL: re-arm the cap timer on every signal and the cap slides out + // with the burst, so the last two assertions here both see nothing. + test('the cap timer is not re-armed within a batch', () => { + const { scope, tick } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + const es = FakeEventSource.last; + const step = RELOAD_QUIET_MS - 100; + es.fire('reload'); // t = 0, the first signal of the batch + tick(step); + es.fire('reload'); // mid-batch: must not push the cap out + tick(step); + es.fire('reload'); // mid-batch again + tick(RELOAD_MAX_HOLD_MS - 2 * step - 1); + assert.deepEqual(a.received, [], 'the cap has not fired one tick early'); + tick(1); + assert.deepEqual(a.received, [{ type: 'reload' }], 'the cap still measures from the first signal'); + }); + + test('a signal after a cap fire starts a NEW batch', () => { + const { scope, tick } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + const es = FakeEventSource.last; + const step = RELOAD_QUIET_MS - 100; + // Sustain a burst until the cap fires, so batch one ends on the CAP rather + // than on a quiet window (which is the case this test is about). + es.fire('reload'); + tick(step); + es.fire('reload'); + tick(step); + es.fire('reload'); + tick(RELOAD_MAX_HOLD_MS - 2 * step); + assert.equal(a.received.length, 1, 'batch one emitted at the cap'); + es.fire('reload'); + tick(RELOAD_QUIET_MS - 1); + assert.equal(a.received.length, 1, 'batch two waits a full quiet window, it does not inherit the old timers'); + tick(1); + assert.equal(a.received.length, 2, 'batch two emitted on its own window'); + }); + + test('an error frame is never debounced', () => { + const { scope } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + const a = fakePort(); + scope.onconnect({ ports: [a.port] }); + FakeEventSource.last.fire('webjs-error', 'FRAME_JSON'); + // No tick: an overlay has to appear at once, and it is not a reload. + assert.deepEqual(a.received, [{ type: 'webjs-error', data: 'FRAME_JSON' }]); + }); + + test('a reload signal clears the cached error immediately, before the debounced emit', () => { + const { scope } = fakeClock(); + startReloadWorker(scope, FakeEventSource, '/__webjs/events'); + FakeEventSource.last.fire('webjs-error', 'FRAME_JSON'); + FakeEventSource.last.fire('reload'); // the rebuild fixed it; the reload is still pending + const late = fakePort(); + scope.onconnect({ ports: [late.port] }); // connects DURING the pending window + assert.equal(late.received.length, 0, 'no overlay for an error the rebuild already fixed'); + }); +}); diff --git a/packages/server/test/dev/public-before-analysis.test.js b/packages/server/test/dev/public-before-analysis.test.js new file mode 100644 index 000000000..ea3e39433 --- /dev/null +++ b/packages/server/test/dev/public-before-analysis.test.js @@ -0,0 +1,161 @@ +/** + * Integration tests for #1397: in DEV, `/public/*` is served BEFORE + * `ensureReady()`. + * + * A static asset needs neither the module graph nor the vendor importmap, but + * it used to be handled inside `handleCore`, which runs after the whole-app + * analysis. On the website app a cold `/public/tailwind.css` measured 1907ms, + * of which 1900ms was `ensureReady()`, which is why the stylesheet is the + * request most exposed to the next `node --watch` restart and why a burst of + * agent edits leaves the page unstyled. + * + * The hoist is DEV ONLY. In prod `/__webjs/ready` already holds traffic off a + * cold instance, and hoisting there would silently un-gate a `/public/*` file + * an app middleware protects. + * + * Assertions are on ORDER, not wall clock: the fixture's root middleware module + * has a top-level await sleep, and `loadMiddleware` runs inside `ensureReady()`, + * so the analysis is deterministically slow with no network involved. + */ +import { test, before, after } from 'node:test'; +import assert from 'node:assert/strict'; +import { mkdtempSync, rmSync, writeFileSync, mkdirSync, utimesSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; + +import { createRequestHandler } from '../../src/dev.js'; + +let tmpRoot; +before(() => { tmpRoot = mkdtempSync(join(tmpdir(), 'webjs-publicearly-')); }); +after(() => { rmSync(tmpRoot, { recursive: true, force: true }); }); + +/** + * @param {{ slowMiddleware?: boolean, regenerate?: boolean }} [opts] + */ +function makeApp(opts = {}) { + const appDir = mkdtempSync(join(tmpRoot, 'app-')); + mkdirSync(join(appDir, 'app'), { recursive: true }); + mkdirSync(join(appDir, 'public'), { recursive: true }); + writeFileSync(join(appDir, 'public', 'a.css'), 'body{color:red}\n'); + writeFileSync(join(appDir, 'public', 'sw.js'), "self.addEventListener('install', () => {});\n"); + writeFileSync(join(appDir, 'public', 'offline.html'), 'offline\n'); + writeFileSync( + join(appDir, 'app', 'page.ts'), + "import { html } from '@webjsdev/core';\nexport default function Page() { return html`

hi

`; }\n", + ); + const pkg = { name: 'public-early-fixture', type: 'module' }; + if (opts.regenerate) { + // #967: an on-request rebuild rule whose command rewrites the served output. + pkg.webjs = { + dev: { + regenerate: [ + { + output: 'public/gen.css', + inputs: ['src/in.css'], + command: `node -e "require('fs').writeFileSync('public/gen.css','body{color:lime}')"`, + }, + ], + }, + }; + mkdirSync(join(appDir, 'src'), { recursive: true }); + writeFileSync(join(appDir, 'src', 'in.css'), 'body{color:lime}\n'); + writeFileSync(join(appDir, 'public', 'gen.css'), '/* stale */\n'); + // Backdate the output so the input is newer, which is what makes it stale. + const past = new Date(Date.now() - 60_000); + utimesSync(join(appDir, 'public', 'gen.css'), past, past); + } + writeFileSync(join(appDir, 'package.json'), JSON.stringify(pkg, null, 2)); + if (opts.slowMiddleware) { + // Top-level await, so the module does not finish evaluating (and therefore + // `loadMiddleware`, and therefore `ensureReady()`, does not resolve) until + // the sleep elapses. It also tags every response it DOES run for, which is + // how the dev/prod gating is asserted below. + writeFileSync( + join(appDir, 'middleware.ts'), + 'await new Promise((r) => setTimeout(r, 500));\n' + + 'export default async function middleware(req: Request, next: () => Promise) {\n' + + ' const res = await next();\n' + + " res.headers.set('x-mw', '1');\n" + + ' return res;\n' + + '}\n', + ); + } + return appDir; +} + +// COUNTERFACTUAL: revert the dev-only `tryServePublicAsset` call ahead of +// `await ensureReady()` in dev.js and the order becomes ['warm', 'public']. +test('dev serves /public/* before the analysis completes', async () => { + const app = await createRequestHandler({ appDir: makeApp({ slowMiddleware: true }), dev: true }); + /** @type {string[]} */ + const order = []; + const publicP = app.handle(new Request('http://x/public/a.css')).then((r) => { order.push('public'); return r; }); + const warmP = app.warmup().then(() => { order.push('warm'); }); + const [res] = await Promise.all([publicP, warmP]); + assert.deepEqual(order, ['public', 'warm'], 'the CSS lands before the analysis finishes'); + assert.equal(res.status, 200); + assert.equal(await res.text(), 'body{color:red}\n', 'and it is the real file'); +}); + +// Pins the settled dev-only gating in BOTH directions. The dev bypass is a +// deliberate, documented trade (the same one the framework statics already make +// in both modes); silently extending it to prod would un-gate a protected asset. +test('root middleware does not run for /public/* in dev, and does in prod', async () => { + const devApp = await createRequestHandler({ appDir: makeApp({ slowMiddleware: true }), dev: true }); + const devRes = await devApp.handle(new Request('http://x/public/a.css')); + assert.equal(devRes.status, 200); + assert.equal(devRes.headers.get('x-mw'), null, 'dev takes the early path, ahead of middleware'); + + const prodApp = await createRequestHandler({ appDir: makeApp({ slowMiddleware: true }), dev: false }); + const prodRes = await prodApp.handle(new Request('http://x/public/a.css')); + assert.equal(prodRes.status, 200); + assert.equal(prodRes.headers.get('x-mw'), '1', 'prod still runs root middleware for a public asset'); +}); + +// COUNTERFACTUAL: drop the containment check from `tryServePublicAsset` and this +// serves the file, which is a directory-traversal hole. +test('the traversal guard travels with the moved code (dev early path)', async () => { + const appDir = makeApp(); + writeFileSync(join(appDir, 'secret.txt'), 'nope\n'); + const app = await createRequestHandler({ appDir, dev: true }); + const res = await app.handle(new Request('http://x/public/%2E%2E/secret.txt')); + assert.equal(res.status, 404, 'a path that escapes appDir/public/ is refused'); + assert.notEqual(await res.text(), 'nope\n'); +}); + +// The `return null` contract: a missing public asset is NOT a short-circuit, so +// it 404s through normal routing exactly as it did before the extraction. +test('a missing /public/* file still falls through to normal routing', async () => { + const app = await createRequestHandler({ appDir: makeApp(), dev: true }); + const res = await app.handle(new Request('http://x/public/nope.png')); + assert.equal(res.status, 404); +}); + +// #830, re-asserted here because the code moved to a shared function. +test('/sw.js and /offline.html still serve at the root through the early path', async () => { + const app = await createRequestHandler({ appDir: makeApp(), dev: true }); + const sw = await app.handle(new Request('http://x/sw.js')); + assert.equal(sw.status, 200); + assert.equal(sw.headers.get('service-worker-allowed'), '/', 'still opts into the root scope'); + const offline = await app.handle(new Request('http://x/offline.html')); + assert.equal(offline.status, 200); + assert.match(await offline.text(), /offline/); +}); + +// #967 still holds ahead of ensureReady: a stale regenerate output is rebuilt +// before it is served, on the early path too. +test('a stale webjs.dev.regenerate output is rebuilt before serving on the early path', async () => { + const app = await createRequestHandler({ appDir: makeApp({ regenerate: true }), dev: true }); + const res = await app.handle(new Request('http://x/public/gen.css')); + assert.equal(res.status, 200); + assert.equal(await res.text(), 'body{color:lime}', 'the stale output was regenerated, not served as-is'); +}); + +// #243: the `?v=` fingerprint still decides the cache header on the early path. +test('?v= still yields immutable on the early path, un-versioned keeps the 1h fallback', async () => { + const app = await createRequestHandler({ appDir: makeApp(), dev: false }); + const versioned = await app.handle(new Request('http://x/public/a.css?v=abc123')); + assert.match(versioned.headers.get('cache-control') || '', /immutable/, 'content-addressed is immutable'); + const plain = await app.handle(new Request('http://x/public/a.css')); + assert.doesNotMatch(plain.headers.get('cache-control') || '', /immutable/, 'un-fingerprinted is not'); +}); diff --git a/packages/server/test/dev/reload-shared-connection.test.js b/packages/server/test/dev/reload-shared-connection.test.js index 4eb3d10ab..1dd21ea35 100644 --- a/packages/server/test/dev/reload-shared-connection.test.js +++ b/packages/server/test/dev/reload-shared-connection.test.js @@ -39,12 +39,19 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E // The primary path is one shared connection through the SharedWorker. assert.match(clientSrc, /new SharedWorker\(/, 'client constructs a SharedWorker'); assert.match(clientSrc, /reload-worker\.js/, 'client points the worker at the worker route'); - // The fallback keeps the original per-tab EventSource where SharedWorker is - // unavailable, and the whole thing is guarded so a construction failure (a - // strict dev CSP) degrades instead of breaking. + // The fallback keeps a per-tab connection where SharedWorker is unavailable, + // and the whole thing is guarded so a construction failure (a strict dev CSP) + // degrades instead of breaking. Since #1397 the fallback runs the SAME relay + // in the tab over a shim port rather than a second copy of the boot-id rule, + // so the client inlines the relay module too and hands it `EventSource`. assert.match(clientSrc, /typeof SharedWorker/, 'client feature-detects SharedWorker'); - assert.match(clientSrc, /new EventSource\(/, 'client keeps an EventSource fallback'); + assert.match(clientSrc, /function startReloadWorker/, 'the client inlines the relay module for the fallback'); + assert.match(clientSrc, /startReloadWorker\(scope, EventSource, "\/__webjs\/events"\)/, 'the fallback runs the relay against the real EventSource'); + assert.match(clientSrc, /scope\.onconnect\(\{ ports: \[\{/, 'and drives it over a shim port'); assert.match(clientSrc, /catch\s*\(_\)\s*\{\s*__webjsDirectEvents/, 'a worker failure falls back'); + // The debounce (#1397) is part of the relay, so it ships in BOTH scripts. + assert.match(clientSrc, /const RELOAD_QUIET_MS/, 'the reload debounce ships in the client fallback'); + assert.match(clientSrc, /const RELOAD_MAX_HOLD_MS/, 'including the max-hold cap'); // The overlay still renders on the main thread (a worker has no DOM). assert.match(clientSrc, /renderDevOverlay/, 'the error overlay still renders in the client'); // ...and it tracks the page actually on screen (#1047). The gate lives in the @@ -67,6 +74,8 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E assert.match(workerSrc, /scope\.onconnect/, 'the relay accepts a port per tab'); assert.match(workerSrc, /lastError = null/, 'the relay clears the cached error on reload'); assert.match(workerSrc, /if \(lastError != null\)/, 'a late-joining tab gets the current error'); + assert.match(workerSrc, /const RELOAD_QUIET_MS/, 'the reload debounce ships in the worker (#1397)'); + assert.ok(!/\bexport\s/.test(workerSrc), 'no export keyword survives into the classic worker script'); }); test('both reload routes 404 in prod (never shipped to a production page)', async () => { @@ -102,13 +111,15 @@ test('the reload client probes the server is up before reloading (no restart fla assert.match(clientSrc, /function __webjsReloadWhenReady/, 'reload is gated on a readiness probe'); assert.match(clientSrc, /fetch\(\"\/__webjs\/version\"/, 'the probe hits the lightweight version endpoint'); // Both the SharedWorker path and the direct fallback route through the gate, - // never a bare location.reload() on a reload signal. - assert.match(clientSrc, /if \(m\.type === 'reload'\) __webjsReloadWhenReady\(\)/, 'the SharedWorker path gates the reload'); - assert.match(clientSrc, /addEventListener\('reload', \(\) => __webjsReloadWhenReady\(\)\)/, 'the direct fallback gates the reload'); - // The direct fallback reloads on a reconnect only when the boot id changed (a - // real restart), never on a same-process transient reconnect. - assert.match(clientSrc, /addEventListener\('hello'/, 'the fallback tracks the boot id via the hello frame'); - assert.match(clientSrc, /if \(lastBoot !== null && e\.data !== lastBoot\) __webjsReloadWhenReady\(\)/, 'only a changed boot id reloads'); + // never a bare location.reload() on a reload signal. Since #1397 the fallback + // reaches it through the shim port the shared relay posts to, which is the + // same message contract the SharedWorker path consumes. + assert.match(clientSrc, /if \(m\.type === 'reload'\) __webjsReloadWhenReady\(\)/, 'both paths gate the reload'); + assert.match(clientSrc, /else if \(m\.type === 'webjs-error'\) __webjsApplyError\(m\.data\)/, 'and both route an error frame to the overlay'); + // The boot-id rule (#893) lives in the relay now, in ONE place, rather than + // being re-implemented in the fallback where the two copies could drift. + assert.match(clientSrc, /if \(lastBoot !== null && e\.data !== lastBoot\) requestReload\(\)/, 'only a changed boot id reloads'); + assert.equal(clientSrc.match(/lastBoot !== null/g).length, 1, 'the boot-id rule exists exactly once'); }); test('the direct-fallback probe carries the base path under a sub-path deploy (#893 + #256)', async () => { diff --git a/test/bun/dev-public-before-warm.mjs b/test/bun/dev-public-before-warm.mjs new file mode 100644 index 000000000..e58588497 --- /dev/null +++ b/test/bun/dev-public-before-warm.mjs @@ -0,0 +1,115 @@ +/** + * Cross-runtime dev static-serve test (#1397): in dev, `GET /public/*` must be + * answered BEFORE the whole-app analysis completes, on BOTH the node:http and + * `Bun.serve` listener shells. + * + * The hoist moves a serve branch earlier in the request path, and the two + * listener shells reach `handle()` differently (the Bun shell may rebuild the + * `Request` in `forwardedRequest` and classifies the response body through + * `readBufferedOrStream` / `compressBufferSync`), so the invariant is worth + * proving on each runtime rather than on Node alone. + * + * The fixture's root middleware module has a top-level await sleep, and + * `loadMiddleware` runs inside `ensureReady()`, so the analysis is + * deterministically slow with no network involved. `/__webjs/ready` stays 503 + * until the analysis lands, which is the observable the assertion pivots on. + * + * COUNTERFACTUAL: revert the dev-only `tryServePublicAsset` call ahead of + * `await ensureReady()` in dev.js and the CSS request cannot resolve until warm + * completes, so `/__webjs/ready` is already 200 by the time it does. + * + * node test/bun/dev-public-before-warm.mjs + * bun test/bun/dev-public-before-warm.mjs + */ +import assert from 'node:assert/strict'; +import { spawn } from 'node:child_process'; +import { mkdtempSync, writeFileSync, mkdirSync, rmSync, symlinkSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { dirname, join, resolve } from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { connect } from 'node:net'; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const ROOT = resolve(__dirname, '../..'); +const CLI = join(ROOT, 'packages/cli/bin/webjs.js'); +const runtime = process.versions.bun ? `bun ${process.versions.bun}` : `node ${process.versions.node}`; +const PORT = 9500 + (process.pid % 240); +const BASE = `http://localhost:${PORT}`; + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + +/** Resolves once the port ACCEPTS a TCP connection, which happens before the + * first request is answered. Polling an HTTP route instead would be circular: + * the route under test is the one whose timing is being measured. */ +function portAccepts() { + return new Promise((res) => { + const s = connect(PORT, 'localhost'); + s.once('connect', () => { s.destroy(); res(true); }); + s.once('error', () => { s.destroy(); res(false); }); + }); +} + +async function until(fn, { timeoutMs, stepMs = 50 }) { + const deadline = Date.now() + timeoutMs; + for (;;) { + try { if (await fn()) return true; } catch { /* keep polling */ } + if (Date.now() > deadline) return false; + await sleep(stepMs); + } +} + +const dir = mkdtempSync(join(tmpdir(), 'webjs-public-warm-')); +let child; +try { + mkdirSync(join(dir, 'app'), { recursive: true }); + mkdirSync(join(dir, 'public'), { recursive: true }); + writeFileSync(join(dir, 'app/page.ts'), "import { html } from '@webjsdev/core';\nexport default () => html`

ok

`;\n"); + writeFileSync(join(dir, 'public/a.css'), 'body{color:red}\n'); + // Top-level await, so the module does not finish evaluating (and therefore + // `loadMiddleware`, and therefore `ensureReady()`, does not resolve) for 1.5s. + writeFileSync( + join(dir, 'middleware.ts'), + 'await new Promise((r) => setTimeout(r, 1500));\n' + + 'export default async function middleware(req: Request, next: () => Promise) { return next(); }\n', + ); + writeFileSync(join(dir, 'package.json'), JSON.stringify({ name: 'public-warm', type: 'module', imports: { '#*': './*' }, webjs: {} })); + mkdirSync(join(dir, 'node_modules/@webjsdev'), { recursive: true }); + symlinkSync(join(ROOT, 'packages/core'), join(dir, 'node_modules/@webjsdev/core')); + symlinkSync(join(ROOT, 'packages/server'), join(dir, 'node_modules/@webjsdev/server')); + + // `--no-hot` runs the server in-process, so the listener shell under test is + // the one this runtime provides rather than a respawned `node --watch` child. + child = spawn(process.execPath, [CLI, 'dev', '--port', String(PORT), '--no-hot'], { + cwd: dir, detached: true, stdio: ['ignore', 'pipe', 'pipe'], + env: { ...process.env, NODE_ENV: 'development' }, + }); + let log = ''; + child.stdout.on('data', (d) => { log += d; }); + child.stderr.on('data', (d) => { log += d; }); + + const listening = await until(portAccepts, { timeoutMs: 30_000 }); + assert.ok(listening, `dev server never listened on ${runtime}\n--- server log ---\n${log}`); + + // One pass: the CSS must answer while readiness is still gated on the + // analysis. Both requests are issued together so neither waits on the other. + const [css, ready] = await Promise.all([ + fetch(`${BASE}/public/a.css`), + fetch(`${BASE}/__webjs/ready`), + ]); + const cssBody = await css.text(); + assert.equal(css.status, 200, `/public/a.css was not served cold on ${runtime}\n--- server log ---\n${log}`); + assert.equal(cssBody, 'body{color:red}\n', `/public/a.css served the wrong bytes on ${runtime}`); + assert.equal( + ready.status, 503, + `the analysis had already completed on ${runtime}, so this proves nothing about ordering\n--- server log ---\n${log}`, + ); + + console.log(`OK dev serves /public/* before the analysis completes on ${runtime} (#1397)`); +} finally { + if (child && child.pid) { + try { process.kill(-child.pid, 'SIGTERM'); } catch { try { child.kill('SIGTERM'); } catch { /* gone */ } } + await sleep(500); + try { process.kill(-child.pid, 'SIGKILL'); } catch { /* gone */ } + } + rmSync(dir, { recursive: true, force: true }); +} diff --git a/test/bun/dev-public-before-warm.test.mjs b/test/bun/dev-public-before-warm.test.mjs new file mode 100644 index 000000000..bf17069b9 --- /dev/null +++ b/test/bun/dev-public-before-warm.test.mjs @@ -0,0 +1,13 @@ +/** + * Run the cross-runtime dev static-serve check (#1397) under WHICHEVER runtime + * runs the suite. `npm test` exercises the node:http shell; CI runs + * `bun test/bun/dev-public-before-warm.mjs` for the `Bun.serve` shell. The + * behaviour script is a plain assert file (`dev-public-before-warm.mjs`, not + * `*.test.mjs`, so the runner does not double-run it); importing it spawns the + * real CLI and throws on any failure. + */ +import { test } from 'node:test'; + +test('dev serves /public/* before the analysis completes on this runtime (#1397)', async () => { + await import('./dev-public-before-warm.mjs'); +}); diff --git a/website/app/docs/middleware/page.ts b/website/app/docs/middleware/page.ts index 2ab21d90f..169cc0358 100644 --- a/website/app/docs/middleware/page.ts +++ b/website/app/docs/middleware/page.ts @@ -8,7 +8,8 @@ export default function Middleware() {

Middleware in WebJs lets you intercept requests before they reach your pages, API routes, or server actions. Use it for authentication, logging, rate limiting, CORS, header injection, or any cross-cutting concern. WebJs supports two levels of middleware: a single root middleware and per-segment middleware scoped to subtrees of your route hierarchy.

Root Middleware

-

Place a middleware.ts at the root of your project (next to app/, not inside it). This middleware runs on every request before WebJs routes it to a page, API route, or server action. Any of middleware.ts, .js, .mts, or .mjs works, and .ts wins if you somehow have more than one.

+

Place a middleware.ts at the root of your project (next to app/, not inside it). This middleware runs on every app request before WebJs routes it to a page, API route, or server action. Any of middleware.ts, .js, .mts, or .mjs works, and .ts wins if you somehow have more than one.

+

Two things are served ahead of it, so root middleware never sees them. The framework's own /__webjs/* assets and health probes bypass it in both dev and production, because they are framework infrastructure your app needs to boot rather than app routes. In development only, static files under /public/* (plus the /sw.js and /offline.html root remaps and /favicon.ico) are served ahead of it as well, so a stylesheet is never queued behind the dev server's startup analysis. In production those static files go through root middleware normally, so a middleware that protects an asset still protects it where it counts.

my-app/ middleware.ts # root middleware: runs on every request app/ From 7c7489392ae6fd5bb69acd8547c4245aa2ed7b5f Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 18:40:23 +0530 Subject: [PATCH 02/13] fix: aim the public traversal test at the vector that reaches the guard The counterfactual did not discriminate: removing the containment check left the test green. `/public/%2E%2E/secret.txt` never enters the public branch at all, because the WHATWG URL parser decodes `%2E%2E` and normalises the dot segment away, so the request arrives as plain `/secret.txt`. The test was observing nothing. An encoded slash survives parsing intact, so `/public/..%2Fsecret.txt` enters the branch with a path `join` then resolves outside appDir/public/, which is what the guard is for. With that vector the test goes red when the guard is removed. The comment this was derived from asserted the opposite ("after URL parsing, which doesn't touch %2E"), so correct it where it now lives rather than carry an inaccurate claim into the extracted function. --- packages/server/src/dev.js | 13 ++++++++----- .../server/test/dev/public-before-analysis.test.js | 13 +++++++++++-- 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/packages/server/src/dev.js b/packages/server/src/dev.js index d65b60683..f3cee1e88 100644 --- a/packages/server/src/dev.js +++ b/packages/server/src/dev.js @@ -2111,14 +2111,17 @@ async function tryServePublicAsset(path, ctx) { const p = path === '/favicon.ico' ? '/public/favicon.ico' : (ROOT_ASSETS[path] || path); const abs = join(appDir, p); // Containment check. `join` normalises `..` segments, so a path - // like `/public/%2E%2E/secret/x.svg` decodes (after URL parsing, - // which doesn't touch `%2E`) to `/public/../secret/x.svg` and - // `join(appDir, ...)` resolves it to `appDir/secret/x.svg`. The - // resulting `abs` could be inside `appDir` but OUTSIDE `appDir/ - // public/`, exposing files the user reasonably thought were + // like `/public/..%2Fsecret/x.svg` decodes to `/public/../secret/ + // x.svg` and `join(appDir, ...)` resolves it to `appDir/secret/ + // x.svg`. The resulting `abs` could be inside `appDir` but OUTSIDE + // `appDir/public/`, exposing files the user reasonably thought were // private under their non-public directories. Reject anything // that doesn't stay under `appDir/public/` (and the favicon // exception, which is already validated above). + // The live vector encodes the SLASH, not the dots: the WHATWG URL + // parser decodes `%2E%2E` and normalises the dot segment away, so a + // `/public/%2E%2E/x` request arrives here as plain `/x` and never + // enters this branch. `..%2F` survives parsing intact and does. const publicRoot = join(appDir, 'public') + sep; if (!abs.startsWith(publicRoot)) { return new Response(null, { status: 404 }); diff --git a/packages/server/test/dev/public-before-analysis.test.js b/packages/server/test/dev/public-before-analysis.test.js index ea3e39433..1a8404552 100644 --- a/packages/server/test/dev/public-before-analysis.test.js +++ b/packages/server/test/dev/public-before-analysis.test.js @@ -113,12 +113,21 @@ test('root middleware does not run for /public/* in dev, and does in prod', asyn }); // COUNTERFACTUAL: drop the containment check from `tryServePublicAsset` and this -// serves the file, which is a directory-traversal hole. +// serves the file, which is a directory-traversal hole. Verified 2026-08-12 at +// 915c662d by removing the guard and watching this go red. +// +// The vector has to use an encoded SLASH (`..%2F`), not encoded dots. The +// WHATWG URL parser decodes `%2E%2E` to `..` and then normalises the dot +// segment away, so `/public/%2E%2E/secret.txt` arrives as `/secret.txt` and +// never enters the public branch at all: asserting on it would pass with the +// guard removed, which is a test that observes nothing. `..%2F` survives +// parsing intact, so the branch is entered with a path that `join` then +// resolves outside `appDir/public/`, which is exactly what the guard is for. test('the traversal guard travels with the moved code (dev early path)', async () => { const appDir = makeApp(); writeFileSync(join(appDir, 'secret.txt'), 'nope\n'); const app = await createRequestHandler({ appDir, dev: true }); - const res = await app.handle(new Request('http://x/public/%2E%2E/secret.txt')); + const res = await app.handle(new Request('http://x/public/..%2Fsecret.txt')); assert.equal(res.status, 404, 'a path that escapes appDir/public/ is refused'); assert.notEqual(await res.text(), 'nope\n'); }); From 0a6fcfbd135e72df5a8efd2e6c246a16853ad51d Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 18:43:33 +0530 Subject: [PATCH 03/13] test: assert the reload events URL where the relay call now carries it The base-path guard matched a literal `new EventSource("")` in the served client. The per-tab fallback now hands that URL to the shared relay instead of constructing the EventSource itself, so both cases went red on a shape change with the invariant intact. They assert the same thing at the call that carries the URL, and the negative widened from "no bare new EventSource(...)" to "no bare /__webjs/events string anywhere in the script", which is the stronger form of what it was guarding. --- packages/server/test/base-path/integration.test.js | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/packages/server/test/base-path/integration.test.js b/packages/server/test/base-path/integration.test.js index bcee558e3..2e4bda9a3 100644 --- a/packages/server/test/base-path/integration.test.js +++ b/packages/server/test/base-path/integration.test.js @@ -206,6 +206,9 @@ test('the dev reload client EventSource URL is base-path-prefixed (#256)', async // Regression: reload.js opens an EventSource to /__webjs/events, a // framework-emitted client URL. A bare path breaks dev live-reload under a // sub-path proxy (the script src was prefixed but the URL inside it was not). + // Since #1397 the per-tab fallback hands that URL to the shared relay rather + // than calling `new EventSource` itself, so the URL is asserted at the call + // that carries it. The invariant is unchanged: no bare path in the script. const appDir = makeApp({ basePath: '/myapp' }); const app = await createRequestHandler({ appDir, dev: true }); await app.warmup(); @@ -215,12 +218,12 @@ test('the dev reload client EventSource URL is base-path-prefixed (#256)', async const src = await res.text(); assert.match( src, - /new EventSource\("\/myapp\/__webjs\/events"\)/, + /startReloadWorker\(scope, EventSource, "\/myapp\/__webjs\/events"\)/, 'the EventSource URL must be prefixed with the base path', ); assert.ok( - !/new EventSource\("\/__webjs\/events"\)/.test(src), - 'the EventSource URL must not be a bare /__webjs/events', + !/"\/__webjs\/events"/.test(src), + 'no bare /__webjs/events URL survives anywhere in the script', ); }); @@ -234,7 +237,7 @@ test('the dev reload client EventSource URL is bare with no basePath (no-op)', a const src = await res.text(); assert.match( src, - /new EventSource\("\/__webjs\/events"\)/, + /startReloadWorker\(scope, EventSource, "\/__webjs\/events"\)/, 'the EventSource URL is the bare path when no basePath is set (byte-identical)', ); }); From 6088c3fbf9ff64eec988eb18e2376d03a91b05c6 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 21:13:33 +0530 Subject: [PATCH 04/13] fix: isolate the shim port from application throws, and close the doc drift Review found five things. The shim port the per-tab fallback hands the relay runs application code synchronously in its postMessage, and the relay deletes a port whose postMessage throws. That heuristic reads a throw as "the tab is gone", which is right for a real MessagePort and wrong here: an overlay render that threw would permanently unsubscribe the tab and silently kill its live reload. The pre-#1397 fallback attached to the EventSource directly, where a handler throw detached nothing, so the guard restores the old behaviour rather than adding a new one. The ?v= test claimed to cover the early path while building the handler with dev: false, so it exercised the prod path instead. It cannot be written the other way, since fileResponse hard-codes no-cache whenever dev is true, so immutable is unobservable there. Retitled to what it actually pins, plus a dev-mode assertion that a fingerprinted asset still serves on the early path. Correcting the middleware prose left two code-block comments two lines below still saying "every request", and the same claim was still on the agent-facing skill reference the scaffold ships, in the root AGENTS.md layout table, and in the blog example's own middleware. Inserting the two constants between the module header and the function orphaned its @param tags onto a constant, and the header no longer mentioned that reloadClientJs inlines this source too. --- .../webjs/references/muscle-memory-gotchas.md | 2 +- AGENTS.md | 2 +- examples/blog/middleware.ts | 2 +- packages/server/src/dev-reload-worker.js | 22 +++++++++++------ packages/server/src/dev.js | 15 ++++++++++-- .../test/dev/public-before-analysis.test.js | 18 ++++++++++++-- .../test/dev/reload-shared-connection.test.js | 24 +++++++++++++++++++ website/app/docs/middleware/page.ts | 4 ++-- 8 files changed, 73 insertions(+), 16 deletions(-) diff --git a/.agents/skills/webjs/references/muscle-memory-gotchas.md b/.agents/skills/webjs/references/muscle-memory-gotchas.md index d6c9c8ecd..2ffdd199f 100644 --- a/.agents/skills/webjs/references/muscle-memory-gotchas.md +++ b/.agents/skills/webjs/references/muscle-memory-gotchas.md @@ -202,7 +202,7 @@ Export `GET` / `POST` / etc. as named async functions `(request, { params }) => ### `middleware.ts` is per-segment and chainable, not one matcher config -The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every request, outermost to innermost. +The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every app request, outermost to innermost. Two things are served ahead of it: the framework's own `/__webjs/*` assets and probes, in both modes, and in DEV only, `/public/*` plus the `/sw.js` / `/offline.html` root remaps and `/favicon.ico`, so a stylesheet is never queued behind the dev startup analysis. In production those static files go through root middleware normally, so a middleware that guards an asset still guards it where it counts. ### No ``, no `next/navigation`, no `next/*` libraries diff --git a/AGENTS.md b/AGENTS.md index cf91902be..892126ec3 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -185,7 +185,7 @@ app/ ROUTING ONLY (thin adapters importing from modules/; /route.js HTTP handler at / /middleware.js per-segment middleware /loading.js auto Suspense boundary -middleware.js root middleware (every request; .ts/.mts/.mjs too) +middleware.js root middleware (every app request; .ts/.mts/.mjs too) readiness.js optional /__webjs/ready check (return false/throw = 503) env.js optional boot-time env validation (schema or validator fn; fails fast) instrumentation.js optional boot-time hook (register(); wire APM via setOnError, #848) diff --git a/examples/blog/middleware.ts b/examples/blog/middleware.ts index a752f1137..e4590fb14 100644 --- a/examples/blog/middleware.ts +++ b/examples/blog/middleware.ts @@ -1,5 +1,5 @@ /** - * Global middleware. Runs on every request before webjs routes it. + * Global middleware. Runs on every app request before webjs routes it. * Return a Response to short-circuit; call next() to continue. * * To add framework sessions: diff --git a/packages/server/src/dev-reload-worker.js b/packages/server/src/dev-reload-worker.js index 61128ce1c..0fa96a368 100644 --- a/packages/server/src/dev-reload-worker.js +++ b/packages/server/src/dev-reload-worker.js @@ -2,19 +2,19 @@ * The dev live-reload SharedWorker relay (#887), the BROWSER half. Kept as a * standalone browser-safe module (no node imports) so the served worker inlines * the EXACT source a browser test drives, with no drift, the same pattern as - * `dev-overlay.js` (#264). `reloadWorkerJs` in dev.js reads this file, strips - * the `export` keyword, and appends a - * `startReloadWorker(self, EventSource, '')` call. + * `dev-overlay.js` (#264). + * + * BOTH served dev scripts inline this file, `export`-stripped. `reloadWorkerJs` + * appends a `startReloadWorker(self, EventSource, '')` call for the + * SharedWorker, and since #1397 `reloadClientJs` inlines it too, so the per-tab + * fallback runs this same relay over a shim port instead of a second copy of + * the boot-id rule and the reload debounce. * * One SharedWorker is shared across every tab of the origin (a SharedWorker is * keyed by its script URL), so it holds the ONE `EventSource` to * `/__webjs/events` and fans each `reload` / `webjs-error` out to every tab over * its `MessagePort`. Tab count never touches the browser's per-host HTTP/1.1 * connection cap, which the per-tab `EventSource` it replaces used to exhaust. - * - * @param {{ onconnect: any }} scope the worker global (`self`) - * @param {new (url: string) => any} EventSourceCtor the `EventSource` constructor - * @param {string} eventsUrl the base-path-aware `/__webjs/events` URL */ /** @@ -42,6 +42,14 @@ export const RELOAD_QUIET_MS = 2000; */ export const RELOAD_MAX_HOLD_MS = 5000; +/** + * @param {{ onconnect: any, setTimeout?: any, clearTimeout?: any }} scope the + * worker global (`self`), or a plain shim object for the per-tab fallback. + * Timers are read off it when it has them, which is what lets a browser test + * drive the debounce on a fake clock. + * @param {new (url: string) => any} EventSourceCtor the `EventSource` constructor + * @param {string} eventsUrl the base-path-aware `/__webjs/events` URL + */ export function startReloadWorker(scope, EventSourceCtor, eventsUrl) { /** @type {Set} */ const ports = new Set(); diff --git a/packages/server/src/dev.js b/packages/server/src/dev.js index f3cee1e88..9a236b136 100644 --- a/packages/server/src/dev.js +++ b/packages/server/src/dev.js @@ -3026,8 +3026,19 @@ function __webjsDirectEvents() { const scope = {}; startReloadWorker(scope, EventSource, ${eventsUrl}); scope.onconnect({ ports: [{ start() {}, postMessage(m) { - if (m.type === 'reload') __webjsReloadWhenReady(); - else if (m.type === 'webjs-error') __webjsApplyError(m.data); + // Nothing may throw out of here. The relay's fanout deletes a port whose + // postMessage throws, which is the right read for a REAL MessagePort (a + // throw there means the tab is gone) and the wrong one for this shim, + // whose postMessage runs application code synchronously: an overlay + // render that threw would permanently unsubscribe this tab and silently + // kill live reload for the rest of the page's life. The old fallback + // attached to the EventSource directly, where a handler throw detached + // nothing, so swallowing here restores that behaviour rather than adding + // a new one. + try { + if (m.type === 'reload') __webjsReloadWhenReady(); + else if (m.type === 'webjs-error') __webjsApplyError(m.data); + } catch (_) { /* a bad frame must not cost this tab its live reload */ } } }] }); } try { diff --git a/packages/server/test/dev/public-before-analysis.test.js b/packages/server/test/dev/public-before-analysis.test.js index 1a8404552..8a95b204d 100644 --- a/packages/server/test/dev/public-before-analysis.test.js +++ b/packages/server/test/dev/public-before-analysis.test.js @@ -160,11 +160,25 @@ test('a stale webjs.dev.regenerate output is rebuilt before serving on the early assert.equal(await res.text(), 'body{color:lime}', 'the stale output was regenerated, not served as-is'); }); -// #243: the `?v=` fingerprint still decides the cache header on the early path. -test('?v= still yields immutable on the early path, un-versioned keeps the 1h fallback', async () => { +// #243: the `?v=` fingerprint still decides the cache header through the +// extracted function. This is the PROD path deliberately, and it is the one +// case in this file that is not about the dev hoist. `fileResponse` hard-codes +// `cache-control: no-cache` whenever `opts.dev` is true, so `immutable` is +// unobservable on the dev early path by construction; asserting it there would +// be a test that cannot fail. What this pins is that the extraction did not +// drop the fingerprint handling on the call site that still serves it. +test('?v= still yields immutable through the extracted serve (prod), un-versioned keeps the 1h fallback', async () => { const app = await createRequestHandler({ appDir: makeApp(), dev: false }); const versioned = await app.handle(new Request('http://x/public/a.css?v=abc123')); assert.match(versioned.headers.get('cache-control') || '', /immutable/, 'content-addressed is immutable'); const plain = await app.handle(new Request('http://x/public/a.css')); assert.doesNotMatch(plain.headers.get('cache-control') || '', /immutable/, 'un-fingerprinted is not'); + + // The dev half of the same concern, which IS on the early path: a `?v=` + // request still resolves and serves rather than falling through, even though + // the header it earns in dev is `no-cache` either way. + const devApp = await createRequestHandler({ appDir: makeApp(), dev: true }); + const devVersioned = await devApp.handle(new Request('http://x/public/a.css?v=abc123')); + assert.equal(devVersioned.status, 200, 'a fingerprinted asset still serves on the dev early path'); + assert.equal(await devVersioned.text(), 'body{color:red}\n'); }); diff --git a/packages/server/test/dev/reload-shared-connection.test.js b/packages/server/test/dev/reload-shared-connection.test.js index 1dd21ea35..23a9f9c4f 100644 --- a/packages/server/test/dev/reload-shared-connection.test.js +++ b/packages/server/test/dev/reload-shared-connection.test.js @@ -48,6 +48,30 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E assert.match(clientSrc, /function startReloadWorker/, 'the client inlines the relay module for the fallback'); assert.match(clientSrc, /startReloadWorker\(scope, EventSource, "\/__webjs\/events"\)/, 'the fallback runs the relay against the real EventSource'); assert.match(clientSrc, /scope\.onconnect\(\{ ports: \[\{/, 'and drives it over a shim port'); + // The shim's postMessage runs application code synchronously, and the relay's + // fanout DELETES a port whose postMessage throws (correct for a real + // MessagePort, where a throw means the tab is gone). Unguarded, an overlay + // render that threw would permanently unsubscribe the tab and silently kill + // its live reload, which the pre-#1397 fallback could not do because it + // attached to the EventSource directly. + // + // Sliced to the fallback function's own body first. A regex over the whole + // client matches the SharedWorker bootstrap's `try { ... } catch` further + // down and passes with the guard removed, which is a test that observes + // nothing (found by running exactly that counterfactual). + const fallbackStart = clientSrc.indexOf('function __webjsDirectEvents()'); + assert.notEqual(fallbackStart, -1, 'the fallback function is in the client'); + const fallbackBody = clientSrc.slice(fallbackStart, clientSrc.indexOf('if (typeof SharedWorker', fallbackStart)); + assert.match( + fallbackBody, + /postMessage\(m\)\s*\{[^]*?try\s*\{/, + 'the shim port opens a try before running any application code', + ); + assert.match(fallbackBody, /\}\s*catch\s*\(_\)/, 'and swallows the throw so the relay cannot drop the tab'); + assert.ok( + fallbackBody.indexOf('try {') < fallbackBody.indexOf('__webjsReloadWhenReady()'), + 'the guard opens BEFORE the reload call, not around something else', + ); assert.match(clientSrc, /catch\s*\(_\)\s*\{\s*__webjsDirectEvents/, 'a worker failure falls back'); // The debounce (#1397) is part of the relay, so it ships in BOTH scripts. assert.match(clientSrc, /const RELOAD_QUIET_MS/, 'the reload debounce ships in the client fallback'); diff --git a/website/app/docs/middleware/page.ts b/website/app/docs/middleware/page.ts index 169cc0358..b4825dbe5 100644 --- a/website/app/docs/middleware/page.ts +++ b/website/app/docs/middleware/page.ts @@ -11,7 +11,7 @@ export default function Middleware() {

Place a middleware.ts at the root of your project (next to app/, not inside it). This middleware runs on every app request before WebJs routes it to a page, API route, or server action. Any of middleware.ts, .js, .mts, or .mjs works, and .ts wins if you somehow have more than one.

Two things are served ahead of it, so root middleware never sees them. The framework's own /__webjs/* assets and health probes bypass it in both dev and production, because they are framework infrastructure your app needs to boot rather than app routes. In development only, static files under /public/* (plus the /sw.js and /offline.html root remaps and /favicon.ico) are served ahead of it as well, so a stylesheet is never queued behind the dev server's startup analysis. In production those static files go through root middleware normally, so a middleware that protects an asset still protects it where it counts.

my-app/ - middleware.ts # root middleware: runs on every request + middleware.ts # root middleware: runs on every app request app/ page.ts api/ @@ -33,7 +33,7 @@ export default async function middleware(

Per-Segment Middleware

Place a middleware.ts inside any directory under app/ to scope it to that subtree. It runs only for requests whose URL matches that segment and its children.

my-app/ - middleware.ts # root: every request + middleware.ts # root: every app request app/ page.ts # /: root + no segment middleware dashboard/ From 5d48e5f9d89ba3012f07de1130114ee8a22cf535 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 21:27:47 +0530 Subject: [PATCH 05/13] fix: report the swallowed fallback throw, and fix the vacuous slice Delta review of the previous commit found four things. The shim port's catch was empty, which hid dev-overlay bugs on that one path. A throw out of an EventSource listener (the old shape) and out of the SharedWorker path's onmessage ten lines below both reach the console, so discarding it here was a new behaviour rather than the restored one the comment claimed. It is reported now. The slice meant to anchor that guard's assertion to the fallback function ended at the bootstrap that follows it, so it still trailed a closing brace plus `try {`, leaving the bootstrap's try inside the slice. The first assertion therefore matched with the guard removed, and the comment asserted a counterfactual that did not hold. The slice now ends at the function's own closing brace, with an assertion that the bootstrap is outside it, and each assertion was re-checked to fail with the guard gone. The architecture page's Request Lifecycle said root middleware runs first and put internal endpoints and static files after it. That was already wrong for /__webjs/* before this PR, and the dev /public/* hoist widens it, so two pages on the same site contradicted each other on the fact this PR exists to correct. The blog example's middleware comment wrote the brand as a lowercase code token in prose, which invariant 11 does not allow. --- examples/blog/middleware.ts | 2 +- packages/server/src/dev.js | 9 ++++++++- .../test/dev/reload-shared-connection.test.js | 17 ++++++++++++++--- website/app/docs/architecture/page.ts | 5 +++-- 4 files changed, 26 insertions(+), 7 deletions(-) diff --git a/examples/blog/middleware.ts b/examples/blog/middleware.ts index e4590fb14..5537f4957 100644 --- a/examples/blog/middleware.ts +++ b/examples/blog/middleware.ts @@ -1,5 +1,5 @@ /** - * Global middleware. Runs on every app request before webjs routes it. + * Global middleware. Runs on every app request before WebJs routes it. * Return a Response to short-circuit; call next() to continue. * * To add framework sessions: diff --git a/packages/server/src/dev.js b/packages/server/src/dev.js index 9a236b136..b4c602def 100644 --- a/packages/server/src/dev.js +++ b/packages/server/src/dev.js @@ -3038,7 +3038,14 @@ function __webjsDirectEvents() { try { if (m.type === 'reload') __webjsReloadWhenReady(); else if (m.type === 'webjs-error') __webjsApplyError(m.data); - } catch (_) { /* a bad frame must not cost this tab its live reload */ } + } catch (_) { + // Reported, never swallowed. Detachment is the only thing being + // prevented: a throw out of an EventSource listener (the old shape) and + // out of the SharedWorker path's onmessage below both surface to the + // console, so discarding it here would hide dev-overlay bugs on this one + // path, which would be a new behaviour rather than a restored one. + console.error('[webjs] dev reload handler threw', _); + } } }] }); } try { diff --git a/packages/server/test/dev/reload-shared-connection.test.js b/packages/server/test/dev/reload-shared-connection.test.js index 23a9f9c4f..c2292e6f9 100644 --- a/packages/server/test/dev/reload-shared-connection.test.js +++ b/packages/server/test/dev/reload-shared-connection.test.js @@ -58,16 +58,27 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E // Sliced to the fallback function's own body first. A regex over the whole // client matches the SharedWorker bootstrap's `try { ... } catch` further // down and passes with the guard removed, which is a test that observes - // nothing (found by running exactly that counterfactual). + // nothing. + // + // The slice must end at the function's OWN closing brace, not at the + // bootstrap that follows it: ending at `indexOf('if (typeof SharedWorker')` + // still trails `}\ntry {`, which leaves the bootstrap's `try` inside the + // slice and the first assertion below vacuous again. Verified by running the + // counterfactual against a reconstructed client: each of the three + // assertions below fails with the guard removed. const fallbackStart = clientSrc.indexOf('function __webjsDirectEvents()'); assert.notEqual(fallbackStart, -1, 'the fallback function is in the client'); - const fallbackBody = clientSrc.slice(fallbackStart, clientSrc.indexOf('if (typeof SharedWorker', fallbackStart)); + const fallbackEnd = clientSrc.indexOf('\n}\n', fallbackStart); + assert.notEqual(fallbackEnd, -1, 'the fallback function closes'); + const fallbackBody = clientSrc.slice(fallbackStart, fallbackEnd + 2); + assert.ok(!/if \(typeof SharedWorker/.test(fallbackBody), 'the slice stops before the bootstrap'); assert.match( fallbackBody, /postMessage\(m\)\s*\{[^]*?try\s*\{/, 'the shim port opens a try before running any application code', ); - assert.match(fallbackBody, /\}\s*catch\s*\(_\)/, 'and swallows the throw so the relay cannot drop the tab'); + assert.match(fallbackBody, /\}\s*catch\s*\(_\)/, 'and catches the throw so the relay cannot drop the tab'); + assert.match(fallbackBody, /console\.error\(/, 'and reports it rather than discarding it'); assert.ok( fallbackBody.indexOf('try {') < fallbackBody.indexOf('__webjsReloadWhenReady()'), 'the guard opens BEFORE the reload call, not around something else', diff --git a/website/app/docs/architecture/page.ts b/website/app/docs/architecture/page.ts index d638e03c5..62a048eed 100644 --- a/website/app/docs/architecture/page.ts +++ b/website/app/docs/architecture/page.ts @@ -103,9 +103,10 @@ import { listPosts } from '#modules/posts/queries/list-posts.server.ts';Request Lifecycle
  1. HTTP request arrives at the Node HTTP server (or HTTP/2 if TLS configured).
    2. -
  2. Root middleware (middleware.ts) runs first if present.
    3. +
  3. Framework-internal assets and probes (/__webjs/*: the core runtime, the dev reload client, downloaded vendor bundles, /__webjs/health and /__webjs/ready) are served here, ahead of everything below. They depend on neither the app analysis nor the vendor importmap, so a cold instance must not gate them. In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
    4. +
  4. Root middleware (middleware.ts) runs next if present, for every request that was not already answered above.
  5. 103 Early Hints sent (prod only) with modulepreload URLs for the matched page.
    6. -
  6. Route matching: the router tries (in order) internal endpoints, static files, user source modules, API routes (route.ts), then page routes.
    7. +
  7. Route matching: the router tries (in order) static files, user source modules, API routes (route.ts), then page routes. In production this is where /public/* is served, so a middleware that guards an asset still guards it.
  8. Segment middleware chain runs (outermost → innermost) for the matched route.
  9. For pages: SSR pipeline runs (load page + layouts, render to HTML, inject DSD, collect metadata, stream response with Suspense).
  10. For API routes: the matched handler function runs, returns a Response.
    11. From 4cf6e893c9d0a8be00b0f65935374fb860d43f21 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 21:37:36 +0530 Subject: [PATCH 06/13] fix: correct the contradicting comment, the -1 comparison, and the action step Second delta review found three things, two of them mine from the round before. The block comment above the shim guard still ended with "swallowing here restores that behaviour", which the previous commit made false on both halves: the code no longer swallows, and swallowing was never the restored behaviour. Two adjacent comments asserted opposite things about the same line, and the stale one came first. Merged into one statement of what is actually being prevented, which is detachment and not reporting. The ordering assertion compared indexOf results directly, and indexOf returns -1 for a missing needle, which is less than any real index. So with the guard removed entirely, the one case the assertion exists for, it passed. Both indices are asserted present first now. The comment above also claimed three assertions where five follow it, so it named the wrong set; it now says how each was checked instead of counting them. The architecture page's new step 2 said everything under /__webjs/* is answered before root middleware. The server-action RPC endpoint is dispatched in handleCore, which is reached through next() after middleware runs, so that was wrong in the direction that matters: it would tell someone gating actions with auth or rate-limit middleware that their middleware does not run. Step 2 now names the set it actually covers, and the action endpoint is back in the routing step with a note that middleware does run for it. --- packages/server/src/dev.js | 28 ++++++++++--------- .../test/dev/reload-shared-connection.test.js | 24 +++++++++++----- website/app/docs/architecture/page.ts | 4 +-- 3 files changed, 34 insertions(+), 22 deletions(-) diff --git a/packages/server/src/dev.js b/packages/server/src/dev.js index b4c602def..8373d3e34 100644 --- a/packages/server/src/dev.js +++ b/packages/server/src/dev.js @@ -3026,24 +3026,26 @@ function __webjsDirectEvents() { const scope = {}; startReloadWorker(scope, EventSource, ${eventsUrl}); scope.onconnect({ ports: [{ start() {}, postMessage(m) { - // Nothing may throw out of here. The relay's fanout deletes a port whose - // postMessage throws, which is the right read for a REAL MessagePort (a - // throw there means the tab is gone) and the wrong one for this shim, - // whose postMessage runs application code synchronously: an overlay - // render that threw would permanently unsubscribe this tab and silently - // kill live reload for the rest of the page's life. The old fallback + // Nothing may throw out of here, and nothing may be silently dropped. + // + // The relay's fanout deletes a port whose postMessage throws, which is the + // right read for a REAL MessagePort (a throw there means the tab is gone) + // and the wrong one for this shim, whose postMessage runs application code + // synchronously: an overlay render that threw would permanently + // unsubscribe this tab and silently kill live reload for the rest of the + // page's life. So the throw is contained here. + // + // Contained, NOT swallowed, and the difference matters. The old fallback // attached to the EventSource directly, where a handler throw detached - // nothing, so swallowing here restores that behaviour rather than adding - // a new one. + // nothing but still reached the console, and so does a throw out of the + // SharedWorker path's onmessage below. Only the DETACHMENT is being + // prevented; discarding the error would make this the one path where a + // dev-overlay bug leaves no trace, which would be a new behaviour rather + // than a restored one. try { if (m.type === 'reload') __webjsReloadWhenReady(); else if (m.type === 'webjs-error') __webjsApplyError(m.data); } catch (_) { - // Reported, never swallowed. Detachment is the only thing being - // prevented: a throw out of an EventSource listener (the old shape) and - // out of the SharedWorker path's onmessage below both surface to the - // console, so discarding it here would hide dev-overlay bugs on this one - // path, which would be a new behaviour rather than a restored one. console.error('[webjs] dev reload handler threw', _); } } }] }); diff --git a/packages/server/test/dev/reload-shared-connection.test.js b/packages/server/test/dev/reload-shared-connection.test.js index c2292e6f9..c614c5f1e 100644 --- a/packages/server/test/dev/reload-shared-connection.test.js +++ b/packages/server/test/dev/reload-shared-connection.test.js @@ -63,9 +63,14 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E // The slice must end at the function's OWN closing brace, not at the // bootstrap that follows it: ending at `indexOf('if (typeof SharedWorker')` // still trails `}\ntry {`, which leaves the bootstrap's `try` inside the - // slice and the first assertion below vacuous again. Verified by running the - // counterfactual against a reconstructed client: each of the three - // assertions below fails with the guard removed. + // slice and the guard assertion vacuous again. + // + // Each assertion below was checked INDIVIDUALLY against the counterfactual, + // not just the file as a whole. Checking the file is what let two vacuous + // assertions through earlier here: the run went red on a later assertion and + // the earlier one was recorded as discriminating without being looked at. + // Removing the guard fails the `try`-present and ordering assertions; + // removing only the `console.error` fails the reporting one. const fallbackStart = clientSrc.indexOf('function __webjsDirectEvents()'); assert.notEqual(fallbackStart, -1, 'the fallback function is in the client'); const fallbackEnd = clientSrc.indexOf('\n}\n', fallbackStart); @@ -79,10 +84,15 @@ test('dev serves the reload SharedWorker, and the client uses it with a direct E ); assert.match(fallbackBody, /\}\s*catch\s*\(_\)/, 'and catches the throw so the relay cannot drop the tab'); assert.match(fallbackBody, /console\.error\(/, 'and reports it rather than discarding it'); - assert.ok( - fallbackBody.indexOf('try {') < fallbackBody.indexOf('__webjsReloadWhenReady()'), - 'the guard opens BEFORE the reload call, not around something else', - ); + // Both indices are asserted present FIRST. `indexOf` returns -1 for a + // missing needle, and -1 is less than any real index, so a bare `<` + // comparison passes when the guard is gone entirely, which is the one case + // this assertion exists for. + const tryAt = fallbackBody.indexOf('try {'); + const reloadAt = fallbackBody.indexOf('__webjsReloadWhenReady()'); + assert.notEqual(tryAt, -1, 'the guard is present at all'); + assert.notEqual(reloadAt, -1, 'the reload call is present at all'); + assert.ok(tryAt < reloadAt, 'the guard opens BEFORE the reload call, not around something else'); assert.match(clientSrc, /catch\s*\(_\)\s*\{\s*__webjsDirectEvents/, 'a worker failure falls back'); // The debounce (#1397) is part of the relay, so it ships in BOTH scripts. assert.match(clientSrc, /const RELOAD_QUIET_MS/, 'the reload debounce ships in the client fallback'); diff --git a/website/app/docs/architecture/page.ts b/website/app/docs/architecture/page.ts index 62a048eed..90f41397a 100644 --- a/website/app/docs/architecture/page.ts +++ b/website/app/docs/architecture/page.ts @@ -103,10 +103,10 @@ import { listPosts } from '#modules/posts/queries/list-posts.server.ts';Request Lifecycle
    1. HTTP request arrives at the Node HTTP server (or HTTP/2 if TLS configured).
      2. -
    2. Framework-internal assets and probes (/__webjs/*: the core runtime, the dev reload client, downloaded vendor bundles, /__webjs/health and /__webjs/ready) are served here, ahead of everything below. They depend on neither the app analysis nor the vendor importmap, so a cold instance must not gate them. In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
      3. +
    3. Some framework-internal assets and probes are answered here, ahead of everything below, because they depend on neither the app analysis nor the vendor importmap and a cold instance must not gate them: the health and readiness probes (/__webjs/health, /__webjs/ready), the build-info probe (/__webjs/version), the core runtime (/__webjs/core/*), the dev reload client and its SharedWorker (/__webjs/reload.js, /__webjs/reload-worker.js), and downloaded vendor bundles (/__webjs/vendor/*). This is a named set, not all of /__webjs/*: the server-action RPC endpoint is NOT in it (see step 5). In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
    4. Root middleware (middleware.ts) runs next if present, for every request that was not already answered above.
    5. 103 Early Hints sent (prod only) with modulepreload URLs for the matched page.
      6. -
    6. Route matching: the router tries (in order) static files, user source modules, API routes (route.ts), then page routes. In production this is where /public/* is served, so a middleware that guards an asset still guards it.
      7. +
    7. Route matching: the router tries (in order) the server-action RPC endpoint (/__webjs/action/<hash>/<fn>), static files, user source modules, API routes (route.ts), then page routes. The action endpoint sits here rather than in step 2, so root middleware DOES run for a server action, which is what lets you gate actions with auth or rate limiting. In production this is also where /public/* is served, so a middleware that guards an asset still guards it.
    8. Segment middleware chain runs (outermost → innermost) for the matched route.
    9. For pages: SSR pipeline runs (load page + layouts, render to HTML, inject DSD, collect metadata, stream response with Suspense).
    10. For API routes: the matched handler function runs, returns a Response.
      11. From 3d57568547259b2ac5caed83d6d93c7744429ac4 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 21:47:34 +0530 Subject: [PATCH 07/13] docs: state the middleware bypass rule instead of a closed list Third delta review found three more members of an enumeration I kept asserting was complete. Every round has found another one, so this stops listing and states the rule: anything the listener shell or the pre-analysis stage answers bypasses root middleware, and everything routed with the app reaches it. The examples stay, but as examples. The two genuinely missing members are real bypasses. A WebSocket upgrade bound for a route.ts exporting WS is intercepted at the server level on both listener shells before the app handler is called, and it is unambiguously an app route, which is the line the reworded "every app request" was drawn to make. The dev SSE stream at /__webjs/events is intercepted the same way, so the previous wording, which said the named set was everything under /__webjs/* except the action endpoint, was wrong in the direction where believing it costs you. Inserting a step into the architecture lifecycle also shifted the list under a "Step 6 above" reference further down the page, which now pointed at segment middleware instead of the SSR pipeline. That reference is by name now, so inserting a step cannot break it again. The lifecycle also listed WebSocket upgrades as a late step, after middleware and route matching, which the same interception contradicts. --- .../webjs/references/muscle-memory-gotchas.md | 2 +- website/app/docs/architecture/page.ts | 14 +++++++------- website/app/docs/middleware/page.ts | 3 ++- 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/.agents/skills/webjs/references/muscle-memory-gotchas.md b/.agents/skills/webjs/references/muscle-memory-gotchas.md index 2ffdd199f..f148e0019 100644 --- a/.agents/skills/webjs/references/muscle-memory-gotchas.md +++ b/.agents/skills/webjs/references/muscle-memory-gotchas.md @@ -202,7 +202,7 @@ Export `GET` / `POST` / etc. as named async functions `(request, { params }) => ### `middleware.ts` is per-segment and chainable, not one matcher config -The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every app request, outermost to innermost. Two things are served ahead of it: the framework's own `/__webjs/*` assets and probes, in both modes, and in DEV only, `/public/*` plus the `/sw.js` / `/offline.html` root remaps and `/favicon.ico`, so a stylesheet is never queued behind the dev startup analysis. In production those static files go through root middleware normally, so a middleware that guards an asset still guards it where it counts. +The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every app request, outermost to innermost. Some requests are answered before it and never reach it, and the RULE is what to remember, not the list: anything the listener shell or the framework's pre-analysis stage answers bypasses root middleware, and everything routed with the app reaches it. That covers WebSocket upgrades bound for a `route.ts` exporting `WS`, the dev SSE stream at `/__webjs/events`, and the framework's own `/__webjs/*` runtime assets and probes; in DEV only it also covers `/public/*` plus the `/sw.js` / `/offline.html` root remaps and `/favicon.ico`, so a stylesheet is never queued behind the dev startup analysis. In production those static files go through root middleware normally. Server actions are routed with the app, so middleware DOES run for an action call, which is what lets you gate actions with auth or rate limiting. ### No ``, no `next/navigation`, no `next/*` libraries diff --git a/website/app/docs/architecture/page.ts b/website/app/docs/architecture/page.ts index 90f41397a..dfde337ba 100644 --- a/website/app/docs/architecture/page.ts +++ b/website/app/docs/architecture/page.ts @@ -102,21 +102,21 @@ import { listPosts } from '#modules/posts/queries/list-posts.server.ts';Request Lifecycle
        -
      1. HTTP request arrives at the Node HTTP server (or HTTP/2 if TLS configured).
        2. -
      2. Some framework-internal assets and probes are answered here, ahead of everything below, because they depend on neither the app analysis nor the vendor importmap and a cold instance must not gate them: the health and readiness probes (/__webjs/health, /__webjs/ready), the build-info probe (/__webjs/version), the core runtime (/__webjs/core/*), the dev reload client and its SharedWorker (/__webjs/reload.js, /__webjs/reload-worker.js), and downloaded vendor bundles (/__webjs/vendor/*). This is a named set, not all of /__webjs/*: the server-action RPC endpoint is NOT in it (see step 5). In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
        3. -
      3. Root middleware (middleware.ts) runs next if present, for every request that was not already answered above.
        4. +
      4. HTTP request arrives at the listener shell (node:http, or Bun.serve on Bun; HTTP/2 if TLS is configured).
        5. +
      5. The listener shell answers what does not fit the request/response model, before the app handler is called at all: a WebSocket upgrade bound for a route.ts that exports WS, and in dev the live-reload SSE stream at /__webjs/events. Both are intercepted on the node and Bun shells alike.
        6. +
      6. Framework-internal assets and probes are answered next, because they depend on neither the app analysis nor the vendor importmap and a cold instance must not gate them: the health, readiness and build-info probes (/__webjs/health, /__webjs/ready, /__webjs/version), the core runtime (/__webjs/core/*), the dev reload client and its SharedWorker (/__webjs/reload.js, /__webjs/reload-worker.js), and downloaded vendor bundles (/__webjs/vendor/*). Not everything under /__webjs/* is here: the server-action RPC endpoint is routed with the app, below. In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
        7. +
      7. Root middleware (middleware.ts) runs next if present, for every request not already answered above. The rule behind the exceptions is worth holding onto rather than the list: anything the listener shell or the pre-analysis stage answers never reaches your middleware, and everything routed with the app does.
      8. 103 Early Hints sent (prod only) with modulepreload URLs for the matched page.
        9. -
      9. Route matching: the router tries (in order) the server-action RPC endpoint (/__webjs/action/<hash>/<fn>), static files, user source modules, API routes (route.ts), then page routes. The action endpoint sits here rather than in step 2, so root middleware DOES run for a server action, which is what lets you gate actions with auth or rate limiting. In production this is also where /public/* is served, so a middleware that guards an asset still guards it.
        10. +
      10. Route matching: the router tries (in order) the server-action RPC endpoint (/__webjs/action/<hash>/<fn>), static files, user source modules, API routes (route.ts), then page routes. The action endpoint is routed here rather than answered early, so root middleware DOES run for a server action, which is what lets you gate actions with auth or rate limiting. In production this is also where /public/* is served, so a middleware that guards an asset still guards it.
      11. Segment middleware chain runs (outermost → innermost) for the matched route.
        12. -
      12. For pages: SSR pipeline runs (load page + layouts, render to HTML, inject DSD, collect metadata, stream response with Suspense).
        13. +
      13. For pages: the SSR pipeline runs (load page + layouts, render to HTML, inject DSD, collect metadata, stream response with Suspense).
      14. For API routes: the matched handler function runs, returns a Response.
        15. -
      15. For WebSocket upgrades: the WS handler is invoked with the ws object + Request.
      16. Response is sent (with compression in prod and cache headers).

      Progressive Enhancement

      - Step 6 above produces real HTML. The SSR pipeline runs every web component's render() on the server, so the component's initial markup is in the response before any script loads. The browser paints content, processes <a> links, and handles <form> submissions before any JavaScript runs. The client router, custom-element upgrades, and Suspense streaming are layered on top of that HTML. They enhance an already-working page, they do not constitute it. + The SSR pipeline above produces real HTML. It runs every web component's render() on the server, so the component's initial markup is in the response before any script loads. The browser paints content, processes <a> links, and handles <form> submissions before any JavaScript runs. The client router, custom-element upgrades, and Suspense streaming are layered on top of that HTML. They enhance an already-working page, they do not constitute it.

      • Read-paths: the SSR'd HTML is the user's first interaction. With JS disabled, content reads, <a> links navigate, and display-only custom elements render correctly.
        • diff --git a/website/app/docs/middleware/page.ts b/website/app/docs/middleware/page.ts index b4825dbe5..74eece3d1 100644 --- a/website/app/docs/middleware/page.ts +++ b/website/app/docs/middleware/page.ts @@ -9,7 +9,8 @@ export default function Middleware() {

        Root Middleware

        Place a middleware.ts at the root of your project (next to app/, not inside it). This middleware runs on every app request before WebJs routes it to a page, API route, or server action. Any of middleware.ts, .js, .mts, or .mjs works, and .ts wins if you somehow have more than one.

        -

        Two things are served ahead of it, so root middleware never sees them. The framework's own /__webjs/* assets and health probes bypass it in both dev and production, because they are framework infrastructure your app needs to boot rather than app routes. In development only, static files under /public/* (plus the /sw.js and /offline.html root remaps and /favicon.ico) are served ahead of it as well, so a stylesheet is never queued behind the dev server's startup analysis. In production those static files go through root middleware normally, so a middleware that protects an asset still protects it where it counts.

        +

        Some requests are answered before it and so never reach it. The rule is worth holding onto rather than a list: anything the listener shell or the framework's pre-analysis stage answers bypasses root middleware, and everything routed with your app reaches it. In practice that means WebSocket upgrades bound for a route.ts exporting WS, the dev live-reload SSE stream at /__webjs/events, and the framework's own /__webjs/* runtime assets and health probes, which your app needs in order to boot at all. In development only, static files under /public/* (plus the /sw.js and /offline.html root remaps and /favicon.ico) are answered there too, so a stylesheet is never queued behind the dev server's startup analysis.

        +

        Two things that look like exceptions are not. In production those /public/* files go through root middleware normally, so a middleware that protects an asset still protects it where it counts. And server actions are routed with your app, not answered early, so root middleware DOES run for an action call: that is what lets you gate actions with auth or rate limiting.

        my-app/ middleware.ts # root middleware: runs on every app request app/ From a20974dddd4ab3ca995db6bbc312b6933090eb4c Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 21:56:09 +0530 Subject: [PATCH 08/13] docs: name the config rewrites and fix the early-hints ordering Fourth delta review found two more ordering facts wrong on the lifecycle. webjs.redirects and webjs.trailingSlash are resolved ahead of the probes and well ahead of root middleware, so a 308 from a redirect rule is answered with the app's own middleware never running. The rule stated last commit does cover them, but only if a reader classifies a config key they wrote themselves as a framework pre-analysis stage, which is exactly the guess that goes the wrong way. So they are called out by name on all three surfaces rather than left to the rule. 103 Early Hints were listed after root middleware and are emitted before it, from inside the node listener shell ahead of the app handler. That is the same listener-shell step the last commit restructured the list around, so it should have moved then. It is also node-only rather than merely prod-only, since Bun.serve has no informational-response API, which now matters because the surrounding steps name both shells. --- .agents/skills/webjs/references/muscle-memory-gotchas.md | 2 +- website/app/docs/architecture/page.ts | 6 +++--- website/app/docs/middleware/page.ts | 1 + 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/.agents/skills/webjs/references/muscle-memory-gotchas.md b/.agents/skills/webjs/references/muscle-memory-gotchas.md index f148e0019..fd552fd2d 100644 --- a/.agents/skills/webjs/references/muscle-memory-gotchas.md +++ b/.agents/skills/webjs/references/muscle-memory-gotchas.md @@ -202,7 +202,7 @@ Export `GET` / `POST` / etc. as named async functions `(request, { params }) => ### `middleware.ts` is per-segment and chainable, not one matcher config -The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every app request, outermost to innermost. Some requests are answered before it and never reach it, and the RULE is what to remember, not the list: anything the listener shell or the framework's pre-analysis stage answers bypasses root middleware, and everything routed with the app reaches it. That covers WebSocket upgrades bound for a `route.ts` exporting `WS`, the dev SSE stream at `/__webjs/events`, and the framework's own `/__webjs/*` runtime assets and probes; in DEV only it also covers `/public/*` plus the `/sw.js` / `/offline.html` root remaps and `/favicon.ico`, so a stylesheet is never queued behind the dev startup analysis. In production those static files go through root middleware normally. Server actions are routed with the app, so middleware DOES run for an action call, which is what lets you gate actions with auth or rate limiting. +The file stays `middleware.ts`, NOT Next 16's renamed `proxy.ts`. WebJs middleware is in-process, chainable, and per-segment (the Remix / Koa model). There is no `export const config = { matcher }` and no single-file restriction. The default export is `async (req, next) => Response`: return a Response to short-circuit, or call `next()` and post-process. Colocate `app/admin/middleware.ts` next to the admin routes and it runs for that subtree only. An optional root `middleware.ts` runs on every app request, outermost to innermost. Some requests are answered before it and never reach it, and the RULE is what to remember, not the list: anything the listener shell or the framework's pre-analysis stage answers bypasses root middleware, and everything routed with the app reaches it. That covers WebSocket upgrades bound for a `route.ts` exporting `WS`, the dev SSE stream at `/__webjs/events`, and the framework's own `/__webjs/*` runtime assets and probes; in DEV only it also covers `/public/*` plus the `/sw.js` / `/offline.html` root remaps and `/favicon.ico`, so a stylesheet is never queued behind the dev startup analysis. In production those static files go through root middleware normally. **`webjs.redirects` and `webjs.trailingSlash` are the case intuition gets wrong**: you configure them, but the framework resolves them ahead of middleware, so a 308 from a redirect rule is answered without root middleware running (redirect in the middleware instead when it has to observe those requests). Server actions go the other way: they are routed with the app, so middleware DOES run for an action call, which is what lets you gate actions with auth or rate limiting. ### No ``, no `next/navigation`, no `next/*` libraries diff --git a/website/app/docs/architecture/page.ts b/website/app/docs/architecture/page.ts index dfde337ba..7f8c322f8 100644 --- a/website/app/docs/architecture/page.ts +++ b/website/app/docs/architecture/page.ts @@ -103,10 +103,10 @@ import { listPosts } from '#modules/posts/queries/list-posts.server.ts';Request Lifecycle
        1. HTTP request arrives at the listener shell (node:http, or Bun.serve on Bun; HTTP/2 if TLS is configured).
          2. -
        2. The listener shell answers what does not fit the request/response model, before the app handler is called at all: a WebSocket upgrade bound for a route.ts that exports WS, and in dev the live-reload SSE stream at /__webjs/events. Both are intercepted on the node and Bun shells alike.
          3. +
        3. The listener shell answers what does not fit the request/response model, before the app handler is called at all: a WebSocket upgrade bound for a route.ts that exports WS, and in dev the live-reload SSE stream at /__webjs/events. Both are intercepted on the node and Bun shells alike. The node shell also emits 103 Early Hints here in production, with modulepreload URLs for the matched page; Bun has no informational-response API, so that step does not exist on the Bun shell.
          4. +
        4. Declarative rewrites from your webjs config are applied: webjs.redirects and webjs.trailingSlash. These are configured by your app but resolved by the framework ahead of everything below, so a 308 from a redirect rule is answered without your root middleware running. That is the case most likely to surprise you, so it is worth remembering specifically.
        5. Framework-internal assets and probes are answered next, because they depend on neither the app analysis nor the vendor importmap and a cold instance must not gate them: the health, readiness and build-info probes (/__webjs/health, /__webjs/ready, /__webjs/version), the core runtime (/__webjs/core/*), the dev reload client and its SharedWorker (/__webjs/reload.js, /__webjs/reload-worker.js), and downloaded vendor bundles (/__webjs/vendor/*). Not everything under /__webjs/* is here: the server-action RPC endpoint is routed with the app, below. In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
          6. -
        6. Root middleware (middleware.ts) runs next if present, for every request not already answered above. The rule behind the exceptions is worth holding onto rather than the list: anything the listener shell or the pre-analysis stage answers never reaches your middleware, and everything routed with the app does.
          7. -
        7. 103 Early Hints sent (prod only) with modulepreload URLs for the matched page.
          8. +
        8. Root middleware (middleware.ts) runs next if present, for every request not already answered above. The rule behind the exceptions is worth holding onto rather than the list: anything the listener shell or the framework's pre-analysis stage answers never reaches your middleware, and everything routed with the app does.
        9. Route matching: the router tries (in order) the server-action RPC endpoint (/__webjs/action/<hash>/<fn>), static files, user source modules, API routes (route.ts), then page routes. The action endpoint is routed here rather than answered early, so root middleware DOES run for a server action, which is what lets you gate actions with auth or rate limiting. In production this is also where /public/* is served, so a middleware that guards an asset still guards it.
        10. Segment middleware chain runs (outermost → innermost) for the matched route.
        11. For pages: the SSR pipeline runs (load page + layouts, render to HTML, inject DSD, collect metadata, stream response with Suspense).
          12. diff --git a/website/app/docs/middleware/page.ts b/website/app/docs/middleware/page.ts index 74eece3d1..ba9949e50 100644 --- a/website/app/docs/middleware/page.ts +++ b/website/app/docs/middleware/page.ts @@ -10,6 +10,7 @@ export default function Middleware() {

          Root Middleware

          Place a middleware.ts at the root of your project (next to app/, not inside it). This middleware runs on every app request before WebJs routes it to a page, API route, or server action. Any of middleware.ts, .js, .mts, or .mjs works, and .ts wins if you somehow have more than one.

          Some requests are answered before it and so never reach it. The rule is worth holding onto rather than a list: anything the listener shell or the framework's pre-analysis stage answers bypasses root middleware, and everything routed with your app reaches it. In practice that means WebSocket upgrades bound for a route.ts exporting WS, the dev live-reload SSE stream at /__webjs/events, and the framework's own /__webjs/* runtime assets and health probes, which your app needs in order to boot at all. In development only, static files under /public/* (plus the /sw.js and /offline.html root remaps and /favicon.ico) are answered there too, so a stylesheet is never queued behind the dev server's startup analysis.

          +

          One case is worth calling out on its own, because the rule gives the right answer and intuition does not: webjs.redirects and webjs.trailingSlash are configured by you, but they are resolved by the framework before your middleware runs. So a request that a redirect rule answers with a 308 never reaches root middleware, and a logging or auth middleware will not see it. If you need middleware to observe those requests, do the redirect in the middleware rather than in config.

          Two things that look like exceptions are not. In production those /public/* files go through root middleware normally, so a middleware that protects an asset still protects it where it counts. And server actions are routed with your app, not answered early, so root middleware DOES run for an action call: that is what lets you gate actions with auth or rate limiting.

          my-app/ middleware.ts # root middleware: runs on every app request From c82f2b7fd118c50fedb1808c06ed78228507226c Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 22:02:38 +0530 Subject: [PATCH 09/13] docs: call the redirect step a redirect, not a rewrite The lifecycle step added last commit was headed "Declarative rewrites", and neither key rewrites anything: webjs.redirects replies with a configured redirect status and a Location header, and webjs.trailingSlash 308-redirects the non-canonical form. A rewrite is a distinct concept the audience for this page brings from Next, where it proxies internally with no client-visible redirect, and WebJs ships no such key, so the heading invented a feature. Every other surface this PR touches already says redirect. --- website/app/docs/architecture/page.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/app/docs/architecture/page.ts b/website/app/docs/architecture/page.ts index 7f8c322f8..6f2e8ef87 100644 --- a/website/app/docs/architecture/page.ts +++ b/website/app/docs/architecture/page.ts @@ -104,7 +104,7 @@ import { listPosts } from '#modules/posts/queries/list-posts.server.ts';
        12. HTTP request arrives at the listener shell (node:http, or Bun.serve on Bun; HTTP/2 if TLS is configured).
        13. The listener shell answers what does not fit the request/response model, before the app handler is called at all: a WebSocket upgrade bound for a route.ts that exports WS, and in dev the live-reload SSE stream at /__webjs/events. Both are intercepted on the node and Bun shells alike. The node shell also emits 103 Early Hints here in production, with modulepreload URLs for the matched page; Bun has no informational-response API, so that step does not exist on the Bun shell.
          14. -
        14. Declarative rewrites from your webjs config are applied: webjs.redirects and webjs.trailingSlash. These are configured by your app but resolved by the framework ahead of everything below, so a 308 from a redirect rule is answered without your root middleware running. That is the case most likely to surprise you, so it is worth remembering specifically.
          15. +
        15. Declarative redirects from your webjs config are answered: webjs.redirects and the webjs.trailingSlash canonical-form policy, both of which reply with a redirect status and a Location header rather than routing the request onward. These are configured by your app but resolved by the framework ahead of everything below, so a 308 from a redirect rule is answered without your root middleware running. That is the case most likely to surprise you, so it is worth remembering specifically.
        16. Framework-internal assets and probes are answered next, because they depend on neither the app analysis nor the vendor importmap and a cold instance must not gate them: the health, readiness and build-info probes (/__webjs/health, /__webjs/ready, /__webjs/version), the core runtime (/__webjs/core/*), the dev reload client and its SharedWorker (/__webjs/reload.js, /__webjs/reload-worker.js), and downloaded vendor bundles (/__webjs/vendor/*). Not everything under /__webjs/* is here: the server-action RPC endpoint is routed with the app, below. In development only, /public/* plus the /sw.js and /offline.html root remaps and /favicon.ico are served here too, so a stylesheet is never queued behind the startup analysis.
        17. Root middleware (middleware.ts) runs next if present, for every request not already answered above. The rule behind the exceptions is worth holding onto rather than the list: anything the listener shell or the framework's pre-analysis stage answers never reaches your middleware, and everything routed with the app does.
        18. Route matching: the router tries (in order) the server-action RPC endpoint (/__webjs/action/<hash>/<fn>), static files, user source modules, API routes (route.ts), then page routes. The action endpoint is routed here rather than answered early, so root middleware DOES run for a server action, which is what lets you gate actions with auth or rate limiting. In production this is also where /public/* is served, so a middleware that guards an asset still guards it.
          19. From 21dfc1d7e3dbc5acbc4b0aa40d15860421c4e20c Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 23:35:33 +0530 Subject: [PATCH 10/13] fix: correct the inverted warm comparison and the shared bun port base Final whole-diff review found three things. The rationale for RELOAD_QUIET_MS said 2000ms is "under the 1900ms analysis warm", which is backwards: 2000 is above 1900. The number carries the argument for picking this value, so an inverted relation is not a stray adjective. The behaviour it describes is real, the debounce lands just past the point where the restarted process has finished warming, so it now says that. Corrected in the module and in the server AGENTS.md row that repeats it. The new bun script took 9500 + pid % 240, byte-identical to dev-reload-retry. Every other dev-server bun script takes a disjoint base on purpose, so that two FILES can never contend whatever their pids, leaving the modulus to separate concurrent runs of one file. Both ship a test wrapper the node runner can schedule concurrently, so the only thing preventing a bind collision was the two pids not being congruent mod 240. The every-request sweep missed the middleware sample on the backend-only docs page, which was the last surviving instance of the claim. --- packages/server/AGENTS.md | 2 +- packages/server/src/dev-reload-worker.js | 8 +++++--- test/bun/dev-public-before-warm.mjs | 7 ++++++- website/app/docs/backend-only/page.ts | 2 +- 4 files changed, 13 insertions(+), 6 deletions(-) diff --git a/packages/server/AGENTS.md b/packages/server/AGENTS.md index 086ae9cf3..7a4a062f0 100644 --- a/packages/server/AGENTS.md +++ b/packages/server/AGENTS.md @@ -30,7 +30,7 @@ with metadata, Suspense, streaming) for HTML, or `api.js` / | File | What it owns | |---|---| -| `dev.js` | The request handler. File serving, TypeScript stripping (Node 24+ built-in `module.stripTypeScriptTypes`, backed by the `amaro` package; non-erasable syntax fails at strip time with a 500), **server-file guardrail**, live reload via SSE. Also the observability seam (#239): `handle()` mints / honors the per-request id (`X-Request-Id` + `setRequestId`), emits the one-line structured access log via `logger.info` after the response (suppressing `/__webjs/*` probe traffic), and routes unhandled errors to the app's `onError` sink (best-effort, threaded into the SSR error path, the action endpoint, middleware, metadata, and the top-level catch); applies conditional GET (#240, via `applyConditionalGet`) as the final funnel step so every cacheable response gets an ETag + honors If-None-Match -> 304; commits the server HTML cache (#241, via `commitHtmlCache`) just before conditional-GET so the store decision sees the final post-middleware response (threads `cspEnabled` into the page `ssrOpts` so a CSP page is never HTML-cached); `produce()` answers the `/__webjs/version` build-info probe. **Root middleware resolution:** `loadMiddleware` tries `middleware.ts` / `.js` / `.mts` / `.mjs` beside `appDir` in that order (`.ts` first, matching the dev supervisor's watch order). It used to look up the single literal `middleware.js`, so a root `middleware.ts`, what the scaffold writes and what the docs document, was silently never loaded: no error, no warning, indistinguishable from an app with no middleware. Tests: `test/dev/root-middleware-resolution.test.js` plus the cross-runtime `test/bun/root-middleware.mjs`. Dev error overlay (#264): `reportDevError(error, info)` builds a frame (via `dev-error.js`) and pushes it to the open tab over the SSE channel as a `webjs-error` event, fed by three sources (the SSR render catch via `ssrOpts.onDevError`, the `tsResponse` strip-failure, and the `rebuild` catch); a successful rebuild clears `state.lastDevError`; the page-render branch stamps its frame with `withBasePath(url.pathname, basePath()) + url.search` so the browser gate can compare it against `location` (the RAW pathname, and the base path put back, since the ingress strip removed it), SKIPS the hook entirely for a request carrying `x-webjs-prefetch: 1` so a speculative link prefetch never raises an overlay or becomes the retained error, and after a successful GET render clears `state.lastDevError` when it is still the same object AND names this url, so the replay cannot resurrect a superseded error while a good render of an unrelated page leaves a still-current one standing (#1047); `reloadClientJs` also calls `installDevOverlayNavSync()` so the overlay tracks the page on screen; `startServer`'s SSE replays the current frame to a freshly-connected tab; `reloadClientJs` renders the dev-only plain-DOM overlay (`textContent` only). **Graceful reload (#893):** a reload never paints into a half-restarted server (Node's `node --watch` briefly kills the process on an app edit), so the client gates every reload on a `/__webjs/version` readiness probe (probe-then-reload, instant under an in-process reload, waits out a restart), and a reconnect after a drop is itself treated as an edit signal so an app edit whose in-process reload frame was killed with the old process still reloads without a manual refresh (the SSE `hello` carries a short `retry: 300` so the reconnect is prompt). The reconnect-reload lives in the `dev-reload-worker.js` relay (shared connection) and the per-tab `__webjsDirectEvents` fallback. **Reload coalescing (#1397):** an agent saves several files a second or two apart, and EACH save produces TWO reload signals (the in-process `reload` frame from the fs.watch rebuild, then a changed boot id when the browser reconnects to the process `node --watch` restarted, measured 429ms apart with 1071ms between saves), so acting on every one reloads into a server about to be killed again and the page ends up unstyled. Both signals now route through ONE debounced emitter in the relay, which reloads once the signals stop for `RELOAD_QUIET_MS` (2000ms, above the measured inter-save gap so a realistic burst collapses, and under the 1900ms analysis warm so the wait overlaps work the reload would have blocked on anyway) or at the latest `RELOAD_MAX_HOLD_MS` (5000ms) after the FIRST signal of the batch, so a sustained burst still repaints rather than freezing on stale content. The cap timer is armed once per batch and never re-armed, which is what makes it measure from the first signal instead of sliding with the burst. `webjs-error` is NOT debounced (an overlay has to appear at once and it is not a reload) and the cached error is cleared on the signal rather than on the debounced emit, so a tab connecting mid-window is not replayed an overlay for an error the rebuild already fixed. The debounce must live browser-side because the server process dies on every edit, so no server-side timer can span a burst; the SharedWorker survives, being keyed by script URL. The per-tab `__webjsDirectEvents` fallback now RUNS THE SAME RELAY over a shim port rather than re-implementing the boot-id rule, so the boot-id rule, the error cache, and the debounce are one implementation with nothing to drift. Each tab still gates its reload on the `/__webjs/version` readiness probe, and `isRegenerateOutputPath` still suppresses a regenerate-output write upstream of the emitter. **Extra watch roots (#894):** `startServer`'s recursive `fs.watch` also follows the dirs in `webjs.dev.watch` (`readDevWatchPathsFromApp`), for content the app reads from OUTSIDE its appDir (blog markdown in a repo-root `blog/`). Dev-only: `reportDevError` early-returns in prod and `/__webjs/reload.js` 404s. **Listener shell (#511):** `startServer` builds a shared `SseHub` + `ListenerContext`, then selects a shell by `serverRuntime()`: `startNodeListener` (in this file, the node:http path: `toWebRequest` -> `app.handle` -> `sendWebResponse`, 103 Early Hints, node WS via `attachWebSocket`, node:http timeouts) on Node, or the dynamically-imported `startBunListener` (`listener-bun.js`) on Bun. The SSE registry/fanout, the live-reload predicate, the WS module loader, and the lifecycle wiring live in `listener-core.js` so the two shells share them. `isCompressible` (used by `sendWebResponse`) also moved there | +| `dev.js` | The request handler. File serving, TypeScript stripping (Node 24+ built-in `module.stripTypeScriptTypes`, backed by the `amaro` package; non-erasable syntax fails at strip time with a 500), **server-file guardrail**, live reload via SSE. Also the observability seam (#239): `handle()` mints / honors the per-request id (`X-Request-Id` + `setRequestId`), emits the one-line structured access log via `logger.info` after the response (suppressing `/__webjs/*` probe traffic), and routes unhandled errors to the app's `onError` sink (best-effort, threaded into the SSR error path, the action endpoint, middleware, metadata, and the top-level catch); applies conditional GET (#240, via `applyConditionalGet`) as the final funnel step so every cacheable response gets an ETag + honors If-None-Match -> 304; commits the server HTML cache (#241, via `commitHtmlCache`) just before conditional-GET so the store decision sees the final post-middleware response (threads `cspEnabled` into the page `ssrOpts` so a CSP page is never HTML-cached); `produce()` answers the `/__webjs/version` build-info probe. **Root middleware resolution:** `loadMiddleware` tries `middleware.ts` / `.js` / `.mts` / `.mjs` beside `appDir` in that order (`.ts` first, matching the dev supervisor's watch order). It used to look up the single literal `middleware.js`, so a root `middleware.ts`, what the scaffold writes and what the docs document, was silently never loaded: no error, no warning, indistinguishable from an app with no middleware. Tests: `test/dev/root-middleware-resolution.test.js` plus the cross-runtime `test/bun/root-middleware.mjs`. Dev error overlay (#264): `reportDevError(error, info)` builds a frame (via `dev-error.js`) and pushes it to the open tab over the SSE channel as a `webjs-error` event, fed by three sources (the SSR render catch via `ssrOpts.onDevError`, the `tsResponse` strip-failure, and the `rebuild` catch); a successful rebuild clears `state.lastDevError`; the page-render branch stamps its frame with `withBasePath(url.pathname, basePath()) + url.search` so the browser gate can compare it against `location` (the RAW pathname, and the base path put back, since the ingress strip removed it), SKIPS the hook entirely for a request carrying `x-webjs-prefetch: 1` so a speculative link prefetch never raises an overlay or becomes the retained error, and after a successful GET render clears `state.lastDevError` when it is still the same object AND names this url, so the replay cannot resurrect a superseded error while a good render of an unrelated page leaves a still-current one standing (#1047); `reloadClientJs` also calls `installDevOverlayNavSync()` so the overlay tracks the page on screen; `startServer`'s SSE replays the current frame to a freshly-connected tab; `reloadClientJs` renders the dev-only plain-DOM overlay (`textContent` only). **Graceful reload (#893):** a reload never paints into a half-restarted server (Node's `node --watch` briefly kills the process on an app edit), so the client gates every reload on a `/__webjs/version` readiness probe (probe-then-reload, instant under an in-process reload, waits out a restart), and a reconnect after a drop is itself treated as an edit signal so an app edit whose in-process reload frame was killed with the old process still reloads without a manual refresh (the SSE `hello` carries a short `retry: 300` so the reconnect is prompt). The reconnect-reload lives in the `dev-reload-worker.js` relay (shared connection) and the per-tab `__webjsDirectEvents` fallback. **Reload coalescing (#1397):** an agent saves several files a second or two apart, and EACH save produces TWO reload signals (the in-process `reload` frame from the fs.watch rebuild, then a changed boot id when the browser reconnects to the process `node --watch` restarted, measured 429ms apart with 1071ms between saves), so acting on every one reloads into a server about to be killed again and the page ends up unstyled. Both signals now route through ONE debounced emitter in the relay, which reloads once the signals stop for `RELOAD_QUIET_MS` (2000ms, above the measured inter-save gap so a realistic burst collapses, and landing just past the 1900ms analysis warm so the wait overlaps work the reload would have blocked on anyway) or at the latest `RELOAD_MAX_HOLD_MS` (5000ms) after the FIRST signal of the batch, so a sustained burst still repaints rather than freezing on stale content. The cap timer is armed once per batch and never re-armed, which is what makes it measure from the first signal instead of sliding with the burst. `webjs-error` is NOT debounced (an overlay has to appear at once and it is not a reload) and the cached error is cleared on the signal rather than on the debounced emit, so a tab connecting mid-window is not replayed an overlay for an error the rebuild already fixed. The debounce must live browser-side because the server process dies on every edit, so no server-side timer can span a burst; the SharedWorker survives, being keyed by script URL. The per-tab `__webjsDirectEvents` fallback now RUNS THE SAME RELAY over a shim port rather than re-implementing the boot-id rule, so the boot-id rule, the error cache, and the debounce are one implementation with nothing to drift. Each tab still gates its reload on the `/__webjs/version` readiness probe, and `isRegenerateOutputPath` still suppresses a regenerate-output write upstream of the emitter. **Extra watch roots (#894):** `startServer`'s recursive `fs.watch` also follows the dirs in `webjs.dev.watch` (`readDevWatchPathsFromApp`), for content the app reads from OUTSIDE its appDir (blog markdown in a repo-root `blog/`). Dev-only: `reportDevError` early-returns in prod and `/__webjs/reload.js` 404s. **Listener shell (#511):** `startServer` builds a shared `SseHub` + `ListenerContext`, then selects a shell by `serverRuntime()`: `startNodeListener` (in this file, the node:http path: `toWebRequest` -> `app.handle` -> `sendWebResponse`, 103 Early Hints, node WS via `attachWebSocket`, node:http timeouts) on Node, or the dynamically-imported `startBunListener` (`listener-bun.js`) on Bun. The SSE registry/fanout, the live-reload predicate, the WS module loader, and the lifecycle wiring live in `listener-core.js` so the two shells share them. `isCompressible` (used by `sendWebResponse`) also moved there | | `router.js` | Scans `app/` once, builds the route table, matches pages + APIs (`buildRouteTable`, `matchPage`, `matchApi`). Page order is set by `compareSpecificity` (#750): POSITIONAL specificity (per URL segment, static `0` < dynamic `1` < catch-all `2` via `segKind`, lexicographic on the kind arrays with shorter-prefix-first), so the catch-all kind is lowest AT ITS POSITION (a literal-prefixed catch-all like `docs/[[...slug]]` outranks an all-dynamic `[org]/[repo]`), NOT a global catch-all-last bucket, then a stable alphabetical `routeDir` tiebreak. This replaces the old coarse 3-bucket `dynScore` whose same-bucket ties resolved by fs-walk order (so `/[org]/[repo]` vs `/[user]/settings` could match the wrong page). `matchPage` returns the first pattern that matches in that deterministic order. The table also carries the app-ROOT convention files that are not router stems: `instrumentationClient` (#1399, resolved here rather than by each caller, because a consumer that built its own table and forgot to attach it lost a browser-bound entry and inverted the vendor pin-is-a-superset invariant) | | `route-types.js` | Route-types generator (#258). `generateRouteTypes(appDir)` reuses `buildRouteTable` to emit the `.d.ts` text that augments `@webjsdev/core` (the `WebjsRoutes` href union + `RouteParamMap` per-route params), backing `webjs types` and the dev-startup emit. Pages-only (a `route.{js,ts}` API path is not a navigable href); strips route groups, excludes `_private`; an optional catch-all `[[...x]]` emits both the without-segment and a normalized `[...x]` href key while keeping the doubled literal as the param-map key. Deterministic (sorted keys). Helpers `routeKeyFromDir` / `dynamicSegments` / `paramTypeForKey` / `webjsRoutesKeysForKey` are exported for unit tests | | `ssr.js` | SSR pipeline: nested layouts, metadata → ``, Suspense streaming, error boundaries. `ssrPage` accepts `actionData` (put on `ctx.actionData` for the page + layouts) and `status` (default 200; the form-action re-render passes 422). Server HTML cache (#241): on a plain GET render it loads the page module once to read `export const revalidate`, serves a cache HIT via `cachedHtmlResponse` (re-minting the build id), and on a miss stamps the `HTML_CACHE_MARKER` so the funnel writes the final body. Skipped for the form-action re-render and partial-nav (`X-Webjs-Have`) requests. **Partial responses are never shared-cacheable (#1140):** a reduced `X-Webjs-Have` body is sliced by a request header, so `privateFragment(res)` rewrites its `Cache-Control` to `private` (stripping `public` / `s-maxage` / `proxy-revalidate`, and any qualified `private="field"`, which per RFC 9111 would leave the response shared-storable). `Vary: X-Webjs-Have` is still sent, but as belt-and-braces: it is not the guarantee, because Cloudflare and others honour only `Accept-Encoding`. The helper is quote-aware (a comma inside `no-cache="Set-Cookie,X-Foo"` is not a separator), fails CLOSED on an absent header (no `Cache-Control` is heuristically shared-storable), and is exported for unit testing (not re-exported from the package index, so it is not public API). The fragment KEEPS its ETag, which is why `conditional-get.js` no longer excludes `private`. A non-200 (the 422 form-action re-render, which carries the submitter's own field values) never inherits the page's `cacheControl`. **Frame subtree render (#253):** after `renderChain`, when the request carries `x-webjs-frame: ` (a `` self-load or a click-driven frame nav) AND the render is non-streamed, it extracts the matching `` subtree from the rendered body via `frame-render.js` and returns ONLY that (byte-equivalent to the client's extraction from a full page, but far fewer bytes); an absent frame id falls through to the full page (the client's `webjs:frame-missing` handles it), and a request with no `x-webjs-frame` header is byte-identical to before. **Vendor modulepreload (#754):** `reachedVendorSpecifiers(graph, shippedEntryFiles, componentUrls, appDir, elidable, serverFiles)` collects the bare specifiers (`bareImports(graph)`) reached by the page's SHIPPED modules. Its walk ROOTS are the boot's actually-shipped set: the caller passes the absolute paths of `moduleUrls` (which already drops INERT page/layout modules and substitutes an IMPORT-ONLY page with its components) plus the rendered `componentUrls`, then it walks the non-elided transitive closure and collects each reached file's bare imports, skipping server files (the `.server.*` suffix AND the `serverFiles` action index). Because the roots are the shipped set, a vendor reached ONLY through a dropped module, a dropped page's SSR-only DIRECT vendor import OR its SSR-only RELATIVE HELPER's vendor, is never collected (pages/layouts are not importable, so nothing that ships reaches them), so there is no over-fetch. `vendorPreloadTargets` (importmap.js) maps that set to `{ href, integrity }`, and `wrapHead` emits one `` per target (no `fp()` rewrite, deduped against the app module/component preloads, with `crossorigin` + `integrity`), flattening the cross-origin CDN waterfall one level. Only reached + non-elided + pinned vendors are hinted. **App-module modulepreload shipped-roots (#780):** `deduplicatedPreloads` (the app-module + component preload walk) roots at the SAME shipped set as the vendor walk. `shippedRoots` (the absolute paths of `moduleUrls`) is computed once and passed to BOTH `deduplicatedPreloads` and `reachedVendorSpecifiers`, NOT the raw `[route.file, ...route.layouts]`. So a module reached ONLY through a dropped page/layout (its SSR-only DIRECT app import OR its SSR-only RELATIVE HELPER) gets no `modulepreload` hint, the app-module analog of the #754 vendor over-fetch. A module that also ships another way (a component shared with a live route, or one reached through an import-only page's substituted components) is still reached via a real shipped root, so its hint stays (no under-fetch). `seen = new Set(moduleUrls)` already excludes the shipped page/layout URLs themselves, so this closes the TRANSITIVE gap only | diff --git a/packages/server/src/dev-reload-worker.js b/packages/server/src/dev-reload-worker.js index 0fa96a368..172197380 100644 --- a/packages/server/src/dev-reload-worker.js +++ b/packages/server/src/dev-reload-worker.js @@ -28,9 +28,11 @@ * * 2000ms is above the measured 1071ms inter-save gap so a realistic burst * collapses to one reload, and deliberately not NEAR it, since a window close - * to that gap fires just as the next restart begins. It is also under the - * 1900ms analysis warm measured on the website app, so on a real app the wait - * overlaps work the reload request would have blocked on anyway. + * to that gap fires just as the next restart begins. It also lands just past + * the 1900ms analysis warm measured on the website app, which is the useful + * place for it: the restarted process kicks off `warmup()` as soon as it + * listens, so the wait overlaps work the reload request would have blocked on + * anyway and the reload arrives at a server that has finished warming. */ export const RELOAD_QUIET_MS = 2000; diff --git a/test/bun/dev-public-before-warm.mjs b/test/bun/dev-public-before-warm.mjs index e58588497..044f4eb92 100644 --- a/test/bun/dev-public-before-warm.mjs +++ b/test/bun/dev-public-before-warm.mjs @@ -33,7 +33,12 @@ const __dirname = dirname(fileURLToPath(import.meta.url)); const ROOT = resolve(__dirname, '../..'); const CLI = join(ROOT, 'packages/cli/bin/webjs.js'); const runtime = process.versions.bun ? `bun ${process.versions.bun}` : `node ${process.versions.node}`; -const PORT = 9500 + (process.pid % 240); +// Own base, disjoint from every other dev-server bun script (dev-reload-retry +// 9500, dev-hot-reload 9700, dev-extra-watch 9750, dev-overlay-scope 9800). +// The modulus separates concurrent RUNS of this file; the distinct base is what +// separates this file from the others, since the node runner can schedule the +// `*.test.mjs` wrappers concurrently in separate child processes. +const PORT = 9850 + (process.pid % 140); const BASE = `http://localhost:${PORT}`; const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); diff --git a/website/app/docs/backend-only/page.ts b/website/app/docs/backend-only/page.ts index e2eb5e8b0..616b4b30d 100644 --- a/website/app/docs/backend-only/page.ts +++ b/website/app/docs/backend-only/page.ts @@ -90,7 +90,7 @@ export async function DELETE(req: Request, { params }: { params: { id: string }

          Middleware for Auth, CORS, Rate Limiting

          Middleware works identically in backend-only mode. Place middleware.ts files at the root or in any segment directory:

          - // middleware.ts (root): logging for every request + // middleware.ts (root): logging for every app request export default async function logger( req: Request, next: () => Promise<Response>, From 6696c33f425383cf9c5602c26fe46432f8e830bb Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 23:42:40 +0530 Subject: [PATCH 11/13] fix: put the bun script's port range above every other dev script The previous commit moved this script to base 9850 and called the bases disjoint. They are not. A base only separates two files if the earlier one's modulus window stops before the later one's base, and these windows run well past: reload-retry reaches 9739, hot-reload 9949, extra-watch 9989, overlay-scope 9979. So 9850-9989 landed inside three of them, where the old 9500-9739 had overlapped two. The change made the collision it claimed to fix more likely, and narrowing the modulus to 140 also raised the same-file rate the comment credited it with lowering. 9989 is the highest port any existing script reaches, so this takes 10000-10255, which cannot collide with any of the four for any pair of pids, and restores a modulus at least as wide as the original for the same-file case. The overlapping ranges among the other four are pre-existing and left alone. --- test/bun/dev-public-before-warm.mjs | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-) diff --git a/test/bun/dev-public-before-warm.mjs b/test/bun/dev-public-before-warm.mjs index 044f4eb92..cbe45de76 100644 --- a/test/bun/dev-public-before-warm.mjs +++ b/test/bun/dev-public-before-warm.mjs @@ -33,12 +33,19 @@ const __dirname = dirname(fileURLToPath(import.meta.url)); const ROOT = resolve(__dirname, '../..'); const CLI = join(ROOT, 'packages/cli/bin/webjs.js'); const runtime = process.versions.bun ? `bun ${process.versions.bun}` : `node ${process.versions.node}`; -// Own base, disjoint from every other dev-server bun script (dev-reload-retry -// 9500, dev-hot-reload 9700, dev-extra-watch 9750, dev-overlay-scope 9800). -// The modulus separates concurrent RUNS of this file; the distinct base is what -// separates this file from the others, since the node runner can schedule the -// `*.test.mjs` wrappers concurrently in separate child processes. -const PORT = 9850 + (process.pid % 140); +// A distinct base is NOT enough on its own, which is the trap here: a base +// only separates two files if the earlier one's modulus window stops before +// the later one's base, and among the existing dev-server scripts it does not. +// Their reachable RANGES are dev-reload-retry 9500-9739, dev-hot-reload +// 9700-9949, dev-extra-watch 9750-9989 and dev-overlay-scope 9800-9979, which +// overlap each other freely. That is pre-existing and not this file's to fix; +// what this file can do is sit entirely ABOVE all of them. 9989 is the highest +// port any of them reaches, so 10000-10255 cannot collide with any of the four +// for any pair of pids. The modulus then separates concurrent RUNS of this +// file, which is the only collision left. Both halves matter, because the node +// runner schedules the `*.test.mjs` wrappers concurrently in separate child +// processes with near-consecutive pids. +const PORT = 10000 + (process.pid % 256); const BASE = `http://localhost:${PORT}`; const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); From 7532ac02558930740d7a09e86892d3e110085493 Mon Sep 17 00:00:00 2001 From: Vivek Date: Wed, 12 Aug 2026 23:49:50 +0530 Subject: [PATCH 12/13] docs: correct what the per-pid port offset is actually for The comment credited the modulus with separating concurrent runs of this file. No runner here produces those: each runner runs a given file once, and the node and bun runs are sequential CI steps. The base is what separates this file from its siblings; the offset is only defensive against a leftover socket lingering in TIME_WAIT, which is what dev-hot-reload.mjs already says about the identical construct. Two sibling files gave incompatible accounts of the same mechanism and this one was wrong. --- test/bun/dev-public-before-warm.mjs | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/test/bun/dev-public-before-warm.mjs b/test/bun/dev-public-before-warm.mjs index cbe45de76..25d53f472 100644 --- a/test/bun/dev-public-before-warm.mjs +++ b/test/bun/dev-public-before-warm.mjs @@ -41,10 +41,12 @@ const runtime = process.versions.bun ? `bun ${process.versions.bun}` : `node ${p // overlap each other freely. That is pre-existing and not this file's to fix; // what this file can do is sit entirely ABOVE all of them. 9989 is the highest // port any of them reaches, so 10000-10255 cannot collide with any of the four -// for any pair of pids. The modulus then separates concurrent RUNS of this -// file, which is the only collision left. Both halves matter, because the node -// runner schedules the `*.test.mjs` wrappers concurrently in separate child -// processes with near-consecutive pids. +// for any pair of pids. The per-pid offset is NOT doing that work and should +// not be credited with it: the node and bun runs are sequential steps and each +// runner runs a given file once, so nothing here races for a port. It is only +// defensive against a leftover socket from a prior run lingering in TIME_WAIT, +// which is the same account `dev-hot-reload.mjs` gives of the identical +// `base + pid % n` construct. const PORT = 10000 + (process.pid % 256); const BASE = `http://localhost:${PORT}`; From 19ef74bfe52a1e3c5356213ee1d71194bd03b98f Mon Sep 17 00:00:00 2001 From: Vivek Date: Thu, 13 Aug 2026 00:43:54 +0530 Subject: [PATCH 13/13] fix: sequence the bun script's fetches so the ordering is observable The script raced its two fetches, and /__webjs/ready answers immediately (it reports unready without blocking on the analysis), so the pair passed whether or not the hoist exists: the ready fetched at t=0 was 503 either way. Proven vacuous by running it against a server without the hoist, where it printed OK. Sequenced, the timeline discriminates. With the hoist the css returns in milliseconds, long before the fixture middleware's sleep releases the analysis, so the ready fetched after it is still 503. Without the hoist the css itself blocks on ensureReady, so by the time it returns the analysis is done and the same fetch reads 200. Verified in both directions: OK against this branch's server on node and bun, and the assertion fails against main's server. The middleware sleep widened from 1500ms to 3000ms so the discrimination has margin over css fetch latency. --- test/bun/dev-public-before-warm.mjs | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/test/bun/dev-public-before-warm.mjs b/test/bun/dev-public-before-warm.mjs index 25d53f472..f6aadf8cc 100644 --- a/test/bun/dev-public-before-warm.mjs +++ b/test/bun/dev-public-before-warm.mjs @@ -80,10 +80,10 @@ try { writeFileSync(join(dir, 'app/page.ts'), "import { html } from '@webjsdev/core';\nexport default () => html`

          ok

          `;\n"); writeFileSync(join(dir, 'public/a.css'), 'body{color:red}\n'); // Top-level await, so the module does not finish evaluating (and therefore - // `loadMiddleware`, and therefore `ensureReady()`, does not resolve) for 1.5s. + // `loadMiddleware`, and therefore `ensureReady()`, does not resolve) for 3s, wide margin over the css fetch latency. writeFileSync( join(dir, 'middleware.ts'), - 'await new Promise((r) => setTimeout(r, 1500));\n' + 'await new Promise((r) => setTimeout(r, 3000));\n' + 'export default async function middleware(req: Request, next: () => Promise) { return next(); }\n', ); writeFileSync(join(dir, 'package.json'), JSON.stringify({ name: 'public-warm', type: 'module', imports: { '#*': './*' }, webjs: {} })); @@ -104,18 +104,25 @@ try { const listening = await until(portAccepts, { timeoutMs: 30_000 }); assert.ok(listening, `dev server never listened on ${runtime}\n--- server log ---\n${log}`); - // One pass: the CSS must answer while readiness is still gated on the - // analysis. Both requests are issued together so neither waits on the other. - const [css, ready] = await Promise.all([ - fetch(`${BASE}/public/a.css`), - fetch(`${BASE}/__webjs/ready`), - ]); + // ORDER of these two fetches is the whole assertion, so they are sequenced, + // never raced. `/__webjs/ready` answers IMMEDIATELY (it reports unready + // without blocking on the analysis), so a concurrent pair passes whether or + // not the hoist exists: the css resolves whenever it resolves and the ready + // fetched at t=0 was 503 either way. That exact vacuous shape shipped first + // and passed against a server WITHOUT the hoist. Sequenced, the timeline + // discriminates: with the hoist the css returns in milliseconds, long before + // the fixture middleware's 3s sleep releases the analysis, so the ready + // fetched AFTER it is still 503; without the hoist the css itself blocks on + // `ensureReady()`, so by the time it returns the analysis is done and the + // same ready fetch reads 200, failing the assertion below. + const css = await fetch(`${BASE}/public/a.css`); const cssBody = await css.text(); + const ready = await fetch(`${BASE}/__webjs/ready`); assert.equal(css.status, 200, `/public/a.css was not served cold on ${runtime}\n--- server log ---\n${log}`); assert.equal(cssBody, 'body{color:red}\n', `/public/a.css served the wrong bytes on ${runtime}`); assert.equal( ready.status, 503, - `the analysis had already completed on ${runtime}, so this proves nothing about ordering\n--- server log ---\n${log}`, + `the analysis had already completed before the CSS was served on ${runtime}, so the hoist is not doing its job\n--- server log ---\n${log}`, ); console.log(`OK dev serves /public/* before the analysis completes on ${runtime} (#1397)`);