From 7f18870ba26707e3faaa859ec35985ea8ce14fba Mon Sep 17 00:00:00 2001 From: Vivek Date: Thu, 13 Aug 2026 17:28:00 +0530 Subject: [PATCH 01/16] feat: refresh page and layout edits in place in dev Every dev edit produced a full page reload: the page blinked, hydrated component state reset, scroll position was lost. Most dev edits do not need that. Pages and layouts never hydrate, so a freshly rendered page is the complete truth for them and nothing in the browser can be stale after a re-render. The dev watcher now classifies the changed file against the module graph and puts a ternary verdict on the SSE reload frame, and the browser picks the lightest correct response: morph the deepest shared boundary for a page edit (scroll and the hydrated state of components outside it survive), replace the whole body for a layout edit (its own markup sits outside every children range), and reload for anything else. The classification walks the graph rather than the path shape, because a page that imports a client-effecting util ships whole and a util under lib/utils reachable from a component is a component edit whose path says nothing. Everything unclassifiable reloads: a wrong morph is a broken page, a wrong reload is a flash. Component edits stay a full reload by design. customElements.define is once-per-tag, so a morph would apply new markup wired to the old class. This hot-swaps no module and does not reopen the deferred HMR question. --- .../references/client-router-and-streaming.md | 16 +- .agents/skills/webjs/references/runtime.md | 5 + AGENTS.md | 5 +- packages/core/AGENTS.md | 2 +- packages/core/index-browser.js | 2 +- packages/core/index.d.ts | 2 +- packages/core/index.js | 2 +- packages/core/src/router-client.d.ts | 9 + packages/core/src/router-client.js | 153 ++++++++++++++-- packages/server/AGENTS.md | 5 +- packages/server/src/component-elision.js | 2 +- packages/server/src/dev-classify.js | 163 +++++++++++++++++ packages/server/src/dev-reload-worker.js | 65 ++++++- packages/server/src/dev.js | 131 ++++++++++++-- packages/server/src/listener-core.js | 18 +- .../server/test/dev/classify-live.test.js | 150 ++++++++++++++++ .../test/dev/classify-watch-path.test.js | 167 ++++++++++++++++++ .../test/dev/reload-shared-connection.test.js | 24 ++- .../test/listener/listener-core.test.js | 35 +++- scripts/run-bun-tests.js | 1 + test/bun/dev-morph-verdict.mjs | 164 +++++++++++++++++ test/bun/dev-morph-verdict.test.mjs | 14 ++ website/app/docs/client-router/page.ts | 13 ++ website/app/docs/runtime/page.ts | 4 + 24 files changed, 1107 insertions(+), 45 deletions(-) create mode 100644 packages/server/src/dev-classify.js create mode 100644 packages/server/test/dev/classify-live.test.js create mode 100644 packages/server/test/dev/classify-watch-path.test.js create mode 100644 test/bun/dev-morph-verdict.mjs create mode 100644 test/bun/dev-morph-verdict.test.mjs diff --git a/.agents/skills/webjs/references/client-router-and-streaming.md b/.agents/skills/webjs/references/client-router-and-streaming.md index 53268ccd5..9718fede5 100644 --- a/.agents/skills/webjs/references/client-router-and-streaming.md +++ b/.agents/skills/webjs/references/client-router-and-streaming.md @@ -2,7 +2,7 @@ ## What This Covers -- The automatic client router (SPA-style partial swaps), how it opts out, and programmatic `navigate()` / `revalidate()`. +- The automatic client router (SPA-style partial swaps), how it opts out, and programmatic `navigate()` / `revalidate()` / `refreshPage()`. - Link prefetch with device-adaptive defaults. - `` partial-swap regions (WebJs's Turbo Frames). - View Transitions opt-in. @@ -56,6 +56,20 @@ revalidate(); // clear the entire snapshot cache The router keeps a URL-keyed snapshot cache (LRU, cap 16) so Back/Forward restores instantly, then refetches in the background. Call `revalidate(path)` after a server action mutates data a cached page depends on. Wire bytes are minimized by an `X-Webjs-Have` header, so the server returns only the divergent layout fragment. Concurrent navigations abort the prior in-flight fetch, and scroll is restored on Back/Forward. +**In-place refresh of the page you are on.** `refreshPage(mode)` re-renders the CURRENT url on the server and applies it without a page load. + +```js +import { refreshPage } from '@webjsdev/core'; +await refreshPage(); // 'page': morph the deepest shared boundary +await refreshPage('shell'); // replace the whole body (the layout's own markup changed) +``` + +It records no history entry and never scrolls, so the reader keeps their place and Back still goes to the previous page. `'page'` morphs the deepest shared boundary, so the outer layout's DOM and the hydrated state of its components survive; `'shell'` replaces the whole body, which is what a LAYOUT change needs, since a layout's own header, nav, and footer sit outside every children range and a boundary morph would leave them untouched. Component instances do not survive a `'shell'` refresh. + +It sends no `X-Webjs-Have`, deliberately: the server short-circuits at the first layout the client already holds, and a same-url request matches every one of them, so the response would omit the very layout that changed. It resolves `false` when it did not apply (the router is disabled, or the fetch failed), so a caller falls back to a full load. + +It does NOT reload changed component modules and cannot: `customElements.define` is once-per-tag and a module url is fetched once per document. A caller whose change touched browser code has to reload. This is exactly why the dev live-reload client calls `refreshPage` for a page or layout edit and `location.reload()` for a component edit (#1398, and see `references/runtime.md` for which dev modes get the refresh). + **Back/Forward scroll restore vs late layout growth.** The router SUPPRESSES the browser's scroll anchoring (`overflow-anchor`) for the duration of a Back/Forward restore, then puts it back. The saved offset was recorded against the page at its SETTLED height, while the DOM the restore swaps in is still shorter until its components upgrade and render. Without the suppression the browser treats that late growth as content appearing above a reader and adds it to the offset the router just replayed, so the reader lands BELOW where they left (the reported case was 763px, exactly the height a page gained after its swap). What follows for an app: - **Do not write your own scroll restore.** A `popstate` listener that calls `scrollTo`, a saved offset in `sessionStorage`, a `scrollIntoView` on a remembered element: all of them fight the router, which already set `history.scrollRestoration = 'manual'` and is the sole authority on scroll during a navigation. If Back lands in the wrong place, that is a framework bug to report, not something to patch in app code. diff --git a/.agents/skills/webjs/references/runtime.md b/.agents/skills/webjs/references/runtime.md index 93f50dd0e..b9f48e319 100644 --- a/.agents/skills/webjs/references/runtime.md +++ b/.agents/skills/webjs/references/runtime.md @@ -35,8 +35,13 @@ Three seams pick a runtime-specific implementation, all inside the framework, no | Hot reload | `node --watch` | `bun --hot` | | WebSocket | the `ws` library | native `Bun.serve` + a bridge adapter | | 103 Early Hints | yes | no (`Bun.serve` has no informational-response API) | +| Dev edit to a page / layout | full reload (the `node --watch` restart replaces the process) | refreshes IN PLACE, no reload (#1398) | | Reverse-proxy headers | `X-Forwarded-Proto` / `X-Forwarded-Host` honored | same | +**The in-place dev refresh (#1398) needs the server process to SURVIVE the edit,** which is the whole of the Node-versus-Bun difference in that row. A page or layout never hydrates, so a freshly rendered page is the complete truth for it and the client router can swap it in without a reload, keeping scroll and (for a page edit) the hydrated state of components outside the changed region. The server classifies the changed file and puts the verdict on the live-reload event, so this needs a process that is still alive to do the classifying. + +Bun's `bun --hot` invalidates modules in place without restarting, so it gets the refresh. Node's `bun --hot` equivalent is `node --watch`, which RESTARTS the process on a change under `app`, `components`, `modules`, `lib`, or `actions`, and a fresh process holds no record of what changed, so those edits are always a full reload. Two Node cases still refresh in place: an edit OUTSIDE those five dirs (`db/schema.server.ts`, a `webjs.dev.watch` content dir), and running `webjs dev --no-hot`, which keeps the server in one process on either runtime. A component edit is a full reload everywhere by design, because `customElements.define` is once-per-tag and swapping fresh markup onto the old class would be worse than the reload. + The 103 Early Hints gap costs only a small first-load latency edge where an edge proxy forwards the 103, never correctness. The `modulepreload` hints still ship in the document head on both runtimes. Behind a TLS-terminating proxy (Railway, Fly, Render, Cloudflare, nginx), both shells rewrite the request URL from `X-Forwarded-Proto` / `X-Forwarded-Host`, so `ctx.url` in a page, `req.url` in a `route.{js,ts}` handler, and every absolute URL you build from either carry the ORIGINAL scheme and host rather than the internal `http://container` hop. A comma-separated chain (a CDN in front of a load balancer) takes the value closest to the client, only `http` and `https` are accepted as a scheme, and a malformed host is ignored rather than failing the request. This was Bun-only broken before #1090, which shipped an `http://` `og:image` on an HTTPS site. diff --git a/AGENTS.md b/AGENTS.md index 892126ec3..5fd8496ee 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -244,6 +244,7 @@ The bare `@webjsdev/core` specifier resolves to a BROWSER bundle dropping server | `asset(path)` | Content-hash a `public/` asset url so a deploy cannot serve stale bytes: `href=${asset('/public/app.css')}` emits `?v=` and is served `immutable` for a year (#1194). Prod-only, `public/` paths only. Use it in a PAGE, LAYOUT, or metadata route, not inside a component that ships: hydration re-renders on the client where there is no resolver, so the hashed url is swapped for the bare path and the asset is fetched twice. Call it inside the render function, since a module-scope call is a side effect that ships the module. Mark only files that change with a DEPLOY (the hash is memoized for the process lifetime). Opt-in on purpose: do NOT mark a `rel=preload` hint whose asset is fetched by CSS `url()`, or the preload can never match. | | `connectWS(url, handlers)` / `richFetch` | Client WebSocket (auto-reconnect, queued sends); content-negotiated rich-type fetch. | | `navigate(url, opts?)` / `revalidate(url?)` | Programmatic client-router nav; evict the BROWSER snapshot cache. | +| `refreshPage(mode?)` | Re-render the CURRENT url on the server and apply it in place, with no reload (#1398). Records no history entry and never scrolls. `'page'` (default) morphs the deepest shared boundary, so hydrated component state outside it survives; `'shell'` replaces the whole body, which a LAYOUT change needs because its own markup sits outside every children range. It reloads no component module (`customElements.define` is once-per-tag), so a caller whose change touched browser code must reload instead. This is what the dev live-reload client calls for a page or layout edit. | | `optimistic(signal, value, action)` / `optimistic(host, { source, update })` | Imperative: set `signal` immediately, run `action`, roll back on error. Declarative (preferred): queue optimistic updates with auto-release via `.add(payload, promise?)`. See `references/client-router-and-streaming.md`. | | `renderStream(payload)` / `WebjsFrame` | `` element-level updates (#248); `` partial-swap regions (#253). See `references/client-router-and-streaming.md`. | | `Metadata` / `PageProps` / `LayoutProps` / `RouteHandlerContext` / `WebjsConfig` (type-only) | Types for metadata, page/layout/route args (`R` narrows `params` against the `webjs types` route union), and the `webjs` config block. See `references/routing-and-pages.md` + `references/built-ins.md`. | @@ -517,7 +518,7 @@ Rules: **always scaffold via `webjs create`** (never hand-roll). **Default to a ## CLI reference ```sh -webjs dev [--port N] [--no-hot] # dev server with live reload (node --watch on Node, bun --hot on Bun). --no-hot runs in-process. Runs webjs.dev.before + webjs.dev.parallel (#550) +webjs dev [--port N] [--no-hot] # dev server with live reload (node --watch on Node, bun --hot on Bun). --no-hot runs in-process. Runs webjs.dev.before + webjs.dev.parallel (#550). A page or layout edit REFRESHES IN PLACE where the server process survives it (#1398); a component edit always reloads webjs start [--port N] # prod server; source IS the runtime, plain HTTP/1.1 (reverse-proxy for TLS + HTTP/2). Runs webjs.start.before first (#550) webjs test [--server] [--browser] [--watch] webjs check [--rules] [--json] # correctness validator (report-only, no autofix); --json for an agent loop @@ -636,6 +637,6 @@ Call it from a client component via a normal import (rewritten to an RPC stub). Not in v1. Do not implement as part of other tasks: - **Bundling and per-route code splitting.** WebJs is **no-build** (the Rails 7 + importmap model); prod perf comes from HTTP/2 multiplex + `` hints, not concatenation. **Do not propose a bundler or `webjs build`.** -- **Vite-grade HMR with state preservation.** Custom elements only `define` once, so full reload is necessary; data reloads are near-instant via `fs.watch` to SSE. +- **Vite-grade HMR with state preservation.** Custom elements only `define` once, so full reload is necessary; data reloads are near-instant via `fs.watch` to SSE. #1398 does not change this: it re-renders on the SERVER and swaps the result, and hot-swaps no module, so a component edit still reloads and `customElements.define` is still once-per-tag. - **React Server Components Flight.** Server actions + `Suspense` streaming cover the need. - **Edge-runtime bundling / full portability** (deployment guidance lives in the docs site at `/docs/deployment`), **i18n, image optimization** (layer libraries on top). diff --git a/packages/core/AGENTS.md b/packages/core/AGENTS.md index 7142f1cc5..adea706d8 100644 --- a/packages/core/AGENTS.md +++ b/packages/core/AGENTS.md @@ -38,7 +38,7 @@ the same output in all three. | `webjs-suspense.js` | The `` component-level streaming boundary element (#471). SSR (`render-server.js`) does the work: `injectDSD`'s `processSuspenseElements` pre-pass reads `.fallback` (carried as `data-webjs-fallback` by `renderTemplate`, because this element is browser-defined, so the walk below finds no class for it and no server-side instance runs `consumePropAttrs`, which leaves `connectedCallback` as the only consumer of a normal `data-webjs-prop-*` binding, far too late for a placeholder that has to be in the first flushed bytes) and, in a streaming context, flushes the fallback as `` while pushing the children to `ctx.pending` for out-of-order streaming (concurrent across boundaries via `Promise.all`); without a streaming context the children render inline (blocking). This client element is layout-neutral (`display:contents`) and the registration home for the soft-nav apply; first-load streaming needs no client runtime (the inline swap script `replaceWith`s the boundary element with the resolved children, which then upgrade). Every swap path (the inline script, the boot `__webjsResolve`, and the soft-nav `applyStreamedResolve`) removes the transient wrapper, so a boundary settles to the same DOM however the page was reached. SSR-inert (defined client-side only) | | `context.js` | Context Protocol: `createContext`, `ContextProvider`, `ContextConsumer`, `ContextRequestEvent` | | `task.js` | `Task` / `TaskStatus` controller for async data in components | -| `router-client.js` | Turbo Drive–style client router; entry: `enableClientRouter` / `navigate`. Also exports `loadFrame(frameEl, url)` (#253), the reusable frame self-load `webjs-frame.js` calls: it fetches `url` as a frame nav (the `x-webjs-frame` header) and applies the matched subtree through the SAME `fetchAndApply` frame-swap path a click uses (no history push / snapshot / optimistic skeleton, since it swaps one region). Post-swap activation of a boundary range goes through `activateSwappedRange` (#1102), the ONE place both tiers (`replaceBoundaryRange`, `swapMarkerRange`) reactivate scripts and upgrade custom elements. Two things it owns and a new call site must keep: it SNAPSHOTS the range before iterating, because `reactivateScripts` replaces a top-level script and a detached node cuts a live `nextSibling` walk (every later node in the range is then silently skipped); and `reactivateScripts` handles container-IS-a-script itself, since `querySelectorAll` never matches the node it is called on. A top-level script therefore re-executes on every swap of its range, INCLUDING one the keyed differ reused by `id`, matching what a descendant script in a reused container has always done. `data-webjs-permanent` splits into two cases and they must NOT be unified (#1252). The marked element IS a script: NEVER exempt, whether the walk reaches it as the container or as a descendant of one (the regraft selector has no tag filter, so a marked script IS preserved by identity and does land in the WeakSet, which is why the exemption is STRICT containment and never reflexive). The regraft also has a both-exist guard, so on the swap that first mounts a route there is no live node to preserve and exempting the inert parsed copy would leave a script that runs on a cold load and never on a soft nav, which is #1102 itself. Script INSIDE a preserved marked element: exempt, because the attribute is subtree-scoped (`diffElementInPlace` already returns early rather than recursing into one) and re-emitting an init script against an instance the author kept alive is a double-initialization. The filter keys on the `regraftedPermanents` WeakSet, which the two regrafts populate on every successful path, so it means ACTUALLY preserved by identity rather than merely carrying the attribute; an attribute-only filter would leave a first-mount permanent element's scripts never running at all | +| `router-client.js` | Turbo Drive–style client router; entry: `enableClientRouter` / `navigate`. Also exports `refreshPage(mode)` (#1398): re-render the CURRENT url on the server and apply it in place, recording no history entry and never scrolling, so the reader keeps their place and Back still goes to the previous page. `'page'` (the default) morphs the deepest shared boundary, so the outer layout's DOM and the hydrated state of its components survive; `'shell'` takes the full-body tier (`swapFullBody`, extracted from `applySwap`'s tail so the background snapshot-restore path and this share one implementation) because a LAYOUT's own markup lives outside every children range and a boundary morph would leave it untouched. It SUPPRESSES `X-Webjs-Have`, which is required rather than an optimisation: the server short-circuits at the first layout whose segment path and route key the client already holds, and a same-URL request matches every one of them, so the response would omit the layouts and a layout edit would be invisible. It also suppresses the outgoing snapshot (it would write the pre-edit page under the url being refreshed, and a later Back would restore it), the optimistic `loading.ts` skeleton (flashing one over content that is already correct is worse than one round trip of stale content), and the prefetch fast path (every cached copy predates the change). It does NOT reload changed component modules and cannot, since `customElements.define` is once-per-tag and a module URL is fetched once per document. `enableClientRouter` publishes it as `globalThis.__webjsRefreshPage` and `disableClientRouter` deletes it, which is how the dev live-reload client (a separate served script with no import of this module) feature-DETECTS it: the global's absence covers both `webjs.clientRouter: false` and a page that ships no component at all, and both fall back to a full reload. Also exports `loadFrame(frameEl, url)` (#253), the reusable frame self-load `webjs-frame.js` calls: it fetches `url` as a frame nav (the `x-webjs-frame` header) and applies the matched subtree through the SAME `fetchAndApply` frame-swap path a click uses (no history push / snapshot / optimistic skeleton, since it swaps one region). Post-swap activation of a boundary range goes through `activateSwappedRange` (#1102), the ONE place both tiers (`replaceBoundaryRange`, `swapMarkerRange`) reactivate scripts and upgrade custom elements. Two things it owns and a new call site must keep: it SNAPSHOTS the range before iterating, because `reactivateScripts` replaces a top-level script and a detached node cuts a live `nextSibling` walk (every later node in the range is then silently skipped); and `reactivateScripts` handles container-IS-a-script itself, since `querySelectorAll` never matches the node it is called on. A top-level script therefore re-executes on every swap of its range, INCLUDING one the keyed differ reused by `id`, matching what a descendant script in a reused container has always done. `data-webjs-permanent` splits into two cases and they must NOT be unified (#1252). The marked element IS a script: NEVER exempt, whether the walk reaches it as the container or as a descendant of one (the regraft selector has no tag filter, so a marked script IS preserved by identity and does land in the WeakSet, which is why the exemption is STRICT containment and never reflexive). The regraft also has a both-exist guard, so on the swap that first mounts a route there is no live node to preserve and exempting the inert parsed copy would leave a script that runs on a cold load and never on a soft nav, which is #1102 itself. Script INSIDE a preserved marked element: exempt, because the attribute is subtree-scoped (`diffElementInPlace` already returns early rather than recursing into one) and re-emitting an init script against an instance the author kept alive is a double-initialization. The filter keys on the `regraftedPermanents` WeakSet, which the two regrafts populate on every successful path, so it means ACTUALLY preserved by identity rather than merely carrying the attribute; an attribute-only filter would leave a first-mount permanent element's scripts never running at all | | `webjs-frame.js` | The `` custom element (a swap anchor; the router does the swap). Adds the `src` + `loading` self-load (#253): an eager (`connectedCallback`) or lazy (viewport, via `lazy-loader.js`'s `observeViewportOnce`) self-fetch through `router-client.js`'s `loadFrame`, with a per-element loaded-URL guard so eager connect / the lazy observer / a `src` mutation never double-fetch. SSR-inert (defined client-side only) | | `webjs-stream.js` | The `` surgical-update element + `renderStream(payload)` (#248). The element self-applies its action on connect via native DOM (append / prepend / before / after / replace / update / remove against a `target` id or `targets` selector), cloning its single `