research: client-router scroll architecture, remove the two-phase restore instead of managing it

[vivek7405]

The question

Can the client router's scroll and history handling be reduced to something
close to Next's, which "just works", without re-opening any client-router
regression already fixed?

Prompted by #1428, where the iOS back-swipe preview is blank. That turned out
not to be a bug in the swap at all, but a downstream consequence of one
architectural decision, and the same decision is the source of most of the
complexity in the scroll path.

The finding, measured

Three frameworks, one real iPhone, same gesture, plus a local reproduction of
the WebJs case served from a branch build.

	push timing	scrollRestoration	gesture preview
WebJs pre-#1410	after the swap	manual	blank
WebJs post-#1410	before the swap	manual	blank
Turbo Drive	before the swap, inside a rAF	manual	blank
Next App Router	after commit	auto	clean
WebJs + ?scrollauto	before the swap	auto	clean

One property explains all five rows. Under auto WebKit records a scroll
position per history entry and composes the gesture preview from it. Under
manual the app takes that over, the browser records nothing, and the preview
has no scroll state to render against.

?scrolllast, which defers the scroll-to-top and therefore tests the CLAMP that
#1406 built its case on, came back negative. So it is not the scroll-to-top
clamping the offset. It is the browser not recording an offset at all.

#1410 was not the fix and could not have been. It shipped Turbo's exact
ordering, and Turbo blanks. The ordering change stands on its own merits and is
not being reverted, but it was aimed at the wrong mechanism.

Why manual is there

Commit 231cd8f8, 17 May 2026, states it:

history.scrollRestoration defaults to 'auto', so the browser tries to
restore scroll on popstate at the same time our SPA does. The two race;
whichever runs last wins; the user sees the correct cached scroll briefly,
then it jumps somewhere else as the second write lands.

So manual silences a race between TWO scroll writers, and the commit says the
shape was copied from Turbo's assumeControlOfScrollRestoration.

Why Next needs no equivalent

Next has one writer. handlePopState dispatches a restore and nothing else.
The restore reducer sets scrollRef: null
(router-reducer/reducers/restore-reducer.ts:54), and InnerScrollHandler
opens with if (scrollRef === null || !scrollRef.current) return
(layout-router.tsx:151-156). Next never scrolls on popstate. With one writer
there is no race, so there is nothing to silence, so auto survives, so the
preview works.

Worth recording because it was our first wrong answer: the <Activity> bfcache
is NOT the explanation. It is gated behind process.env.__NEXT_CACHE_COMPONENTS
(layout-router.tsx:754) and MAX_BF_CACHE_ENTRIES is 1 without that flag, so
default Next preserves no inactive trees.

The actual root cause in WebJs

WebJs cannot simply drop its own restore, because its restore is TWO-PHASE:
insert the snapshot HTML (short), then let custom elements upgrade and re-render
(taller). Native restoration lands against the short version. Next's back render
is single-phase, one React commit from its client cache, so the height is right
when the browser restores.

That single fact is upstream of everything:

• manual, to stop the browser fighting a restore we had to own
• the blank gesture preview, because manual stops the recording
• scroll.js, 259 lines of anchoring suppression, catch-up chase, restore
  generations, and two-frame deferral
• nav-scroll-anchor-restore.test.js, 861 lines and 23 tests, every one of them
  a rule about growth that arrives late
• dogfood: back-button scroll restores ~763px too low on pages that grow after swap #1310 and fix: back-button restore survives late layout growth #1313 themselves

The problem is not scroll-restoration policy. It is that the restored DOM
changes height after insertion.

The proposed architecture

Record the settled height in the snapshot and reserve it across the swap.

snapshot-cache.js:11 currently records { html, scrollX, scrollY }. Add the
settled scrollHeight. On a popstate restore, reserve that height on the
document before applySwap, then release it once the upgrade settles.

Sequence, for a same-document traversal:

1. The browser restores scroll as part of the traversal, against the outgoing
   document, which is still tall enough for the offset to be reachable
2. popstate fires, and the router reserves the recorded height
3. applySwap inserts the shorter incoming DOM, which cannot clamp the scroll,
   because the reservation holds the height
4. Components upgrade, the real content reaches the recorded height, and the
   reservation is released

Consequences:

• Native auto restoration lands correctly, so the router needs no restore of
  its own, so there is ONE writer, exactly Next's property
• manual is no longer needed, so WebKit records per-entry offsets again, so
  the gesture preview is fixed, which closes dogfood: iOS back-swipe still blank after #1410; A/B the snapshot timing on-device #1428 and dogfood: iOS back-swipe gesture flashes a blank page (client router) #641
• Late growth cannot move the reader, because there is no window during which
  the document is short. dogfood: back-button scroll restores ~763px too low on pages that grow after swap #1310 and fix: back-button restore survives late layout growth #1313 become structurally impossible rather
  than continuously managed
• Most of scroll.js and the restore branch in navigator.js can go

What this must not break, and whether it does

Every client-router regression with a live guard, checked against the proposal.

Issue	What it guards	Affected?
#1310 / #1313	back restore survives late layout growth	Mechanism replaced, guarantee preserved. See caveat below
#1406 / #1410	history recorded before the swap	No. Independent of scroll, and kept
#601	forward nav scroll is instant under scroll-behavior: smooth	No. Forward scroll-to-top stays ours and is unchanged
#639	query params preserved on every nav path	No
#610	sticky header flicker on iOS	No. Fixed in CSS with position: fixed
#1015	route-keyed comment boundary swap	No
#1114	full-load fallback is observable	No
#936 / #906	CSS drop, hydrated-component corruption	No
#1102 / #1252	script reactivation, permanent-element descendants	No
#1046	page-scoped meta leak in the head merge	No
#1398	refreshPage preserves scroll, records no entry	No, but it has a scroll assertion, so it is on the verify list

The caveat, stated plainly. "Not regressing previously fixed regressions"
means preserving the BEHAVIOURS, not the tests. The 23 tests in
nav-scroll-anchor-restore.test.js mostly assert the MECHANISM (anchoring
windows opening and closing, catch-up chases, restore generations), and that
mechanism would no longer exist. So they cannot all be kept. The migration is:
keep and re-point the OUTCOME tests, which is that a reader returning to a page
that grows late lands at the recorded offset, and delete the ones that assert
machinery being removed. Anyone reviewing this must be told that a shrinking
test count here is the intended result and not lost coverage.

Risks, none of them settled yet

• Browser restore timing is an assumption. The design assumes the browser
  restores as part of the traversal, before popstate, against the outgoing
  document. That is the load-bearing claim and it is not yet measured.
• A changed viewport between visits (a rotated phone, a resized window)
  makes the recorded height wrong. Release-on-settle bounds the damage, but the
  reservation needs a sane cap.
• Where the reservation goes. A min-height on the wrong element interacts
  with sticky and fixed positioning, which is exactly the neighbourhood of dogfood: mobile navbar flickers on forward nav (backdrop-blur sticky header) #610.
• A page that legitimately shrinks must not be held tall past settle.

Decision

Pursue the height reservation. It is the only option that removes the class of
bug rather than managing it, it is small, it fits the existing snapshot
structure, and it converges on Next's one-writer property, which is what "just
works" actually means here.

Two alternatives were weighed and rejected:

A transient flip (auto normally, manual only across the restore) treats
the symptom. It keeps both writers, keeps all 259 lines of scroll.js, and adds
a per-entry inheritance hazard, since history.scrollRestoration is a property
of the current entry and a new entry inherits it, so a flip that is not restored
before the next forward navigation silently brings the blank back on the second
nav. Kept as the fallback if the reservation fails on-device.

A live-DOM bfcache (keep the outgoing nodes, re-show on Back) would work and
is what Activity does, but it fights the web-components model. Detaching a
custom element fires disconnectedCallback and re-attaching fires
connectedCallback, so every component author inherits lifecycle churn, and
holding two pages in one document invites duplicate ids plus focus and form
problems.

How this gets verified, given the record

Four hypotheses in this investigation were wrong before being measured, and two
device rounds were void because the setup was not verified first. So the
reservation ships as a guarded lever, gets the #1310 guards run under it across
Chromium, Firefox, and WebKit locally, and gets one on-device cell for the
preview, before a single line of it becomes default behaviour.

The implementation is separate tracked work.

[vivek7405]

Deep dive: the measurement trail, including the wrong turns

Recorded because four hypotheses in a row were wrong before anyone measured
them, and two device rounds produced nothing because the setup was never
verified. The wrong turns are the reusable part.

Void round 1: world.hey.com

Chosen as a production Hotwire site to compare against. It ships NO Turbo. The
only script on /dhh is Plausible analytics, and the "Hotwire" text on the page
is bio copy linking to hotwired.dev. Every link there is a full document
navigation using WebKit's cross-document snapshot, which is a different
mechanism from a same-document pushState entry. A clean back-swipe there was
guaranteed and meant nothing.

Void round 2: same-page anchors

The next attempt used turbo.hotwired.dev/handbook/drive, which is genuinely
Turbo-driven, but clicked links from the table of contents at the top of the
page. Those are href="#form-submissions" style fragments, 46 of them, and
Turbo does not intercept same-page anchors. No pushState was involved.

The valid run used the link at 98.8% of the document, "Next: Smooth page
refreshes with morphing", which changes the URL path. Confirmed Turbo really
was driving it: a 93KB Turbo bundle carrying scrollRestoration and
pushState, data-turbo-visit-direction="forward" in the served HTML, and no
data-turbo="false" opt-out anywhere.

Void round 3: the control mistaken for a lever

The first local run used plain /blog with no query param, which is the
control, not a lever cell. It reproduced the bug, which is useful, but it says
nothing about any candidate fix.

The resolution trap that would have wasted a deploy

The local rig serves the website app from the branch worktree over the LAN to a
real iPhone. A linked worktree's node_modules/@webjsdev/core points at the
PRIMARY checkout, so every bare specifier resolves to main and the app serves
a bundle with no lever in it while looking entirely normal. Repointed at the
worktree's own packages/core, then confirmed scrollauto was present in the
served core bundle AND in the served HTML before the device was touched.

Verifying the lever took effect was done in-page rather than assumed: the loaded
document reported __webjsDiag.scrollauto: true and
history.scrollRestoration: "auto".

Deep dive: why a blanket auto is not the fix

?scrollauto clears the preview, and on-device the scroll position also looked
correct on both the swipe and the Back button, which suggested the May 2026 race
might no longer be reachable.

It is. The default was flipped to auto locally and the routing browser suite
run against it. Five failures per engine on Chromium, Firefox, AND WebKit. Two
were the lever's own tests asserting the default that had just been inverted.
The other three are real:

• late growth above the viewport does not push the reader down
• a revalidation that answers before the growth still holds the reader
• anchoring WORKS again once the window has closed

That is #1310, reproducing deterministically on all three engines.

The device looked clean because /blog is not the pathological case. #1310
needs late layout growth, where the snapshot restores at the settled height
while the swapped-in DOM is still short until components upgrade. On /blog
that gap is small enough not to show. The browser tests construct the case
deliberately, which is exactly why a green device does not license shipping.

So both halves are measured, and neither current option is shippable:

	preview	#1310 restore
manual, today	blank	correct
auto, blanket	clean	broken

That is what makes the reservation the interesting option rather than a
preference. It is the only one that can satisfy both, because it removes the
short-document window that both the clamp and the anchoring machinery exist to
handle.

Cross-references

• dogfood: iOS back-swipe still blank after #1410; A/B the snapshot timing on-device #1428, the reported bug and the verdict thread
• fix: record the history entry before the swap, not after #1410 and dogfood: record history before the swap so iOS back-swipe is not blank #1406, the ordering fix and the reasoning it was built on
• dogfood: iOS back-swipe gesture flashes a blank page (client router) #641, the first report, closed with no fix and superseded by dogfood: record history before the swap so iOS back-swipe is not blank #1406
• chore: on-device isolation flags for the #610 iOS nav repaint (diagnostic) #637 and fix: use position:fixed for the blog header to end the iOS nav flicker (#610) #640, the guarded-lever method this reused, which isolated dogfood: mobile navbar flickers on forward nav (backdrop-blur sticky header) #610
• dogfood: back-button scroll restores ~763px too low on pages that grow after swap #1310 and fix: back-button restore survives late layout growth #1313, the scroll restore behaviour the design must preserve
• 231cd8f8, the 17 May 2026 commit that introduced manual and states why

[vivek7405]

Correction: the Navigation API design in this record is a measured dead end

Recording this against the body above so the next reader does not implement it.

What was built. Traverses moved onto navigation.addEventListener('navigate')
with event.intercept(), the Remix 3 model, on the reasoning that intercept()
defers the UA's scroll restoration until the handler settles. It worked in the
harness: 222 routing browser tests green on Chromium, Firefox and WebKit.

Why it is gone. On a real iPhone the interception BROKE the very preview it
was meant to fix. Whatever scroll: 'manual' on an interception signals to
WebKit, it costs the gesture preview the same way the history-wide mode does.
The whole path was removed rather than left dormant.

What shipped instead (PR #1430): leave history.scrollRestoration alone, and
absorb the UA's own late restore inside the restore window that is already open,
by writing back an off-target programmatic displacement for its span. The
Navigation API is not involved.

Two claims in the body above are wrong

1. "dogfood: back-button scroll restores ~763px too low on pages that grow after swap #1310 and fix: back-button restore survives late layout growth #1313 become structurally impossible rather than continuously
   managed." Too strong. Height reservation addresses the CLAMP; the dogfood: back-button scroll restores ~763px too low on pages that grow after swap #1310
   double-count is scroll ANCHORING responding to growth above the viewport,
   which a total-height reservation does not prevent. scroll.js stays, and it
   grew slightly.
2. "The browser restores as part of the traversal, before popstate, against
   the outgoing document." Measured false. Instrumented on the rig:
   scrollY is 0 at popstate handler entry and the recorded offset by the next
   frame, so the UA's restore lands a frame AFTER the handler. That is what made
   the in-window write-back the right shape, and it is the opposite of the
   ordering assumed here.

Method note worth more than either

Two on-device rounds reported failure against code that was never running: the
rig's content-hashed bundle URL is memoized at server boot and served
immutable, so rebuilding dist underneath a running webjs start left the
device on its cached copy. Verifying by grepping the bundle on the server's disk
proves what a NEW client would get and says nothing about a device that already
has one. Restart the server, read the ?v= the HTML advertises, fetch THAT url.