Add --state-in-memory: keep the guest register file out of SSA

[dougchansan]

The LLVM backend loads every state slot a region touches into an alloca at entry and lets mem2reg promote it. That is the textbook move, and on this workload it loses to the C backend.

Why promotion backfires here

Promoting a whole guest register file gives the allocator far more simultaneously live values than the machine has registers, so it spills them straight back. Measured on GM4E01, one boot chunk carries 631 state phi nodes across 228 basic blocks, 20+ per block in hot loops:

%state106.0 = phi i32 [ %186, %fallback_repeat_check ], [ %85, ... ]
%state105.0 = phi i32 [ %185, %fallback_repeat_check ], [ %84, ... ]
%state103.0 = phi i32 [ %183, %fallback_repeat_check ], [ %82, ... ]
...

x86-64 has 16 GPRs. The live set is many times that, so the result is phi construction, a much larger function and worse allocation — arriving back at values in memory, with more instructions around them. The C backend never had this problem because generated C operates directly on ctx->gpr[N] and clang hoists only what pays, locally.

The change

--state-in-memory points state_[slot] straight into CPUState. Every load and store site works unchanged; what disappears is the entry prologue and, with it, the materialization barriers — those exist only to flush values that were hoisted, and nothing is hoisted.

Same chunk, with the option on: 0 state phis, getelementptr straight off ctx. LLVM still forwards stores to loads and keeps values in registers where that pays; it is simply no longer forced to keep the entire register file live across a region.

Measurements

Same DOL per title, same module compiler (clang 20.1.8, -O2 -flto=thin), bench-interleaved 6 forward + 6 reversed pairs, machine idle, no build overlapping a measurement. speed is the guest clock ratio; fps is the median present rate over the sample window.

With --state-in-memory, against the C backend

platform	title / scene	C speed / fps	arm speed / fps	ratio	pairs	vs current LLVM
x86-64	GM4E01 race.sav	1.0597 / 63.53	1.3208 / 79.10	1.2464	12/12	1.46x
x86-64	GLME01 foyer.sav	1.7026 / 50.57	1.9482 / 58.03	1.1442	12/12	1.37x
x86-64	SOUE01 gameplay.sav	1.6724 / 49.71	1.8620 / 55.56	1.1134	12/12	1.57x
AArch64 Pi 4	GLME01 cold boot	0.1920 / 5.27	0.2490 / 6.98	1.3035	6/6	—

Every measurable title, every pair. All are ratios against the C backend, measured directly rather than chained. On AArch64 the same option measures 1.1327 against the unmodified LLVM backend (6/6 pairs), so the win is not an artefact of the C control.

Module size falls on every title:

title	before	after
GM4E01	389.1 MB	161.0 MB
GLME01	204.9 MB	91.9 MB
SOUE01	548.0 MB	221.8 MB
GLME01 (AArch64)	126.0 MB	78.7 MB

The gain is larger on x86-64 than on AArch64, which is what the register-pressure explanation predicts: AArch64's 31 GPRs already absorb much of the promoted live set — which is also why the LLVM backend was already ahead of C there and behind it on x86-64. The asymmetry is the mechanism showing through.

Backend landscape across the whole test set, x86-64

Context for where the backend sits per title. These arms are the existing backend and two configurations of the AOT branch (#16).

title	arm	speed	fps	ratio vs C	pairs
GM4E01 Mario Kart	C control	1.0411	62.37	1.0000	—
	LLVM (pre-rewrite)	0.5441	32.50	0.4990	0/12
	LLVM (current)	0.8793	52.67	0.8533	0/12
	AOT branch, fixed-chunk	1.1459	68.69	1.1007	10/12
	AOT branch, region	1.1243	67.67	1.1181	11/12
	--state-in-memory	1.3208	79.10	1.2464	12/12
GLME01 Luigi's Mansion	C control	1.6660	49.44	1.0000	—
	LLVM (pre-rewrite)	1.0017	29.64	0.5800	0/12
	LLVM (current)	1.4516	43.13	0.8330	0/12
	AOT branch, region	1.7388	51.79	1.0437	10/12
	--state-in-memory	1.9482	58.03	1.1442	12/12
SOUE01 Skyward Sword	C control	1.6724	49.71	1.0000	—
	LLVM (current)	1.1481	33.94	0.7110	1/12
	AOT branch	does not render	—	—	—
	--state-in-memory	1.8620	55.56	1.1134	12/12
GC6E01 Colosseum	C control	—	54.67	—	—
	all LLVM backends	—	0.34–0.85	not measurable	—

Two titles do not yield a usable LLVM number, and neither is affected by this change:

• SOUE01 Skyward Sword renders under the current LLVM backend (0.7110) and under this change (1.1134), but not under the pre-rewrite backend or the AOT branch — both advance frame_count to an identical 17684 while presenting exactly 1 frame.
• GC6E01 Colosseum does not render under any LLVM backend, on x86-64 or AArch64, at cold boot or from a savestate, including on freshly extracted retail disc data. The guest runs at full speed with the video interrupt ticking normally while frame_count stays frozen. Worth flagging for anyone benchmarking that title: speed counts retired guest cycles, so a non-rendering module rewards itself on that metric — on speed alone it reads as a 3.58x win over C. fps and sample count are what expose it.

An Apple Silicon run is in progress and will follow as a comment.

Correctness detail

Three slot families keep their allocas. CR0..CR7 are 4-bit nibbles of CPUState.cr, XER_CA..XER_SO are individual bits of xer, and XER itself writes only the low 29 bits while preserving the flag bits. storeContext/loadContext pack and unpack them, so they have no standalone storage to point at. Pointing them at raw addresses corrupts CPUState — it produced Invalid read from 0x00000010 and a green screen at 0 fps. slotIsPacked() keeps them on the promoted path.

Cache key

The option participates in the object cache key. It changes emitted code, so an entry built with it must not collide with one built without it. Without that, toggling the flag returns the other configuration's objects and you end up benchmarking a module against itself — the emitter never runs, so nothing looks wrong.

Notes

Off by default.

Skipping the PromoteMemToReg call alone achieves nothing: optimizeModule() runs the standard -O2 pipeline before emission and SROA/mem2reg promote the allocas anyway. Verified byte-identical IR either way. The allocas have to not exist.

Tests: 26/28 pass. codegen_compile and llvm_generated_compile fail identically on an unmodified build in this environment (rc.exe not on PATH in the nested build), so they are not related to this change.

Build notes, both environmental rather than code dependencies of this change:

• Linking against an LLVM built without BPF/WebAssembly/NVPTX needs Register only the LLVM targets this build can emit for, and build the bench on Windows #17, which gates target registration on what the build can actually emit for. An LLVM with all targets enabled links main as-is.
• Emitting SOUE01 at all needs Route fallback blocks through per-edge trampolines so the phi matches the CFG #18. Without it all 9917 objects emit and the verifier then rejects the module with PHINode should have one entry for each predecessor of its parent basic block! on %fallback_pc — the defect Route fallback blocks through per-edge trampolines so the phi matches the CFG #18 fixes. The SOUE01 row above was measured with Route fallback blocks through per-edge trampolines so the phi matches the CFG #18 merged.

[dougchansan]

Filled in the remaining titles. --state-in-memory now measured on every title that produces a usable LLVM number, and it wins on all of them.

platform	title / scene	C speed / fps	arm speed / fps	ratio	pairs	vs current LLVM
x86-64	GM4E01 race.sav	1.0597 / 63.53	1.3208 / 79.10	1.2464	12/12	1.46x
x86-64	GLME01 foyer.sav	1.7026 / 50.57	1.9482 / 58.03	1.1442	12/12	1.37x
x86-64	SOUE01 gameplay.sav	1.6724 / 49.71	1.8620 / 55.56	1.1134	12/12	1.57x
AArch64 Pi 4	GLME01 cold boot	0.1920 / 5.27	0.2490 / 6.98	1.3035	6/6	—

12/12 pairs on every x86-64 title, 6/6 on AArch64. Same C control per title as every other arm measured on it, same module compiler, interleaved forward and reversed blocks.

Two things worth calling out from getting SOUE01 measured:

It needs #18 to emit at all. Without it, all 9917 objects emit and the module verifier then rejects the IR:

PHINode should have one entry for each predecessor of its parent basic block!
  %fallback_pc = phi i32 [ %entry_pc, %cold_entry ], [ -2143668300, %fallback_edge ], ...

That is the defect #18 fixes, quoted verbatim in its own commit message, down to the same 9917 chunk count. With #18 merged it emits clean. This is a prerequisite for measuring the title, not a dependency of the change itself.

It renders, which the AOT branch never managed. On this Wii title the AOT branch (#16) advances frame_count to 17684 while presenting exactly 1 frame — it runs the guest but never draws. --state-in-memory renders it normally at 55.56 fps. So the two approaches are not equivalent on Broadway.

Module sizes with the option on: GM4E01 389.1 → 161.0 MB, GLME01 204.9 → 91.9 MB, SOUE01 548.0 → 221.8 MB, GLME01 on AArch64 126.0 → 78.7 MB.

GC6E01 Colosseum is still excluded and unaffected by this change — no LLVM backend renders it on either architecture, including on freshly extracted retail disc data.

[dougchansan]

Apple Silicon result, and it is a negative one. Reporting it because it changes how the option should be used, not despite that.

Measured against the LLVM baseline (not the C backend) on an M-series Mac, macOS 26.1, Luigi's Mansion, 12 interleaved pairs, 60s sample after 20s warmup, null graphics:

	median fps	range
LLVM baseline	70.626	69.8 – 71.7
--state-in-memory	65.986	65.8 – 67.5

Median ratio 0.9387 — 6.1% slower. 0 of 12 pairs favour the change. The ranges do not overlap.

Validity checks, since a negative deserves the same scrutiny as a positive:

• The two modules are genuinely different builds: chunk_0017 md5 bf776b5b… vs f8a3dd88…
• The dylib shrinks 135,442,696 → 84,254,600 bytes (−37.8%), matching the size coefficient this change shows elsewhere (GLME01 on AArch64 is −37.5%), so the option is definitely active
• EmulationSpeed = 0.0000 on both arms, so neither is pinned at the 59.94 target
• 0 of 12 runs discarded

Where that leaves the change, all figures against the current LLVM backend:

platform	title	effect
x86-64	GM4E01 / GLME01 / SOUE01	+46% / +37% / +57%
AArch64 Cortex-A72 (Pi 4)	GLME01	+13%
Apple Silicon M-series	GLME01	−6.1%

A hypothesis consistent with that ordering, though not verified by looking at spill counts on the Apple target: the change pays where the register file cannot be kept in machine registers. x86-64 has 16 GPRs and gains most; Cortex-A72 has 31 and gains less; Apple Silicon has 31 architectural registers but a much wider machine with far deeper renaming, so it tolerates the promoted live set and the extra CPUState traffic is not repaid.

This is the clearest argument for the option defaulting off, which is how it is submitted. It is a per-target tuning knob rather than a universal win, and anyone enabling it should measure on their target.

Separately, a note on ARM coverage in this PR: I attempted a second AArch64 title (GM4E01 on the Pi) and discarded it. Cold-boot Mario Kart on the Pi is not a controlled scene — the driver lands on logos, title or attract demo depending on timing, and ratios spanned 0.30–1.22 with the C control alone swinging 7.69–25.01 fps. A single smoke run had suggested a large win; the paired runs showed that was scene lottery. AArch64 coverage here therefore rests on GLME01 alone, and pinning a second ARM title would need a Pi-native savestate since savestates do not restore across architectures.

[dougchansan]

Second AArch64 title, and it corroborates the first.

GM4E01 Mario Kart, Raspberry Pi 4, from a pinned savestate, 3 forward + 3 reversed pairs, 70s each:

pair	C speed / fps	--state-in-memory speed / fps	ratio
fwd1	0.1098 / 6.66	0.1596 / 9.52	1.4533
fwd2	0.1119 / 6.75	0.1559 / 9.39	1.3930
fwd3	0.1118 / 6.68	0.1615 / 9.71	1.4450
rev1	0.1118 / 6.71	0.1540 / 9.38	1.3774
rev2	0.1104 / 6.70	0.1541 / 9.31	1.3966
rev3	0.1115 / 6.74	0.1566 / 9.42	1.4041

Median 1.4003 vs the C backend, 6/6 pairs, spread 1.3774–1.4533.

A note on method, because the first attempt at this title was thrown away. Cold-booting Mario Kart on the Pi is not a controlled scene — the driver lands on logos, title or the attract demo depending on timing, and the ratios spanned 0.30–1.22 with the C control alone swinging 7.69–25.01 fps. A single smoke run had suggested a large win, and it was scene lottery. Windows savestates do not restore on aarch64, so the Pi needed its own: the game was driven into a race with the automation pad and a savestate captured there. With the scene pinned, the C control now varies by under 1% between runs (6.66–6.75 fps).

Updated platform picture, all against the C backend:

platform	title	ratio	pairs
x86-64	GM4E01	1.2464	12/12
x86-64	GLME01	1.1442	12/12
x86-64	SOUE01	1.1134	12/12
AArch64 Cortex-A72	GLME01	1.3035	6/6
AArch64 Cortex-A72	GM4E01	1.4003	6/6
Apple Silicon M-series	GLME01	0.9307 (regression)	0/12

Five titles-platforms favour the change at every pair; Apple Silicon regresses at every pair. The Apple Silicon result has since been repeated as an independent second series (0.9387 at a 60s window, 0.9307 at 90s, 0/24 pairs across both), so that regression is stable rather than noise.

[dougchansan]

Apple Silicon (AArch64) follow-up: the regression is split evenly between GPRs and FPRs

On Apple Silicon this option is a regression, not a win. Luigi's Mansion
(GLME01), pinned foyer.sav, interleaved paired runs (6 forward + 6 reversed),
per-config object caches, median of per-pair ratios.

Full option vs baseline on this title/state measured 0.9307 in an earlier
series. To find out which half of the state costs the time, I built an
intermediate arm that moves only the GPRs into CPUState and leaves FPR/PS1
SSA-promoted.

A. baseline -> GPR-only

pair	order	baseline	gpr	ratio
1	fwd	71.674	68.501	0.9557
2	fwd	71.591	68.669	0.9592
3	fwd	71.388	68.937	0.9657
4	rev	71.571	69.864	0.9761
5	rev	71.321	69.379	0.9728
6	rev	73.107	69.675	0.9531

median 0.9625, 6/6 pairs favour baseline.

B. GPR-only -> full option

pair	order	gpr	full	ratio
1	fwd	69.639	67.956	0.9758
2	fwd	69.856	67.441	0.9654
3	fwd	70.051	67.558	0.9644
4	rev	69.955	66.941	0.9569
5	rev	69.719	66.989	0.9608
6	rev	69.067	67.367	0.9754

median 0.9649, 6/6 pairs favour GPR-only.

Decomposition. 0.9625 x 0.9649 = 0.9285, against 0.9307 measured
independently end-to-end -- consistent within run-to-run noise (0.2%).

So the ~7% Apple Silicon cost is split almost exactly in half: ~3.75 points from
moving GPRs into CPUState, ~3.5 points from moving FPR/PS1. Neither half
dominates, which argues against a targeted fix (e.g. keeping FPRs promoted) and
in favour of leaving the option off by default on AArch64 -- as it is.

Scope note: this decomposition is GLME01/foyer.sav only. Whether the
regression generalises to other titles and scenes on Apple Silicon is still open.

[dougchansan]

Incorporated upstream: 1bec355 carries --state-in-memory (with the packed-slot guard) integrated with the native-ABI work, so this branch is superseded. Closing. One caveat moved to #24: current main measures well below this branch on both benchmarked titles, so the option's measured wins (1.25x x86-64, 1.30x AArch64) should be re-validated once that regression is understood.