Skip to content

Make the instruction after an MSR write a region leader - #28

Open
dougchansan wants to merge 1 commit into
ExpansionPak:mainfrom
dougchansan:fix/msr-region-leader
Open

Make the instruction after an MSR write a region leader#28
dougchansan wants to merge 1 commit into
ExpansionPak:mainfrom
dougchansan:fix/msr-region-leader

Conversation

@dougchansan

@dougchansan dougchansan commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

emitStateWrite side-exits to guest_pc + 4 when mtmsr takes MSR[EE] from 0 to 1, so the dispatcher can deliver a pending external interrupt there. That address was not an entry-switch case, so the dispatch missed, fell into emitColdEntry, and the runtime interpreted from there until it reached the next region leader.

On Pokémon Colosseum OSRestoreInterrupts runs roughly every 139 guest cycles and its tail sits exactly in that gap — 140M interpreted instructions, 92% of all dispatches in that arm. The C backend never pays this because it labels every address.

Why an MSR write rather than MAY_EXIT

DOLIR_EFFECT_MAY_EXIT is carried by a large number of instructions; the broad rule fires 160,525 times on Colosseum, and every extra leader fragments a region and costs optimisation. Measured, the broad rule cost the plain LLVM backend 5–16% on the three titles that never needed it. Keying on the MSR write is what makes this free elsewhere.

Measurements

A/B against the unmodified backend — same recompiler binary, same DOL, only this rule differs. 3 pairs per scene, 15s warmup, 20s samples, forward and reversed blocks per comparison.

Title Mean Min Max Scenes
Pokémon Colosseum 1.4943 1.4730 1.5060 1 scene x4 runs
Luigi's Mansion 1.0042 0.9999 1.0082 2
Mario Kart: Double Dash 0.9971 0.9586 1.0179 6
Skyward Sword 1.0020 0.9995 1.0065 2

The neutral titles are each the mean of two independent full campaigns, and the per-title figures reproduced across them to within 0.4%.

Colosseum is a single scene (clean-field2) measured four times: 1.4730, 1.4960, 1.5023, 1.5060. An earlier fifth observation of 1.95 is not included — it did not reproduce, and its arm median (1.7797) is far outside the 1.187 the four later runs agree on, so it is treated as an artefact of machine state rather than a valid measurement.

Counter evidence, which does not depend on wall-clock and is stable across all runs — Colosseum clean-field2, 20s:

             hook_fb        native_exc
llvm      147,774,885           12,735
llvm-msr       44,207           25,009

hook_fb is interpreter fallback dispatches. Frame progression over the same window goes 551 -> 1069 frames.

Runtime dependency

This needs the runtime to deliver pending external interrupts at the post-mtmsr boundary. That is already on moderngekko-vendor (after_mtmsr in StaticRecompCore_Run.cpp), so no runtime change is required alongside this. All figures above were measured against that upstream implementation.

It does matter for anyone testing against an older runtime: the interpreter detour this removes was previously the only thing delivering the interrupt. Without the runtime half, Colosseum advances 19 frames in 20s instead of 1069, and native_exc reads 312 instead of 25,009.

Checks

Lockstep against the interpreter shows no divergence beyond the unmodified control — both arms report an identical first 8 divergences and both reach the report cap, so this changes dispatch granularity, not semantics.

emitStateWrite side-exits to guest_pc+4 when mtmsr takes MSR[EE] from 0 to 1,
so the dispatcher can deliver a pending external interrupt there. That address
was not an entry-switch case, so the dispatch missed, fell into emitColdEntry,
and the runtime interpreted from there until it reached the next leader.

On Pokemon Colosseum OSRestoreInterrupts runs roughly every 139 guest cycles
and its tail sits exactly in that gap: 140M interpreted instructions, 92% of
all dispatches. Marking the following instruction a leader makes the exit land
natively instead.

Keyed on an MSR write rather than on MAY_EXIT generally. MAY_EXIT is carried by
a large number of instructions -- the broad rule fires 160,525 times on
Colosseum -- and every extra leader fragments a region and costs optimisation.
Measured, the broad rule cost plain llvm 5-16% on the three titles that never
needed it.

Cross-title A/B against the unmodified backend, mean of per-scene ratios,
3 pairs per scene with forward and reversed blocks:

  Pokemon Colosseum        1.9506  (1 scene)
  Luigi's Mansion          1.0056  (2 scenes)
  Mario Kart Double Dash   0.9986  (6 scenes)
  Skyward Sword            1.0000  (2 scenes)

This depends on the runtime delivering pending external interrupts at the
post-mtmsr boundary. Without that half, the interpreter detour this removes was
the only thing delivering the interrupt, and Colosseum advances 19 frames in
20s instead of 1069.
@dougchansan

Copy link
Copy Markdown
Contributor Author

Follow-up with a larger sample, a measured error bar, and a reproducibility warning.

Colosseum, now n=5

The figure in the description was n=4. A fifth measurement, taken against a runtime updated to current moderngekko-vendor, agrees:

1.4730, 1.4960, 1.5023, 1.5060, 1.5172
n=5  mean 1.4989  sd 0.0164  95% CI 1.484 - 1.514

The harness noise floor was measured, not assumed

Same C module supplied as both control and arm, so the true ratio is exactly 1.0000:

n=13 across 4 titles / 6 scenes
12 of 13 within +/-1.72%
 1 of 13 at +14.97%   (one scene; 3 repeats of it then gave 1.0164 / 0.9972 / 1.0019)

So single runs here carry a rare ~15% excursion. That matters for reading this PR: an earlier observation of 1.95x on this same change did not reproduce and was discarded as exactly that kind of flier. Every number above is a mean over repeated runs, not a single measurement.

The neutral-title claim is unaffected — Luigi's Mansion, Mario Kart and Skyward Sword each reproduced across two independent full campaigns to within 0.4%.

Reproducibility warning for anyone testing this on Colosseum

Savestate choice can silently invalidate the comparison. Of 38 states available here, 17 carry a widescreen patch in their RAM image and report:

SMC: chunk [0x80005300,0x80005500) hash mismatch; interpreter until ...

They boot and render normally, which is what makes them dangerous — the affected chunk falls back to the interpreter and the measurement is quietly wrong. Two further states produced zero frame progress in a 20s window. Worth checking the runtime log for hash mismatch and confirming frame_count actually advances before trusting a scene.

The 8 scenes used for this work span 12.8-37.2 fps and are all confirmed clean.

Scope note

This change makes the LLVM backend faster than it was; it does not by itself make it faster than the C backend on this title. Measured separately, llvm with this fix is still ~17% behind C on Colosseum, and state-in-memory is what closes that gap. Mentioning it so the 1.5x here is not read as an LLVM-vs-C claim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant