Skip to content

ntdll: 64-bit images with a fixed base below 4 GB, TEB ceiling, one child boot at a time; pin wine and FEX - #187

Merged
willfaust merged 4 commits into
willfaust:mainfrom
spitefulowl:fixed-base-and-wow64-fixes
Oct 4, 2026
Merged

willfaust merged 4 commits into
willfaust:mainfrom
spitefulowl:fixed-base-and-wow64-fixes

Conversation

@spitefulowl

@spitefulowl spitefulowl commented Oct 3, 2026 •

Copy link
Copy Markdown
Contributor

Low priority. Depends on willfaust/wine#24 and willfaust/FEX#12 (see Merge order).

Updated onto main (3f87f36)

Rebased onto the merged batch. No commit was dropped. The only overlap is with #141 and 19cbce3:

  • Give a pseudo-process child's emulator and pool copies its own ntdll #141 added the same ios_jit_alias_pushback_peb (the process whose FEX holds the alias-push callback), set in unixcall_ios_push_jit_aliases. That one declaration is kept. This PR no longer declares or sets its own copy and only reads upstream's.
  • ios_push_subfloor_window loads the callback once with acquire, as 19cbce3 does. It then compares the window's owner with that process. The registration path stores the process before its release store of the callback.
  • The catch-up loop of a registering emulator pushes the windows of the registering process and the shared ones.
  • The new arity of ios_subfloor_enum / ios_push_subfloor_window also changes their stubs in tests/host/check-child-ntdll-alias.py (Give a pseudo-process child's emulator and pool copies its own ntdll #141).

Commits

1. Run 64-bit images with a fixed base below 4 GB

Background: a 64-bit PE whose fixed base is below 4 GB and whose relocations are stripped (the default output of Delphi Win64 and older MinGW toolchains, typically at 0x400000) cannot be mapped at its base on iOS. It is mapped high, and its low addresses are served by the sub-floor window.

Such a program failed at every step. Each fix below showed its own log line in a device run on 2026-10-01, and the failure it targeted did not recur:

  • ml1195, load. The header now names the real base, so the loader no longer refuses the image with c0000018.
  • ml1201, VirtualProtect/VirtualQuery. On the image's own low addresses these are translated to the real mapping.
  • ml1202, FlushInstructionCache. On a low address it flushes the real mapping instead of SEGV-ing.
  • ml1203, unwind data. The unix side answers NtQueryVirtualMemory class 1006 for ntdll ARM64EC: unwind data for x64 code running at a fixed base below 4 GB wine#24, so exceptions raised at low RIPs are handled.
  • ml1204. A relocatable DLL with the same preferred base no longer re-points a running image's window.
  • ml1205, windows per process. Windows carry the owner's PEB, so two processes with images at the same low base no longer run each other's code. A window goes only to the FEX of its own process: the callback's process is upstream's ios_jit_alias_pushback_peb. Windows are released at process exit, and the table grows from 8 to 32 slots.
  • ml1206. A stripped fixed-base exe starts at the low address of its entry point.
  • ml1208. The store emulator handles CASB/CASH in all forms. A misaligned halfword is emulated without atomicity.

Kill switches: MADEIRA_SUBFLOOR_VM, MADEIRA_SUBFLOOR_SEH and MADEIRA_SUBFLOOR_LOWENTRY (default on).

Fixes found in review, built only:

  • The KUSER_SHARED_DATA window is no longer translated, so VirtualProtect(0x7ffe0000, PAGE_READWRITE) fails as on Windows.
  • ios_subfloor_window_held reads the holder's headers fault-safe.

2. Reserve 64-bit TEB blocks without the WoW64 address ceiling (ml1210)

user_space_wow_limit is session-wide here. After any 32-bit process, a 64-bit thread's next TEB block was requested below 4 GB, where iOS maps nothing, so a 64-bit program could not create its 33rd thread. The ceiling now applies to WoW TEB blocks only.

Verified on the device: new 64-bit TEBs come from a high block, and no more threads are killed.

3. Child processes boot one at a time (ml1213)

A child process is a thread of the app that works through session-wide state during start-up. Two children created a few ms apart corrupted each other's start-up ([init-peb] ... MISMATCH), and both died. A mutex now covers a child's unix boot.

  • Verified on the device: the waits are logged, children boot in turn, and the mismatch is gone.
  • Built only: a child that dies through pthread_exit during init releases the lock through a pthread key destructor. A TSan harness on macOS confirms it.
  • Not covered: a child whose boot hangs without exiting still holds the lock. The cost of serialising boots was not measured on Steam or CEF helper bursts.

4. Pin wine and FEX

The pins move to the heads of:

Pins: wine f4bbccf9499, FEX c9fe957cf0d (rebased onto the FEX commit main pins now).

Known gaps (from review)

Merge order

The pins point at the head commits of willfaust/wine#24 and willfaust/FEX#12, which so far exist only in the spitefulowl forks. Merge those two first.

  • If they are merged with merge commits, the pinned commits are then in willfaust/wine and willfaust/FEX unchanged, and this PR can follow as is.
  • If they are squashed or rebased, I will move the pins to the resulting commits before this one is merged.

Rebuild (no binaries in this PR)

  • libntdll_unix.a (build/ntdll-unix/build.sh)
  • arm64ec-windows/ntdll.dll (build/wine-pe/build-ntdll.sh)
  • arm64ec-windows/xtajit64.dll (build/fex-arm64ec/build.sh)
  • aarch64-windows/xtajit.dll (build/fex-wow64/build.sh)

Checks

  • clang -fsyntax-only with build.sh's flags passes for the changed files of every commit (Xcode 27 / iOS SDK 27.0). At the head, they give the same warnings as on main.
  • tests/host gives the same results as on main. check-child-ntdll-alias, check-image-reload and check-iat-sync-owner pass. The tests that fail here also fail on main on this host (Linux headers, or submodules not checked out).
  • ConfigCatalog is regenerated and current.

🤖 Generated with Claude Code

spitefulowl and others added 4 commits October 4, 2026 13:08
A 64-bit PE whose fixed base is below 4 GB and whose relocations are
stripped (the default output of Delphi Win64 and older MinGW toolchains,
typically at 0x400000) cannot be mapped at its base on iOS, where nothing
is mapped below 4 GB. It is mapped high and its low addresses are served by
the ml938 sub-floor window (fault emulation, FEX's inline translation).
Such a program failed at every step; each fix below showed its own log line
in a device run on 2026-10-01 and the failure it targeted did not recur:

- ml1195, load: without a dynamic base the server hands out no map
  address, so the image was placed high with its header still naming the
  low base, and the loader refused it ("failed to create main module",
  c0000018). The header now names the real base; nothing is relocated.
  (run 6: "ml1195: ... ImageBase rewritten", the program started)
- ml1201, VirtualProtect/VirtualQuery on the image's own low addresses
  found no view (STATUS_INVALID_PARAMETER); a program patching 5 bytes of
  its own code died with a Delphi exception. A span inside one of this
  process's image windows is translated to the real mapping, as ml966 does
  for low allocations. (run 11: "ml1201: PROTECT ... -> real ...", status 0)
- ml1202, FlushInstructionCache on a low address ran `ic ivau` on unmapped
  memory and SEGV'd; it now flushes the real mapping of an image window or
  low allocation and skips other low addresses. (run 12)
- ml1203, unwind data: the PE-side RtlLookupFunctionTable (wine fork,
  pinned at the end of this branch) asks for the window of a low pc
  through the private NtQueryVirtualMemory class 1006 added here, so
  exceptions raised at low RIPs are no longer all unhandled. (run 13)
- ml1204, a relocatable DLL with the same preferred base as a running
  fixed-base image no longer re-points that image's window; the program had
  gone on running the DLL's bytes. (run 14)
- ml1205, windows per process: all pseudo-processes share one address
  space, and two processes with images at the same low base replaced each
  other's window and ran each other's code. Windows now carry the owner's
  PEB, lookups use the faulting or calling thread's PEB, each process's FEX
  gets only its own windows, they are released at process exit, and the
  table grows from 8 to 32 slots with free-slot reuse. Which process's FEX
  holds the callback is ios_jit_alias_pushback_peb, which upstream added
  for the same reason (willfaust#141); a push compares it after its single acquire
  load of the callback (19cbce3), and the catch-up loop of a registering
  emulator pushes that process's own windows and the shared ones. The
  window functions change arity (ios_subfloor_enum,
  ios_push_subfloor_window, ios_register_subfloor_image), and every caller
  declares them with a block-scope extern, which is why virtual_ios.c,
  signal_arm64_ios.c and server_ios.c change in this one commit, and the
  stubs of tests/host/check-child-ntdll-alias.py with them. (run 15: two
  windows with two owners, "released with its process")
- ml1206, a stripped fixed-base exe starts at the LOW address of its entry
  point: its absolute pointers are low, and code that combines a
  RIP-relative base with an absolute pointer (MinGW's pseudo-relocator)
  computed wrong targets. (run 15: "starts at its low entry")
- ml1208, the store emulator handles CASB/CASH in all acquire/release
  forms, which FEX emits for a byte or word `lock cmpxchg`; one on such an
  image's .bss was an unhandled encoding (c0000005). A misaligned halfword
  is emulated without atomicity (x86 keeps a split-lock word CAS atomic, so
  a concurrent writer could be lost there; rare). (run 17)

Kill switches (one cached getenv each, default on, catalog regenerated):
MADEIRA_SUBFLOOR_VM, MADEIRA_SUBFLOOR_SEH, MADEIRA_SUBFLOOR_LOWENTRY.

Fixes found in review, built only, not yet run on the device:
- ios_subfloor_image_translate skipped only other processes' windows and so
  also translated the shared KUSER_SHARED_DATA window (owner NULL):
  VirtualQuery(0x7ffe0000) answered with the real page and
  VirtualProtect(0x7ffe0000, PAGE_READWRITE) made it writable, which
  Windows refuses. It now skips that window (an image window registered
  without an owner is still translated, as before).
- ios_subfloor_window_held read the holder image's DOS/NT headers directly
  under virtual_mutex; an image that made its header page PAGE_NOACCESS
  would fault in unix code. The reads go through mach_vm_read_overwrite.
- The doc comment of ios_lowalloc_translate is back above that function.
- The ml1205 comment in ios_push_subfloor_window now states the known gap:
  a window registered after another process replaced the single FEX
  callback is not pushed to its owner's FEX (not a regression).

Tagged ml1195, ml1201, ml1202, ml1203, ml1204, ml1205, ml1206, ml1208.

Needs the wine and FEX branches of the same name (pinned in the last
commit): wine dlls/ntdll/unwind.c for ml1203, FEX's ml1207 invalidation of
both names of a window's code. clang -fsyntax-only with the build.sh flags:
no new warnings in the three files.

Rebuild: libntdll_unix.a (build/ntdll-unix/build.sh); nothing committed.

Signed-off-by: spitefulowl <spitefulowll@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
virtual_alloc_teb() reserves TEBs in blocks of 32 and passes
user_space_wow_limit as the zero-bits ceiling of a new block. That limit is
session-wide: the first 32-bit process sets it (0xffffffff), and from then
on a 64-bit thread's next TEB block was requested below 4 GB, where iOS
maps nothing. Once a 64-bit process had used up its first TEB block, every
new thread of it failed (the server saw EOF on its request fd and killed
the thread), so a 64-bit program started after any 32-bit one could not
create its 33rd thread.

The ceiling now applies to WoW TEB blocks only. In upstream Wine every
process has its own user_space_wow_limit, which stays 0 in a 64-bit one.

Tagged ml1210.

Status: verified on the device (2026-10-01, run 18 of that series: new
64-bit TEBs came from a high block, e.g. 0x1551c0000, and no more threads
were killed). clang -fsyntax-only with the build.sh flags: no new warnings.

Rebuild: libntdll_unix.a (build/ntdll-unix/build.sh); nothing committed.

Signed-off-by: spitefulowl <spitefulowll@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
On iOS a child process is a thread of the app: wine_ios_child_main runs on
the new process's own pthread and works through session-wide state (the
global PEB, main_image_info and its restore, the WoW window binding). Two
children created a few ms apart corrupted each other's start-up: a program
started three 32-bit children within 3 ms, the log showed
`[init-peb] thread_peb=0x800310000 global_peb=0xa00310000 <-- MISMATCH`, the
first child mapped its image into the second child's window and started
with that child's PEB and entry point, the second started at explorer's
entry, and both died in their 32-bit ntdll.

A mutex now covers a child's unix boot, from wine_ios_child_main's entry to
server_init_process_done, just before the jump into guest code. It is also
released by CHILD_BOOT_FAIL and after wine_ios_child_main returns or exits
(process_ios.c). Waits are logged (first 16).

Fix found in review, not yet run on the device: a booting child can also
end through pthread_exit (fatal_error and fatal_perror during init in
loader_ios.c and server_ios.c, abort_thread -> pthread_exit_wrapper in
thread_ios.c), which runs none of those unlocks. A child that failed that
way (e.g. a 32-bit child whose wow64 ntdll cannot be loaded) would have left
the lock held, and every later CreateProcess of the session, 64-bit ones
too, would have waited forever. Rather than adding an unlock to each of
those exits, the lock is paired with a pthread key whose value is set while
the thread holds it; the key's destructor runs on that thread as it exits
and releases the lock (it logs a line when it does). It reads only its
argument, so it does not depend on thread-local storage still being there.
A scratch harness on macOS (TSan) confirms the lock is free after a holder
pthread_exit()s, and that lock/unlock/unlock and a plain return leave it
free too.

Not covered (review): a child whose boot blocks without exiting still holds
the lock, and child boots are now serialised; neither was measured on Steam
or CEF helper bursts.

Tagged ml1213.

Status: the serialisation is verified on the device (2026-10-01, run 20 of
that series: "[Wine child] ml1213 ...: another child is booting, waiting",
children booted in turn, no `[init-peb] ... MISMATCH`, which run 19 had).
clang -fsyntax-only with the build.sh flags: no new warnings.

Rebuild: libntdll_unix.a (build/ntdll-unix/build.sh); nothing committed.

Signed-off-by: spitefulowl <spitefulowll@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Moves the submodule pointers to the spitefulowl forks' branches of the same
name:

wine f4bbccf9499  ntdll ARM64EC: unwind data for x64 code running at a
                  fixed base below 4 GB (ml1203): RtlLookupFunctionTable
                  asks the unix side (information class 1006, added in this
                  branch) for the window of a low pc. Verified on the
                  device.
FEX  9b263c00a80  WOW64/iOS: TerminateThread keeps the target's JIT state
                  (ml1200): a 32-bit TerminateThread on a running thread
                  freed its JIT state while it still ran, and the fault
                  storm ended the whole app. Verified on the device.
FEX  c9fe957cf0d  ARM64EC/iOS: invalidate sub-floor image code under both
                  of its names (ml1207). Verified on the device.

The pins point at the spitefulowl forks until the submodule changes are
merged upstream (the FEX ones are local to the iOS port and not meant for
FEX-Emu). The config catalog does not change with them.

Rebuild from these sources (never committed): arm64ec-windows/ntdll.dll
(build/wine-pe/build-ntdll.sh), arm64ec-windows/xtajit64.dll
(build/fex-arm64ec/build.sh) and aarch64-windows/xtajit.dll
(build/fex-wow64/build.sh).

Signed-off-by: spitefulowl <spitefulowll@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@spitefulowl
spitefulowl force-pushed the fixed-base-and-wow64-fixes branch from 18e3e6e to ab45990 Compare October 4, 2026 10:48
@willfaust
willfaust merged commit a695805 into willfaust:main Oct 4, 2026
willfaust added a commit that referenced this pull request Oct 4, 2026
…iling, one child boot at a time

Without its pin commit; the wine and FEX pins are set with the fork merges.

# Conflicts:
#	app/Madeira/ConfigCatalog.generated.swift
willfaust added a commit that referenced this pull request Oct 4, 2026
…y session

- One child boot at a time (#187) now has a kill switch,
  MADEIRA_CHILD_BOOT_SERIAL=0, and waits at most 10 s for another child's
  boot: one that hangs without exiting no longer stops every later
  CreateProcess of the session.
- FEX_MADEIRA_HOSTPROBE is published for every session, not only when the
  bundle has the i386 set: the ARM64EC FEX module now reads it too, which is
  what keeps 64-bit games on A12/A13 from 0xC000001D.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
willfaust added a commit that referenced this pull request Oct 4, 2026
Wine and FEX runtime fixes, DXMT texture and memory fixes, audio, the swap
tier and library launch options (#178-#187 and willfaust/wine#22-#24,
willfaust/FEX#11-#12, willfaust/dxmt#16-#17).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
spitefulowl added a commit to spitefulowl/Madeira that referenced this pull request Oct 4, 2026
Integration branch of the spitefulowl fork, never for upstream: the
topic branches not merged upstream yet, on bbbf8d0 (upstream main
after it merged the pull requests willfaust#178-willfaust#184, willfaust#186 and willfaust#187):

- controllers-and-keyboard
- display-resolution
- misc-app-fixes-later
- build-and-bundle

.gitmodules points at the spitefulowl forks (branch staging) and the
gitlinks at their staging heads: wine 47fa28e18b8, FEX 7d3b59477a8;
dxmt and madeira-dock stay at upstream main's pins (dxmt has no
unmerged topic branch left). dxmt-correctness-memory-later is empty
now: dxmt#16 carries its pin, and ml1222 is parked in the dxmt fork
branch experiment/dxmt-shader-ir.

Signed-off-by: spitefulowl <spitefulowll@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
c-gow added a commit to c-gow/Madeira that referenced this pull request Oct 6, 2026
172 upstream commits: Wine builtins (sspicli and 17 more), audio pacing and
silent GetBuffer (willfaust#180), TEB ceiling and one child boot at a time (willfaust#187),
CPU fixes, the GPU meter, controller output, saves and shortcuts, native
D3D12 work, and new wine, FEX and DXMT pins.

Conflicts:
- arm64ec-windows/gdiplus.dll, mlang.dll: upstream's builds (at the new
  wine pin) replace the fork's MADEIRA port copies.
- Madeira-Bridging-Header.h: both declarations kept (madeira_frame_count
  and the GPU meter).
- Library.swift (performance overlay): upstream's GPU-meter version, with
  the fork's madeira_frame_count() so OpenGL frames still count.
- process_ios.c: the fork's LuaJIT swap restore and upstream's conhost
  refusal and Steam session log, side by side.
- sysparams_ios.c: upstream's wording of the same virtual-monitor name fix
  the fork had ported.
- project.pbxproj: the fork's OpenGL and MadeiraConfig entries and
  upstream's PadOutput entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Connor Gow <cmgow08@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants