bklibcvenv: fix /proc/self/exe via load-time PT_INTERP rewrite#357
Merged
Conversation
Replace the explicit-ld.so shell wrapper with a static stub that re-points each sealed binary's ELF interpreter at the bundled loader, fixing self-locating tools while keeping bottles relocatable. The old wrapper ran the bundled libc by invoking the loader explicitly (`ld.so --library-path … ./real`). That works for ordinary tools but breaks any program that finds itself via /proc/self/exe: the kernel exec's the *loader*, so /proc/self/exe points at ld.so, not the program. clang re-execing itself as `clang -cc1` is the canonical victim (#346) — it computed its InstalledDir as the glibc dir and could never find cc1. The relocatability tax is what makes this hard. PT_INTERP must be an absolute path to a real ELF loader: the kernel reads it before ld.so exists, so it never expands $ORIGIN and won't accept a relative path or a #! script. An absolute path can't be baked at build time either — the install prefix is unknown, and pkgm can move a bottle after install. So fix it up at load time, mirroring bkpyvenv. A tiny stub (bkinterp, built from share/brewkit/bkinterp.zig) sits in front of each binary; on every run it locates itself via /proc/self/exe and, only if the stored PT_INTERP is stale, re-pokes it in place to <prefix>/lib/<libc>/<ldso>, then exec's the real binary directly so /proc/self/exe is correct. The check is a ~6µs read; the poke fires once per install (or after a move). The stub is statically linked against musl, so it has no PT_INTERP or libc of its own — the one thing guaranteed to run before any libc fixup. Layout after seal: real binaries move to libexec/ (a direct child of prefix, same depth as bin/, so $ORIGIN-relative RPATHs — including cross-package $ORIGIN/../../../../other-pkg — stay valid); each bin/ entry becomes a symlink to libexec/bkinterp; alias symlinks (clang -> clang-22) are mirrored into libexec/ for the stub's argv[0] lookup. Constraints encoded here, each learned by hitting it: * The reserved PT_INTERP slot is sized to exactly PATH_MAX. The kernel rejects (ENOEXEC) any binary whose PT_INTERP p_filesz exceeds PATH_MAX; a runtime install path can't exceed PATH_MAX either, so a PATH_MAX slot always fits. * The placeholder ends with the real loader basename, which the stub recovers via basename(current-interp) to glob lib/*/<ldso>. This also makes it arch-correct per binary (ld-linux-x86-64.so.2, ld-linux-aarch64.so.1, …). * The bundled-libc rpath is forced to DT_RPATH, not DT_RUNPATH: RUNPATH applies only to a binary's direct NEEDED libs, so a transitively-loaded lib (libcurl pulling libpthread.so.0) would skip the bundle. DT_RPATH applies process-wide and is searched before LD_LIBRARY_PATH, so the host can't inject an older libc. * The stub closes the target fd before execve, or the kernel returns ETXTBSY (it opens read-write to poke). bkinterp.zig is built with zig pinned exactly (ziglang.org=0.15.2, via the shebang) because the source tracks a specific stdlib API; bump the pin and the source together. zig bundles its own musl sysroot, so the build needs nothing on the host but the zig package. Proven end-to-end on llvm.org/mingw-w64 + bundled glibc-2.43: sealed clang runs against the bundle, InstalledDir resolves to the real binary in libexec/ (not the loader), cc1 runs, and a full cross-compile produces a PE32+ executable. Per-invocation stub overhead is ~0.1ms. Replaces share/brewkit/libcvenv-wrapper.sh (removed). Refs: #344 (RFC), #346 (the /proc/self/exe wall), pkgxdev/pantry@5354c73f (the inline-in-glibc-recipe origin).
There was a problem hiding this comment.
Pull request overview
This PR updates bklibcvenv to stop using an explicit ld.so --library-path ... wrapper (which breaks /proc/self/exe self-location) and instead introduces a small front-stub (bkinterp) that rewrites each sealed binary’s PT_INTERP at load time to the bundled loader, then execves the real binary so /proc/self/exe resolves correctly.
Changes:
- Remove the POSIX shell wrapper template (
share/brewkit/libcvenv-wrapper.sh). - Add
share/brewkit/bkinterp.zig, a stub that locates the install prefix at runtime, fixesPT_INTERPif stale, and execs the real tool. - Update
libexec/bklibcvenvto build/installbkinterp, move sealed binaries intolibexec/, reserve aPATH_MAX-sizedPT_INTERPslot, and symlinkbin/*to the stub.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| share/brewkit/libcvenv-wrapper.sh | Removes the legacy shell-wrapper approach. |
| share/brewkit/bkinterp.zig | Adds a runtime PT_INTERP fixup + exec stub to preserve /proc/self/exe semantics. |
| libexec/bklibcvenv | Switches sealing to the bkinterp model; reserves PT_INTERP space and rewrites rpath/interpreter during seal. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+30
to
+47
| var hdr: [64]u8 = undefined; | ||
| _ = f.preadAll(&hdr, 0) catch return 126; | ||
| const phoff = std.mem.readInt(u64, hdr[0x20..0x28], .little); | ||
| const phentsize = std.mem.readInt(u16, hdr[0x36..0x38], .little); | ||
| const phnum = std.mem.readInt(u16, hdr[0x38..0x3a], .little); | ||
|
|
||
| var i: u16 = 0; | ||
| var ioff: u64 = 0; | ||
| var isz: u64 = 0; | ||
| while (i < phnum) : (i += 1) { | ||
| var ph: [56]u8 = undefined; | ||
| _ = f.preadAll(&ph, phoff + @as(u64, i) * phentsize) catch return 126; | ||
| if (std.mem.readInt(u32, ph[0..4], .little) == 3) { // PT_INTERP | ||
| ioff = std.mem.readInt(u64, ph[8..16], .little); | ||
| isz = std.mem.readInt(u64, ph[0x20..0x28], .little); | ||
| break; | ||
| } | ||
| } |
Comment on lines
+69
to
+76
| if (want) |w| { | ||
| if (!std.mem.eql(u8, cur, w) and w.len + 1 <= isz) { | ||
| const buf = a.alloc(u8, isz) catch return 127; | ||
| @memset(buf, 0); | ||
| @memcpy(buf[0..w.len], w); | ||
| _ = f.pwriteAll(buf, ioff) catch {}; | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace the explicit-ld.so shell wrapper with a static stub that re-points each sealed binary's ELF interpreter at the bundled loader, fixing self-locating tools while keeping bottles relocatable.
The old wrapper ran the bundled libc by invoking the loader explicitly (
ld.so --library-path … ./real). That works for ordinary tools but breaks any program that finds itself via /proc/self/exe: the kernel exec's the loader, so /proc/self/exe points at ld.so, not the program. clang re-execing itself asclang -cc1is the canonical victim (#346) — it computed its InstalledDir as the glibc dir and could never find cc1.The relocatability tax is what makes this hard. PT_INTERP must be an absolute path to a real ELF loader: the kernel reads it before ld.so exists, so it never expands $ORIGIN and won't accept a relative path or a #! script. An absolute path can't be baked at build time either — the install prefix is unknown, and pkgm can move a bottle after install.
So fix it up at load time, mirroring bkpyvenv. A tiny stub (bkinterp, built from share/brewkit/bkinterp.zig) sits in front of each binary; on every run it locates itself via /proc/self/exe and, only if the stored PT_INTERP is stale, re-pokes it in place to /lib//, then exec's the real binary directly so /proc/self/exe is correct. The check is a ~6µs read; the poke fires once per install (or after a move). The stub is statically linked against musl, so it has no PT_INTERP or libc of its own — the one thing guaranteed to run before any libc fixup.
Layout after seal: real binaries move to libexec/ (a direct child of prefix, same depth as bin/, so $ORIGIN-relative RPATHs — including cross-package $ORIGIN/../../../../other-pkg — stay valid); each bin/ entry becomes a symlink to libexec/bkinterp; alias symlinks (clang -> clang-22) are mirrored into libexec/ for the stub's argv[0] lookup.
Constraints encoded here, each learned by hitting it:
bkinterp.zig is built with zig pinned exactly (ziglang.org=0.15.2, via the shebang) because the source tracks a specific stdlib API; bump the pin and the source together. zig bundles its own musl sysroot, so the build needs nothing on the host but the zig package.
Proven end-to-end on llvm.org/mingw-w64 + bundled glibc-2.43: sealed clang runs against the bundle, InstalledDir resolves to the real binary in libexec/ (not the loader), cc1 runs, and a full cross-compile produces a PE32+ executable. Per-invocation stub overhead is ~0.1ms.
Replaces share/brewkit/libcvenv-wrapper.sh (removed).
Refs: #344 (RFC), #346 (the /proc/self/exe wall),
pkgxdev/pantry@5354c73f (the inline-in-glibc-recipe origin).