scripts/build.sh / make fail under the Nix flake dev shell on macOS; cross-arch builds unsupported

[kriswill]

Version

codebase-memory-mcp dev (built from source, main @ 7824e50)

Platform

macOS (Apple Silicon)

Install channel

Built from source

Binary variant

standard

What happened, and what did you expect?

In the Nix flake dev shell (nix develop) on macOS, scripts/build.sh aborts before compiling anything: it prints WARNING: clang (…) is x86_64 only — cannot build native arm64 and exits 1 — even though the Nix clang targets arm64-apple-darwin natively and builds fine. I expected scripts/build.sh (the "Build from Source" flow in CONTRIBUTING) to produce build/c/codebase-memory-mcp. The same false negative hits test.sh/lint.sh/repro.sh (shared scripts/env.sh).

Separately, a cross-architecture build (scripts/build.sh --arch x86_64) does not produce an x86_64 binary, and once the compiler check is bypassed the build fails to compile with unknown type name 'git_allocator'.

nix build works only because the flake's buildPhase already bypasses build.sh and calls make directly (with a comment acknowledging the file-check failure), so this is specific to the documented developer workflow, not the packaged build.

Reproduction

Public repo: this repository (DeusData/codebase-memory-mcp @ main). Host: macOS, Apple Silicon. Toolchain: nixpkgs clang-wrapper 21.x in the flake dev shell (CC=clang CXX=clang++); clang --version → Target: arm64-apple-darwin.

1. nix develop
2. scripts/build.sh
   • Actual: aborts with WARNING: clang (…) is x86_64 only — cannot build native arm64, exit 1.
   • Expected: builds build/c/codebase-memory-mcp (native arm64).
3. Cross-arch: scripts/build.sh --arch x86_64
   • Actual: no working x86_64 binary (compiler check, then link failure — see Logs).
4. Bypass the check the way the flake does — make -f Makefile.cbm cbm:
   • Actual: compile error internal/cbm/cbm.c:347: unknown type name 'git_allocator' (only when libgit2 is on the pkg-config path, i.e. inside the Nix shell).

Logs

# 1) scripts/build.sh
=== build.sh: os=darwin arch=arm64 cores=10 cc=clang cxx=clang++ ===
  ui=false version=dev
WARNING: clang (/nix/store/…-clang-wrapper-21.x/bin/clang) is x86_64 only — cannot build native arm64
  Install arm64 GCC: arch -arm64 /opt/homebrew/bin/brew install gcc
  Or override: scripts/test.sh --arch x86_64
# exit 1

# 2) cross-arch link (after the arch check is satisfied)
ld: warning: ignoring file …/zlib-1.3.2/lib/libz.dylib, building for macOS-x86_64 but … arm64
ld: warning: ignoring file …/libgit2-1.9.4/lib/libgit2.dylib, building for macOS-x86_64 but … arm64
Undefined symbols for architecture x86_64
ld: symbol(s) not found for architecture x86_64

# 3) make -f Makefile.cbm cbm
internal/cbm/cbm.c:347:12: error: unknown type name 'git_allocator'
  347 |     static git_allocator cbm_git_alloc = {
      |            ^

Diagnostics trajectory (memory / performance / leak issues)

N/A — build/toolchain bug, not a memory or performance issue.

Project scale (if relevant)

N/A

Confirmations

• I searched existing issues and this is not a duplicate.
• My reproduction uses shareable code (a public OSS repository — this repo itself).

[kriswill]

Root cause

1. Compiler check is a false negative under Nix. verify_compiler() in scripts/env.sh runs file on the clang path and greps the output for arm64/universal. Nix's clang is a bash wrapper script, so file reports … ASCII text executable — matching neither — and the check concludes "x86_64 only" and exits before any compile, even though the wrapper drives a clang whose default target is arm64-apple-darwin. (flake.nix's buildPhase already documents this exact failure and works around it by calling make directly.)

2. No explicit -arch, so cross-builds can't work. Architecture selection relies on ARCH_PREFIX="arch -<arch>" make; Makefile.cbm carries no -arch flags. Nix's clang has a fixed target triple, so arch -x86_64 make still emits arm64. And even with correct targeting, the dev shell ships host-arch-only zlib/libgit2, so an x86_64 link can't resolve symbols.

3. Latent libgit2 include bug. cbm.c uses git_allocator but includes only <git2.h>, which does not pull in <git2/sys/alloc.h> where the type is declared (it's a sys/ header). It's compiled out on a bare macOS toolchain (HAVE_LIBGIT2 undefined), but the Nix shell provides libgit2, so the block compiles and fails. (Arguably a separable fix.)

Proposed fix (build-system change — requesting maintainer sign-off per CONTRIBUTING)

1. Replace the file-based arch check with a capability probe: compile + link a trivial program with -arch <target> and trust the result — correct for wrapper scripts (Nix, ccache) and cross-compilers.
2. Make arch targeting explicit and toolchain-agnostic: export ARCHFLAGS="-arch <arch>" from env.sh and fold it into CC/CXX in Makefile.cbm via override (so it reaches every compile and link, including the vendored objects, and survives a command-line CC=); drop the now-redundant arch -<arch> prefix from build/test/lint/repro.
3. Add an optional cross flake dev shell that puts the target-arch zlib + libgit2 on the link path, so nix develop .#cross + scripts/build.sh --arch x86_64 links a working x86_64 binary (running it on Apple Silicon still needs Rosetta).
4. Add #include <git2/sys/alloc.h> in cbm.c.

A candidate implementation is ready and can be opened as a PR referencing this issue once the approach is approved. Happy to split the git_allocator include fix and/or the nixpkgs lock bump into separate PRs if preferred.

[DeusData]

Thanks for the thorough root-cause breakdown. I’ve triaged this under build-from-source / developer UX.

The native Nix clang-wrapper false negative is clear: a file check on the wrapper is the wrong signal. A capability probe is the right shape. For implementation, I’d prefer keeping the git2/sys/alloc.h include fix separate from the broader arch-probe / cross-arch shell work if practical, because the include bug is a narrow build fix while cross-arch support has dependency-architecture constraints that need dedicated validation.

[kriswill]

@DeusData as requested, I've split-out each part of the fixes for this issue. when stack-merged, the entire macos build chain will properly compile and cross-arch compile. all lints and tests work locally under the flake's nix dev shell.