Support Windows and Linux x86_64 binary builds and release artifacts

[inureyes]

Summary

Extend mlxcel's release build matrix to cover Linux x86_64 and Windows in addition to the current macOS aarch64 + Linux aarch64 (CUDA) targets. This aligns the distribution surface with the Backend.AI:GO desktop application and makes mlxcel deployable to customer sites that standardize on Linux x86_64 (the dominant CUDA host architecture in production).

Background

Today's releases ship two artifact families:

• mlxcel-macos-aarch64.zip — Apple Silicon
• mlxcel-linux-aarch64-cuda13-{gb10,gh200} — Linux aarch64 + CUDA (Grace Blackwell / Grace Hopper)

The README prerequisites already claim "Linux (aarch64 or x86_64)" support, but no published binary exists for x86_64 today. Windows is not built at all, although the runtime distribution layout (MLX upstream's mlx/backend/cuda/delayload.cpp delay-load mechanism + NVIDIA DLL bundling, ~1.0–1.3 GB total) has been worked out.

[Update 2026-06-25: the Linux x86_64 + CUDA artifact now ships (since v0.3.0, PR #262). Windows remains unbuilt and is tracked in detail by #58.]

Two motivations:

1. Customer fit — Most enterprise CUDA hosts (RTX, A100, L40, H100/H200 outside of GH200 Grace systems) are Linux x86_64. Without a published x86_64 binary, those sites cannot adopt mlxcel without building from source.
2. Backend.AI:GO parity — The Backend.AI:GO desktop application supports macOS + Windows + Linux x86_64. mlxcel should match so the same runtime can serve any Backend.AI:GO target.

Proposed Solution

Split the work into three deliverables.

A. Linux x86_64 + CUDA build

Status (2026-06-25): Done (PR #262). The mlxcel-linux-x86_64-cuda13.zip CLI + server artifacts ship from v0.3.0 onward (present in latest v0.3.3). CI runs a 10-token GPU smoke test on an RTX 5060 (sm_120) self-hosted runner. The CUDA arch list is 80;86;89;90a;100;120 (Ampere through Blackwell), broader than the proposal below.

• Add a release artifact mlxcel-linux-x86_64-cuda13.{tar.gz|zip} produced from an x86_64 + NVIDIA host (build-only is acceptable for the first iteration; smoke test on a real GPU is preferred).
• Decide on the target CUDA architectures — SM 80/86/89/90 covers Ampere through Hopper; the existing CUDA arch matrix in README.md already documents non-Hopper quantization limitations.
• Verify src/lib/mlxcel-core/build.rs and the bundled MLX C++ build produce equivalent output on x86_64 — expected to be mechanical because Linux paths already differentiate from macOS via #[cfg(target_os = "linux")].

B. Windows + CUDA build

Status (2026-06-25): Open, tracked in detail by #58 (native Windows + CUDA feasibility spike + porting plan). The bullets below map to #58's Layer 2 (build.rs Windows branches, static-link delay-load replication) and Phases 2-5. Upstream picture per #58: MLX now builds and CI-tests Windows CPU, and the CUDA backend ships Windows delay-load plumbing (including the static-library case), but there is no Windows+CUDA CI, so it is plumbed yet unproven.

• Extend src/lib/mlxcel-core/build.rs with #[cfg(target_os = "windows")] branches:
  • MSVC toolchain detection (vs. GCC/Clang on Linux)
  • CUDA_PATH env var (Windows convention) in addition to CUDA_HOME
  • Link against cudart.lib and friends; static-vs-dynamic linkage decision mirrors MLX's Windows build
• Add Windows handling in the bundled MLX C++ build for OpenBLAS FetchContent and the delay-load DLL compile definitions (MLX_CUDA_BIN_DIR, MLX_CUDNN_BIN_DIR).
• Package the Windows artifact (mlxcel.exe + nvidia/cublas/bin/... etc.) per the runtime distribution layout. Bundle size is expected to be ~1.0–1.3 GB because of cuBLAS + NVRTC + cuDNN — fits within the 2 GB-per-asset GitHub Release limit.
• Code signing — investigate signtool.exe / Azure Code Signing, or ship unsigned for the first release with a documented SmartScreen warning.
• Confirm all crates compile on x86_64-pc-windows-msvc: tokenizers, safetensors, axum, cxx, sentencepiece-sys, and the multimodal stack (ffmpeg / video frame extraction may need a Windows-friendly variant).

C. Documentation + README updates

Status (2026-06-25): Partial. CUDA arch coverage and the Linux aarch64/x86_64 differentiation are documented in docs/installation.md (Supported-platforms table :13-18, release section :120-122, arch selection :97-122). The Windows build guide and the README Windows enumeration remain pending #58.

• Update README.md prerequisites to enumerate Linux aarch64, Linux x86_64, and Windows distinctly rather than the current "Linux (aarch64 or x86_64)" shorthand.
• Add a docs/windows-build-guide.md for developers building from source on Windows.
• Update the CUDA architecture compatibility table to reflect the broader x86_64 GPU coverage (RTX 30/40/50 series, A100, L40, H100).

Implementation Notes

• Code surface that already touches Windows
  • src/distributed/rdma_capabilities.rs already has cfg!(target_os = "windows") branches. The rest of src/distributed/ should be audited for Windows-specific socket / transport quirks before claiming pipeline parallelism works on Windows.
  • No other target_os = "windows" gates exist today. (Confirmed 2026-06-25; feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58's Layer 3 audit measures ~79 Unix-API sites total, 39 in src/distributed/, with all hard primitives io_uring/kqueue/RDMA inside the distributed module.)
• MLX upstream Windows status — MLX has the delay-load CUDA mechanism (mlx/backend/cuda/delayload.cpp), but full Windows-only correctness may require additional upstream patches. [Update 2026-06-25: see feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58; upstream MLX now CI-tests Windows CPU and ships CUDA delay-load plumbing including the static-library case, but has no Windows+CUDA CI yet.]
• Self-hosted runner needs — x86_64 + NVIDIA Linux runner (RTX class is enough for build, but actual smoke testing benefits from Hopper or Blackwell). Windows + NVIDIA runner ideally has the same.
• Build-only vs. test-on-platform — For the first iteration, building plus a smoke test (model load + 10-token generate) on each platform is sufficient. Full benchmark + parity runs can land in a follow-up.
• Rust target triples
  • Linux x86_64: x86_64-unknown-linux-gnu
  • Windows: x86_64-pc-windows-msvc

The release pipeline lives in the development repository and not this mirror; this issue tracks the user-visible deliverable. Implementation discussion is welcome here.

[Update 2026-06-25: the release workflow .github/workflows/release.yml runs in this repo and now produces the macOS, Linux aarch64, and Linux x86_64 CUDA artifacts (PR #262).]

Acceptance Criteria

• Release publishes mlxcel-linux-x86_64-cuda13.{tar.gz|zip} (feat(ci): add Linux x86_64 + aarch64 CUDA release builds with bundled CCCL #262; ships from v0.3.0, present in v0.3.3 as .zip for both mlxcel and mlxcel-server, each with .sha256)
• Release publishes mlxcel-windows-x86_64-cuda13.zip with bundled CUDA runtime DLLs
  • Tracked by feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58; not yet built (no Windows job in .github/workflows/release.yml).
• Smoke test (mlxcel generate with a 0.6B 4-bit model, 10 tokens) succeeds on Linux x86_64 + NVIDIA GPU (feat(ci): add Linux x86_64 + aarch64 CUDA release builds with bundled CCCL #262; CI step "Smoke test (10-token generate on GPU)" downloads mlx-community/Qwen3-0.6B-4bit and runs mlxcel generate -n 10 on an RTX 5060 / sm_120 self-hosted runner)
• Smoke test succeeds on Windows + NVIDIA GPU
  • Blocked on Deliverable B / feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58.
• README.md prerequisites section enumerates Linux aarch64, Linux x86_64, and Windows distinctly
  • Done: docs/installation.md differentiates Linux aarch64 (GH200/GB200/GB10) and x86_64 (Ampere through Blackwell) in the Supported-platforms table (:13-18) and release section (:120-122); README.md:26 names Linux x86_64 + aarch64.
  • Remaining: Windows is still listed only as "not documented here"; no distinct Windows prerequisites yet (pending feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58).
• docs/windows-build-guide.md walks through a developer-side Windows build
  • Not created; depends on feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58 producing a working Windows build.
• CUDA architecture compatibility table reflects x86_64 GPU coverage (Ampere, Ada, Hopper, Blackwell) (feat(ci): add Linux x86_64 + aarch64 CUDA release builds with bundled CCCL #262; docs/installation.md:97-122 documents the x86_64 list 80;86;89;90a;100;120)
• Code-signing strategy decided for the Windows binary (signed / unsigned with documented warning / deferred)
  • Pending feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58 packaging; macOS uses rcodesign today, Windows signtool / Azure Code Signing undecided.

---

Original Suggestion

Let's match the compatibility matrix with Backend.AI:GO desktop application, and also make it available to our customer sites which often use Linux x86-64 environments.

---

Refresh log

• 2026-06-25: Refreshed against current code and releases (issue created 2026-05-18).
  • Deliverable A (Linux x86_64 + CUDA) is done: PR feat(ci): add Linux x86_64 + aarch64 CUDA release builds with bundled CCCL #262 added the build-linux-x86_64-cuda release job; mlxcel-linux-x86_64-cuda13.zip (CLI + server) ships from v0.3.0 through v0.3.3, and CI runs a 10-token GPU smoke test (mlx-community/Qwen3-0.6B-4bit) on an RTX 5060 (sm_120) runner. Checked off acceptance criteria 1, 3, and 7.
  • Deliverable B (Windows + CUDA) remains open and is now tracked in detail by feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58 (feasibility spike + porting plan); acceptance criteria 2, 4, 6, and 8 stay unchecked with pointers to feat: Native Windows + CUDA build feasibility spike and porting plan (x86_64-pc-windows-msvc) #58.
  • Deliverable C is partial: CUDA arch coverage and Linux aarch64/x86_64 differentiation are documented in docs/installation.md; the Windows build guide and the README Windows enumeration are still pending.
  • Recorded the CUDA arch decision: x86_64 targets 80;86;89;90a;100;120 (Ampere through Blackwell), broader than the originally proposed SM 80/86/89/90.
  • Marked stale claims with dated updates rather than deleting them: the x86_64 binary is now published, and the release workflow (.github/workflows/release.yml) runs in this repo.