Skip to content

docs(install): add consolidated GPU backends & build options guide (#2314)#2315

Open
Anai-Guo wants to merge 1 commit into
abetlen:mainfrom
Anai-Guo:docs/gpu-backend-guide
Open

docs(install): add consolidated GPU backends & build options guide (#2314)#2315
Anai-Guo wants to merge 1 commit into
abetlen:mainfrom
Anai-Guo:docs/gpu-backend-guide

Conversation

@Anai-Guo

Copy link
Copy Markdown
Contributor

Summary

Adds a single, consolidated GPU Backends & Build Options page under Installation Guides, addressing #2314.

Today the build/GPU information is spread across the README and many issues, so users on newer GPUs or uncommon CUDA versions repeatedly hit the same build failures. This page gathers it into one place:

  • CMAKE_ARGS mapping table — one row per backend (CPU/OpenBLAS, CUDA, Metal, HIP/ROCm, Vulkan, SYCL, RPC) with the exact flag and requirements.
  • Pre-built wheel matrix — every published --extra-index-url tag (cu118cu132, metal, rocm72, hip-radeon, vulkan) with the compute-capability notes from the README.
  • Verifying GPU offload is active — uses the existing bindings llama_cpp.llama_supports_gpu_offload() and llama_cpp.llama_print_system_info(), plus the n_gpu_layers=-1, verbose=True log check, so users can confirm a build isn't silently CPU-only.
  • Platform notes linking the existing macOS (Metal) guide and the Windows w64devkit workaround.

All flags, wheel tags, and compute-capability ranges are taken verbatim from the current README's Supported Backends section — no new claims. The page is wired into mkdocs.yml nav next to the macOS guide.

Test plan

  • mkdocs build renders the new install/gpu.md page and nav entry without warnings
  • Verify the llama_supports_gpu_offload() / llama_print_system_info() snippet matches the public API

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant