Hundreds of models & providers. One command to find what runs on your hardware.

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

PMetal: high-performance Apple Silicon framework for local LLM inference, LoRA/QLoRA fine-tuning, serving, quantization, and MLX/Metal acceleration.

The Easiest Rust Interface for Local LLMs and an Interface for Deterministic Signals from Probabilistic LLM Vibes

Camelid: a Rust-native local inference backend with evidence-gated model compatibility.

The Natural Language Shell integrates OpenAI's GPTs, Anthropic's Claude, or local GGUF-formatted LLMs directly into the terminal experience, allowing operators to describe their tasks in either POSIX commands or fluent human language

A utility to inspect, validate, sign and verify machine learning model files.

A fast terminal native app (TUI) and CLI with init wizard for launching local LLMs with zero overhead

llm_utils: Basic LLM tools, best practices, and minimal abstraction.

AI inference library for mobile devices

A small utility library for parsing GGUF file info

An open-source, high-performance local AI inference engine.

a GGUF file parser

Inspect LLM's logprobs and perplexity over a piece of text, or compare two LLMs (like a git diff)

Apple Neural Engine (ANE) LLM inference engine — reverse-engineered private APIs, Metal GPU shaders, hybrid ANE+GPU+CPU on Apple Silicon. 32 tok/s matching llama.cpp, 3.6 TFLOPS fused ANE mega-kernels.

Pure Rust tokenizer for GGUF models - llama.cpp compatible

The Private Agent OS — search files, run AI agents, connect to 10,000+ tools via the complete protocol stack (MCP, AG-UI, A2UI, A2A). Zero cloud. Zero telemetry. On-device inference.

handle gguf files

Agentic AI server in Rust. Multi-provider LLM routing, tool calling, RAG, MCP, multi-tenant workflows.

LLM inference engine for AMD RDNA4 — Rust + Vulkan compute shaders, gguf & native FP8.