mgorshkov · mgorshkov · Sep 18, 2026
diff --git a/.vscodeignore b/.vscodeignore
@@ -8,6 +8,7 @@ tsconfig.json
 **/*.cpp
 **/*.hpp
 **/CMakeLists.txt
+tests/**
 build/**
 !build/Release/*.node
 !.vscode/launch.json

diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -6,7 +6,7 @@ cmake_minimum_required(VERSION 3.18.0)
 # (https://github.com/mgorshkov/tinycoder-inference) and is consumed here via
 # FetchContent as the tinycoder_core static library. This tree keeps only the
 # N-API bridge (src/cpp/bridge) and the TypeScript extension sources.
-project(tinycoder VERSION 0.4.0 DESCRIPTION "High-Performant Local AI coding agent" LANGUAGES CXX)
+project(tinycoder VERSION 0.5.0 DESCRIPTION "High-Performant Local AI coding agent" LANGUAGES CXX)
 enable_testing()
 
 # ---- Options ----

diff --git a/README.md b/README.md
@@ -119,56 +119,65 @@ All weights are stored in their **native quantized format** in memory and dequan
 - **Generate code** — generate from comments/descriptions
 - **Streaming output** — see tokens as they're generated
 - **Status bar** — model status indicator with periodic updates
+- **Autonomous coding agent** — ReAct (Reasoning + Acting) harness that reads/writes
+  files, runs terminal commands and searches the codebase through structured
+  JSON tool calls (`TinyCoder: Run Agent`)
+
+## Getting a Model
+
+TinyCoder loads **GGUF v3** files from the model families below. Every model in the
+table has been load-verified and benchmarked on the project's reference hardware
+(i7-4790K + RTX 2080 Ti 11 GB); "GPU mode" is what the engine automatically
+selects on that card.
+
+| Model | Quantization | Size (approx) | GPU mode on 11 GB | Notes |
+|-------|-------------|---------------|-------------------|-------|
+| `Qwen2.5-Coder-1.5B` | Q2_K | ~0.7 GB | full offload | Lightweight coding, 132 tg tok/s |
+| `Qwen2.5-Coder-1.5B` | IQ3_XXS (imat) | ~0.6 GB | full offload | Recommended balanced |
+| `Qwen2.5-Coder-7B` | IQ2_S | ~2.4 GB | full offload | Ultra-compact 7B |
+| `Qwen2.5-Coder-7B` | IQ3_XXS (imat) | ~2.9 GB | full offload | Recommended 7B |
+| `Gemma 4 (26B-A4B)` | Q4_K_XL (qat-UD) | ~14 GB | full offload (fits) | Instruction-tuned MoE |
+| `Qwen3.6-35B-A3B` | UD-IQ1_M | ~9.4 GB | full offload | Ultra-compact 35B MoE, 13 tg tok/s |
+| `Qwen3.6-35B-A3B` | UD-IQ2_M | ~10.7 GB | hybrid (experts CPU) | Ultra-compact MoE |
+| `Qwen3.6-35B-A3B` | UD-Q4_K_M | ~20.6 GB | hybrid (experts CPU) | **9.7 tg tok/s decode — recommended** |
+| `Qwen3.6-27B` | UD-Q4_K_XL | ~16.7 GB | partial offload (34/65) | Dense qwen35 |
+| `Qwen3.8-27B` | UD-Q4_K_M | ~15.3 GB | partial offload (37/65) | Dense qwen35 with IQ4_NL/Q8_K kernels |
+| `Ornith-1.5-35B-A3B` | Q8_0 | ~34.4 GB | hybrid (experts CPU) | Full-precision MoE |
+| `Ornith-1.5-35B` | Q4_K_M | ~20.2 GB | hybrid (experts CPU) | 41-layer qwen35moe variant |
+
+Download a `.gguf` file and point TinyCoder at it:
+- **In VS Code**: set `tinycoder.modelPath` in settings, or use
+  *TinyCoder: Open Agent Options* → Model → Browse, then *Load Model*.
+- **CLI / API**: set `TINYCODER_MODEL_PATH` or pass the path to `Model::load()`.
+
+> The Gemma 4 *coding* files (`gemma4-coding-*`) hit a loader gap ("missing attention
+> weights for layer 5" — the file names its attention tensors differently), and the
+> `gemma-2-*` files use the `gemma2` architecture, which is not yet supported; both
+> print a clear load error rather than producing wrong output.
 
 ## Project Structure
 
 ```
 tinycoder/
-├── include/                        # C++ headers
-│   ├── ModelConfig.hpp             # Model & inference configuration
-│   ├── GGUFLoader.hpp              # GGUF v3 file format loader
-│   ├── GGMLDequantize.hpp          # Multi-type dequantization (Q5_K, IQ3_XXS, etc.)
-│   ├── IQ3XXS.hpp                  # Legacy IQ3_XXS block-level dequantization
-│   ├── LMHead.hpp                  # LM head computation (CPU OpenMP path)
-│   ├── LMHeadCUDA.hpp              # LM head CUDA (cublasSgemv) interface
-│   ├── Model.hpp                   # Transformer model (forward, generate, KV cache)
-│   ├── SIMDMatMulVec.hpp           # SIMD-accelerated dot product & accumulate
-│   └── Tokenizer.hpp               # BPE tokenizer (Qwen2.5)
+├── CMakeLists.txt                  # Builds the N-API addon; fetches tinycoder-inference
 ├── src/
-│   ├── cpp/                        # C++ source
-│   │   ├── core/                   # Engine core (compiled once into tinycoder_core)
-│   │   │   ├── ChatTemplateRenderer.cpp  # Chat template formatting
-│   │   │   ├── GGUFLoader.cpp            # GGUF v3 reader (metadata + tensor data)
-│   │   │   ├── GridTables.cpp            # IQ2_S grid lookup table (1024 entries)
-│   │   │   ├── GridTablesIQ3S.cpp        # IQ3_S grid lookup table (512 entries)
-│   │   │   ├── Model.cpp                 # Transformer model (forward, generate)
-│   │   │   ├── ModelDebug.cpp            # Debug helpers for the model
-│   │   │   ├── ModelForward.cpp          # Forward pass implementation
-│   │   │   ├── ModelForwardDebug.cpp     # Forward pass debug helpers
-│   │   │   ├── ModelGeneration.cpp       # Generation loop
-│   │   │   ├── ModelInternal.cpp         # Internal model helpers
-│   │   │   ├── ModelLoad.cpp             # Model loading & weight prep
-│   │   │   ├── ModelMoE.cpp              # Mixture-of-Experts layers
-│   │   │   ├── ModelPrimitives.cpp       # Core primitives
-│   │   │   ├── ModelSampling.cpp         # Token sampling
-│   │   │   ├── QuantizedEmbedding.cpp    # Quantized token embedding dequantization
-│   │   │   ├── QuantizedMatrix.cpp       # Quantized matrix-vector multiply (CUDA/CPU)
-│   │   │   ├── SIMDMatMulVec.cpp         # SIMD dispatch (AVX2/AVX-512/scalar)
-│   │   │   ├── SIMDMatMulVecAVX2.cpp     # AVX2 SIMD kernels
-│   │   │   ├── SIMDMatMulVecAVX512.cpp   # AVX-512 SIMD kernels
-│   │   │   ├── ThreadPool.cpp            # Thread pool for parallel loops
-│   │   │   ├── Tokenizer.cpp             # BPE tokenizer (GGUF embedded + file loading)
-│   │   │   └── LMHeadCUDA.cu             # CUDA LM head (cublasSgemv)
-│   │   └── bridge/                 # N-API native addon (load, generate, status)
-│   │       └── Bridge.cpp
+│   ├── cpp/bridge/                 # N-API native addon (load, generate, status)
+│   │   └── Bridge.cpp
 │   └── ts/                         # TypeScript source
 │       ├── extension.ts            # VS Code extension entry (commands, status bar)
 │       ├── panel.ts                # WebView chat panel (standalone + sidebar)
-│       └── nativeBridge.ts         # Native addon wrapper (async load/generate)
-├── unit_tests/                     # Unit tests
-│   ├── CMakeLists.txt              # Test build config
-│   └── ModelTest.cpp               # Model loading & inference tests
-├── CMakeLists.txt                  # CMake build (with np fetch, N-API, CUDA)
+│       ├── nativeBridge.ts         # Native addon wrapper (async load/generate)
+│       └── agent/                  # Autonomous agent harness (ReAct)
+│           ├── agentLoop.ts        # Core ReAct loop engine
+│           ├── tools.ts            # Tool JSON schemas + VS Code tool executors
+│           ├── prompt.ts           # System prompt + tool protocol builder
+│           ├── chatTemplate.ts     # Conversation → prompt renderer
+│           ├── inference.ts        # callAPInference abstraction (swappable backend)
+│           ├── compaction.ts       # Token pruning / history compaction
+│           ├── config.ts           # Agent config (settings loader/saver)
+│           ├── runner.ts           # Shared "run agent" helper (model ensure + loop)
+│           └── optionsPanel.ts     # Agent options webview
+├── tests/                          # Pure Node smoke/integration tests (test:agent)
 ├── package.json                    # VS Code extension manifest
 ├── tsconfig.json                   # TypeScript configuration
 ├── scripts/
@@ -177,9 +186,7 @@ tinycoder/
 ├── media/
 │   ├── icon.png                    # Extension icon
 │   └── icon.svg                    # Extension icon (vector)
-└── .vscode/
-    ├── launch.json                 # Debug configurations
-    └── tasks.json                  # Build tasks
+└── plans/                          # Design documents
 ```
 
 ## Dependencies

diff --git a/media/tinycoder.jpg b/media/tinycoder.jpg
diff --git a/package.json b/package.json
@@ -2,7 +2,7 @@
   "name": "tinycoder",
   "displayName": "TinyCoder AI",
   "description": "Local AI coding assistant powered by Qwen2.5-Coder, Gemma 4, and Qwen3.6 GGUF models. Native C++ inference with AVX2/AVX-512/AMX SIMD acceleration and CUDA GPU support. Runs entirely offline on limited resources.",
-  "version": "0.4.0",
+  "version": "0.5.0",
   "publisher": "tinycoder",
   "license": "MIT",
   "repository": {
@@ -25,6 +25,8 @@
     "code generation",
     "qwen",
     "qwen2.5-coder",
+    "qwen3.6",
+    "qwen3.8",
     "gemma4",
     "llm",
     "local ai",
@@ -53,7 +55,10 @@
     "onCommand:tinycoder.generateCode",
     "onCommand:tinycoder.loadModel",
     "onCommand:tinycoder.showStatus",
-    "onCommand:tinycoder.inferFromTerminal"
+    "onCommand:tinycoder.inferFromTerminal",
+    "onCommand:tinycoder.agentRun",
+    "onCommand:tinycoder.openAgentOptions",
+    "onCommand:tinycoder.openSettings"
   ],
   "main": "./out/extension.js",
   "contributes": {
@@ -75,6 +80,20 @@
         }
       ]
     },
+    "menus": {
+      "view/title": [
+        {
+          "command": "tinycoder.openAgentOptions",
+          "when": "view == tinycoder.chat",
+          "group": "navigation"
+        },
+        {
+          "command": "tinycoder.openSettings",
+          "when": "view == tinycoder.chat",
+          "group": "navigation@1"
+        }
+      ]
+    },
     "commands": [
       {
         "command": "tinycoder.openPanel",
@@ -103,6 +122,20 @@
       {
         "command": "tinycoder.inferFromTerminal",
         "title": "TinyCoder: Infer from Terminal Selection"
+      },
+      {
+        "command": "tinycoder.agentRun",
+        "title": "TinyCoder: Run Agent (Autonomous Coding)"
+      },
+      {
+        "command": "tinycoder.openAgentOptions",
+        "title": "TinyCoder: Open Agent Options",
+        "icon": "$(gear)"
+      },
+      {
+        "command": "tinycoder.openSettings",
+        "title": "TinyCoder: Open Settings",
+        "icon": "$(settings-gear)"
       }
     ],
     "keybindings": [
@@ -127,7 +160,7 @@
       "properties": {
         "tinycoder.modelPath": {
           "type": "string",
-          "default": "",
+          "default": "/data/models/qwen/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf",
           "description": "Path to the GGUF model file"
         },
         "tinycoder.nThreads": {
@@ -185,6 +218,73 @@
           "minimum": 0,
           "maximum": 1024,
           "description": "Number of recent tokens to consider for repetition penalty"
+        },
+        "tinycoder.agent.maxIterations": {
+          "type": "number",
+          "default": 8,
+          "minimum": 1,
+          "maximum": 100,
+          "description": "Maximum ReAct loop iterations (tool calls + follow-ups) before forcing a final answer"
+        },
+        "tinycoder.agent.compactionThreshold": {
+          "type": "number",
+          "default": 24,
+          "minimum": 4,
+          "maximum": 1000,
+          "description": "History depth (messages) at which older tool results are truncated for context management"
+        },
+        "tinycoder.agent.compactionPreserveRecent": {
+          "type": "number",
+          "default": 8,
+          "minimum": 0,
+          "maximum": 200,
+          "description": "How many of the most recent messages to keep verbatim during compaction"
+        },
+        "tinycoder.agent.workspaceRoot": {
+          "type": "string",
+          "default": "",
+          "description": "Root directory the agent may operate on. Empty = first workspace folder"
+        },
+        "tinycoder.agent.terminalName": {
+          "type": "string",
+          "default": "TinyCoder Agent",
+          "description": "Name of the VS Code terminal used for runTerminalCommand"
+        },
+        "tinycoder.agent.terminalMode": {
+          "type": "string",
+          "default": "vscode",
+          "enum": ["vscode", "child", "auto"],
+          "description": "vscode: createTerminal + output capture. child: shell subprocess (deterministic capture). auto: try vscode, fall back to child"
+        },
+        "tinycoder.agent.systemPrompt": {
+          "type": "string",
+          "default": "",
+          "description": "Optional base system prompt override. The tool protocol is always appended"
+        },
+        "tinycoder.agent.enableReadFile": {
+          "type": "boolean",
+          "default": true,
+          "description": "Expose the readFile tool to the agent"
+        },
+        "tinycoder.agent.enableWriteFile": {
+          "type": "boolean",
+          "default": true,
+          "description": "Expose the writeFile tool to the agent"
+        },
+        "tinycoder.agent.enablePatchFile": {
+          "type": "boolean",
+          "default": true,
+          "description": "Expose the patchFile tool to the agent"
+        },
+        "tinycoder.agent.enableTerminal": {
+          "type": "boolean",
+          "default": true,
+          "description": "Expose the executeTerminalCommand tool to the agent"
+        },
+        "tinycoder.agent.enableSearch": {
+          "type": "boolean",
+          "default": true,
+          "description": "Expose the searchCodebase tool to the agent"
         }
       }
     }
@@ -199,7 +299,8 @@
     "build:native:cuda": "cmake -B build -DCMAKE_BUILD_TYPE=Release -DENABLE_CUDA=ON && cmake --build build --config Release",
     "lint": "eslint src --ext ts",
     "pretest": "npm run compile",
-    "test": "node ./out/test/runTest.js"
+    "test": "node ./out/test/runTest.js",
+    "test:agent": "npm run compile && node tests/agentSmoke.test.js && node tests/agentLoop.test.js"
   },
   "devDependencies": {
     "@types/node": "^20.11.0",

diff --git a/plans/tinycoder_plan.md b/plans/tinycoder_plan.md
@@ -16,11 +16,12 @@ All common inference parameters are configurable: **temperature, max‑tokens, t
 |---|---|
 | **Model loading** | Load a single *.gguf* model via a file‑picker. Only one model is active at a time. |
 | **Sidebar chat** | Webview panel (see [`src/ts/panel.ts`](src/ts/panel.ts:1)) for interactive messaging. |
-| **Command‑palette actions** | `tinycoder.explainCode`, `tinycoder.completeCode`, `tinycoder.generateCode`, `tinycoder.loadModel`. |
+| **Command‑palette actions** | `tinycoder.explainCode`, `tinycoder.completeCode`, `tinycoder.generateCode`, `tinycoder.loadModel`, `tinycoder.agentRun`, `tinycoder.openAgentOptions`. |
 | **Status‑bar item** | Shows *model loaded* / *inference ready* and allows quick reload. |
 | **Inference settings UI** | Settings contribution (`package.json`) with inputs for temperature, max‑tokens, top‑p, repeat‑penalty. |
 | **Terminal integration** | New terminal command `TinyCoder: Infer Selected Text` that reads the current selection, sends it to the native bridge, and streams output to the terminal. |
 | **Native bridge** | C++/CUDA backend (see [`src/cpp/bridge/*.cpp`](src/cpp/bridge/Bridge.cpp:1)) exposing `loadModel`, `runInference`. |
+| **Agent harness (ReAct)** | Autonomous coding agent: JSON tool-call loop (`readFile`/`writeFile`/`patchFile`/`executeTerminalCommand`/`searchCodebase`), tool definitions as JSON schemas, conversation compaction, and an options webview (`tinycoder.openAgentOptions`). Lives in [`src/ts/agent/`](src/ts/agent/types.ts:1). |
 
 ---