🧠 RepoMind Tracker

A complete, context-aware AI coding assistant that helps you talk to, understand, and visualize your GitHub codebases.

Powered by Qwen 2.5 Coder, RAG (FAISS + LlamaIndex), and a Custom Node-Graph Pipeline.

📑 Table of Contents

1. Overview
2. Features
3. Under the Hood
4. Graphic Demonstrations
5. Tech Stack
6. Installation
7. Usage
8. API Documentation
9. Project Structure
10. Contributing
11. License

---

📖 Overview

🤔 The Problem

Navigating and understanding a large, unfamiliar, or legacy GitHub codebase can take days. Developers struggle to figure out where functions are defined, how dependencies trace back globally, and what the overall architecture looks like before writing a single line of code.

💡 The Solution

RepoMind clones any public GitHub repository, parses the entire Abstract Syntax Tree (AST), and embeds the code into a semantic vector space using FAISS and HuggingFace models. You can then chat directly with the code and generate graphical node-based visualizations (Directory Structure, Dependencies, and Call Graphs) to instantly understand how the repository ticks.

👥 Who is this for?

• Software Engineers joining a new project or onboarding to a massive microservice.
• Code Reviewers needing extra architectural context on large pull requests.
• Open Source Contributors trying to find the exact file to fix a bug in a multi-thousand-file repository.

---

🌟 Features

• ✨ RAG-Powered Chat: Ask questions about your code natively. The local LLM will scan your files and construct prompts grounded purely in your repository's logic.
• ⚡ Lightning Fast Ingestion: Utilizes highly optimized ThreadPoolExecutor parallel processing to blaze through massive codebases in seconds.
• 🧠 Conversational Memory & Control: Follow-up questions work! The built-in History Tab allows you to view contextual memory and reset the chat context instantly to prevent token bloat.
• 🌍 Multi-Language AST Support: Robust Abstract Syntax Tree parsing for Python, JavaScript, TypeScript, and Dart codebases.
• 🗺️ Architecture Graphs: Visualize your codebase instantly. Generates pure JSON endpoints and interactive Streamlit UI tabs for:
  • Repository Directory Trees
  • Function Call Graphs (who calls whom across files)
  • Modular Dependency Tracking (import hierarchies)
• 🚀 GPU Accelerated: Built-in CUDA support via PyTorch to efficiently embed thousands of chunks.
• 🔒 100% Local Privacy: Runs entirely on your hardware via Ollama. No proprietary/enterprise code is ever sent to OpenAI, Anthropic, or external API providers.

---

⚙️ Under the Hood

When you submit a GitHub URL, RepoMind executes a highly optimized pipeline:

1. Repository Ingestion: Clones the repo locally to cloned_repos/.
2. Multi-Language AST Parsing: Uses native ast modules and advanced regex trees to logically break down files across Python, JS/TS, and Dart.
3. Parallel Text Chunking: Fragments the code into semantically coherent overlapping chunks across all CPU cores simultaneously.
4. Vector Embedding: Uses BAAI/bge-small-en-v1.5 to generate dense vector embeddings with batch processing, leveraging GPU acceleration if available.
5. LlamaIndex Indexing: Builds a single-pass VectorStoreIndex directly from code chunks — no redundant FAISS double-indexing.
6. LlamaIndex Query Engine: Routes user queries through an Ollama-hosted LLM (qwen2.5-coder) using highly tuned, token-efficient context windows.

---

🎨 Graphic Demonstrations

1. End-to-End Pipeline (RAG + code graphs)

An overview of how RepoMind ingests code, embeds chunks into FAISS, and answers questions grounded in retrieved context.

What happens in the pipeline?

RepoMind: 1. clones your GitHub repo into `cloned_repos/` 2. extracts code chunks via traversal + AST-aware chunking 3. embeds chunks into a LlamaIndex VectorStoreIndex (single pass) 4. answers questions using Ollama + LlamaIndex, constrained by retrieved chunks 5. generates graph data (structure/call/dependencies) for the UI

2. Streamlit UI

💬 Chat — ChatGPT-style Interface

Talk directly to your codebase with a scrollable chat container and pinned input.

Ingesting Repository	Repo Loaded & Ready

🤖 AI-Powered Conversation

Ask questions and get context-aware answers grounded in your codebase.

📊 Visualize — Interactive Code Graphs

Generate file structure trees, function call graphs, and dependency webs natively inside the UI.

📂 File Structure	🔗 Call Graph	🌐 Dependencies

---

🛠️ Tech Stack

Language: Python 3.10+

Core Technologies:

• UI Frontend: Streamlit (app.py)
• Backend API: Flask (visualization/api.py)
• Local LLM Engine: Ollama running Qwen 2.5 Coder
• Vector Database: FAISS (Facebook AI Similarity Search)
• RAG Orchestration: LlamaIndex & Langchain
• Embeddings Model: BAAI/bge-small-en-v1.5 (fast, lightweight)
• Acceleration: PyTorch (CUDA 12.1+ Support)

---

💻 System Requirements

To run the local embeddings and the Ollama LLM smoothly, your system should meet these specifications:

• OS: Windows 10/11, macOS (M-series recommended), or Linux
• RAM: 8 GB minimum (16 GB highly recommended for parsing large repositories seamlessly)
• Storage: ~10 GB free space (to download Ollama LLMs and cache HuggingFace embeddings)
• CPU: Modern multi-core bridging (Intel i5/Ryzen 5 or better) for parallelized AST chunking
• GPU (Optional but Recommended): 6GB+ VRAM (NVIDIA) for PyTorch CUDA acceleration to index massive codebases instantly. If a GPU is not available, RepoMind will gracefully fall back to CPU.

---

🚀 Installation

Prerequisites

1. Python 3.10+
2. Ollama (Must be installed and running as a background service)
3. Git (For cloning target repositories)

Step-by-Step Setup

1. Clone the repository

git clone https://github.com/Yash-Kavaiya/RepoMind.git
cd RepoMind

2. Create a Virtual Environment (Highly Recommended)

python -m venv .venv

# On Windows (Command Prompt or PowerShell):
.venv\Scripts\activate

# On Mac/Linux:
source .venv/bin/activate

3. Install Dependencies

For standard CPU environments:

pip install -r requirements.txt

🔥 For NVIDIA GPU Acceleration (Recommended for 10x faster indexing!):

# Remove CPU-only torch versions
pip uninstall -y torch torchvision torchaudio

# Install PyTorch with CUDA 12.1 support
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

(Verify GPU installation by running python -c "import torch; print(torch.cuda.is_available())". It should print True.)

4. Start the Local LLM In a new terminal window (keep it running), pull and start the model:

ollama run qwen2.5-coder:3b

⏱️ Try it in ~2 minutes

1. Start the UI:

python -m streamlit run app.py

2. Open http://localhost:8501 in your browser.
3. Paste a GitHub repo URL (for example: https://github.com/pallets/flask) into the sidebar and click Load Repository.
4. Ask a question like:

Sample prompts

- `Where is the request routing logic located?` - `Explain the function that handles incoming requests.` - `Show the call graph for the main entry point.`

---

💻 Usage

RepoMind consists of two parallel systems. You can run either the Frontend UI or the Backend Visualization API depending on your needs.

Option A: The Streamlit Chat Application

To interact with the conversational AI, ingest repositories, and query code logic natively menus:

python -m streamlit run app.py

Workflow:

1. Open http://localhost:8501 in your browser.
2. In the sidebar, paste a valid GitHub URL (e.g., https://github.com/pallets/flask).
3. Click Load Repository and wait for the ingestion process (FAISS indexing) to complete.
4. Use the Chat box to ask contextual questions:
   • "Where is the core routing logic located?"
   • "Explain the render_template function and optimize it."

Option B: The Application Visualization API (Flask)

If you want to extract raw architectural intelligence to build your own frontend UI (like React Flow or D3.js):

python -m visualization.api

The Flask server boots up natively on http://localhost:5000.

---

🔌 API Documentation

If running the Flask backend, you can execute GET/POST requests to retrieve codebase intelligence.

Base URL: http://localhost:5000

For repository-based endpoints, provide an absolute path via repo_path: (Replace C:\path\to\repo with the absolute path of the locally cloned repository stored in cloned_repos/.)

HTTP Method	Endpoint Path	Description	Response Type
GET	/repo/structure?repo_path=<path>	Generates a deep hierarchical tree dictionary of all project files and folders.	application/json
GET	/repo/call-graph?repo_path=<path>	Identifies functions and returns node/edge pairs detailing which functions call which.	application/json
GET	/repo/dependencies?repo_path=<path>	Analyzes Python import statements to map file-level dependencies.	application/json
GET	/chat/history	Retrieves the current session's chat memory array.	application/json
POST	/chat/reset	Purges conversation history memory for a fresh context window.	application/json

Run a quick API test (curl)

```bash curl "http://localhost:5000/repo/structure?repo_path=C:/path/to/cloned_repos/" ```

---

📂 Project Structure

RepoMind/
├── app.py                      # Main Streamlit Chat Interface
├── architecture-diagram.png    # High-level pipeline illustration (shown in README)
├── cloned_repos/               # Local clones of target GitHub repos
├── faiss_indices/              # Persistent FAISS index storage
├── weights/                    # HuggingFace / embedding cache (created automatically)
├── rag_101/                    # LlamaIndex + retrieval orchestration
│   ├── client.py
│   ├── rag.py
│   └── retriever.py
├── repo_ingestion/             # Pipeline to clone, chunk, and embed code
│   ├── code_chunker.py
│   ├── embedding_store.py
│   ├── file_traversal.py
│   └── github_handler.py
├── visualization/              # AST parsing + graph generation + API
│   ├── api.py                  # Flask backend exposing JSON endpoints
│   ├── repo_structure.py      # Directory tree builder
│   ├── call_graph.py          # Function call graph builder
│   ├── dependency_graph.py   # Import dependency graph builder
│   ├── streamlit_viz.py       # Render helpers for Streamlit
│   └── ast_analyzers/         # Language-specific analyzers
├── memory/                     # Conversation memory
│   └── chat_memory.py
├── tests/                      # Automated tests
└── requirements.txt            # Python dependency locks

---

🛟 Troubleshooting

Ollama errors (model not found / connection refused)

Run `ollama run qwen2.5-coder:3b` first (in a separate terminal) and keep the service running.

Ingestion takes a long time

The first run includes cloning + chunking + embedding. Try a smaller repo to validate the setup, then scale up.

API returns 400/404

Double-check that `repo_path` is an existing absolute directory and that you’re using the correct parameter name: `repo_path` (not `path`).

Windows curl quoting issues

Use forward slashes in paths (for example `C:/Users/...`) and URL-encode spaces if needed.

---

❓ FAQ

Is everything local / privacy-friendly?

Yes. The Streamlit app calls Ollama locally; it does not require sending your code to OpenAI/Anthropic/etc.

Does the API support multiple users?

Currently the API uses a single global chat memory instance, so isolation per user/session is not implemented yet.

Do FAISS indices get reused?

The ingestion pipeline persists indices to `faiss_indices/`, but the current UI rebuilds during ingestion. (The project includes index-loading utilities for future reuse.)

---

🤝 Contributing

Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

---

📄 License

Distributed under the MIT License. See the LICENSE file for more information.

(back to top)

---

Made with ❤️ by Akarsh
Empowering developers to understand codebases instantly.