Codebase Intelligence RAG

Codebase Intelligence RAG is a focused RAG/ML engineering project for asking grounded questions about GitHub codebases. It is not a full-stack product: Streamlit is only a thin demo layer, and the core work is repository ingestion, code-aware retrieval, reranking, context construction, and grounded generation with exact citations.

Why Basic Vector Search Is Not Enough For Code

Code questions often depend on exact identifiers, file paths, symbols, and relationships. A semantic vector search might understand "authentication", but miss an exact function such as generate_token or a browser API such as IntersectionObserver. This project combines semantic retrieval, BM25 lexical retrieval, metadata filters, and lightweight graph expansion so code is retrieved as structured evidence instead of anonymous text chunks.

Architecture

GitHub Repository
  -> Repository Ingestion
  -> Source Code Parsing
  -> Code-aware Chunking
  -> Metadata Extraction
  -> Local Embeddings
  -> Local Vector Index
  -> Vector Search + BM25 + Graph Context
  -> Hybrid Retrieval with Reciprocal Rank Fusion
  -> Reranking
  -> Top-K Context Construction
  -> LLM or Retrieval-only Fallback
  -> Grounded Answer
  -> File + Line Citations

Repository Ingestion

rag/ingestion.py validates GitHub HTTPS URLs, creates safe repository names, avoids recloning repositories that already exist in data/, and skips directories such as .git, node_modules, venv, __pycache__, dist, build, and coverage. It also skips secrets, .env files, oversized files, and unsupported/binary files.

Code Parsing

rag/parser.py uses Tree-sitter to extract functions, classes, methods, imports, calls, API endpoints, database interactions, symbols, file paths, languages, and line ranges. Python, JavaScript, TypeScript, Java, C++, and Go are supported, but the implementation stays conservative and retrieval-focused.

Code-aware Chunking

rag/chunker.py creates module, class, function, and method chunks. Each chunk preserves:

• repository id
• file path
• language
• symbol name
• symbol type
• parent class when available
• start line and end line
• source code
• imports and calls

This makes citations possible and keeps code boundaries meaningful for retrieval.

Embeddings And Vector Search

Embeddings use FastEmbed locally by default with BAAI/bge-small-en-v1.5. No OpenAI key is required for indexing or retrieval.

rag/vector_store.py implements a local NumPy cosine-similarity index persisted under data/vector_index. There is no Qdrant server, Docker service, or external vector database requirement.

BM25

rag/bm25.py implements BM25 for exact lexical retrieval. This is critical for identifiers like authenticate_user, Navbar, fetchProducts, generate_token, and IntersectionObserver.

Hybrid Retrieval

rag/retrieval.py combines vector and BM25 results with Reciprocal Rank Fusion. It also stores source-level score details so the Streamlit UI can show how vector, BM25, and graph candidates contributed to the final ranking.

Query Processing

Queries are transformed into retrieval-oriented terms while avoiding common stopwords. For example:

Does this project have user authentication?

can expand toward:

authentication, login, jwt, token, session, credentials, middleware

Exact identifiers such as generate_token are preserved.

Graph-based Context Expansion

rag/graph.py builds a lightweight NetworkX graph with relationships such as:

• file defines class/function
• class contains method
• function calls function
• file imports module
• endpoint exposes handler

Retrieval can expand from a matched chunk to nearby callers, callees, containing classes, imports, and endpoints using a bounded graph depth. The goal is to show that code entities are related, not isolated text blocks.

Reranking

rag/reranker.py defaults to an understandable lexical reranker based on query overlap and candidate score. A FastEmbed cross-encoder reranker can be enabled with RERANKER_PROVIDER=fastembed, but the default remains lightweight and easy to explain.

Context Construction

rag/context.py deduplicates chunks, preserves file and line citations, controls context size, and formats evidence like:

[1] src/auth.py:20-42
Symbol: authenticate_user (function)

The LLM never receives the entire repository.

LLM Generation And Grounding

rag/llm.py supports:

• LLM_PROVIDER=gemini with GEMINI_API_KEY
• LLM_PROVIDER=openai with OPENAI_API_KEY
• LLM_PROVIDER=openai-compatible with LLM_API_URL, LLM_API_KEY, and LLM_MODEL

If no provider is configured, the app remains useful in retrieval-only mode and says: LLM generation unavailable; showing retrieved evidence.

The grounded prompt tells the model to answer only from retrieved repository context, avoid invented files/functions/line numbers, distinguish evidence from inference, cite exact file ranges, and say when evidence is insufficient.

Citations

Every retrieved chunk carries exact source metadata. Answers and sources cite real repository locations such as:

• script.js:1-74
• src/auth.py:20-42
• src/utils/token.js:10-25

Negative answers should be based on missing evidence, not assumptions.

Streamlit UI

Run:

streamlit run streamlit_app.py

The UI includes:

• repository URL input
• Index Repository button
• repository name, file count, chunk count, languages, indexing time
• question input
• retrieval strategy: Hybrid, Hybrid + Graph, Vector, BM25
• optional filters: language, symbol type, file path
• answer
• sources
• retrieved code context
• retrieval details with strategy, query terms, filters, source scores, and final ranking

Installation

python -m venv venv
.\venv\Scripts\activate
pip install -r requirements.txt
streamlit run streamlit_app.py

Copy .env.example if you want LLM generation. Embeddings and retrieval work without an LLM key.

Example Questions

• Where is the website's navigation implemented?
• How does the website reveal sections as the user scrolls?
• Where is IntersectionObserver used?
• Which files are involved in the website animations?
• Trace what happens when a section becomes visible.
• Does this project implement JWT authentication?
• What files would I need to modify to add a new product section?

Evaluation

Run:

python -m evaluation.run_evaluation

The evaluator reports Hit Rate, Recall@K, Precision@K, and MRR for BM25, vector, hybrid, hybrid + graph, and full reranked retrieval.

Tests

pytest

Tests cover ingestion, parser extraction, chunk metadata, BM25, vector retrieval, hybrid retrieval, metadata filtering, graph expansion, citations, context construction, LLM request construction, and no-LLM evidence fallback.

How This Differs From A Basic RAG Chatbot

Basic RAG:

documents -> text chunks -> embeddings -> vector search -> LLM

This project: Deployed Link - https://codebase-intelligence-rag-hqfteuyappwhonm8yje4agk.streamlit.app/

code -> AST/code entities -> metadata-rich chunks -> semantic + lexical retrieval
     -> graph relationships -> hybrid ranking -> bounded context -> grounded generation

The result is better suited for code because it preserves symbols, exact identifiers, line numbers, file paths, and dependency relationships.

Limitations

Static parsing is conservative. Dynamic imports, runtime framework behavior, minified/generated code, and ambiguous calls may not be fully resolved. The graph is intentionally lightweight and local. LLM generation requires a configured provider, but retrieval and citations work without one.