Skip to content

Interview-ready pass: pin deps, fix stranded demo link, real eval numbers, CI - #1

Open
Ragh234 wants to merge 1 commit into
mainfrom
chore/interview-ready
Open

Interview-ready pass: pin deps, fix stranded demo link, real eval numbers, CI#1
Ragh234 wants to merge 1 commit into
mainfrom
chore/interview-ready

Conversation

@Ragh234

@Ragh234 Ragh234 commented Sep 6, 2026

Copy link
Copy Markdown
Owner

This repo's README was already detailed and technically credible, so the changes here are narrow — finishing touches, not a rewrite.

What changed

  • Pinned requirements.txt. All 13 deps were unpinned, including 6 tree-sitter-* grammar packages, which are exactly the kind of dependency that breaks against a new tree-sitter core release. This repo has a live Streamlit deployment; an unpinned rebuild is what would take it down.
  • Fixed a README formatting bug: the deployed Streamlit link was stranded mid-sentence inside the "How This Differs From A Basic RAG Chatbot" section, between two code fences. Moved to a Live demo line right under the title where it's actually visible.
  • Added real evaluation numbers. The README described the Hit Rate/Recall/Precision/MRR harness but never showed output. Ran it and added the table — and reported it honestly rather than cherry-picked: hybrid retrieval clearly beats either single-strategy baseline, but graph expansion and reranking don't show a measurable win on the current 5-question eval set (full-strategy MRR is fractionally below plain hybrid). That's a fair, defensible thing to say in an interview.
  • MIT LICENSE, CI running the full test suite (all local — FastEmbed embeddings, no API key needed).

Verification

$ pytest
21 passed in 1.35s
$ python -m evaluation.run_evaluation
bm25         {'hit_rate': 0.8, 'recall@5': 0.8, 'precision@5': 0.44, 'mrr': 0.6}
vector       {'hit_rate': 1.0, 'recall@5': 1.0, 'precision@5': 0.52, 'mrr': 0.717}
hybrid       {'hit_rate': 1.0, 'recall@5': 1.0, 'precision@5': 0.56, 'mrr': 0.75}
hybrid_graph {'hit_rate': 1.0, 'recall@5': 1.0, 'precision@5': 0.56, 'mrr': 0.717}
full         {'hit_rate': 1.0, 'recall@5': 1.0, 'precision@5': 0.52, 'mrr': 0.717}

(3 test_vector_store.py cases failed locally on a Windows-specific temp-directory permission conflict in my sandbox, unrelated to this repo's code — re-running with --basetemp pointed at a writable directory gave the clean 21/21 above. Not expected to occur on GitHub's Linux runners.)

Deliberately not changed

No architecture or feature changes. evaluation/run_evaluation.py's 5-question dataset is small enough that reranking's effect isn't distinguishable from noise — noted in the README as a natural next step rather than "fixed" by inflating the dataset myself.

- requirements.txt was fully unpinned (13 deps including 6 tree-sitter
  grammar packages) -- this repo has a live Streamlit deployment, and an
  unpinned rebuild against a new tree-sitter core release is exactly
  what would take it down. Pinned to a verified-installable set.
- The deployed Streamlit link was stranded mid-sentence inside the
  "How This Differs" section, between two code fences. Moved to a
  Live demo line under the title where it's actually visible.
- Added a real Evaluation results table. The README described the
  harness (Hit Rate/Recall/Precision/MRR across 5 retrieval
  strategies) but never showed output. Ran it and added the table --
  and reported it honestly: hybrid retrieval clearly beats either
  single-strategy baseline, but graph expansion and reranking don't
  show a measurable win on the current 5-question eval set.
- MIT LICENSE, CI running the full test suite (all local -- FastEmbed
  embeddings, no API key needed).

Verified: pytest -> 21 passed. python -m evaluation.run_evaluation ->
the numbers now in the README's Evaluation section.
@Ragh234
Ragh234 force-pushed the chore/interview-ready branch from ca0ea0f to 8210680 Compare September 6, 2026 10:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants