fix(rag): align retrieval quality knobs with reviewbot — minScore 0.4 + bm25 rerank + title-led query (#GAP-2) - #1124
Merged
Merged
Conversation
… + bm25 rerank + title-led query (#GAP-2)
The RAG retrieval seam used minScore 0 (off) and no reranker, keeping
low-relevance cosine 'neighbours' as 'relevant code' — itself a source
of false positives. Match reviewbot's core config: pass minScore 0.4 +
reranker 'bm25' on every retrieve call, and prepend the PR title to the
query text for recall parity (reviewbot queries `${title}\n${diff}`).
- rag-wire: buildRagQuery prepends the (trimmed) PR title; buildReviewRagContext
passes minScore 0.4 + reranker bm25 (caller-supplied reranker still wins).
- processors: feed the PR title into the retrieval-context build.
- tests: assert the title leads the query + retrieveContext is called with
minScore 0.4 / reranker bm25 (caller override honored).
Code knobs only — RAG stays per-repo opt-in (flag + allowlist); the operator
enables + warms the index via runtime config separately.
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1124 +/- ##
=======================================
Coverage 94.83% 94.84%
=======================================
Files 155 155
Lines 18708 18711 +3
Branches 6772 6773 +1
=======================================
+ Hits 17742 17746 +4
Misses 395 395
+ Partials 571 570 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JSONbored
added a commit
that referenced
this pull request
Jun 24, 2026
…_RAG=true) (#1126) Flips GITTENSORY_REVIEW_RAG false->true so the reviewer retrieves related-code context (with the #1124 quality knobs: minScore 0.4 + bm25 rerank + title-led query). Fail-safe: retrieval degrades to '' until the Vectorize index warms (this flag also activates ingestion: the 6-hourly cron fan-out + the merged-PR incremental path). Still gated by the GITTENSORY_REVIEW_REPOS allowlist. (#GAP-2)
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Aligns the gittensory codebase-RAG retrieval seam (
src/review/rag-wire.ts) with reviewbot's core config, fixing GAP-2 (the code half).Before, the retrieve call used
minScore: 0(off) and no reranker, so low-relevance cosine "neighbours" were injected as "RELEVANT EXISTING CODE" — noise that itself drives false positives. The query also omitted the PR title, hurting recall vs. reviewbot (which embeds${title}\n${diff}).Changes
src/review/rag-wire.tsbuildRagQuery(files, title?)— PREPENDS the (trimmed) PR title to the query text; additive, omitted cleanly when blank/absent.buildReviewRagContext(...)— always passesminScore: 0.4+reranker: "bm25"toretrieveContext(matches reviewbotrag: { minScore: 0.4, reranker: "bm25" }). A caller-suppliedrerankerstill wins (e.g. forcing"off");minScoreis fixed.src/queue/processors.ts— feeds the PR title into the retrieval-context build at the (already flag+allowlist-gated) review call site.test/unit/rag-wiring.test.ts— new assertions: title leads the query;retrieveContextinvoked withminScore: 0.4/reranker: "bm25"; caller override honored; blank-title handling.Scope / safety
Code knobs ONLY. RAG stays per-repo opt-in (
isRagEnabled(env)flag AND the convergence repo allowlist) — this does not force-enable RAG globally; the operator enables + warms the index via runtime config separately. Fully fail-safe path unchanged.Note: Task A (repoint public-stats to the live
audit_eventsledger) was already shipped onmainin #1114, so this branch contains only the RAG (Task B) work.Verification
npx tsc --noEmit— clean (exactOptionalPropertyTypes+noUncheckedIndexedAccess)npx vitest run— 3546 passed, 1 skipped (215 files); rag-wiring suite 16 → 21 tests