feat: add stale embedding reindex mode

[Steve-too]

Closes #310.

Summary

• add stale support to backfill_embeddings with scanned/reembedded/skipped counts
• wire --stale through CLI, MCP, and JSONL reindex surfaces
• reject force + stale together and fall back to full re-embed on model mismatch

Tests

• PYTHONPATH=src python3 -m pytest tests/embeddings/test_migration.py tests/embeddings/test_search.py::test_mcp_kb_reindex_embeddings tests/embeddings/test_search.py::test_jsonl_kb_reindex_embeddings tests/embeddings/test_cli.py::test_reindex_embeddings_backfills -q

Summary by CodeRabbit

• New Features

  • Added a new reindexing option to re-embed only stale or missing content.
  • Reindex responses now include detailed scan, re-embed, and skip counts.
  • Reindex tools now report the active embedding model alongside results.

• Bug Fixes

  • Prevented conflicting reindex options from being used together.
  • Improved reindex behavior so unchanged items are skipped when appropriate.
  • Re-embedding now better handles model changes and content drift.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds a --stale mode to embedding reindexing: backfill_embeddings gains model-mismatch detection and content-hash-based skip logic, returning structured scan/reembed/skip counts instead of a single count. The CLI, MCP tool, and JSONL handler are updated to expose --stale, enforce mutual exclusivity with --force, and surface the new result shape.

Changes

Stale re-embed feature

Layer / File(s)	Summary
Migration core: mismatch detection and stale backfill logic src/vouch/embeddings/migration.py	Adds detect_mismatch to compare stored vs. current embedder identity, extends backfill_embeddings with a stale parameter, force/stale mutual-exclusivity validation, content-hash-based per-artifact skip logic, and returns a dict with scanned, reembedded, skipped, model, and model_mismatch instead of an integer.
CLI, MCP tool, and JSONL handler wiring src/vouch/cli.py, src/vouch/server.py, src/vouch/jsonl_server.py	Adds --stale/--no-stale and equivalent stale options across the CLI, MCP tool (kb_reindex_embeddings), and JSONL handler, each validating force/stale mutual exclusivity, passing stale into backfill_embeddings, and merging the richer result dict (with touched derived from reembedded) into their outputs.
Test coverage for stale backfill and stricter assertions tests/embeddings/test_migration.py, tests/embeddings/test_search.py	Adds tests for unchanged-skip, drift-triggered reembed, model-mismatch full reembed, and force/stale exclusivity; tightens existing MCP/JSONL reindex tests to assert touched == reembedded and scanned >= 1.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

• vouchdev/vouch#42: Earlier work on detect_mismatch and backfill_embeddings model-identity handling in the same file, extended here with stale-based logic.
• vouchdev/vouch#44: Prior plumbing for kb_reindex_embeddings and backfill_embeddings/force behavior that this PR builds on.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly reflects the main change: adding a stale mode for embedding reindexing.
Linked Issues check	✅ Passed	The change matches #310: stale reindexing is wired through CLI, MCP, and JSONL, and returns scanned/reembedded/skipped counts.
Out of Scope Changes check	✅ Passed	The modified files stay within the stale reindexing feature, with no unrelated feature or refactor visible in the summary.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Steve-too]

Closing this because the branch is based on Steve-too's fork baseline, which still contains the embedding reindex surface that #310 targets, while current upstream main has diverged and no longer contains that surface. That makes the PR conflict-heavy and not a clean merge candidate as opened.

[coderabbitai]

🧹 Nitpick comments (1)

src/vouch/embeddings/migration.py (1)

69-95: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Skip-decision logic is duplicated between migration.py and storage.py::_embed_and_store.

should_embed here recomputes content_hash(text) and calls index_db.get_embedding to compare hash/model, but store._embed_and_store (when called with force=force_run=False) performs the exact same lookup + comparison again internally before actually re-encoding. This means every non-skipped, non-forced artifact triggers two separate get_embedding DB round-trips and two hash computations, and the two skip conditions can silently diverge over time since they're maintained independently.

Consider extracting the "is this artifact stale?" predicate into a single shared helper (e.g. in embeddings/base.py) used by both backfill_embeddings and _embed_and_store, or pass the already-fetched existing/live_hash into _embed_and_store to avoid recomputation.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/vouch/embeddings/migration.py` around lines 69 - 95, The skip/staleness
check is duplicated between backfill_embeddings in migration.py and
store._embed_and_store in storage.py, causing redundant DB lookups and hash
recomputation. Refactor the “should this artifact be embedded?” logic into a
shared helper (for example in embeddings/base.py) or have backfill_embeddings
pass the already-fetched existing record and live_hash into _embed_and_store so
both paths use the same predicate. Update both backfill_embeddings and
_embed_and_store to call the shared logic and keep the force_run behavior
consistent.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/vouch/embeddings/migration.py`:
- Around line 69-95: The skip/staleness check is duplicated between
backfill_embeddings in migration.py and store._embed_and_store in storage.py,
causing redundant DB lookups and hash recomputation. Refactor the “should this
artifact be embedded?” logic into a shared helper (for example in
embeddings/base.py) or have backfill_embeddings pass the already-fetched
existing record and live_hash into _embed_and_store so both paths use the same
predicate. Update both backfill_embeddings and _embed_and_store to call the
shared logic and keep the force_run behavior consistent.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: de909106-84c8-437e-b167-6a830a656a0a

📥 Commits

Reviewing files that changed from the base of the PR and between 8ae2806 and b5f0102.

📒 Files selected for processing (6)

• src/vouch/cli.py
• src/vouch/embeddings/migration.py
• src/vouch/jsonl_server.py
• src/vouch/server.py
• tests/embeddings/test_migration.py
• tests/embeddings/test_search.py