fix(models): require Claim.evidence to be non-empty at the model layer - #82
Merged
plind-junior merged 4 commits intoMay 27, 2026
Merged
Conversation
vouchdev#81) The 'claims must cite sources' guarantee (README §'Why this exists' point 3; CONTRIBUTING §'Things we won't merge') used to live only in proposals.propose_claim, so every other write path silently accepted Claim(evidence=[]) and landed an uncited claim: - store.put_claim direct: existence-check loop iterates zero times. - store.update_claim: writes the YAML without re-validating. - bundle.import_apply via _validate_content: defers to Claim.model_validate, which accepted evidence=[] because the model had no min-length constraint. Add @field_validator('evidence') on Claim — raises ValueError when the list is empty. Closes all three bypass paths in one place. store.update_claim additionally re-validates via Claim.model_validate(claim.model_dump()) before persisting, so in-place mutation (c.evidence = []; store.update_claim(c)) raises before the YAML hits disk — the field validator only fires at construction time, not on attribute assignment. Four regression tests: - test_claim_model_rejects_empty_evidence (tests/test_storage.py) — Claim(evidence=[]) raises pydantic.ValidationError. - test_put_claim_rejects_empty_evidence — store.put_claim raises; no claims/<id>.yaml is written. - test_update_claim_rejects_empty_evidence — in-place mutation + update_claim raises; the on-disk YAML is unchanged. - test_import_rejects_uncited_claim (tests/test_bundle.py) — a schema-valid bundle whose claim YAML has evidence: [] is rejected by import_check (schema validation issue) and import_apply raises before writing. The existing guard in proposals.propose_claim becomes a redundant user-facing error message and is left in place for the friendlier CLI/JSONL error string. No on-disk-layout, schema, or bundle-format change; data the model never should have accepted now raises.
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughMove the "claims must have citations" invariant into the Claim Pydantic model, re-run validation on update, make lint resilient to legacy uncited claim YAMLs, and add tests plus a changelog entry documenting behavior and migration impact. ChangesClaim evidence validation enforcement
Possibly related PRs
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/vouch/storage.py (1)
321-327: ⚡ Quick winUse the validated Claim instead of discarding it.
The re-validation on line 326 correctly catches invalid mutations, but the validated result is discarded. Line 327 still dumps the original
claimobject. For the current_at_least_one_citationvalidator this works fine (it only validates, doesn't transform), but if future validators transform data, those transformations would be lost.♻️ Refactor to use the validated result
- # Re-validate the in-memory Claim before persisting so model - # invariants (e.g. evidence must be non-empty — see `#81`) hold - # even when a caller mutated fields in place after get_claim(). - # The Claim model's field validators only run at construction - # time; mutation alone bypasses them unless we round-trip. - Claim.model_validate(claim.model_dump(mode="json")) - self._claim_path(claim.id).write_text(_yaml_dump(claim.model_dump(mode="json"))) + # Re-validate the in-memory Claim before persisting so model + # invariants (e.g. evidence must be non-empty — see `#81`) hold + # even when a caller mutated fields in place after get_claim(). + # The Claim model's field validators only run at construction + # time; mutation alone bypasses them unless we round-trip. + validated = Claim.model_validate(claim.model_dump(mode="json")) + self._claim_path(claim.id).write_text(_yaml_dump(validated.model_dump(mode="json")))This also eliminates the double serialization of the claim object.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/vouch/storage.py` around lines 321 - 327, The code currently calls Claim.model_validate(claim.model_dump(...)) but discards the validated return value and still writes the original claim; change this to capture the validated model (e.g., validated = Claim.model_validate(claim.model_dump(mode="json"))) and then persist the validated model by calling self._claim_path(claim.id).write_text(_yaml_dump(validated.model_dump(mode="json"))), so any validator transformations are preserved and you avoid double-serializing the original object.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/test_bundle.py`:
- Around line 338-343: Tighten the assertions to verify the citation-specific
error about empty evidence rather than any schema failure: update the
diff.issues check to assert that at least one issue string contains both "schema
validation failed" and the citation invariant text (e.g., "cite at least one"),
and update the pytest.raises match to include both substrings (or a regex
matching "schema validation failed" and "cite at least one") when calling
bundle.import_apply; reference the existing call sites bundle.import_check,
diff.issues, and bundle.import_apply to locate the assertions to change.
In `@tests/test_storage.py`:
- Around line 122-129: The test currently constructs Claim(id="c1",
text="uncited", evidence=[]) which triggers model validation before
store.put_claim is called; change the test to construct a valid Claim (e.g.,
evidence with at least one item), then mutate that Claim's evidence to an empty
list and call store.put_claim(…) expecting ValidationError; keep the final
assertion that (store.kb_dir / "claims" / "c1.yaml").exists() is false and
reference the test function name test_put_claim_rejects_empty_evidence and the
store.put_claim and Claim symbols when making the change.
---
Nitpick comments:
In `@src/vouch/storage.py`:
- Around line 321-327: The code currently calls
Claim.model_validate(claim.model_dump(...)) but discards the validated return
value and still writes the original claim; change this to capture the validated
model (e.g., validated = Claim.model_validate(claim.model_dump(mode="json")))
and then persist the validated model by calling
self._claim_path(claim.id).write_text(_yaml_dump(validated.model_dump(mode="json"))),
so any validator transformations are preserved and you avoid double-serializing
the original object.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d8ff8dfd-d140-40b8-9788-d75291902b43
📒 Files selected for processing (5)
CHANGELOG.mdsrc/vouch/models.pysrc/vouch/storage.pytests/test_bundle.pytests/test_storage.py
Comment on lines
+338
to
+343
| diff = bundle.import_check(store.kb_dir, bundle_path) | ||
| assert not diff.ok | ||
| assert any("schema validation failed" in i for i in diff.issues), diff.issues | ||
|
|
||
| with pytest.raises(RuntimeError, match="schema validation failed"): | ||
| bundle.import_apply(store.kb_dir, bundle_path) |
There was a problem hiding this comment.
Assert the citation-specific error, not only a generic schema failure.
The current checks can pass on unrelated schema errors. To keep this regression targeted to empty evidence, assert that the message also includes the citation invariant text (e.g., "cite at least one").
Suggested assertion tightening
diff = bundle.import_check(store.kb_dir, bundle_path)
assert not diff.ok
- assert any("schema validation failed" in i for i in diff.issues), diff.issues
+ assert any(
+ "schema validation failed" in i and "cite at least one" in i
+ for i in diff.issues
+ ), diff.issues
- with pytest.raises(RuntimeError, match="schema validation failed"):
+ with pytest.raises(RuntimeError, match=r"schema validation failed.*cite at least one"):
bundle.import_apply(store.kb_dir, bundle_path)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_bundle.py` around lines 338 - 343, Tighten the assertions to
verify the citation-specific error about empty evidence rather than any schema
failure: update the diff.issues check to assert that at least one issue string
contains both "schema validation failed" and the citation invariant text (e.g.,
"cite at least one"), and update the pytest.raises match to include both
substrings (or a regex matching "schema validation failed" and "cite at least
one") when calling bundle.import_apply; reference the existing call sites
bundle.import_check, diff.issues, and bundle.import_apply to locate the
assertions to change.
Comment on lines
+122
to
+129
| def test_put_claim_rejects_empty_evidence(store: KBStore) -> None: | ||
| """Regression for #81: store.put_claim is a direct write path | ||
| that used to silently accept Claim(evidence=[]) because the | ||
| only existence-check loop iterated zero times. The model-level | ||
| validator now fires before put_claim is even called.""" | ||
| with pytest.raises(ValidationError, match="cite at least one"): | ||
| store.put_claim(Claim(id="c1", text="uncited", evidence=[])) | ||
| assert not (store.kb_dir / "claims" / "c1.yaml").exists() |
There was a problem hiding this comment.
This test can pass without ever exercising store.put_claim validation.
On Line 128, Claim(id="c1", text="uncited", evidence=[]) raises before store.put_claim(...) is called, so this does not verify the write-path behavior. Use a valid claim, mutate evidence to [], then call store.put_claim to catch mutation-based bypasses.
Proposed test adjustment
def test_put_claim_rejects_empty_evidence(store: KBStore) -> None:
@@
- with pytest.raises(ValidationError, match="cite at least one"):
- store.put_claim(Claim(id="c1", text="uncited", evidence=[]))
+ src = store.put_source(b"e")
+ claim = Claim(id="c1", text="uncited", evidence=[src.id])
+ claim.evidence = []
+ with pytest.raises(ValidationError, match="cite at least one"):
+ store.put_claim(claim)
assert not (store.kb_dir / "claims" / "c1.yaml").exists()📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| def test_put_claim_rejects_empty_evidence(store: KBStore) -> None: | |
| """Regression for #81: store.put_claim is a direct write path | |
| that used to silently accept Claim(evidence=[]) because the | |
| only existence-check loop iterated zero times. The model-level | |
| validator now fires before put_claim is even called.""" | |
| with pytest.raises(ValidationError, match="cite at least one"): | |
| store.put_claim(Claim(id="c1", text="uncited", evidence=[])) | |
| assert not (store.kb_dir / "claims" / "c1.yaml").exists() | |
| def test_put_claim_rejects_empty_evidence(store: KBStore) -> None: | |
| """Regression for `#81`: store.put_claim is a direct write path | |
| that used to silently accept Claim(evidence=[]) because the | |
| only existence-check loop iterated zero times. The model-level | |
| validator now fires before put_claim is even called.""" | |
| src = store.put_source(b"e") | |
| claim = Claim(id="c1", text="uncited", evidence=[src.id]) | |
| claim.evidence = [] | |
| with pytest.raises(ValidationError, match="cite at least one"): | |
| store.put_claim(claim) | |
| assert not (store.kb_dir / "claims" / "c1.yaml").exists() |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/test_storage.py` around lines 122 - 129, The test currently constructs
Claim(id="c1", text="uncited", evidence=[]) which triggers model validation
before store.put_claim is called; change the test to construct a valid Claim
(e.g., evidence with at least one item), then mutate that Claim's evidence to an
empty list and call store.put_claim(…) expecting ValidationError; keep the final
assertion that (store.kb_dir / "claims" / "c1.yaml").exists() is false and
reference the test function name test_put_claim_rejects_empty_evidence and the
store.put_claim and Claim symbols when making the change.
Member
|
My question before merging: since the validator now also fires when claims are read back, a KB that already has an uncited claim from the old bug would start throwing on |
…hdev#82 review) The new Claim.evidence min-citation validator (vouchdev#81) also fires when claims are read back from disk. A KB that has a pre-existing uncited claims/<id>.yaml from before the fix would otherwise crash vouch lint / vouch doctor with a bare pydantic.ValidationError deep in store.list_claims(). Add _load_claims_for_lint(), a per-file iteration that catches pydantic.ValidationError (and any other load error) and surfaces each bad file as a Finding with code='invalid_claim' and an explicit repair hint: 'edit the YAML to add a citation, or delete the file'. lint() also stops calling status() to populate counts — status() calls the strict store.list_claims() which would re-raise on the same files — and builds the counts dict inline from the safely-loaded valid claims. Regression test in tests/test_health.py: - test_lint_surfaces_legacy_uncited_claim_yaml_without_crashing hand-crafts a claims/legacy.yaml with evidence: [] (matches the on-disk shape an older buggy write path would have left), asserts vouch lint runs to completion, surfaces invalid_claim in findings with the repair-hint message, and that the well-formed sibling claim is still discovered. CHANGELOG migration note expanded to describe the repair hint.
Contributor
Author
|
Good catch — agreed that a bare What changed in 5c881de:
The strict validator on the write paths is unchanged — every new Regression test ( CHANGELOG migration note updated to describe the repair hint. Full suite: ruff clean, 5/5 |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/vouch/health.py`:
- Around line 143-160: The counts dict mixes non-int types causing type errors:
remove or move "kb_dir" out of HealthReport.counts and ensure every value in
counts is an int (e.g., replace "index_present": (store.kb_dir /
index_db.DB_FILENAME).exists() with "index_present": int((store.kb_dir /
index_db.DB_FILENAME).exists()) and convert any other non-int entries
similarly); update construction near the end of the function that builds counts
and return HealthReport(ok=ok, findings=findings, counts=counts) (or add a
separate field on HealthReport for kb_dir if you need to keep it).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 415ab105-dd99-42ba-b9fb-86063ead19fa
📒 Files selected for processing (3)
CHANGELOG.mdsrc/vouch/health.pytests/test_health.py
✅ Files skipped from review due to trivial changes (1)
- CHANGELOG.md
The new inline-built counts in lint() (from 5c881de) is a literal dict with mixed value types (str/int/bool), which mypy correctly inferred as dict[str, object] and rejected against the HealthReport.counts: dict[str, int] annotation. The original status() returned the same mixed dict via an untyped 'dict' return, which masked the mismatch — the narrow type was effectively never checked at the call site. Widen counts to dict[str, Any] to match runtime reality. Also tighten status()'s return annotation from 'dict' to 'dict[str, Any]' for consistency. No caller does arithmetic on counts values; they just echo or pass through, so the widening is risk-free.
Member
|
Create PR against the test branch |
Contributor
Author
|
@plind-junior — retargeted to |
Member
|
Fix the conflict, other than that LGTM! |
Contributor
Author
Conflict resolved. |
jsdevninja
pushed a commit
to jsdevninja/vouch
that referenced
this pull request
May 28, 2026
…hdev#82 review) The new Claim.evidence min-citation validator (vouchdev#81) also fires when claims are read back from disk. A KB that has a pre-existing uncited claims/<id>.yaml from before the fix would otherwise crash vouch lint / vouch doctor with a bare pydantic.ValidationError deep in store.list_claims(). Add _load_claims_for_lint(), a per-file iteration that catches pydantic.ValidationError (and any other load error) and surfaces each bad file as a Finding with code='invalid_claim' and an explicit repair hint: 'edit the YAML to add a citation, or delete the file'. lint() also stops calling status() to populate counts — status() calls the strict store.list_claims() which would re-raise on the same files — and builds the counts dict inline from the safely-loaded valid claims. Regression test in tests/test_health.py: - test_lint_surfaces_legacy_uncited_claim_yaml_without_crashing hand-crafts a claims/legacy.yaml with evidence: [] (matches the on-disk shape an older buggy write path would have left), asserts vouch lint runs to completion, surfaces invalid_claim in findings with the repair-hint message, and that the well-formed sibling claim is still discovered. CHANGELOG migration note expanded to describe the repair hint.
3 tasks
This was referenced Jun 1, 2026
3 tasks
This was referenced Jun 3, 2026
3 tasks
joaovictor712
added a commit
to joaovictor712/vouch
that referenced
this pull request
Jun 16, 2026
Claim's four graph-reference fields — entities, supersedes, superseded_by, contradicts — were validated by no write path. put_claim checked only claim.evidence; update_claim re-validated the model (no KB access, so no ref check); bundle.import_apply writes claim YAML straight to disk. The vouchdev#124 graph-integrity fix closed Relation.source/target/evidence and Page.entities/sources but, as its own storage.py comment shows, skipped the Claim's own reference fields — even though fsck already declares dangling_supersedes / dangling_superseded_by / dangling_contradicts as error-severity findings. The invariant was articulated but enforced by no writer (same shape as vouchdev#81 / vouchdev#123). - storage.KBStore._validate_claim_refs: entities -> entity ids, supersedes/contradicts/superseded_by -> claim ids. Called from both put_claim and update_claim (the latter closes the in-place-mutation reach path, mirroring the vouchdev#82 re-validation fix). - bundle.import_check: extend the claim branch of the graph-integrity pass with the same checks, matching the existing page-ref checks, so a bundle can't land a dangling claim ref through import_apply's direct write. - Honest lifecycle writes are unaffected: supersede/contradict load both ends via get_claim before linking, so their refs always resolve. Regression tests in test_storage.py, test_bundle.py, and test_health.py; the fsck/CLI dangling-chain tests now write poisoned YAML directly to disk to reproduce the legacy on-disk state fsck must still surface. Closes vouchdev#196.
3 tasks
plind-junior
pushed a commit
that referenced
this pull request
Jul 6, 2026
the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (#81/#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes #155
plind-junior
added a commit
that referenced
this pull request
Jul 7, 2026
* fix(server): block path traversal in register_source_from_path kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes #10 * fix(cli): translate domain errors into clean ClickException output The CLI handlers for approve and reject caught only (ArtifactNotFoundError, ValueError) but proposals.approve() / proposals.reject() raise ProposalError -- a RuntimeError subclass. The four propose-* shortcuts caught nothing at all. Result: a double- approve, an empty rejection reason, an empty claim text, or an unknown source id surfaced a raw Python traceback instead of the one-line `Error: ...` the rest of the CLI emits. Add a small `_cli_errors` context manager that translates ArtifactNotFoundError / ValueError / ProposalError / LifecycleError into click.ClickException, and apply it to every command that calls into proposals, lifecycle, sessions, or storage. The MCP and JSONL servers already do the equivalent in their own envelopes; this brings the human-facing surface in line. Adds tests/test_cli.py with regressions for approve, reject, propose- claim, propose-entity, and show. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * fix(verify): catch ArtifactNotFoundError and OSError on stored read verify_source() caught FileNotFoundError, but store.read_source_content() raises ArtifactNotFoundError (a KeyError subclass) when the content blob is missing -- so the "stored content missing" graceful path never ran and a single broken source crashed the entire verify_all() sweep, breaking `vouch source verify` and `vouch doctor`. The same call can also raise OSError (permission denied, TOCTOU race between exists() and read_bytes(), underlying I/O error) which was likewise unhandled. Catch both: the existing "missing" path keeps its note, and OSError surfaces as "stored content unreadable: <reason>". Adds three regression tests: - missing-content blob -> graceful per-source failure - unreadable stored content (monkeypatched PermissionError) -> graceful per-source failure with the underlying reason in the note - mixed sweep with one good + one broken source -> verify_all returns both results instead of aborting at the first failure Fixes #30 * chore(ci): unblock lint/type/test gates so the CLI fix can land The CI workflow runs `ruff check`, `mypy`, then `pytest`, and the fix/cli-clean-domain-errors branch was failing all three on pre-existing main-branch issues unrelated to the CLI change itself. * ruff: add `from e` to the seven `raise ValueError(...) from FileExistsError` re-raises added by the recent exclusive-create guards in storage.put_* (B904), and let ruff re-sort the cli.py import block (I001). * mypy: narrow the `kind` value flowing into `ContextItem(type=...)` with a `Literal` cast so the strict-typed field accepts what the search backends actually return. * pytest: `session_end()` mutates a session that `session_start()` has already written, but `put_session()` now uses exclusive create and rejects the second write. Add `KBStore.update_session()` mirroring `update_claim` and have `session_end` call it; existing test_sessions coverage now passes again. No behavior change for the CLI surface — these are infrastructure fixes so the existing test_sessions / lint / type assertions pass on this branch. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * fix(server): close TOCTOU window between path check and read CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * docs: add banner diagram to README * docs(spec): semantic search as primary retrieval backend Approved design for feat/semantic-search. Embedding (sentence- transformers all-mpnet-base-v2) becomes the primary search backend with FTS5 as fallback. Synchronous-at-write indexing across all six artifact types (claim, page, source, entity, relation, evidence). Maximally functional scope (~3000 LOC): pluggable model adapter registry, sqlite-vec ANN with NumPy fallback, cross-encoder rerank, HyDE query expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG eval harness, and full CLI/MCP/JSONL parity. The writing-plans step will turn the rollout order in section 16 into concrete phased tasks. * docs(plan): semantic search implementation plan 32-task TDD plan for the semantic-search feature: foundation, storage, write hooks across all six artifact types, semantic-primary search integration in MCP/JSONL/CLI, RRF fusion + hybrid, cross-encoder rerank, HyDE expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG scorer, end-to-end integration test, and user docs. Each task: failing test, minimal implementation, passing test, commit. MockEmbedder test double keeps the unit suite fast; the real model is exercised only under @pytest.mark.integration. The evaluation module is named scorer.py rather than eval.py to avoid shadowing the Python builtin and to keep static analysers quiet; the user-facing CLI subcommand remains `vouch eval embedding` (the Click group is registered under the name "eval", with the Python identifier eval_group). * feat(embeddings): add optional-deps extras and pytest markers * feat(embeddings): create package skeleton * feat(embeddings): Embedder ABC, registry, content_hash, MockEmbedder * feat(embeddings): sentence-transformers all-mpnet-base-v2 default adapter * feat(embeddings): sentence-transformers MiniLM-L6 alternative adapter * feat(embeddings): fastembed BGE alternative (no-torch) adapter * fix(ci): satisfy ruff SIM105 + add mypy overrides for optional deps CI ran ruff which flagged SIM105 on the three try/except ImportError guard blocks in src/vouch/embeddings/__init__.py. Replace each with `contextlib.suppress(ImportError)` -- same semantics, satisfies ruff. Also add mypy overrides for numpy / sqlite_vec / sentence_transformers / fastembed so the type check passes in the base [dev] CI install where those optional extras aren't present. The embedding code paths that import these libraries are only reached when the extras are installed at runtime; for the static type check the missing stubs are noise. * fix(ci): skip embeddings test suite when numpy isn't installed CI runs `pip install -e '.[dev]'` which deliberately excludes the optional `[embeddings]` extras. tests/embeddings/test_*.py modules import numpy at top level, so pytest collection fails with ModuleNotFoundError before any test can be deselected. Add tests/embeddings/conftest.py with `pytest.importorskip("numpy")` so the entire embeddings test directory skips gracefully when numpy is absent. Once `pip install vouch[embeddings]` (or numpy itself) is present, the skip is a no-op and the tests run normally. * fix(bundle): skip Pydantic validation for opaque source content files `_validate_content` in bundle.py uses the path's first directory component to look up a Pydantic validator. For sources, both `sources/<sha>/meta.yaml` (the Source model) and `sources/<sha>/content` (raw opaque bytes) hit the same "sources" key, so the validator was being run on the raw content bytes and failing with "1 validation error for Source". Add an early return for any non-meta.yaml path under `sources/` so opaque content bytes are not Pydantic-validated. Only the Source metadata file is checked, which matches the original intent of the PR #13 validation. Unblocks three pre-existing test_bundle.py failures inherited from upstream main. * fix(embeddings): MockEmbedder uses uint32 scaling to avoid NaN/Inf Per CodeRabbit on PR #37: decoding sha256 chunks as raw float32 (via struct.unpack("<f", ...)) produced NaN/Inf for ~1 in 2^7 chunks because most 4-byte patterns aren't finite IEEE754 floats. Norm overflow then made unit normalization unreliable and broke cosine similarity asserts in downstream phases (the dedup test in Phase 6 already had to work around this with a custom _HashEmbedder). Decode each chunk as an unsigned 32-bit integer, scale to [-1.0, 1.0], accumulate in float64 for exact norm, and cast to float32 on return. Deterministic, finite, well-behaved. * feat(embeddings): state.db schema for embedding storage + put/get helpers * feat(embeddings): NumPy brute-force cosine search over embedding_index * feat(embeddings): sqlite-vec ANN path with NumPy fallback * feat(embeddings): query embedding LRU cache * style: fix ruff E402/SIM105/E702/I001 violations from Phase 2 storage tasks * fix(embeddings): use outer-query for WHERE on cosine alias in search_embedding Per CodeRabbit Critical on PR #39/40/43: SQLite does not allow a column alias defined in the SELECT list to be referenced in the WHERE clause of the same query. The old form SELECT ... 1.0 - vec_distance_cosine(vec, ?) AS score ... WHERE ... AND score >= ? raised `no such column: score` at runtime; the catch-all `except sqlite3.OperationalError` silently swallowed it and made every call fall through to the NumPy brute-force path, which works but defeats the whole point of installing sqlite-vec. Wrap the projection in a subquery so the WHERE clause sees the alias. No behavior change for callers that have already been using the fallback (NumPy still produces correct results); the ANN code path now actually runs when sqlite-vec is loaded. * feat(embeddings): write-time embedding hook + wire put_claim * feat(embeddings): search_semantic wrapper with query cache * feat(embeddings): hook all six artifact write paths * feat(embeddings): kb_search defaults to embedding primary, fts5 fallback * feat(embeddings): re-embed on update_claim * feat(embeddings): JSONL _h_search parity with MCP semantic-primary dispatch * style: fix ruff SIM105/E501/I001/RUF059 violations from Phase 3 * fix(ci): silence mypy import-untyped for forward-ref fusion import The hybrid backend branch in kb_search (server.py) and _h_search (jsonl_server.py) lazily imports vouch.embeddings.fusion (added in Phase 5). On Phase 4's branch that module doesn't exist yet, so mypy emits import-untyped. Mark each import with `# type: ignore` and let ruff auto-format the line. * fix(ci): silence mypy import-untyped for forward-ref dedup import The KBStore._embed_and_store helper lazily imports vouch.embeddings.dedup (added in Phase 6). On Phase 3's branch that module does not yet exist, so mypy treats it as untyped and errors out. Mark the import with `# type: ignore[import-not-found,import-untyped, unused-ignore]` so the type check passes here and stays silent once Phase 6 lands. * spec: cut dated 2026-05-21 snapshot; promote schema generator to script * ci(mypy): collapse fusion import to single line for type-ignore * feat(embeddings): RRF/weighted/normalized fusion strategies * docs: add 'What ships today' table to README * kb: approve review-gate fact * docs: add 'When to use vouch' section * fix(cli): honor VOUCH_AGENT in _whoami for consistent actor attribution The CLI propose-* commands and most actor sites called _whoami(), which only checked VOUCH_USER and the OS user — VOUCH_AGENT was silently ignored. The MCP and JSONL servers (server.py, jsonl_server.py) and session_start already honour VOUCH_AGENT, so multi-agent attribution broke as soon as you used the CLI to propose. Prefer VOUCH_AGENT over VOUCH_USER over the OS user so the recorded proposed_by / audit actor matches across all three transports. * docs: add captured example session walkthrough Real propose -> review -> commit -> retrieve loop captured from a sandbox run on 2026-05-21. Includes on-disk YAML, audit log lines, and an honest note about the literal substring backend limitation pre-embeddings. * docs: add screen recording of full vouch loop (VHS tape + GIF) * docs: embed demo GIF under Quick start in README * fix(embeddings): address CodeRabbit review on hybrid + fusion + write hook Bundles the substantive correctness fixes flagged by CodeRabbit on PR #41. CI was already green; these are latent bugs the review caught. storage._embed_and_store - Re-embed when the active embedder model changes, not only when the content hash changes. The previous short-circuit kept stale vectors on disk after a model swap, mixing document embeddings from one model with queries from another. - Truly best-effort: catch any exception during encode/put/meta/dedup so a hook failure can't bubble back to the caller and leave an artifact persisted-but-API-errored. index_db.reset - Also clears embedding_index, query_embedding_cache, embedding_dupes, and embedding_* keys from index_meta. Otherwise `vouch index` left orphaned hits behind after artifacts were removed. index_db.search_embedding (Python fallback) - Normalize the stored vector by its own L2 norm before scoring, so rankings match the sqlite-vec `1 - vec_distance_cosine` path. The previous raw dot product made magnitude leak into the score. index_db.search_semantic - Invalidate the query-vector cache entry when the embedder dim no longer matches what's on disk; otherwise a model swap returned stale vectors living in the wrong space. server.kb_search + jsonl_server._h_search (hybrid branch) - Hybrid now passes min_score into search_semantic (consistent with embedding/auto branches) and wraps the FTS lookup in try/except so an FTS5 error doesn't fail the whole request. - JSONL transport rejects unknown backend values with a clear error instead of silently returning []; matches MCP transport behavior. embeddings.fusion - rrf_fuse validates limit >= 0 and k >= 0 (k = -rank would divide by zero). weighted_fuse validates limit >= 0. Tests - Three new fusion guard tests cover the negative-limit and negative-k paths. * fix(lifecycle): only catch missing-artifact errors in cite() the bare except was treating every error as a missing citation, which hid real bugs. narrow it to ArtifactNotFoundError so unexpected failures actually surface. * fix(context): only catch missing-artifact errors in citation lookup same issue as cite() — a bare except was hiding real errors behind an empty citation list. narrow it to ArtifactNotFoundError. * refactor(sessions): keep tracebacks when crystallize() approve fails we were dropping the traceback and only stringifying the error, which made it painful to diagnose anything other than ProposalError. now logs via logger.exception and includes error_type in the failures list. * fix(context): only catch sqlite errors when FTS5 falls back the retrieval helper had a bare except that would also swallow bugs in the substring fallback path or in our own code. narrow it to sqlite3.Error so only the intended cases (FTS5 missing, db missing, schema mismatch) trigger the fallback. * refactor(health): use typed status filter for pending proposal count we were listing every proposal and comparing p.status.value == 'pending' as a string. list_proposals() already takes a ProposalStatus filter — use it. shorter, type-checked, and avoids the stringly-typed compare. * chore(capabilities): pin pluginApi compat baseline, fail CI on host-compat drift * feat(dual-solve): surface engine logs * fix(sessions): make crystallize retry idempotent for summary pages Partial crystallize runs that already wrote session-{id} summary pages failed on retry with "page already exists". Upsert the summary page and derive its artifact list from all approved session proposals. Fixes #139 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(models): reject empty text/name/title on claim/entity/page the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (#81/#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes #155 * refactor(models): extract shared _require_non_empty validator helper address review on #300: the three #155 field validators (Claim.text / Entity.name / Page.title) were structurally identical strip-checks. extract a module-level `_require_non_empty(v, label)` helper they all delegate to, keeping the per-field error messages. also fix a mid-entry sentence casing in the CHANGELOG. no behaviour change. * fix: resolve RPC traceback leakage on internal errors Unexpected exceptions in handle_request() included full stack traces in the JSON error envelope, exposing paths and internals to HTTP /rpc clients. Log server-side and return message only. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(review): kb.triage_pending — advisory triage scoring for the pending queue a long `kb.list_pending` forces the reviewer to reconstruct, per proposal, whether the claim fits the existing kb, whether its citations resolve, whether it duplicates something already filed, and whether it contradicts an approved claim. this adds an optional triage pass that scores each pending proposal on those four signals and attaches a `_meta.vouch_triage` block (recommendation/score/signals/rationale) to help a reviewer prioritize, without ever deciding anything itself. read-only by construction: the pass never calls proposals.approve/reject, store.put_*, or store.move_proposal_to_decided — a human still calls kb.approve/kb.reject. citation_quality reuses proposals._payload_block_reason; duplication_risk reuses the propose-time embedding similarity path (embeddings.similarity.find_similar_on_propose) and degrades to a difflib heuristic when the embeddings extra isn't installed. fit uses a separate, lower-threshold embedding search so a near-duplicate hit doesn't also inflate fit and cancel out its own duplication penalty. opt-in via `triage.enabled: true` in config.yaml (default false). registered at all four kb.* surface sites (server.py, jsonl_server.py, capabilities.py, cli.py) plus `vouch triage [proposal-id...]` with `--json` and `--reverse`. * feat(diff): register kb.diff at all four kb.* surface sites vouch diff shipped CLI-only in 0.1.0, an explicit non-goal at the time ("MCP/JSONL parity ... kb.* surface unchanged"). that leaves it the only read method skipping the four-site registration convention documented in CLAUDE.md, so agents talking MCP/JSONL have no way to fetch a revision diff. adds the kb_diff MCP tool, the kb.diff JSONL handler, and the capabilities.METHODS entry, next to the other by-id read tools (kb_read_claim/kb_read_page) with the same unrestricted-read posture. also makes new_id optional for a superseded claim: diff_artifacts and the CLI both resolve it from superseded_by when omitted, erroring clearly when there's no successor (pages still require an explicit new_id — they have no successor pointer). closes #327. * fix(jsonl): define module logger after the import block #361 inserted `_log = logging.getLogger(...)` between two import groups in jsonl_server.py. that statement-among-imports trips ruff's E402 on every import that follows it, so `ruff check src tests` — the lint gate in ci — now fails on the whole repo. move the logger definition below the imports (where storage.py already keeps its own module logger); no behaviour change. * feat(adapters): toml_merge install strategy for codex config.toml .codex/config.toml is codex's primary config file, so the plain-copy path silently skipped any project where codex was already configured and vouch never got wired. add a toml_merge entry flag mirroring json_merge: parse the existing destination with tomllib, deep-merge the template's tables into it (existing user values always win on conflict), and write back. flip the codex T1 entry to toml_merge. writing uses a minimal hand-rolled serializer (tables, arrays, inline tables in arrays, scalars, datetimes) so the dependency set stays unchanged; the output must survive a tomllib round-trip back to the merged data, and anything the serializer can't faithfully re-emit degrades to skipped rather than risking the user's config. closes #384 * feat(adapters): codex T2 agents.md fenced snippet codex reads AGENTS.md for project instructions the way cursor does, but the codex adapter stopped at T1, so a codex session got the kb tools with no standing guidance on recall-first or the review gate. ship adapters/codex/AGENTS.md.snippet as a T2 tier with the standard fence markers, kept in lockstep with cursor's snippet modulo the host name (enforced by a sync test). also close the edited-fence gap in _install_fenced per the ticket's acceptance criteria: a fence body that drifted from the shipped snippet is replaced within the markers (reported as merged) instead of being skipped forever, while user content outside the fence stays untouched. a begin marker without an end marker is treated as corrupt and left alone. closes #385 * feat(adapters): codex T3 skills mirroring the vouch slash commands claude-code T3 ships nine slash commands; codex users got none of the guided flows. ship them as codex skills under the project-local .codex/skills/ as a T3 tier. scope decision the ticket asked to resolve first: codex loads custom prompts only from ~/.codex/prompts/ (user-global) and has deprecated them upstream in favour of skills, which do have a project-local home at <project>/.codex/skills/ in trusted projects. the #179 rule forbids a project-scoped install from touching home-directory state, so prompts are out and skills are in — recorded in the manifest comment and the adapter readme, with a manual-copy pointer for users who still want ~/.codex/prompts. the SKILL.md files are referenced from the openclaw mirror rather than duplicated (same reuse pattern openclaw itself applies to the claude-code commands), so one edit updates every host. a parametrized sync test asserts each installed codex skill body equals the matching claude-code command body, and a scope test asserts every installed path stays under the project target. closes #386 * feat(capture): ingest codex session rollouts into review-gated summaries session auto-capture was claude-code-only: hooks drive capture observe and capture finalize live. codex has no hook stream, but it persists every session as a rollout jsonl under $CODEX_HOME/sessions containing user messages, tool calls, and outputs — everything the existing rollup needs, just after the fact. new cli command `vouch capture ingest-codex [<rollout> | --latest]` parses one rollout through a small dedicated parser (codex_rollout.py) that maps function_call records into the same observation shape capture.observe produces — shell commands with failure detection from exit codes, apply_patch heredocs surfaced as file edits, mcp tools under their own names, session mechanics skipped — then reuses the existing build_summary_body -> propose_page rollup. one code path from observation to proposal, two front doors. the rollout format is not a stable public contract: unknown record types are tolerated, and unreadable, compressed, or meta-less files degrade to a CodexRolloutError with an actionable message and a non-zero exit, never a stack trace. re-ingesting a session is a no-op keyed on the rollout's session id; --latest resolves the newest rollout whose recorded cwd matches the current project. proposals are attributed to the codex actor (VOUCH_AGENT wins when set), respect capture's enabled/min_observations config, and never touch approve(). fixture rollouts use placeholder data only, enforced by a test. closes #387 * feat(adapters): codex T4 hook wiring for automatic session capture with `vouch capture ingest-codex` available, codex can self-capture the way claude-code T4 does. the ticket assumed the `notify` setting in config.toml, but codex only honours notify in user-global config — it is a restricted key in project-local layers — and the #179 rule forbids touching ~/.codex. codex's hooks system is the project-local equivalent: `.codex/hooks.json` (loaded in trusted projects) fires Stop when a turn completes, with a payload carrying the session id and transcript path. T4 ships that file through json_merge, so an existing user hooks.json is deep-merged into, never overwritten, and re-runs don't duplicate the hook. the handler is a new --hook mode on ingest-codex: read the stop payload from stdin, resolve the session's rollout (transcript_path when present, else by session id in the rollout filename), ingest idempotently, and exit 0 no matter what — the same never-break-the- host rule capture observe follows. stop fires per turn, not per session end, so the finalize policy the ticket left open is resolved as idempotent re-ingest: the dedup guard now refreshes a session's still-PENDING proposal in place when the rollout grew (same id, updated summary, audit-logged), reports an unchanged rollout as a no-op, and never resurrects a proposal a human already decided. storage gains update_proposal, a pure-I/O in-place rewrite of a pending proposal file. wiring only — everything still lands through propose_page; nothing touches approve(). closes #388 * test(adapters): live codex install gate against the real cli tests/test_openclaw_plugin_load_real.py set the pattern: exercise the real host cli when it's on PATH, skip cleanly where it isn't. the codex adapter had no equivalent, so regressions in the shipped config only surfaced when a user hit them — the old T1 silent-skip no-op is exactly the class of bug a live gate would have caught. the new suite installs the adapter into a temp project at T4, marks the project trusted inside an isolated CODEX_HOME, and asserts through `codex mcp list --json` that the vouch server is visible: next to a pre-seeded unrelated server on the merge path, alone on the fresh path, and absent for an untrusted project (the config must stay inert where codex says it does). a full-tier check covers the fenced AGENTS.md, the skills, and the Stop hook, and an isolation check pins every written artifact under the temp project. assertions target the observable contract — server listed, config parses, snippet present — not codex internals, so version bumps shouldn't break the suite. no network, no credentials, never touches the real ~/.codex; runs in the normal pytest invocation. closes #389 * docs(adapters): codex adapter docs reflect the tiered install three surfaces went stale once the codex tier work landed. the adapters table row described codex as a one-file config.toml snippet; it now lists the tiered install. the codex adapter readme led with the manual ~/.codex edit; it now leads with `vouch install-mcp codex`, explains the project-local .codex/config.toml choice and the trust requirement, summarizes what each tier adds (mcp wire / AGENTS.md snippet / skills / capture hook) with the merge-safety guarantees, and keeps the manual edit as an explicit fallback section. getting-started treated codex as a one-liner aside inside the claude instructions; the install section now shows both hosts side by side. the prompts-location decision from the T3 ticket and the notify restriction from the T4 ticket stay documented in their sections. no code; placeholder examples only. closes #390 * docs(mcp): design a friendlier mcp surface — profiles + auto-recall the closest competitor (pmb) exposes 10 mcp tools by default and hides the rest behind a profile flag; vouch shows all 58 every turn, which is the main first-touch friendliness gap. this spec adds a minimal-by-default tool profile, fusion-by-default retrieval, a per-prompt auto-recall hook, and compact tool descriptions. all of it is presentation-only — the review gate, the yaml storage, and the protocol method surface are untouched. it also closes the current 2-of-3 surface-parity gap as a bonus. * docs(mcp): plan the friendlier-mcp-surface build six TDD tasks: tool profiles (minimal default), real mcp↔methods parity, fusion-by-default retrieval, near-duplicate drop, per-prompt auto-recall hook, and compact tool descriptions. reconcile the spec: minimal is 8 tools (kb_capabilities kept as the discovery escape hatch) and the recency multiplier is deferred (per-hit timestamps aren't plumbed through _retrieve). * feat(mcp): add tool profiles, expose a minimal surface by default agents saw all 58 kb.* tools every turn — the main first-touch friendliness gap vs pmb, which exposes ~10 by default. add a profile layer applied in run_stdio: minimal (8 core tools) by default, standard (16), or full (58), selected by VOUCH_TOOL_PROFILE / config mcp.tool_profile. exposure only — the protocol surface, jsonl/cli, and the review gate are unchanged. * fix(mcp): harden profile resolution against a non-dict config section a bare `mcp:` key in config.yaml parses to None in YAML, and a non-dict mcp section is also possible from hand-edited configs. the previous config.get("mcp", {}).get("tool_profile") chain only supplied the {} default when the key was absent, so a present-but-None or non-dict value passed through and the chained .get raised AttributeError. run_stdio calls resolve_profile_name outside its try/except, so this would crash vouch serve instead of degrading to the minimal default. guard each nesting level with isinstance, matching the established reflex_cfg pattern in salience.py. * test(capabilities): assert mcp tool set matches the method list the parity test only compared capabilities.METHODS to the jsonl handlers; the 58 mcp tools were never checked, so mcp drift passed ci. enumerate the unfiltered server tools and assert they equal METHODS — the real 3-surface check for the mcp side. * feat(retrieval): fuse embedding + fts5 by default instead of a waterfall _retrieve tried embedding, then fts5, then substring, returning the first non-empty list — so lexical and semantic hits never combined. auto and hybrid now fuse both retrievers with the already-built rrf_fuse and tag hits "hybrid"; explicit embedding/fts5/substring pins are unchanged. existing KBs (config says "auto") benefit with no migration. * fix(volunteer): treat hybrid relevance as rank-relative, not pre-normalized normalize_relevance() grouped "hybrid" with "embedding", returning the raw score unclamped-by-batch on the assumption it was already a 0-1 similarity. now that _retrieve's auto/hybrid path tags fused hits "hybrid" with a rrf_fuse score (bounded by ~2/(k+rank), typically 0.01-0.03), that assumption stopped holding: every fused relevance landed far below DEFAULT_THRESHOLD (0.85), so kb.volunteer_context stopped firing for any KB using the new default backend. batch-normalize hybrid like fts5 and substring instead — restores the confidence-gated push channel (#236). * feat(retrieval): drop near-duplicate items from the context pack a fused pack could surface the same fact from two claims. add a cheap greedy jaccard pass (>=0.85 over the first 40 tokens) that keeps the highest-scored of a near-duplicate cluster, so an agent never reads the same thing twice. * fix(retrieval): dedupe by score so the highest-scored near-dup wins _dedupe_near_duplicates assumed its input arrived in descending-score order, which only holds for the plain retrieval path. when build_context_pack runs with expand_graph=True, graph neighbours are appended in bfs order with decayed scores after the sorted main items, so a higher-scored neighbour could be dropped in favor of an earlier, lower-scored main item. sort inside the function so the invariant holds regardless of caller order. * feat(hooks): inject per-prompt kb context via a userpromptsubmit hook recall used to be either a session-start firehose or an explicit tool call. add a pure, never-raising helper + a hidden `vouch context-hook` command that reads the host's prompt payload on stdin and prints an additionalContext envelope, and wire it into the claude-code adapter's UserPromptSubmit hook — so relevant, approved knowledge is injected every turn with zero tool calls. * fix(hooks): never raise on non-dict payload or missing kb a reviewer reproduced two crash paths that violated task 5's own contract that the prompt hook must never block a turn: build_claude_prompt_hook raised AttributeError on non-dict JSON (null/number/bool/array/string all decode fine but lack .get), and the context-hook CLI command called _load_store(), whose sys.exit(2) on a missing KB is a SystemExit that slips past `except Exception` and exits the process nonzero. guard the payload with an isinstance check before .get, log a breadcrumb when build_context_pack fails so a broken KB isn't silently indistinguishable from "no hits", and swap _load_store() for the existing non-exiting _capture_store() helper so the command has no sys.exit on its path at all. * feat(mcp): serve one-line tool descriptions under non-full profiles full docstrings for every exposed tool are paid on every turn. under minimal and standard, trim each tool's description to its first line (full keeps the complete docstrings), cutting the per-turn context cost. * docs(mcp): document tool profiles note VOUCH_TOOL_PROFILE / mcp.tool_profile (default minimal, values minimal|standard|full) in the transports reference and the claude-code adapter guide; this repo has no mintlify/ tree yet so both live under docs/ and adapters/claude-code/ instead. * fix(retrieval): dedupe by score but keep caller order for the pack _dedupe_near_duplicates sorted by score and returned that score-sorted order, which fought graph-expansion: build_context_pack appends decayed-score neighbours after the ranked hits, and fused primary hits now carry small rrf scores. re-sorting let an irrelevant depth-2 neighbour outrank a real match, and the max_chars tail-pop then evicted the real match instead of the neighbour. keep the keep-decision in descending-score order so the highest-scored member of a near-duplicate cluster still survives, but return survivors in the caller's original order so budget eviction drops the tail (appended neighbours), not the ranked hits. * docs(changelog): note minimal mcp profile, fusion, auto-recall summarize this branch's user-visible changes under [Unreleased]: the minimal-by-default mcp tool profile, rrf fusion for auto/hybrid retrieval, and the per-prompt auto-recall hook in the claude-code adapter. * docs(readme): note the fourth hook, per-prompt recall, and lean mcp profile install-mcp now writes a fourth hook (UserPromptSubmit -> vouch context-hook), and recall fires per prompt, not only at session start. also note the kb.* mcp surface defaults to a lean profile that widens via VOUCH_TOOL_PROFILE. * fix(capabilities): read openclaw.compat from package.json (#417) the host-compat drift check (#237) read openclaw.compat.pluginApi from openclaw.plugin.json. that block moved to package.json when the plugin packaging was split, and openclaw.* is now banned from the manifest (enforced by test_manifest_carries_no_dead_dialect_fields). the #237 reader was left pointing at the old, now-empty location, so host_compat silently degraded to {} and both host-compat assertions failed on the test branch — which every open pr targeting test inherited, including #413. repoint _load_host_compat and the test helpers at package.json. the openclaw.compat.pluginApi key path is identical in both files, so this is purely a source-file change with no behaviour difference. * chore(repo): untrack owner-local web/ and .claude/ (#413) these two directories were committed by accident and reach every first-time contributor on clone. neither is project source: - web/ is the netlify-deployed marketing landing page. it is shipped via the netlify cli and is not meant to live in the repo tree. - .claude/ is a personal claude code config: a settings.json with owner-specific hooks, plus command files that are byte-identical duplicates of adapters/claude-code/.claude/commands/. untrack both (files stay on disk) and gitignore them, anchored to the repo root so src/vouch/web and adapters/claude-code/.claude stay tracked. also ignore coverage.xml and the .netlify/ and .playwright-mcp/ tool-scratch dirs. * chore(merge): resolve changelog and version conflicts with main * chore(merge): update changelog to resolve merge conflicts * chore(merge): update to main's latest versions for clean merge --------- Co-authored-by: plind-junior <138900956+dripsmvcp@users.noreply.github.com> Co-authored-by: alpurkan17 <alpurkan9@gmail.com> Co-authored-by: Tet-9 <igeparallax@gmail.com> Co-authored-by: Yaroslav98214 <diakovichyaroslav30@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: minion1227 <romantymkiv1999@gmail.com> Co-authored-by: RealDiligent <brave.challenge007@gmail.com> Co-authored-by: jsdevninja <topit89807@gmail.com>
plind-junior
added a commit
that referenced
this pull request
Jul 7, 2026
* fix(server): block path traversal in register_source_from_path kb.register_source_from_path read the contents of any file the process could access, letting an agent register /etc/passwd, ~/.ssh/id_rsa, ~/.aws/credentials etc. as a "source" and then retrieve the bytes via kb.cite or kb.list_sources. Resolve the path (following symlinks) and require it to be inside the KB root before reading. Adds KBStore.resolve_under_root() so both the JSONL handler and the MCP tool share one containment check. Fixes #10 * fix(cli): translate domain errors into clean ClickException output The CLI handlers for approve and reject caught only (ArtifactNotFoundError, ValueError) but proposals.approve() / proposals.reject() raise ProposalError -- a RuntimeError subclass. The four propose-* shortcuts caught nothing at all. Result: a double- approve, an empty rejection reason, an empty claim text, or an unknown source id surfaced a raw Python traceback instead of the one-line `Error: ...` the rest of the CLI emits. Add a small `_cli_errors` context manager that translates ArtifactNotFoundError / ValueError / ProposalError / LifecycleError into click.ClickException, and apply it to every command that calls into proposals, lifecycle, sessions, or storage. The MCP and JSONL servers already do the equivalent in their own envelopes; this brings the human-facing surface in line. Adds tests/test_cli.py with regressions for approve, reject, propose- claim, propose-entity, and show. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * fix(verify): catch ArtifactNotFoundError and OSError on stored read verify_source() caught FileNotFoundError, but store.read_source_content() raises ArtifactNotFoundError (a KeyError subclass) when the content blob is missing -- so the "stored content missing" graceful path never ran and a single broken source crashed the entire verify_all() sweep, breaking `vouch source verify` and `vouch doctor`. The same call can also raise OSError (permission denied, TOCTOU race between exists() and read_bytes(), underlying I/O error) which was likewise unhandled. Catch both: the existing "missing" path keeps its note, and OSError surfaces as "stored content unreadable: <reason>". Adds three regression tests: - missing-content blob -> graceful per-source failure - unreadable stored content (monkeypatched PermissionError) -> graceful per-source failure with the underlying reason in the note - mixed sweep with one good + one broken source -> verify_all returns both results instead of aborting at the first failure Fixes #30 * chore(ci): unblock lint/type/test gates so the CLI fix can land The CI workflow runs `ruff check`, `mypy`, then `pytest`, and the fix/cli-clean-domain-errors branch was failing all three on pre-existing main-branch issues unrelated to the CLI change itself. * ruff: add `from e` to the seven `raise ValueError(...) from FileExistsError` re-raises added by the recent exclusive-create guards in storage.put_* (B904), and let ruff re-sort the cli.py import block (I001). * mypy: narrow the `kind` value flowing into `ContextItem(type=...)` with a `Literal` cast so the strict-typed field accepts what the search backends actually return. * pytest: `session_end()` mutates a session that `session_start()` has already written, but `put_session()` now uses exclusive create and rejects the second write. Add `KBStore.update_session()` mirroring `update_claim` and have `session_end` call it; existing test_sessions coverage now passes again. No behavior change for the CLI surface — these are infrastructure fixes so the existing test_sessions / lint / type assertions pass on this branch. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * fix(server): close TOCTOU window between path check and read CodeRabbit on PR #28 noted that resolve_under_root() only validated a pathname snapshot. Callers then re-opened the same name with .is_file() and .read_bytes(), so an attacker who can swap the resolved path for a symlink between the containment check and the read can still exfiltrate an out-of-root file via kb.register_source_from_path. Collapse the validate-then-read into a single trusted helper `KBStore.read_under_root(path)` that returns `(resolved, bytes)`: 1. Path.resolve() chases pre-existing symlinks and the resulting target is checked for containment (existing behaviour -- legitimate in-root symlinks still work). 2. The read goes through `os.open(resolved, O_RDONLY | O_NOFOLLOW)` so a fresh symlink placed at the resolved name *after* the check fails with ELOOP rather than following the swap. 3. `fstat` + `S_ISREG` rejects directories / device nodes / pipes atomically on the same fd, replacing the racy `is_file()` test the callers used to do. Both register_source_from_path handlers (MCP + JSONL) switch to the new helper and drop their now-redundant follow-up checks. Adds tests: - symlink swapped into the resolved name -> rejected via ELOOP - directory at a valid path -> rejected via S_ISREG Existing "outside the root" and "inside the root" tests still pass. * chore(lint): chain FileExistsError cause in storage put_ methods ruff B904 requires `raise ... from err` inside an except block so the original exception is preserved on the chained __cause__. Without it, the CI lint step rejects the file. The seven sites are pre-existing (put_claim, put_page, put_entity, put_relation, put_evidence, put_session, put_proposal); this PR inherited the failure from main rather than introducing it, but the path traversal fix cannot land until lint is green, so the cleanup happens here. No behavior change: the chained exception carries .__cause__ but the public ValueError message and type are unchanged. * chore(ci): unblock mypy + test gates for path-traversal PR The B904 lint failures were already addressed in 2a439a5, but the CI matrix still fails on two pre-existing main-branch issues: * mypy: context.py:64 passed `kind: str` into ContextItem.type, which is Literal["claim","page","entity","relation","source"]. Narrow it with a typing.cast so strict mode accepts the search-backend output. * pytest: session_end() mutates a session that session_start() already wrote, but put_session() switched to exclusive create and rejects the second write. Add KBStore.update_session() mirroring update_claim and have session_end call it; test_sessions coverage passes again. No behavior change for the path-traversal fix itself. * docs: add banner diagram to README * docs(spec): semantic search as primary retrieval backend Approved design for feat/semantic-search. Embedding (sentence- transformers all-mpnet-base-v2) becomes the primary search backend with FTS5 as fallback. Synchronous-at-write indexing across all six artifact types (claim, page, source, entity, relation, evidence). Maximally functional scope (~3000 LOC): pluggable model adapter registry, sqlite-vec ANN with NumPy fallback, cross-encoder rerank, HyDE query expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG eval harness, and full CLI/MCP/JSONL parity. The writing-plans step will turn the rollout order in section 16 into concrete phased tasks. * docs(plan): semantic search implementation plan 32-task TDD plan for the semantic-search feature: foundation, storage, write hooks across all six artifact types, semantic-primary search integration in MCP/JSONL/CLI, RRF fusion + hybrid, cross-encoder rerank, HyDE expansion, ingest-time duplicate detection, model-identity migration, recall/MRR/nDCG scorer, end-to-end integration test, and user docs. Each task: failing test, minimal implementation, passing test, commit. MockEmbedder test double keeps the unit suite fast; the real model is exercised only under @pytest.mark.integration. The evaluation module is named scorer.py rather than eval.py to avoid shadowing the Python builtin and to keep static analysers quiet; the user-facing CLI subcommand remains `vouch eval embedding` (the Click group is registered under the name "eval", with the Python identifier eval_group). * feat(embeddings): add optional-deps extras and pytest markers * feat(embeddings): create package skeleton * feat(embeddings): Embedder ABC, registry, content_hash, MockEmbedder * feat(embeddings): sentence-transformers all-mpnet-base-v2 default adapter * feat(embeddings): sentence-transformers MiniLM-L6 alternative adapter * feat(embeddings): fastembed BGE alternative (no-torch) adapter * fix(ci): satisfy ruff SIM105 + add mypy overrides for optional deps CI ran ruff which flagged SIM105 on the three try/except ImportError guard blocks in src/vouch/embeddings/__init__.py. Replace each with `contextlib.suppress(ImportError)` -- same semantics, satisfies ruff. Also add mypy overrides for numpy / sqlite_vec / sentence_transformers / fastembed so the type check passes in the base [dev] CI install where those optional extras aren't present. The embedding code paths that import these libraries are only reached when the extras are installed at runtime; for the static type check the missing stubs are noise. * fix(ci): skip embeddings test suite when numpy isn't installed CI runs `pip install -e '.[dev]'` which deliberately excludes the optional `[embeddings]` extras. tests/embeddings/test_*.py modules import numpy at top level, so pytest collection fails with ModuleNotFoundError before any test can be deselected. Add tests/embeddings/conftest.py with `pytest.importorskip("numpy")` so the entire embeddings test directory skips gracefully when numpy is absent. Once `pip install vouch[embeddings]` (or numpy itself) is present, the skip is a no-op and the tests run normally. * fix(bundle): skip Pydantic validation for opaque source content files `_validate_content` in bundle.py uses the path's first directory component to look up a Pydantic validator. For sources, both `sources/<sha>/meta.yaml` (the Source model) and `sources/<sha>/content` (raw opaque bytes) hit the same "sources" key, so the validator was being run on the raw content bytes and failing with "1 validation error for Source". Add an early return for any non-meta.yaml path under `sources/` so opaque content bytes are not Pydantic-validated. Only the Source metadata file is checked, which matches the original intent of the PR #13 validation. Unblocks three pre-existing test_bundle.py failures inherited from upstream main. * fix(embeddings): MockEmbedder uses uint32 scaling to avoid NaN/Inf Per CodeRabbit on PR #37: decoding sha256 chunks as raw float32 (via struct.unpack("<f", ...)) produced NaN/Inf for ~1 in 2^7 chunks because most 4-byte patterns aren't finite IEEE754 floats. Norm overflow then made unit normalization unreliable and broke cosine similarity asserts in downstream phases (the dedup test in Phase 6 already had to work around this with a custom _HashEmbedder). Decode each chunk as an unsigned 32-bit integer, scale to [-1.0, 1.0], accumulate in float64 for exact norm, and cast to float32 on return. Deterministic, finite, well-behaved. * feat(embeddings): state.db schema for embedding storage + put/get helpers * feat(embeddings): NumPy brute-force cosine search over embedding_index * feat(embeddings): sqlite-vec ANN path with NumPy fallback * feat(embeddings): query embedding LRU cache * style: fix ruff E402/SIM105/E702/I001 violations from Phase 2 storage tasks * fix(embeddings): use outer-query for WHERE on cosine alias in search_embedding Per CodeRabbit Critical on PR #39/40/43: SQLite does not allow a column alias defined in the SELECT list to be referenced in the WHERE clause of the same query. The old form SELECT ... 1.0 - vec_distance_cosine(vec, ?) AS score ... WHERE ... AND score >= ? raised `no such column: score` at runtime; the catch-all `except sqlite3.OperationalError` silently swallowed it and made every call fall through to the NumPy brute-force path, which works but defeats the whole point of installing sqlite-vec. Wrap the projection in a subquery so the WHERE clause sees the alias. No behavior change for callers that have already been using the fallback (NumPy still produces correct results); the ANN code path now actually runs when sqlite-vec is loaded. * feat(embeddings): write-time embedding hook + wire put_claim * feat(embeddings): search_semantic wrapper with query cache * feat(embeddings): hook all six artifact write paths * feat(embeddings): kb_search defaults to embedding primary, fts5 fallback * feat(embeddings): re-embed on update_claim * feat(embeddings): JSONL _h_search parity with MCP semantic-primary dispatch * style: fix ruff SIM105/E501/I001/RUF059 violations from Phase 3 * fix(ci): silence mypy import-untyped for forward-ref fusion import The hybrid backend branch in kb_search (server.py) and _h_search (jsonl_server.py) lazily imports vouch.embeddings.fusion (added in Phase 5). On Phase 4's branch that module doesn't exist yet, so mypy emits import-untyped. Mark each import with `# type: ignore` and let ruff auto-format the line. * fix(ci): silence mypy import-untyped for forward-ref dedup import The KBStore._embed_and_store helper lazily imports vouch.embeddings.dedup (added in Phase 6). On Phase 3's branch that module does not yet exist, so mypy treats it as untyped and errors out. Mark the import with `# type: ignore[import-not-found,import-untyped, unused-ignore]` so the type check passes here and stays silent once Phase 6 lands. * spec: cut dated 2026-05-21 snapshot; promote schema generator to script * ci(mypy): collapse fusion import to single line for type-ignore * feat(embeddings): RRF/weighted/normalized fusion strategies * docs: add 'What ships today' table to README * kb: approve review-gate fact * docs: add 'When to use vouch' section * fix(cli): honor VOUCH_AGENT in _whoami for consistent actor attribution The CLI propose-* commands and most actor sites called _whoami(), which only checked VOUCH_USER and the OS user — VOUCH_AGENT was silently ignored. The MCP and JSONL servers (server.py, jsonl_server.py) and session_start already honour VOUCH_AGENT, so multi-agent attribution broke as soon as you used the CLI to propose. Prefer VOUCH_AGENT over VOUCH_USER over the OS user so the recorded proposed_by / audit actor matches across all three transports. * docs: add captured example session walkthrough Real propose -> review -> commit -> retrieve loop captured from a sandbox run on 2026-05-21. Includes on-disk YAML, audit log lines, and an honest note about the literal substring backend limitation pre-embeddings. * docs: add screen recording of full vouch loop (VHS tape + GIF) * docs: embed demo GIF under Quick start in README * fix(embeddings): address CodeRabbit review on hybrid + fusion + write hook Bundles the substantive correctness fixes flagged by CodeRabbit on PR #41. CI was already green; these are latent bugs the review caught. storage._embed_and_store - Re-embed when the active embedder model changes, not only when the content hash changes. The previous short-circuit kept stale vectors on disk after a model swap, mixing document embeddings from one model with queries from another. - Truly best-effort: catch any exception during encode/put/meta/dedup so a hook failure can't bubble back to the caller and leave an artifact persisted-but-API-errored. index_db.reset - Also clears embedding_index, query_embedding_cache, embedding_dupes, and embedding_* keys from index_meta. Otherwise `vouch index` left orphaned hits behind after artifacts were removed. index_db.search_embedding (Python fallback) - Normalize the stored vector by its own L2 norm before scoring, so rankings match the sqlite-vec `1 - vec_distance_cosine` path. The previous raw dot product made magnitude leak into the score. index_db.search_semantic - Invalidate the query-vector cache entry when the embedder dim no longer matches what's on disk; otherwise a model swap returned stale vectors living in the wrong space. server.kb_search + jsonl_server._h_search (hybrid branch) - Hybrid now passes min_score into search_semantic (consistent with embedding/auto branches) and wraps the FTS lookup in try/except so an FTS5 error doesn't fail the whole request. - JSONL transport rejects unknown backend values with a clear error instead of silently returning []; matches MCP transport behavior. embeddings.fusion - rrf_fuse validates limit >= 0 and k >= 0 (k = -rank would divide by zero). weighted_fuse validates limit >= 0. Tests - Three new fusion guard tests cover the negative-limit and negative-k paths. * fix(lifecycle): only catch missing-artifact errors in cite() the bare except was treating every error as a missing citation, which hid real bugs. narrow it to ArtifactNotFoundError so unexpected failures actually surface. * fix(context): only catch missing-artifact errors in citation lookup same issue as cite() — a bare except was hiding real errors behind an empty citation list. narrow it to ArtifactNotFoundError. * refactor(sessions): keep tracebacks when crystallize() approve fails we were dropping the traceback and only stringifying the error, which made it painful to diagnose anything other than ProposalError. now logs via logger.exception and includes error_type in the failures list. * fix(context): only catch sqlite errors when FTS5 falls back the retrieval helper had a bare except that would also swallow bugs in the substring fallback path or in our own code. narrow it to sqlite3.Error so only the intended cases (FTS5 missing, db missing, schema mismatch) trigger the fallback. * refactor(health): use typed status filter for pending proposal count we were listing every proposal and comparing p.status.value == 'pending' as a string. list_proposals() already takes a ProposalStatus filter — use it. shorter, type-checked, and avoids the stringly-typed compare. * chore(capabilities): pin pluginApi compat baseline, fail CI on host-compat drift * feat(dual-solve): surface engine logs * fix(sessions): make crystallize retry idempotent for summary pages Partial crystallize runs that already wrote session-{id} summary pages failed on retry with "page already exists". Upsert the summary page and derive its artifact list from all approved session proposals. Fixes #139 Co-authored-by: Cursor <cursoragent@cursor.com> * fix(models): reject empty text/name/title on claim/entity/page the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (#81/#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes #155 * refactor(models): extract shared _require_non_empty validator helper address review on #300: the three #155 field validators (Claim.text / Entity.name / Page.title) were structurally identical strip-checks. extract a module-level `_require_non_empty(v, label)` helper they all delegate to, keeping the per-field error messages. also fix a mid-entry sentence casing in the CHANGELOG. no behaviour change. * fix: resolve RPC traceback leakage on internal errors Unexpected exceptions in handle_request() included full stack traces in the JSON error envelope, exposing paths and internals to HTTP /rpc clients. Log server-side and return message only. Co-authored-by: Cursor <cursoragent@cursor.com> * feat(review): kb.triage_pending — advisory triage scoring for the pending queue a long `kb.list_pending` forces the reviewer to reconstruct, per proposal, whether the claim fits the existing kb, whether its citations resolve, whether it duplicates something already filed, and whether it contradicts an approved claim. this adds an optional triage pass that scores each pending proposal on those four signals and attaches a `_meta.vouch_triage` block (recommendation/score/signals/rationale) to help a reviewer prioritize, without ever deciding anything itself. read-only by construction: the pass never calls proposals.approve/reject, store.put_*, or store.move_proposal_to_decided — a human still calls kb.approve/kb.reject. citation_quality reuses proposals._payload_block_reason; duplication_risk reuses the propose-time embedding similarity path (embeddings.similarity.find_similar_on_propose) and degrades to a difflib heuristic when the embeddings extra isn't installed. fit uses a separate, lower-threshold embedding search so a near-duplicate hit doesn't also inflate fit and cancel out its own duplication penalty. opt-in via `triage.enabled: true` in config.yaml (default false). registered at all four kb.* surface sites (server.py, jsonl_server.py, capabilities.py, cli.py) plus `vouch triage [proposal-id...]` with `--json` and `--reverse`. * feat(diff): register kb.diff at all four kb.* surface sites vouch diff shipped CLI-only in 0.1.0, an explicit non-goal at the time ("MCP/JSONL parity ... kb.* surface unchanged"). that leaves it the only read method skipping the four-site registration convention documented in CLAUDE.md, so agents talking MCP/JSONL have no way to fetch a revision diff. adds the kb_diff MCP tool, the kb.diff JSONL handler, and the capabilities.METHODS entry, next to the other by-id read tools (kb_read_claim/kb_read_page) with the same unrestricted-read posture. also makes new_id optional for a superseded claim: diff_artifacts and the CLI both resolve it from superseded_by when omitted, erroring clearly when there's no successor (pages still require an explicit new_id — they have no successor pointer). closes #327. * fix(jsonl): define module logger after the import block #361 inserted `_log = logging.getLogger(...)` between two import groups in jsonl_server.py. that statement-among-imports trips ruff's E402 on every import that follows it, so `ruff check src tests` — the lint gate in ci — now fails on the whole repo. move the logger definition below the imports (where storage.py already keeps its own module logger); no behaviour change. * feat(adapters): toml_merge install strategy for codex config.toml .codex/config.toml is codex's primary config file, so the plain-copy path silently skipped any project where codex was already configured and vouch never got wired. add a toml_merge entry flag mirroring json_merge: parse the existing destination with tomllib, deep-merge the template's tables into it (existing user values always win on conflict), and write back. flip the codex T1 entry to toml_merge. writing uses a minimal hand-rolled serializer (tables, arrays, inline tables in arrays, scalars, datetimes) so the dependency set stays unchanged; the output must survive a tomllib round-trip back to the merged data, and anything the serializer can't faithfully re-emit degrades to skipped rather than risking the user's config. closes #384 * feat(adapters): codex T2 agents.md fenced snippet codex reads AGENTS.md for project instructions the way cursor does, but the codex adapter stopped at T1, so a codex session got the kb tools with no standing guidance on recall-first or the review gate. ship adapters/codex/AGENTS.md.snippet as a T2 tier with the standard fence markers, kept in lockstep with cursor's snippet modulo the host name (enforced by a sync test). also close the edited-fence gap in _install_fenced per the ticket's acceptance criteria: a fence body that drifted from the shipped snippet is replaced within the markers (reported as merged) instead of being skipped forever, while user content outside the fence stays untouched. a begin marker without an end marker is treated as corrupt and left alone. closes #385 * feat(adapters): codex T3 skills mirroring the vouch slash commands claude-code T3 ships nine slash commands; codex users got none of the guided flows. ship them as codex skills under the project-local .codex/skills/ as a T3 tier. scope decision the ticket asked to resolve first: codex loads custom prompts only from ~/.codex/prompts/ (user-global) and has deprecated them upstream in favour of skills, which do have a project-local home at <project>/.codex/skills/ in trusted projects. the #179 rule forbids a project-scoped install from touching home-directory state, so prompts are out and skills are in — recorded in the manifest comment and the adapter readme, with a manual-copy pointer for users who still want ~/.codex/prompts. the SKILL.md files are referenced from the openclaw mirror rather than duplicated (same reuse pattern openclaw itself applies to the claude-code commands), so one edit updates every host. a parametrized sync test asserts each installed codex skill body equals the matching claude-code command body, and a scope test asserts every installed path stays under the project target. closes #386 * feat(capture): ingest codex session rollouts into review-gated summaries session auto-capture was claude-code-only: hooks drive capture observe and capture finalize live. codex has no hook stream, but it persists every session as a rollout jsonl under $CODEX_HOME/sessions containing user messages, tool calls, and outputs — everything the existing rollup needs, just after the fact. new cli command `vouch capture ingest-codex [<rollout> | --latest]` parses one rollout through a small dedicated parser (codex_rollout.py) that maps function_call records into the same observation shape capture.observe produces — shell commands with failure detection from exit codes, apply_patch heredocs surfaced as file edits, mcp tools under their own names, session mechanics skipped — then reuses the existing build_summary_body -> propose_page rollup. one code path from observation to proposal, two front doors. the rollout format is not a stable public contract: unknown record types are tolerated, and unreadable, compressed, or meta-less files degrade to a CodexRolloutError with an actionable message and a non-zero exit, never a stack trace. re-ingesting a session is a no-op keyed on the rollout's session id; --latest resolves the newest rollout whose recorded cwd matches the current project. proposals are attributed to the codex actor (VOUCH_AGENT wins when set), respect capture's enabled/min_observations config, and never touch approve(). fixture rollouts use placeholder data only, enforced by a test. closes #387 * feat(adapters): codex T4 hook wiring for automatic session capture with `vouch capture ingest-codex` available, codex can self-capture the way claude-code T4 does. the ticket assumed the `notify` setting in config.toml, but codex only honours notify in user-global config — it is a restricted key in project-local layers — and the #179 rule forbids touching ~/.codex. codex's hooks system is the project-local equivalent: `.codex/hooks.json` (loaded in trusted projects) fires Stop when a turn completes, with a payload carrying the session id and transcript path. T4 ships that file through json_merge, so an existing user hooks.json is deep-merged into, never overwritten, and re-runs don't duplicate the hook. the handler is a new --hook mode on ingest-codex: read the stop payload from stdin, resolve the session's rollout (transcript_path when present, else by session id in the rollout filename), ingest idempotently, and exit 0 no matter what — the same never-break-the- host rule capture observe follows. stop fires per turn, not per session end, so the finalize policy the ticket left open is resolved as idempotent re-ingest: the dedup guard now refreshes a session's still-PENDING proposal in place when the rollout grew (same id, updated summary, audit-logged), reports an unchanged rollout as a no-op, and never resurrects a proposal a human already decided. storage gains update_proposal, a pure-I/O in-place rewrite of a pending proposal file. wiring only — everything still lands through propose_page; nothing touches approve(). closes #388 * test(adapters): live codex install gate against the real cli tests/test_openclaw_plugin_load_real.py set the pattern: exercise the real host cli when it's on PATH, skip cleanly where it isn't. the codex adapter had no equivalent, so regressions in the shipped config only surfaced when a user hit them — the old T1 silent-skip no-op is exactly the class of bug a live gate would have caught. the new suite installs the adapter into a temp project at T4, marks the project trusted inside an isolated CODEX_HOME, and asserts through `codex mcp list --json` that the vouch server is visible: next to a pre-seeded unrelated server on the merge path, alone on the fresh path, and absent for an untrusted project (the config must stay inert where codex says it does). a full-tier check covers the fenced AGENTS.md, the skills, and the Stop hook, and an isolation check pins every written artifact under the temp project. assertions target the observable contract — server listed, config parses, snippet present — not codex internals, so version bumps shouldn't break the suite. no network, no credentials, never touches the real ~/.codex; runs in the normal pytest invocation. closes #389 * docs(adapters): codex adapter docs reflect the tiered install three surfaces went stale once the codex tier work landed. the adapters table row described codex as a one-file config.toml snippet; it now lists the tiered install. the codex adapter readme led with the manual ~/.codex edit; it now leads with `vouch install-mcp codex`, explains the project-local .codex/config.toml choice and the trust requirement, summarizes what each tier adds (mcp wire / AGENTS.md snippet / skills / capture hook) with the merge-safety guarantees, and keeps the manual edit as an explicit fallback section. getting-started treated codex as a one-liner aside inside the claude instructions; the install section now shows both hosts side by side. the prompts-location decision from the T3 ticket and the notify restriction from the T4 ticket stay documented in their sections. no code; placeholder examples only. closes #390 * docs(mcp): design a friendlier mcp surface — profiles + auto-recall the closest competitor (pmb) exposes 10 mcp tools by default and hides the rest behind a profile flag; vouch shows all 58 every turn, which is the main first-touch friendliness gap. this spec adds a minimal-by-default tool profile, fusion-by-default retrieval, a per-prompt auto-recall hook, and compact tool descriptions. all of it is presentation-only — the review gate, the yaml storage, and the protocol method surface are untouched. it also closes the current 2-of-3 surface-parity gap as a bonus. * docs(mcp): plan the friendlier-mcp-surface build six TDD tasks: tool profiles (minimal default), real mcp↔methods parity, fusion-by-default retrieval, near-duplicate drop, per-prompt auto-recall hook, and compact tool descriptions. reconcile the spec: minimal is 8 tools (kb_capabilities kept as the discovery escape hatch) and the recency multiplier is deferred (per-hit timestamps aren't plumbed through _retrieve). * feat(mcp): add tool profiles, expose a minimal surface by default agents saw all 58 kb.* tools every turn — the main first-touch friendliness gap vs pmb, which exposes ~10 by default. add a profile layer applied in run_stdio: minimal (8 core tools) by default, standard (16), or full (58), selected by VOUCH_TOOL_PROFILE / config mcp.tool_profile. exposure only — the protocol surface, jsonl/cli, and the review gate are unchanged. * fix(mcp): harden profile resolution against a non-dict config section a bare `mcp:` key in config.yaml parses to None in YAML, and a non-dict mcp section is also possible from hand-edited configs. the previous config.get("mcp", {}).get("tool_profile") chain only supplied the {} default when the key was absent, so a present-but-None or non-dict value passed through and the chained .get raised AttributeError. run_stdio calls resolve_profile_name outside its try/except, so this would crash vouch serve instead of degrading to the minimal default. guard each nesting level with isinstance, matching the established reflex_cfg pattern in salience.py. * test(capabilities): assert mcp tool set matches the method list the parity test only compared capabilities.METHODS to the jsonl handlers; the 58 mcp tools were never checked, so mcp drift passed ci. enumerate the unfiltered server tools and assert they equal METHODS — the real 3-surface check for the mcp side. * feat(retrieval): fuse embedding + fts5 by default instead of a waterfall _retrieve tried embedding, then fts5, then substring, returning the first non-empty list — so lexical and semantic hits never combined. auto and hybrid now fuse both retrievers with the already-built rrf_fuse and tag hits "hybrid"; explicit embedding/fts5/substring pins are unchanged. existing KBs (config says "auto") benefit with no migration. * fix(volunteer): treat hybrid relevance as rank-relative, not pre-normalized normalize_relevance() grouped "hybrid" with "embedding", returning the raw score unclamped-by-batch on the assumption it was already a 0-1 similarity. now that _retrieve's auto/hybrid path tags fused hits "hybrid" with a rrf_fuse score (bounded by ~2/(k+rank), typically 0.01-0.03), that assumption stopped holding: every fused relevance landed far below DEFAULT_THRESHOLD (0.85), so kb.volunteer_context stopped firing for any KB using the new default backend. batch-normalize hybrid like fts5 and substring instead — restores the confidence-gated push channel (#236). * feat(retrieval): drop near-duplicate items from the context pack a fused pack could surface the same fact from two claims. add a cheap greedy jaccard pass (>=0.85 over the first 40 tokens) that keeps the highest-scored of a near-duplicate cluster, so an agent never reads the same thing twice. * fix(retrieval): dedupe by score so the highest-scored near-dup wins _dedupe_near_duplicates assumed its input arrived in descending-score order, which only holds for the plain retrieval path. when build_context_pack runs with expand_graph=True, graph neighbours are appended in bfs order with decayed scores after the sorted main items, so a higher-scored neighbour could be dropped in favor of an earlier, lower-scored main item. sort inside the function so the invariant holds regardless of caller order. * feat(hooks): inject per-prompt kb context via a userpromptsubmit hook recall used to be either a session-start firehose or an explicit tool call. add a pure, never-raising helper + a hidden `vouch context-hook` command that reads the host's prompt payload on stdin and prints an additionalContext envelope, and wire it into the claude-code adapter's UserPromptSubmit hook — so relevant, approved knowledge is injected every turn with zero tool calls. * fix(hooks): never raise on non-dict payload or missing kb a reviewer reproduced two crash paths that violated task 5's own contract that the prompt hook must never block a turn: build_claude_prompt_hook raised AttributeError on non-dict JSON (null/number/bool/array/string all decode fine but lack .get), and the context-hook CLI command called _load_store(), whose sys.exit(2) on a missing KB is a SystemExit that slips past `except Exception` and exits the process nonzero. guard the payload with an isinstance check before .get, log a breadcrumb when build_context_pack fails so a broken KB isn't silently indistinguishable from "no hits", and swap _load_store() for the existing non-exiting _capture_store() helper so the command has no sys.exit on its path at all. * feat(mcp): serve one-line tool descriptions under non-full profiles full docstrings for every exposed tool are paid on every turn. under minimal and standard, trim each tool's description to its first line (full keeps the complete docstrings), cutting the per-turn context cost. * docs(mcp): document tool profiles note VOUCH_TOOL_PROFILE / mcp.tool_profile (default minimal, values minimal|standard|full) in the transports reference and the claude-code adapter guide; this repo has no mintlify/ tree yet so both live under docs/ and adapters/claude-code/ instead. * fix(retrieval): dedupe by score but keep caller order for the pack _dedupe_near_duplicates sorted by score and returned that score-sorted order, which fought graph-expansion: build_context_pack appends decayed-score neighbours after the ranked hits, and fused primary hits now carry small rrf scores. re-sorting let an irrelevant depth-2 neighbour outrank a real match, and the max_chars tail-pop then evicted the real match instead of the neighbour. keep the keep-decision in descending-score order so the highest-scored member of a near-duplicate cluster still survives, but return survivors in the caller's original order so budget eviction drops the tail (appended neighbours), not the ranked hits. * docs(changelog): note minimal mcp profile, fusion, auto-recall summarize this branch's user-visible changes under [Unreleased]: the minimal-by-default mcp tool profile, rrf fusion for auto/hybrid retrieval, and the per-prompt auto-recall hook in the claude-code adapter. * docs(readme): note the fourth hook, per-prompt recall, and lean mcp profile install-mcp now writes a fourth hook (UserPromptSubmit -> vouch context-hook), and recall fires per prompt, not only at session start. also note the kb.* mcp surface defaults to a lean profile that widens via VOUCH_TOOL_PROFILE. * fix(capabilities): read openclaw.compat from package.json (#417) the host-compat drift check (#237) read openclaw.compat.pluginApi from openclaw.plugin.json. that block moved to package.json when the plugin packaging was split, and openclaw.* is now banned from the manifest (enforced by test_manifest_carries_no_dead_dialect_fields). the #237 reader was left pointing at the old, now-empty location, so host_compat silently degraded to {} and both host-compat assertions failed on the test branch — which every open pr targeting test inherited, including #413. repoint _load_host_compat and the test helpers at package.json. the openclaw.compat.pluginApi key path is identical in both files, so this is purely a source-file change with no behaviour difference. * chore(repo): untrack owner-local web/ and .claude/ (#413) these two directories were committed by accident and reach every first-time contributor on clone. neither is project source: - web/ is the netlify-deployed marketing landing page. it is shipped via the netlify cli and is not meant to live in the repo tree. - .claude/ is a personal claude code config: a settings.json with owner-specific hooks, plus command files that are byte-identical duplicates of adapters/claude-code/.claude/commands/. untrack both (files stay on disk) and gitignore them, anchored to the repo root so src/vouch/web and adapters/claude-code/.claude stay tracked. also ignore coverage.xml and the .netlify/ and .playwright-mcp/ tool-scratch dirs. * chore(merge): resolve changelog and version conflicts with main * chore(merge): update changelog to resolve merge conflicts * chore(merge): update to main's latest versions for clean merge * docs(readme): restore incubated by gittensor acknowledgment * chore: remove banner.svg and its reference in README * chore: remove desktop, spec, migrations dirs and pre-commit config * fix(ci): restore missing storage and context constants Add back KB_FORMAT_VERSION, SCHEMA_VERSION, SCHEMA_VERSION_FILENAME, _starter_config to storage.py and _RETRACTED_CLAIM_STATUSES to context.py to fix import errors in test suite. * fix(ci): restore coherent backing modules to match callers the test-branch merges (2ddd1f9 onward) took main's older, smaller storage.py / context.py / index_db.py / cli.py / server.py / health.py / jsonl_server.py / bundle.py / sessions.py while keeping the newer caller modules (lifecycle, proposals, graph, notify, sync, provenance, codex_rollout, triage). that desync left 31 mypy errors — callers referencing methods the truncated backing no longer defined (put_relation_idempotent, _validate_claim_refs, update_page, update_proposal, index_prov_edge, get_meta) — so the test job failed at mypy before pytest ever ran. restore src/ and tests/ to bb06565, the last coherent snapshot, where the fuller backing matches the callers. this also brings back the reindex cli command the recall-eval job invokes (the truncated cli only had index), fixing the second red job. keep __version__ at 1.2.2 so the four version sites stay in lockstep. --------- Co-authored-by: plind-junior <138900956+dripsmvcp@users.noreply.github.com> Co-authored-by: alpurkan17 <alpurkan9@gmail.com> Co-authored-by: Tet-9 <igeparallax@gmail.com> Co-authored-by: Yaroslav98214 <diakovichyaroslav30@gmail.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: minion1227 <romantymkiv1999@gmail.com> Co-authored-by: RealDiligent <brave.challenge007@gmail.com> Co-authored-by: jsdevninja <topit89807@gmail.com>
dripsmvcp
pushed a commit
to dripsmvcp/vouch
that referenced
this pull request
Jul 10, 2026
the non-empty contract for `Claim.text`, `Entity.name`, and `Page.title` lived only in the `propose_*` helpers, so `store.put_*`, `store.update_*`, and bundle/sync import (via `_validate_content`) all silently accepted empty or whitespace-only values and landed artifacts carrying zero of the field's semantic content. add `@field_validator`s on each field, mirroring the existing `Claim.evidence` min-citation validator (vouchdev#81/vouchdev#82): they reject blank input at construction time, so every write path — direct construction, put/update, and import — inherits the check at once. the validators gate on emptiness only and preserve surrounding whitespace of non-blank values. `Source.locator` is deliberately left out of scope — its format question is bigger and deserves its own issue, as the report notes. closes vouchdev#155
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
Claimmodel has no min-evidence validator(
src/vouch/models.py:184-187), so aClaimwithevidence: []isschema-valid. The README and CONTRIBUTING contract —
is enforced in exactly one place,
proposals.propose_claim(
src/vouch/proposals.py:89-90). Every other write path silentlyaccepts an uncited claim:
store.put_claimdirect (src/vouch/storage.py:283-291) —the existence-check loop iterates zero times.
store.update_claim— writes the YAML without re-validating.bundle.import_apply—_validate_contentdefers toClaim.model_validate, which acceptsevidence: []; themalicious bundle passes
import_check(ok=True) and lands theuncited claim under the legitimate
bundle_id, with a cleanbundle.importaudit event.CONTRIBUTING explicitly lists this as a won't merge category:
— except the enforcement was never on the model in the first
place, so every write path except the one explicit gate already
qualifies as the relaxed shape the team has stated they reject.
End-to-end repro on
main(uncited_repro.py):Fix
src/vouch/models.py, on theClaimmodel:That closes the direct-construction path, the bundle import path
(both go through
Claim(...)/Claim.model_validate(...)), andthe
propose_claimpath (the existing string-comparison guardbecomes a redundant nicer-error message and is left in place).
The in-place mutation path (
c.evidence = []; store.update_claim(c))needs one extra line:
store.update_claimnow doesClaim.model_validate(claim.model_dump(mode="json"))beforepersisting. Pydantic field validators only fire at construction
time, so mutation alone bypasses them; the round-trip catches it
before the YAML hits disk.
The existing user-facing guard in
proposals.propose_claimisleft as a friendlier CLI/JSONL error message. No on-disk-layout,
schema, or bundle-format change — data the model never should
have accepted now raises.
Tests
Four regression tests:
test_claim_model_rejects_empty_evidence(tests/test_storage.py)—
Claim(evidence=[])raisespydantic.ValidationErrorwithmessage containing
"cite at least one".test_put_claim_rejects_empty_evidence—store.put_claimraises;no
claims/<id>.yamlis written.test_update_claim_rejects_empty_evidence— in-place mutationfollowed by
store.update_claimraises; the on-disk YAML isbyte-for-byte unchanged.
test_import_rejects_uncited_claim(tests/test_bundle.py) —a manifest-consistent bundle whose claim YAML has
evidence: []is rejected by
import_check(with aschema validation failed:issue), and
import_applyraisesRuntimeError("refusing to import: schema validation failed: ...")before writing.All 4 pass on Windows (Python 3.12.13). Lint clean
(
ruff check src tests). Full suite shows the same pre-existingWindows-only failures present on
main(O_NOFOLLOWinstorage.py, bundle round-trip CRLF) — both untouched by this PR.Test count: 152 → 156 (+4 new, 0 regressions).
Audited every existing
Claim(...)call site intests/beforeadding the validator — none construct claims with empty evidence,
so the new gate breaks no existing test.
Fixes #81
Summary by CodeRabbit
Bug Fixes
Documentation
Tests
Health