Test - #486
Merged
Merged
Conversation
the atom the fidelity pivot rests on: a claim's evidence can now carry a byte-offset span [byte_start, byte_end) into the cited source's raw bytes, and receipts.verify_receipt checks the quoted text against the source bytes by string comparison alone — no llm, no judge. the quoted span is in the source at those offsets or it is not. three receipt states, kept distinct because the review gate treats them differently: VERIFIED (span decodes to exactly the quote), FORGED (span claimed but out of range / inverted / undecodable / mismatched, or the source is absent from the kb), NO_RECEIPT (no offsets to verify). byte offsets rather than char offsets because sources are stored as raw content-addressed bytes and utf-8 char indices diverge from byte indices at the first multibyte codepoint. verify_evidence loads the source from the store and delegates; a receipt whose source is missing is reported forged, never verified, so an unverifiable citation can never read as approved. purely additive — two optional evidence fields and one new module, no existing path changes behaviour yet. this is the primitive phase d's auto-approve will call.
the quote step of phase a's retrieve-then-quote loop. locate_span finds the byte offsets of a quote's first exact occurrence in a source's raw bytes, or None when the quote is not present verbatim. the match is deliberately exact and case-sensitive — no normalization, no fuzzy match — because the receipt's value is that it is checkable by string comparison, so a paraphrase must fail to locate and be dropped. receipt_for_quote composes the locator with the schema: it returns an Evidence whose byte-offset receipt is guaranteed to verify against the same bytes, or None to drop an unquotable claim — the mechanical form of "drops any claim it cannot quote." a property test ties the two halves together: anything the locator produces verifies VERIFIED.
propose_quoted_claim wires the span locator into the gated write path: given a source and the quote that supports a claim, it locates the verbatim span, stores a receipt-backed Evidence, and files a normal claim proposal citing it. when the quote is not in the source it returns None and files nothing — the mechanical form of "drops any claim it cannot quote." the write still goes through propose_claim and the review gate; what is new is that the filed claim now carries a byte-offset receipt the gate can verify by string comparison. intake is idempotent: receipt_for_quote mints a content-addressed evidence id from the span, so re-filing the same span reuses the existing Evidence instead of duplicating it. layering kept honest — proposals depends on the receipts primitive, not the reverse. receipt_for_quote's evidence_id is now optional (content-addressed default).
evaluate_claim_receipts is the function phase d's auto-approve will call: it returns approve=True only when a claim cites at least one thing and every citation is a receipt that verifies. a forged receipt, a bare source id (no byte-offset span), an unknown id, or an empty citation list all reject, with reasons naming each failure. the verdict is the conjunction of per-citation string comparisons — no llm, no judge. this completes the verification machinery for "human leaves the loop": schema -> per-evidence verify -> receipt-backed intake -> per-claim verdict. what it deliberately does NOT do yet is touch approve(); wiring the verdict into automatic approval is the phase d step, gated behind the step 0 gate-integrity and step 1 concurrency work so the auto-gate is not forgeable or race-corruptible.
phase d of the fidelity pivot: the human leaves the loop. when review.auto_approve_on_receipt is on, a claim whose byte-offset receipts all verify clears self-approval with no human — the mechanical string comparison (evaluate_claim_receipts, no llm, no judge) is the reviewer. a claim that cannot quote its source (bare source id, forged or missing receipt) does not qualify and still falls through to the human gate: the gate degrades to asking, it never rubber-stamps. off by default, so the review gate stays on until a kb opts in. _approval_block_reason gains the receipt path alongside the trusted-agent opt-out; _review_config dries the config.yaml load the block reason and the new drain share; _claim_receipts_verify wraps the verdict. auto_approve_receipts(store) drains the pending queue — approves every receipt-verified claim, leaves everything else pending — which is what makes "run vouch and it just captures knowledge, no review" real end to end: ingest a source, quote it into receipt-backed claims, auto-approve with no human, recall the knowledge (including from a reopened kb). builds on the phase-a receipt machinery (span-receipts).
phase b: the capture step with no human, so the loop runs without a hand-written quote. extract.segment_source deterministically splits an ingested source into verbatim quotable spans; extract_receipt_claims files each as a claim that quotes itself, so its byte-offset receipt verifies by construction; ingest_source runs the whole loop — store the doc, extract the claims, and (when the gate is on) auto-approve every one whose receipt verifies. new `vouch ingest FILE`: "run vouch on a doc and it captures the knowledge." with review.auto_approve_on_receipt on, verifying claims are approved with no human and are immediately recallable via vouch search / recall; without the gate they are filed pending for review. a span that is not verbatim in the source is dropped, never trusted. deterministic and llm-free by design: the receipt check is the guardrail, so the pipeline runs in the base install and under test with no external command. selection — which spans are worth a claim — is a later quality knob; today it is every substantive sentence (ingesting a 28kb doc yields ~500 claims), which proves the pipeline but is not yet compression. builds on phase d's auto_approve_receipts.
the evidence model gained byte_start/byte_end for span receipts but schemas/evidence.schema.json was not regenerated, so the schema-check drift gate failed. regenerated with scripts/gen_schemas.py; only the evidence schema changed.
Feat/receipt auto approve
📝 WalkthroughWalkthroughAdds byte-offset evidence receipts, deterministic source-to-claim extraction, receipt-backed proposal gating, optional auto-approval, and a ChangesReceipt-backed source ingestion
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CLI as vouch ingest
participant Extract as extract.ingest_source
participant Store as KBStore
participant Proposals as proposals
participant Receipts as receipts
CLI->>Extract: ingest source bytes
Extract->>Store: store Source
Extract->>Proposals: file quoted claim proposals
Proposals->>Receipts: create and verify byte receipts
Extract->>Proposals: auto-approve verified proposals
Proposals->>Store: persist approved Claims
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (3 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
tests/test_extract.py (1)
61-64: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valuePrefer updating the config rather than overwriting it.
Overwriting the entire configuration file drops all other default config keys (such as
require_human_approval,capture,retrieval, etc.). While this passes in isolation, it can become brittle if downstream components are ever updated to strictly require those keys. Consider modifying only the target string.💡 Proposed refactor to safely modify the config
- store.config_path.write_text( - "review:\n auto_approve_on_receipt: true\n", encoding="utf-8" - ) + config_text = store.config_path.read_text(encoding="utf-8") + config_text = config_text.replace("auto_approve_on_receipt: false", "auto_approve_on_receipt: true") + store.config_path.write_text(config_text, encoding="utf-8")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/test_extract.py` around lines 61 - 64, Update test_ingest_source_auto_approves_and_is_recallable to modify only review.auto_approve_on_receipt in the existing configuration, preserving all other default config keys instead of overwriting the file. Reuse the current config content or established configuration-update mechanism before writing the modified value.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/vouch/receipts.py`:
- Around line 87-93: Update the quote encoding logic in the receipt-matching
function around needle = quote.encode("utf-8) to catch UnicodeEncodeError for
surrogate-containing quotes and return None. Preserve the existing empty-needle
and not-found handling for successfully encoded quotes.
- Around line 63-67: Update the range validation guarding receipt span
extraction to reject any negative start offset, including manually constructed
or mocked receipts, while preserving the existing out-of-range handling and
error result in the surrounding receipt verification flow.
---
Nitpick comments:
In `@tests/test_extract.py`:
- Around line 61-64: Update test_ingest_source_auto_approves_and_is_recallable
to modify only review.auto_approve_on_receipt in the existing configuration,
preserving all other default config keys instead of overwriting the file. Reuse
the current config content or established configuration-update mechanism before
writing the modified value.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 9f240202-f976-4b76-a6b3-5572c96f841f
📒 Files selected for processing (11)
schemas/evidence.schema.jsonsrc/vouch/cli.pysrc/vouch/extract.pysrc/vouch/models.pysrc/vouch/proposals.pysrc/vouch/receipts.pysrc/vouch/storage.pytests/test_extract.pytests/test_proposals.pytests/test_receipt_auto_approve.pytests/test_receipts.py
Comment on lines
+63
to
+67
| if start > end or end > len(source_bytes): | ||
| return ReceiptResult( | ||
| ReceiptStatus.FORGED, | ||
| f"span [{start}:{end}) out of range for {len(source_bytes)} bytes", | ||
| ) |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Defensively guard against negative start offsets.
While Pydantic enforces ge=0 during normal initialization, a manual instantiation (e.g. via model_construct or test mocks) could bypass this. In Python, a negative start offset evaluates from the end of the sequence. If start and end are both negative and start < end, this check passes and extracts bytes from the end of the source, potentially verifying a forged receipt.
🛡️ Proposed fix to explicitly reject negative offsets
- if start > end or end > len(source_bytes):
+ if start < 0 or start > end or end > len(source_bytes):
return ReceiptResult(
ReceiptStatus.FORGED,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| if start > end or end > len(source_bytes): | |
| return ReceiptResult( | |
| ReceiptStatus.FORGED, | |
| f"span [{start}:{end}) out of range for {len(source_bytes)} bytes", | |
| ) | |
| if start < 0 or start > end or end > len(source_bytes): | |
| return ReceiptResult( | |
| ReceiptStatus.FORGED, | |
| f"span [{start}:{end}) out of range for {len(source_bytes)} bytes", | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vouch/receipts.py` around lines 63 - 67, Update the range validation
guarding receipt span extraction to reject any negative start offset, including
manually constructed or mocked receipts, while preserving the existing
out-of-range handling and error result in the surrounding receipt verification
flow.
Comment on lines
+87
to
+93
| needle = quote.encode("utf-8") | ||
| if not needle: | ||
| return None | ||
| idx = source_bytes.find(needle) | ||
| if idx < 0: | ||
| return None | ||
| return (idx, idx + len(needle)) |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
Handle potential UnicodeEncodeError from surrogate halves.
If a quote contains surrogate halves (e.g., a hallucination from an LLM proposing a claim in an upstream path), .encode("utf-8") will raise a UnicodeEncodeError and crash the process. Catching the exception and returning None gracefully drops the unquotable claim.
🚑 Proposed fix to catch encode errors
- needle = quote.encode("utf-8")
- if not needle:
- return None
+ try:
+ needle = quote.encode("utf-8")
+ if not needle:
+ return None
+ except UnicodeEncodeError:
+ return None
idx = source_bytes.find(needle)📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| needle = quote.encode("utf-8") | |
| if not needle: | |
| return None | |
| idx = source_bytes.find(needle) | |
| if idx < 0: | |
| return None | |
| return (idx, idx + len(needle)) | |
| try: | |
| needle = quote.encode("utf-8") | |
| if not needle: | |
| return None | |
| except UnicodeEncodeError: | |
| return None | |
| idx = source_bytes.find(needle) | |
| if idx < 0: | |
| return None | |
| return (idx, idx + len(needle)) |
🧰 Tools
🪛 ast-grep (0.44.1)
[warning] 89-89: XPath query is request-/variable-derived; use parameterized XPath to prevent injection.
Context: source_bytes.find(needle)
Note: [CWE-643] Improper Neutralization of Data within XPath Expressions ('XPath Injection').
(xpath-injection-python)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/vouch/receipts.py` around lines 87 - 93, Update the quote encoding logic
in the receipt-matching function around needle = quote.encode("utf-8) to catch
UnicodeEncodeError for surrogate-containing quotes and return None. Preserve the
existing empty-needle and not-found handling for successfully encoded quotes.
This was referenced Jul 16, 2026
This was referenced Jul 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Why
What might break
VEP
Tests
make checkpasses locally (lint + mypy + pytest)CHANGELOG.mdupdated under## [Unreleased]Summary by CodeRabbit
vouch ingestto import files, extract quoted claims, and report approved or pending results.