Validator feedback names the fix, not just the rule (T0.4) - #38
Merged
Conversation
The rejection feedback IS the next prompt — compose.ts hands these messages straight back — and the retry budget is one attempt before the deterministic fallback takes over. A model made to re-derive the value it just got wrong usually gets it wrong again, so every violation that has a deterministic fix now ends by naming it: the sorted token order, the exact deadline, the chainId, the allowed tokens, the offered instructions, the block to copy. Two that are more than a value: I2 says to DIVIDE the budget between strategies (the "added rather than divided" failure the loop was built for), and an empty instruction menu reads as "omit the slot" rather than as a truncated sentence — no guard has an encoder on this venue. Where no deterministic fix exists — which template suits the intent — the message states the rule and stops. Invented advice would be worse. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #27. Tier 0 of the prompt-building ladder (#29) — the independent, validator-side one.
Why
compose()feeds violation messages straight back as the next prompt, and there is exactly one retry before the deterministic fallback takes over. Several messages stated the rule without the value that satisfies it, leaving a 7B model to re-derive what it just got wrong. Runtime cost is zero — these strings are only built on rejection — and the validator still rejects, never rewrites (F2 §4): naming the fix is guidance for the next attempt, not mutation of the old one.What changed
Every violation with a deterministic fix now ends by naming it:
set "chainId": 8453Two are more than a value. I2 says to divide rather than repeat the budget — that is the "split by adding instead of dividing" failure the loop was built for. And an empty menu now reads as an answer ("nothing is offered here on this venue — omit the guards slot") instead of a truncated sentence; no guard has an encoder on this venue, so that is the honest fix.
Where no deterministic fix exists — which template suits the intent — the message states the rule and stops.
recommendation.ts's parse errors were audited too (they flow into the same feedback channel) and left alone: the prompt'sOUTPUT_SCHEMAblock already carries their fix, and duplicating it in the error text would just grow the retry prompt.Verification
validate.test.ts(13 new: one per code, each asserting the corrective literal — the sorted array, the timestamp, the address list — not just that a violation fired). Full SDK suite 144/144, typecheck clean.feeBps: 30(0.3% in the 10000-scale a model knows) is a valid integer in [0, 1e9) and correctly does not fire — so the scale hint belongs to the out-of-range message, where it can actually help.Not in this PR
The prompt-side Tier 0 work (#24, #25, #26) — separate PR, since it touches the prompt contract and bumps its version.
🤖 Generated with Claude Code