Skip to content

Validator feedback names the fix, not just the rule (T0.4) - #38

Merged
GustavoSena merged 1 commit into
mainfrom
gs/validator-feedback-states-the-fix
Jul 26, 2026
Merged

Validator feedback names the fix, not just the rule (T0.4)#38
GustavoSena merged 1 commit into
mainfrom
gs/validator-feedback-states-the-fix

Conversation

@GustavoSena

Copy link
Copy Markdown
Collaborator

Closes #27. Tier 0 of the prompt-building ladder (#29) — the independent, validator-side one.

Why

compose() feeds violation messages straight back as the next prompt, and there is exactly one retry before the deterministic fallback takes over. Several messages stated the rule without the value that satisfies it, leaving a 7B model to re-derive what it just got wrong. Runtime cost is zero — these strings are only built on rejection — and the validator still rejects, never rewrites (F2 §4): naming the fix is guidance for the next attempt, not mutation of the old one.

What changed

Every violation with a deterministic fix now ends by naming it:

Code Now also says
I1 the tokens the user actually selected (symbol + address)
I2 divide the cap between strategies; the total must be ≤ cap
I3 the permitted count, both directions
I4 set "chainId": 8453
I5 the offered instruction — or "omit the slot slot" when the menu is empty
I5 (feeBps) the scale that actually trips a model: out of 1e9, so 0.3% is 3000000
I7 the exact timestamp to use
I8 the known template ids
I10 the corrected token array and "move each virtualAmount with its token"
I11 what a valid amount looks like / that a leg must be positive
I12 the block to copy

Two are more than a value. I2 says to divide rather than repeat the budget — that is the "split by adding instead of dividing" failure the loop was built for. And an empty menu now reads as an answer ("nothing is offered here on this venue — omit the guards slot") instead of a truncated sentence; no guard has an encoder on this venue, so that is the honest fix.

Where no deterministic fix exists — which template suits the intent — the message states the rule and stops.

recommendation.ts's parse errors were audited too (they flow into the same feedback channel) and left alone: the prompt's OUTPUT_SCHEMA block already carries their fix, and duplicating it in the error text would just grow the retry prompt.

Verification

  • 39/39 in validate.test.ts (13 new: one per code, each asserting the corrective literal — the sorted array, the timestamp, the address list — not just that a violation fired). Full SDK suite 144/144, typecheck clean.
  • One new test pins a real subtlety: feeBps: 30 (0.3% in the 10000-scale a model knows) is a valid integer in [0, 1e9) and correctly does not fire — so the scale hint belongs to the out-of-range message, where it can actually help.

Not in this PR

The prompt-side Tier 0 work (#24, #25, #26) — separate PR, since it touches the prompt contract and bumps its version.

🤖 Generated with Claude Code

The rejection feedback IS the next prompt — compose.ts hands these
messages straight back — and the retry budget is one attempt before the
deterministic fallback takes over. A model made to re-derive the value it
just got wrong usually gets it wrong again, so every violation that has a
deterministic fix now ends by naming it: the sorted token order, the exact
deadline, the chainId, the allowed tokens, the offered instructions, the
block to copy.

Two that are more than a value: I2 says to DIVIDE the budget between
strategies (the "added rather than divided" failure the loop was built
for), and an empty instruction menu reads as "omit the slot" rather than
as a truncated sentence — no guard has an encoder on this venue.

Where no deterministic fix exists — which template suits the intent — the
message states the rule and stops. Invented advice would be worse.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prompt T0.4 — rejection feedback that states the fix, not just the rule

1 participant