Skip to content

Prompt T0.4 — rejection feedback that states the fix, not just the rule #27

Description

@GustavoSena

Prompt T0.4 — Rejection feedback that states the fix, not just the rule

Part of prompt-building Tier 0. Principle: the model echoes; deterministic code computes.

Why

The compose loop is validator-driven: a violating attempt is rejected and the violation messages are fed back verbatim as the re-infer prompt (compose.ts: feedback = violations.map(v => ${v.code}: ${v.message})). With MAX_COMPOSE_ATTEMPTS = 2, the single retry is precious — and today several messages state the rule without stating the value that satisfies it, leaving a 7B model to re-derive what it already got wrong once. Wherever the fix is deterministic, the message should contain the literal value to use, so the retry becomes an echo task.

This costs nothing at runtime (messages are only built on rejection) and touches no invariant logic — message strings and tests only. The validator still rejects, never rewrites (F2 §4); putting the correct value in the feedback text is guidance for the next attempt, not mutation of the old one.

What

Audit every Violation message in packages/arbitration-sdk/src/validate.ts (and the parse-error strings in recommendation.ts that flow into the same feedback channel) and, where a deterministic fix exists, append it:

Code Today (gist) Add
I4 chainId X != expected Y — set "chainId": Y
I7 deadline X not within (now, now+max] — use Z where Z = now + maxDeadlineSec (the same "use ${deadlineMax}" default the prompt already states)
I10 tokens not in canonical ascending order (A then B) the full correctly-sorted token list to copy
I11 virtualAmount is not a decimal string / is zero — use a strictly positive decimal string, e.g. "0.25"
I1 token T is not in the user's budget — allowed tokens: <the budget's addresses+symbols>
I2 totals exceed budget C — per-token headroom: cap C, your sum S; scale the amounts down so the sum ≤ C
I12 observedBlock stale/ahead — set "observedBlock": <ctx value>, "observedAt": <ctx value> (the values the prompt already carries)
I8 templateId unknown — known ids: <TEMPLATE_IDS>
I5 instruction not offered — offered: <the relevant OPTIONS list> (already partially done for curves; make it uniform for fee/guards/params)

Notes:

  • Some fixes need request/chain-state values the validator already receives (q, s) — no new plumbing.
  • Keep messages one line each; the feedback block is part of the next prompt and prompt bloat is the 7B's enemy.
  • Where NO deterministic fix exists (e.g. which template to pick), leave the message as-is — do not fabricate advice.

Acceptance criteria

  • For each code above, a unit test triggers the violation and asserts the message contains the literal corrective value (address list, timestamp, sorted order…).
  • Message construction stays pure (values from q/s/the rejected recommendation only).
  • Optional but recommended: one live A/B-ish observation via the compose CLI — force a rejection (e.g. tight maxDeadlineSec) and note in the PR whether attempt 2 now passes. Anecdotal is fine; the unit tests are the gate.

Coordination

Independent of the prompt-refactor PR (validator-side only). No PROMPT_VERSION bump needed unless the feedback framing line in the prompt itself changes.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions