Summary
Introduce a StateVerifier (name flexible) so task success is judged by expected final state, not a single gold tool trajectory. Different valid call sequences that reach the same state should pass.
Priority
priority:p1 — required for Milestone 8 unified evaluation quality (pairs with #76 / #77).
Suggested labels
enhancement, priority:p1, area:probes, area:cli, research
Acceptance criteria
Non-goals
- Requiring exact gold trajectories
- Auto-executing destructive MCP tools unless explicitly enabled
Related
Summary
Introduce a
StateVerifier(name flexible) so task success is judged by expected final state, not a single gold tool trajectory. Different valid call sequences that reach the same state should pass.Priority
priority:p1— required for Milestone 8 unified evaluation quality (pairs with #76 / #77).Suggested labels
enhancement,priority:p1,area:probes,area:cli,researchAcceptance criteria
expected_state(deterministic key/value checks preferred)evalreport when present; offline/fake tests only in CIdocs/final-state.md(or probes.md section)Non-goals
Related