ANS-1121: add optimizer judge model flag - #36
Conversation
Greptile SummaryAdds optimizer model-role selection and documents its fallback behavior.
Confidence Score: 5/5The PR appears safe to merge with the model-role behavior correctly implemented, tested, and documented. No actionable new issues remain, and the previous documentation finding is fully addressed by explicitly describing the analysis-model fallback order. Important Files Changed
Reviews (2): Last reviewed commit: "docs: clarify optimizer model precedence" | Re-trigger Greptile |
| The shared `--model` defaults the eval agent, judge, trace analyzer, and | ||
| component author. Use `--eval-model`, `--judge-model`, `--analysis-model`, or | ||
| `--proposal-model` for explicit role overrides. Without a shared model or | ||
| `--judge-model`, each case keeps its configured judge. |
There was a problem hiding this comment.
Analysis fallback is undocumented
The role description presents --proposal-model and --analysis-model as independent overrides, but the CLI resolves the analysis model from --analysis-model, then --proposal-model, then the shared --model. Without this detail, users may unintentionally select their proposal model for trace analysis.
| The shared `--model` defaults the eval agent, judge, trace analyzer, and | |
| component author. Use `--eval-model`, `--judge-model`, `--analysis-model`, or | |
| `--proposal-model` for explicit role overrides. Without a shared model or | |
| `--judge-model`, each case keeps its configured judge. | |
| The shared `--model` defaults the eval agent, judge, trace analyzer, and | |
| component author. Use `--eval-model`, `--judge-model`, `--analysis-model`, or | |
| `--proposal-model` for explicit role overrides. Without `--analysis-model`, | |
| trace analysis uses `--proposal-model` before falling back to the shared model. | |
| Without a shared model or `--judge-model`, each case keeps its configured judge. |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
|
@greptileai review |
Summary
--judge-modelas an explicit optimizer role override--modelapply to eval, judge, trace analysis, and component author rolesValidation
npm test— 72 passedgit diff --check— passedCompanion to the answerlayer-core ANS-1121 implementation.