Repository navigation
feat(codemode): name the closest tool in unknown-tool errors - #50455
Merged
Merged
Conversation
When a model calls a tool that does not exist, the error said only that it may have been removed or renamed. Rank the guessed name against the index (scoped to the namespace when that part is real) and name the best match: Unknown tool 'github.get-me'. Did you mean tools.github.get_me? The ranker moves out of the search tool into rank() so both can use it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Calling a tool that does not exist now names the closest real one:
The guessed name is ranked against the index with the same scorer
searchuses. When the first segment is a real namespace, the match is scoped to it and the namespace word is dropped from the query, socloudflare.searchdoes not get pointed at a GitHub tool. The follow-up line is now just "Use search to find available tools."; the old "may have been removed or renamed" was speculation.Why
In a tool-discovery eval, models occasionally type a plausible name instead of searching (
get-me,dependabot_list_alerts,list_starred_repositories_for_authenticated_user,get_user_accounts). The generic error costs a search and another round trip; naming the right tool ends it in one step. All four real examples from the last run resolve correctly.How
The scoring loop moves out of
makeSearchToolintorank(entries, query), used by both search andresolve.