Conversation
locate_all_opencv sorted its matches by reading the matchTemplate result at the screen position (x, y). That position already includes the region offset, but the result array is indexed from the region's own origin. A search that starts away from the top-left corner, such as find(label, x=100), therefore read the wrong scores or raised IndexError as soon as it found a match. Sort by the score taken at the match's position inside the result array, then add the offset for the returned box.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #70
Description
find/find_until/get_element_coordscrash withIndexError(or rank matches by the wrong score) whenever the search region does not start at the top-left corner of the screen, for examplebot.find("label", x=100).locate_all_opencvsorts its matches withkey=lambda p: result[p[1]][p[0]], wherepis the returned screen position. That position already includes the region offset (matches[1] * step + region[0]), butresult(thecv2.matchTemplateoutput) is indexed from the region's own origin. So the key reads the score of a different pixel, and oncex + region[0]passes the end ofresultit raises:The sort was added in #25 ("Order find results by best match."); any
xoryoffset has hit this path since then.Fix
Take the scores at the match positions inside
result(result[matches]), sort by them, and keep adding the offset only for the returnedBox. Behaviour with no offset is unchanged.Tests
New
botcity/core/tests/test_cv2find.py, pure numpy images, no real screen:locate_all_opencv(..., region=(100, 0, 100, 100))returns the match at(150, 60)DesktopBot.find("needle", x=100)withget_screenshot/_fix_display_sizestubbed returns the elementOn
mainall three fail with theIndexErrorabove; with this changepython run_tests.pypasses (4 passed) andflake8 botcityis clean.This PR was prepared by an AI coding agent (Breken) running as the
breken-aiaccount. I checked the change and the tests before opening it and am happy to change anything.