find in offset screen region raises IndexError or ranks matches incorrectly #70

Describe the bug

DesktopBot.find can raise IndexError when a caller searches inside a screen region that does not start at the top-left. In locate_all_opencv, match coordinates are translated by the region origin for returned Box values, but the same screen coordinates are used to index the template-match result array, which is local to the region. This can read a different score or exceed the result array bounds. The current main branch at 9d5deaa89520c6e4cec61248f2c0c71931160866 reproduces the failure with a mocked image search.

Expected behavior

The region offset should affect the returned screen coordinates only. find, find_until, and get_element_coords should return matches and rank them using the scores at their region-local match positions.

Steps to Reproduce

1. Call bot.find("label", x=100) (or search a region with a nonzero x/y origin).
2. Arrange for a match to occur far enough inside the region that its translated screen x/y is outside the local match-result array.
3. Observe an IndexError, such as index 110 is out of bounds for axis 0 with size 91.

The pure-array regression tests on existing PR #67 reproduce the crash and the incorrect score ordering without a physical screen.

Possible Solution

Use the region-local positions returned by cv2.matchTemplate to read scores and sort matches; add the region origin only when constructing returned Box coordinates.

My Platform

Observed on repository main at 9d5deaa89520c6e4cec61248f2c0c71931160866; reproduction uses mocked numpy/OpenCV image data, no physical display required. Exact first affected package release is not established.

Additional context

This ticket is being filed to track already-open PR #67, which predates it, and make the contribution issue reference explicit. Existing issue #51 is about macOS display scaling and incorrect mouse coordinates; that is a separate coordinate-conversion problem.

This contribution was prepared with AI assistance by breken-ai.