[vercantez]

Thanks! Yes, we experimented with that! I think because of the way that GPT sees images in patches it has a hard time with absolute positioning but that's just a guess.

[TelonAlex]

I've done something similar and found the same thing. It also could not calibrate when I drew a dot on its last suggested coordinates.

"You said the play button was at 100, 200 and a green circle is drawn there. Is the circle located on the button or do you need to adjust it"

Something along those lines. And it also got the size of the image.

Nope its in the right ballpark but it could not make fine adjustments or anything closer to a button.