Hacker News new | ask | show | jobs
by omneity 965 days ago
UIPath[0] and many others do. It's actually super common, both needed to support desktop apps built using custom UI toolkits (from memory, xaml and others actually give you something DOM-like to hook into, but others like javafx don't) and it's also useful since a lot of enterprise software runs within VDI/VNC/... so all you have (at a low level) is a "screenshot" and the ability to click on coordinates or trigger keyboard events. [1] is what surface automation looks like in practice.

[0]: https://www.uipath.com/product/ai-computer-vision-for-rpa

[1]: https://images.ctfassets.net/5965pury2lcm/4hwXQpRTQkIIAy2q8k...