Hacker News new | ask | show | jobs
by Jefro118 1714 days ago
Identifying the elements within a GUI image (e.g. this is a button, that's an input field, etc.). I want this myself for a tool I'm building to turn Figma designs into code but it's also useful for things like automated testing. There are a bunch of papers on this already but no good public version that I can find. Probably companies like UIPath already have a sophisticated version of this internally. If you could do this and turn it into an API it would be quite valuable I think.