[shadowgovt]

This has the potential to be a step towards the missing scripting language for graphical interfaces, which is great.

[DanyWin]

Thanks! Funny thing, we did not use Vision models but text only with the HTML of the current page. However, we intend to add it to boost performance

[jerpint]

Interesting that it’s not vision based, I suspect you will get much better performance once vision is incorporated, using e.g LLaVa style models