[zomgbbq]

I would love to use this for E2E testing. It would be great to make all my assertions with high level descriptions so tests are resilient to UI changes.

Seems similar to the Amazon Nova Act API which is still in research preview.

[jbarber]

This is harder than you might expect because it's hard to tell whether a passing test is a false positive (i.e. the test passed, but it should have failed).

It's also hard to convey to the testing system what is an acceptable level of change in the UI - what the testing system thinks is ok, you might consider broken.

There are quite a few companies out there trying to solve this problem, including my previous employer https://rainforestqa.com

[nextworddev]

LLM as judge

[sumedh]

Use Playwright and some AI model to write Playwright script, running those scripts will be much faster.