[devjab]

I think it’s less of an issue for e2e testing because e2e testing sucks. If teams did it well in general you would be completely correct, but in many places a LLM will be better even if it hallucinates. As such I think there will be a decent market for products like this, even if they aren’t may not even really be testing what you think they are testing. Simply because that may well be way better than the e2e testing many places already do.

In many cases you’re correct though. We have a few libraries where we won’t use Typescript because even though it might transpire 99% correctly, the fact that we have to check, is too much work for it to be worth our time in those cases. I think LLMs are similar, once in a while you’re not going to want them because checking their work takes too much resources, but for a lot of stuff you can use them. Especially if your e2e testing is really just pseudo jobbing because some middle manager wanted it, which it unfortunately is far too often. If you work in such a place you’re going to recommend the path of least resistance and if that’s LLM powered then it’s LLM powered.

On the less bleak and pessimistic side, if the LLM e2e output is good enough to be less resource consuming, even if you have to go over it, then it’s still a good business case.