[nixpulvis]

Knowing what you want is both the fun and the hard part of most software development. That and understanding the environment you are operating on. Both of these things are not something I feel an AI has a strong advantage with given the fact that you need to provide this information.

Perhaps I can try to say, "Please decide what browser features I should use to maximize profit.", and it will actually do some proper research and contract out studies and all that... but I don't see that happening yet. Even still, the point of validation remains.

I can't help but draw a parallel with automated theorem provers. Sure they resolve "true", but then who validates the validators? It's a never ending cycle.

At the end of the day, you must learn to love and trust your tools and then stand by the joint creations.

[auggierose]

In the case of automated theorem provers, they should generate a certificate together with "true", so their proof can be checked. And that is also the answer to checking AI: let them generate a proof that what they did was correct. Of course, that only works if you ask the AI for a theorem, but a lot of tasks can be framed that way.