[Retr0id]

> Let us suppose some AI came up with an algorithm A to solve a specific problem P. To ensure that A solves P, the AI has to execute A to test its functionality.

Not necessarily. It could (in theory) prove through formal methods that A solves P.

[PartiallyTyped]

Taking this a step further, if the AI could write Daphne, it could rely on invariants and pre and post conditions for this proof, no?