[gmueckl]

Understanding an existing solution is very different from trying to find it. We get taught things at school that seem simple and obvious to us now, but were extremely hard to discover or invent.

I see these training approaches as a tool that makes machines search the space of potential solutions for certain types of problems more efficient than humans. The next logical step is to try and understand these solutions and improve upon them.

Consider adversarial input. Right now we cannot tell which classes of adverserial inputs can exist for a given NN. We can only try to find representative examples. If you had a good enough understanding of how the NN works internally, you have a chance to derive the full set of adversarial inputs or - maybe -prove their absence.