[nl]

Trivially, you could then "copy" the weights that a ANN would learn into a kernel and obtain the same accuracy.

Sure.

I'm not sure I agree with this bit in theory.

No one really agrees with it in theory - I'm not aware of a good theoretical explanation as to why some deep networks are easier to train. And yet there is a growing body of real, generalized practical hints which work pretty reliably.

This is pretty exciting! There is undiscovered ideas here. But it is unsatisfactory from the theoretical sense at the moment.