[tmpmov]

For those interested AutoML-Zero cites "Evolving neural networks through augmenting topologies" (2002) among other "learning to learn" papers and is worth a read if you have time and inclination.

For those with more background and time, would any mind bridging the 18 year gap succinctly? A quick look at the paper reveals solution space constraints (assuming for speed), discovering better optimizers, and specific to the AutoML-Zero paper: symbolic discovery.