[choppaface]

The authors draw from traditional genetic algorithms (NEAT http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf ) but this appears to be one of the first papers (at least recently) where proposed architectures are not trained but rather given random weights. The authors here try to qualitatively distill the role of architecture (vs optimization) in neural net research.

The work here is in a similar vein as the Lottery Ticket Hypothesis ( https://arxiv.org/pdf/1803.03635.pdf ), which found that deep nets (for vision) contain discriminative sub-networks at initialization time (due to random initialization), before training ever starts.

While the authors of this work on architecture search say they hope to inspire the discovery of new architectures, a more immediately striking result of their work is that they get functioning systems from doing something “stupid” (i.e. not optimizing weights).

[kolar]

Genetic Programming tries to do almost the same: ie. generate random programs/expression trees etc and then select the best one. NN gives you two adventages over that: you can optimize weights and do it at scale with current hardware - without those two I can't see the difference.