[mlajtos]

I think the argument is that before ResNets, the depth of the network was constrained and could not be scaled easily. With ResNets (and of course Highway Nets; hi Jürgen!) the depth is just another hyperparam.

[mirker]

In between those, VGG was parameterized by layers, though severely limited in scope (16-19 layers).