[oneshot908]

AlexNet #s IMO provide an excellent ballpark estimate of how well balanced compute and communication are in terms of both the framework and the underlying platform.

A platform that runs AlexNet well has excellent computation performance for the convolution layers but it also has excellent algorithms/communication for parallelizing the model/data by whatever means.

Networks that attempt to minimize computation and/or communication are cool, but they should be considered in that light IMO.

It's also a great estimate of the low-end for strong scaling. There's a lot of bread and butter machine learning at this level in my experience.