[canjobear]

Around the time the phrase "deep learning" came into vogue, the advances were indeed in training deeper networks, not wider. Later on it turned out that shallow wide networks are sufficient for many problems. (Also, it turned out the pre-training tricks that people came up with for training deep networks weren't really necessary either.)