[laretluval]

Fwiw the "deep learning" advances in NLP have typically still been from shallow networks, almost always less than 10 layers and usually more like 2.