[mckoss]

It does seem he is conflating "progress" with "investment". Yes, the world is spending exponentially more compute each year since 2012 on training networks. The marvel is that neural architectures are scaling to more complex problems without much architectural change. But this is not an argument that AI is getting more efficient or productive over time and hence we can expect exponential performance improvements (like Moore's law).