[OceanKing]

I read this paper as commenting on the second-order derivative of benchmark progress with respect to DL techniques. The trends presented start with 2012-era models with 2012 data, and end with 2020 results for 2020-era models. Thus the extrapolation on training costs accounts for the current pace+style of progress and innovation in all relevant subareas of ML. To me it seems that it’s saying “if research continues in the same trends as it has from 2012 to 2020, here is where we will end up in 2035.”

In other words, in order for us to buckle the trend we would need to start innovating in ways that are unlike the ways that got us from AlexNet to here.