[abetusk]

I'm shocked there's so much push back on this here. This was written in 2019 but it applies even more so today. All the recent advances in AI for the past decade have, as a base requirement, massive amounts of compute. Not a proof but it should be a big red flag that if you don't believe Sutton, you should maybe consider.

Put it this way. Do you believe premature optimization is the root of all evil? Then why do you believe that some subtle, intricate optimization for machine intelligence will win out over brute compute force? It's not that there's optimizations to be done, it's knowing what optimizations will yield the most value after the space is explored with better computing capabilities.

To me, this is almost like a generalized "Proebstings Law" [0], where compiler optimizations give a doubling every 18 years compared, to say, some type of generalized Moore's law which give roughly a doubling in compute every 1.5 to 2 years.

[0] https://zeux.io/2022/01/08/on-proebstings-law/