[danaris]

But we also hit serious diminishing returns effects—in the last decade for hardware, in the last 2-3 years for LLMs.

Just because things have advanced in leaps and bounds for X amount of time does not mean they will continue to do so at the same (or increasing) rates indefinitely.

The problem is that just what will end up being the thing that advances dramatically in the next few years is often very unpredictable, especially if one is looking at surface-level trends over the last few years.

Given what we, who have a better understanding of them than "wow, look how far they've come since 2018", can see, LLMs seem unlikely to advance by another 100x over the next several years. Frankly, another 100% seems unlikely. (Though of course, quantifying LLM performance is a tricky proposition to begin with...it is, to a large extent, a qualitative endeavor.)