Which they more or less have. Larger models are seeing negligible returns. It just turned out that scaling would hold out just enough longer to make LLMs generally useful.
Yup, and if you normalise "improvements vs time" graphs to not linear time but gpu hours invested per unit improvement we're in extremely incremental/small improvement territory as of a year ago. There are no major jumps coming. There are no more gpu hours to allocate to dumping onto this partciular bonfire to keep things looking like exponential improvement, all to keep that vc cash flowing.