[versteegen]

You forget that, at that time, OpenAI pushed SoTA LLMs massively forward by scaling them up ~3-4 orders of magnitude when others didn't think that would work or weren't willing to spend the money. But not just that. Following their example, Google and nVidia also attempted to scale up transformers but without really managing to push the SoTA.

So, I agree instead with meowface, and think it could even have been a 5+ year delay rather than 2 or 3. If you look at breakthroughs rather than incremental improvements, 5 years is not a long timescale. (And if OpenAI hadn't have made their breakthroughs, production of the highest-end GPUs/TPUs would be nowhere near where it is today.)

(I'm not attempting to justify OpenAI's structure or behaviour, just want to comment on one point.)