I guess what you are saying would probably have been said by AI skeptics in the 70s, but LLMs provided a quantum leap. Yes, progress is often asymptotic and governed by diminishing returns, but discontinuous breakthrough must also be factored in.
Please tell me what quantum leap was provided by LLMs.
Please inform me of any developments that made current LLMs possible.
I contend that there are none. Witness the actual transformer kernel technologies over the last 20 years and try to find a single new one.
Neural Networks? that's 90's technology.
Scale is the only new factor I can think of.
This is an investor-dollar driven attempt to use brute-force to deliver "magical" results when the fundamental technology is being mis-represented to the general public, to CTOs, and even to Developers.
Technological advances tend to happen in jumps. We are no doubt approaching a local optima right now, but it won't be long until another major advancement propels things forward again. We've seen the same pattern in ML for decades.
Please name me one technological advance of major import in the fundamental transformer kernel space that has occurred in the last decade that has any import at all on today's LLMs.