[Workaccount2]

That industry will only form if LLMs don't improve from here. But the evidence, both theoretical and empirical, is quite the opposite. In fact one of the core reasons transformers gained so much traction is because they scale so well.

If nothing really changes in 3-5 years, then I'd call it a flop. But the writing is on the wall that "scale = smarts", and what we have today still looks like a foundational stage for LLM's.

[namaria]

> In fact one of the core reasons transformers gained so much traction is because they scale so well.

> If nothing really changes in 3-5 years, then I'd call it a flop

Transformers have been used for what 6 years now? Will you in 6 years say "I'll decide if they don't change the world in another 6 years?"

[Workaccount2]

If the difference between now and 6 years in the future is the same as the difference between now and 6 years ago, a lot of people here will be eating their hats.

[namaria]

Why? What exactly have we got for the (how many hundred) billions of dollars poured into GPUs running transformers over the past 6 years?

[Workaccount2]

You don't believe that models 100x better than today (OG transformers were pretty bad) would be fruitful for society?

[mattgreenrocks]

Self-driving cars have been 3-5 years away for what, a decade now?

[Workaccount2]

I never paid much attention to Elon.