[taurath]

> LLMs with sufficient scaling can become frontier AI researchers kicking off the exponential

The cost is already outrunning the benefit to a massive amount, and the predicted expotential is not here yet. I predict it'll always be around the corner, a $1T model won't get there, but it will "look promising", but we'll sadly run out of money for the $10T or $100T model..

[openquery]

The only thing the $1T model needs to do is find some algorithmic speedup which allows it to be trained at $100B. I'm not saying that's easy or that it will happen but I just don't see why not.

[taurath]

I don't understand the logic here - a methodology to increase efficiency by 90% in training doesn't exist until it does - could you explain what you mean by "I don't see why it won't exist"? Are you seeing consistent gains by some process?