Hacker News new | ask | show | jobs
by gnfargbl 23 days ago
You're looking at the status quo and ignoring the trajectory. The best current open models are about as good as closed models from ~1.5 generations ago. The rate of improvement of all models is converging to zero. It follows that in a few generations, open models inferencing will be about as good as closed model inferencing.

The problem is going to become that there's no incentive for anyone to run the stupidly-expensive training phase. May God have mercy on the stock market.
2 comments
zild3d 22 days ago
>The rate of improvement of all models is converging to zero.

Curious where you draw this conclusion from? Most benchmarks still show continual steady progress https://metr.org/time-horizons/

link
gnfargbl 22 days ago
To me it seems like a first-year physics scaling laws problem. To get linear improvements in capability, you appear to need need exponential (or at least superlinear) increases in model size. We have no technical nor business solution for that kind of scaling, so the long-term outcome is obvious.
link
NitpickLawyer 22 days ago
> The rate of improvement of all models is converging to zero.

That's so obviously not true that I don't even think it's worth the energy to even debate it. It's been said for years, yet here we are, constantly improving. People really don't get RL / the bitter lesson, do they?

> It follows that in a few generations, open models inferencing will be about as good as closed model inferencing.

Not a chance. There's hundreds of billions of dollars on one side, and oom less on the other. There's also scaling laws and information theory. No matter how good, a 30B model will not be able to be better than a 3T+ model, all things being equal.

You are mistaking models becoming "good enough" for an increasingly number of tasks, which I agree is happening, with SotA models stagnating, hitting walls etc. That will not happen for many many years to come.

link