[ldjkfkdsjnv]

Yeah I'm also getting suspicious. Also, all of the models (opus, llama3, gpt4, gemini pro) are converging to similar levels of performance. If it was true that the scaling hypothesis was true, we would see a greater divergence of model performance

[bigyikes]

Plot model performance over the last 10 years and show me where the convergence is.

The graph looks like an exponential and is still increasing.

Every exponential is a sigmoid in disguise, but I don’t think there has been enough time to say the curve has flattened.

[MP_1729]

Two pushbacks.

1- The mania only started post Nov 22. And the huge investments since then didn't meant substantial progress since GPT-4 launch in March 22. 2- We are running out of high quality tokens in 2024. (per Epoch AI)

[dwaltrip]

GPT-4 launch was barely 1 year ago. Give the investments a few years to pay off.

I've heard multiple reports that training runs costing ~$1 billion are in the the works at the major labs, and that the results will come in the next year or so. Let's see what that brings.

As for the tokens, they will find more quality tokens. It's like oil or other raw resources. There are more sources out there if you keep searching.