[LouisSayers]

> You can't possibly know that, given that we don't actually understand how LLMs work on a high level.

It's a fair assumption to make however - basically 80/20 rule.

AI research isn't a new thing and I bet you could go back 40/50 years where they thought they were about to have a massive breakthrough to human level intelligence.

> GPT-4 is three months old and you're confident that its working principle cannot be extended further? Where do you get that confidence from?

I'm guessing from actually using it.

GPT4 is super impressive and helpful in a practical way, but having used it myself for a while now I get this feeling also. It feels a bit like "it's been fed everything we have, with all the techniques we have, now what?"

[cornel_io]

There are dozens and maybe hundreds of different approaches that could theoretically get around the limitations of GPT4 that merely haven't been trained at scale yet. There is absolutely no lack of ideas in this space, including potentially revolutionary ones, but they take time and money to prove out.

[LouisSayers]

I'm sure there are lots of ideas, but it doesn't mean they're any good or will necessarily transform AI to the next level.

It's going to take time to figure out what works and what doesn't.

There's a reason why Sam Altman is saying they're not training GPT5, and it's not because they think GPT4 is good enough.