[ben_w]

> Don't think LLM itself will ever be able to out compete competent human + LLM

Perhaps, perhaps not. The best performing chess AI, are not improved by having a human team up with them. The best performing Go AI, not yet.

LLMs are the new hotness in a fast-moving field, and LLMs may well get replaced next year by something that can't reasonably be described with those initials. But if they don't, then how far can the current Transformer style stuff go? They're already on-par with university students in many subjects just by themselves, which is something I have to keep repeating because I've still not properly internalised it. I don't know their upper limits, and I don't think anyone really does.

[withinboredom]

Oh man. Want to know an LLM's limits? Try discussing a new language feature you want to build for an established language. Even more fun is trying to discuss a language feature that doesn't exist yet, even after you provide relevant documentation and examples. It cannot do it. It gets stuck in a rut because the "right" answer is no longer statistically significant. It will get stuck in a local min/max that it cannot easily escape from.

[ben_w]

> Want to know an LLM's limits?

Not a specific LLM's limits, the limits of LLMs as an architecture.

[withinboredom]

This is a limit of an LLM's architecture. It is based on statistics and can only answer statistical questions. If you want it to provide non-probable answers, an LLM won't work.

[jdietrich]

>It is based on statistics and can only answer statistical questions.

"LLM" isn't an architecture. The transformer architecture used by all the leading LLMs is Turing complete.

https://jmlr.org/papers/volume22/20-302/20-302.pdf

[ben_w]

Careful, statistics is a place where you need to be very careful about what exactly you mean: https://en.wikipedia.org/wiki/Bertrand_paradox_(probability)

Your brain is also based on statistics. We also get stuck in a rut because the "right" answer is no longer statistically significant.

And yet this is not what limits our cognition.

Current LLMs are slow to update with new info, which is why they have cut-off dates so far in the past. Can that be improved to learn as fast (from as little data) as we do? Where's the optimal point on inferring from decreasing data before they show the same cognitive biases we do?

(Should they be improved, or would doing that simply bring in the same race dynamics as SEO?)

[withinboredom]

Even humans are not good at this. The US military has a test (DLAB) to figure out how good you are at taking in new information in regards to language -- to determine if it is worth teaching you new languages. Some humans are pretty good at this type of thing, but not all. Some humans can't even wrap their heads around algebra but will sell you a vacuum cleaner before you even realize you bought it.

The problem with LLMs is that there is one and it is always the same. Sure, you can get different ones and train your own, to a degree.

[fire_lake]

> They're already on-par with university students in many subjects just by themselves, which is something I have to keep repeating because I've still not properly internalised it.

That’s because it’s not really true. There are glimpses of this but it trips up too often.

[ben_w]

So do the students :D