[avidiax]

Present LLMs are quite good at interpolating, in fact, too good.

That's the source of hallucinations. A path can be found between A and B, even if A is the 12th century Chinese royal court and B is the Easter bunny.

Interpolation and rote knowledge are still very useful. Most cognitive tasks are like this.

The thing that LLMs are not presently good at is extrapolation. You can train an LLM on pre-1904 literature, but you won't get special relativity from it, at least not without a human to prompt it in just the right way.

You can have an LLM provide a "novel math proof", but you are necessarily discarding 100 or 1,000 "novel math mistakes". The process is more like a guided walk (like the A* algorithm), with human supervision and intervention, not an autonomous math genius.

"They" are, of course, working on it. But the present implementation has some severe structural limitations (such as an inability for new or discovered information to affect model weights) that make LLMs as a human replacement incomplete.

[antonvs]

> You can train an LLM on pre-1904 literature, but you won't get special relativity from it, at least not without a human to prompt it in just the right way.

At least 99.999% of humans aren't capable of producing special relativity either. If the bar for AGI is "must be at least as smart as Albert Einstein", one has to wonder why the deck is being stacked so unreasonably.

> LLMs as a human replacement

"Human replacement" and AGI don't seem like perfect synonyms to me.

It seems to me that "AGI" does a better job of revealing the biases of the people using the term than identifying a specific set of capabilities.

[avidiax]

It's not just special relativity that's out of reach. It's generally difficult for an LLM to do anything novel, i.e. produce a new hypothesis from scientific data that fits no existing hypothesis, or create an algorithm with a new lower bound on runtime, or debug a proprietary system that makes unusual design assumptions.