[antonvs]

> You can train an LLM on pre-1904 literature, but you won't get special relativity from it, at least not without a human to prompt it in just the right way.

At least 99.999% of humans aren't capable of producing special relativity either. If the bar for AGI is "must be at least as smart as Albert Einstein", one has to wonder why the deck is being stacked so unreasonably.

> LLMs as a human replacement

"Human replacement" and AGI don't seem like perfect synonyms to me.

It seems to me that "AGI" does a better job of revealing the biases of the people using the term than identifying a specific set of capabilities.

[avidiax]

It's not just special relativity that's out of reach. It's generally difficult for an LLM to do anything novel, i.e. produce a new hypothesis from scientific data that fits no existing hypothesis, or create an algorithm with a new lower bound on runtime, or debug a proprietary system that makes unusual design assumptions.