[FeepingCreature]

All current LLMs are based on the premise that magic is real and you can wish intelligence into existence; it's called "scaling laws" and "emergent capabilities".

Recursive self-improvement isn't "maybe magic is real", it's "maybe the magic we already know about stays magical as we cast our spells with more mana."

[z7]

Doesn't this line of reasoning imply that human intelligence is magical, i.e. is not the result of scaling/emergence?

[FeepingCreature]

Reality is never magical, by definition. Magic just means that we are using something without understanding it.

Whatever our brains are doing internally isn't magical. But it's magic to us because we don't know how it works. So too with current LLMs.

My point is we're already doing things with LLMs that we don't understand and that we didn't think were attainable until two years ago. We don't know how to do superintelligence and recursive self-improvement... but we're off in uncharted territory already, and I think there's a lot more grounds for positive uncertainty about self-improvement than there was before GPT-3.

[actionfromafar]

The answer is a firm maybe, depending on some factors.