[bjackman]

Yes I think if this works it's one of the strongest signals yet that LLMs as they currently exist have a fairly general form of intelligence.

I am pretty sympathetic to the field of "AI Safety" and I worry a lot about the implications of agent-like general intelligences. This post gives me a lot to ponder.

What are the implications of the fact that even AIs that are not agent-like at heart can apparently be told "please simulate an agent-like AI"? I really don't know. Should we consider it as an "inner AI" with its own objectives? How can we determine what those objectives are?

Instinctively it feels much less concerning than an AI with a direct action->perception feedback loop but who knows. AI is fucking weird.

What a thrilling time to be alive!