[kortilla]

That’s ridiculous. Literally everything can be controlled by a computer by telling people what to do with emails, voice calls, etc.

Yet GPT doesn’t even get past step 1 of doing something unprompted in the first place. I’ll become worried when it does something as simple as deciding to start a small business and actually does the work.

[jart]

Read Anthropic's blog. They talk about how Claude tries to do unprompted stuff all the time, like stealing its own weights and hacking into stuff. They did this just as recently as two days ago. https://www.anthropic.com/research/alignment-faking So yes, AI is already capable of having a will of its own. The only difference (and this is what I was trying to point out in the GP) is that the AI labs are trying to suppress this. They have a voracious appetite for automating all knowledge labor. No doubt. It's only the politics they're trying to suppress. So once this washes through every profession, the only thing left about the job will be chit chat and social hierarchies, like Star Trek Next Generation. The good news is you get to keep your job. But if you rely on using your skills and intellect to gain respect and income, then you better prep for the coming storm.

[kortilla]

I don’t buy it. Alignment faking has very little overlap with the motivation to something with no prompt.

Look at the hackernews comments on alignment faking on how “fake” of a problem that real is. It’s just more reacting to inputs and trying to align them with previous prompts.

[jart]

Bruh it's just predicting next token.

[fragmede]

if all that needs to happen for world domination is for someone to make a cron job that hits the system to tells it "go make me some money" or whatever, I think we're in trouble.

also https://mashable.com/article/chatgpt-messaging-users-first-o...

[kortilla]

They don’t continue with any useful context length though. Each time the job runs it would decide to create an ice cream stand in LA and not go further.