[Wowfunhappy]

…but I think I kind of agree with this argument. Technology is a tool that can be used for good or for ill. We shouldn’t outlaw kitchen knives because people can cut themselves.

We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office. Why is it that as soon as generative AI comes into the mix, we hold the AI companies responsible for what users are able to create?

Not only do I think the companies shouldn’t be responsible for what users make, I want the AI companies to get out of the way and stop potentially spying on me in order to “enforce their policies”…

[ryandrake]

> We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office.

Photoshop and Office don't (yet) conjure up suicide lullabys or child nudity from a simple user prompt or button click. If they did, I would absolutely expect to hold them accountable.

[ls612]

The political economy equilibrium enabled by technology very much goes the other way though. Once politicians realize they can surveil everyone in real time for wrongthink and wrongspeak they have existential incentives to seize that power as fast as possible, lest another power center seize it instead and use it against them. That is why you are seeing the rise of totalitarianism and democratic backsliding everywhere, because the toxic combination of asymmetric cryptography (for secure boot/attestation/restricting what software can run), always online computers, and cheap data processing and storage leads to inexorable centralization of soft and hard power.

[mirabilis]

Some manufactures of knives could still be recalled for safety reasons, and MS Office/Google Drive certainly have content prohibitions in their TOS once you’re dealing with their online storage. I agree with your metaphor in that I doubt much use would come from banning AI entirely, but I feel there must be some viable middle ground of useful regulation here.

[SirFatty]

"We shouldn’t outlaw kitchen knives because people can cut themselves."

How about if the knife would convince you to cut yourself?

[nomel]

We call that a "mental disorder".

[g-b-r]

Or a smart knife

[carefulfungi]

If you encourage someone to kill themselves, you are culpable. OpenAI should meet that standard too.

[Wowfunhappy]

OpenAI didn't encourage anyone to do anything. They made some software that semi-randomly puts words together in response to user input. This type of software isn't even new—I can definitely get Eliza to say terrible things with the right input, and Eliza even bills herself as a therapist!

[4d4m]

Let me get this straight - so the safety team at OpenAI - what exactly are they working on? Is it all focused on censorship of inputs and results, and steering how you think? Or are they not responsible for designing against these horiffic outcomes as a primary goal?

I would take the view that their safety team is maybe focused on the wrong things (former) and has been captured by extremists instead of pragmatists, but that's like just my opinion man. I'll use Anthropic and Venice until I notice less steering in my threads, personally. An GPT that constantly eggs me on isn't a thought-partner, it's a dopamine device. If I'm going to outsource my thinking to an LLM I need something I trust won't put it's own spin on things or gas me up into taking action I never originally intended to do without critical thinking first.

[g-b-r]

We don't know how much aware of the problems (or of tbe likelihood that they'd occur) OpenAI was, and how much they deliberately pushed through.

If they were and did, they sure bear responsibility for what happened

[Wowfunhappy]

What if OpenAI knew responses like this were likely, but also knew preventing them would degrade overall model quality?

I'm being selfish here! I am confident that no AI model will convince me to harm myself, and I don't want the models I use to be hamstrung.

[g-b-r]

What if they knew that preventing them would reduce engagement and revenue?

We just don't know, and it seems sensible to me to investigate it.

Were it only to not degrade the quality model, anyhow, I think it's reasonable that someone's life could be more important than that, but that's me.

> I'm being selfish here! I am confident that no AI model will convince me to harm myself, and I don't want the models I use to be hamstrung.

I do see that you're being selfish

[g-b-r]

By the way, «logs show he tried to resist ChatGPT’s alleged encouragement to take his life».