[gacgacgac]

I don't think you understand the concern. Or at least nothing you've communicated suggests you understand it.

ChatGPT should never produce images like this. Full stop. Prompted or not, it should refuse. Now we know it's possible to walk around the gate and get it to comply. Are there other, genuinely harmful images that it should never produce? Deepfake revenge porn? Images of specific people being brutalized? I'd argue those absolutely can be harmful to someone. Well now there's evidence the "never produce this" wall can be overcome. It's only a matter of time before genuinely harmful imagery is generated.

[samlinnfer]

It may be harmful to someone if shared and sent with malicious intent, but more damage has been done with pens, keyboard and words. Start banning pens that let people write hurtful things next. Ban Photoshop after because someone can get hurt with a manipulated image.

[gacgacgac]

If you think these tools are remotely comparable, you haven't been paying attention.

[andsoitis]

> ChatGPT should never produce images like this. Full stop. Prompted or not, it should refuse.

Why not?

[gacgacgac]

Because the company has said they won't. I'm not making a value judgement about what images should exist here, I'm making a "the company said it shouldn't be capable of producing that output, then it does" argument. Thats a bug.