Hacker News new | ask | show | jobs
by bobsomers 14 days ago
And for generating an absolutely gargantuan amount of CSAM and non-consensual sexualized images, but yeah, exfiltrating data too.
4 comments

You're the one wanting to generate that though
No other model is so easy to generate such things. No model is so negligent in adding safeguards. I've seen it generate such things in response to a post that was clearly labeled as a 4th grader. The person you are talking to is responding to instances like that. They're not asking for it, that's obnoxiously silly and disingenuous.
Sounds like you've tried generating such things. You've seen things people wouldn't believe. Inappropriate AI images off the shoulder of Orion, etc.

Now the counter-weight over-compensation. Insisting the tool "doesn't have enough safeguards". Could it be that our AI usage innocence is related to how much we criticise the tool for not policing us enough?

Meanwhile the rest of us have never seen such images because we're not looking for those images, or trying to prompt for those images, or thinking about those images. How about the people using these tools take responsibility, rather than forcing colossal layers of safeguards on everything and everyone?

This is in headlines, dude. This smear attempt is intellectually dishonest and eyeroll inducing.
You said: "I've seen it generate such things"

By your own admission, you've seen it generate such things.

Meanwhile, I've never seen it generate such things because I don't write sicko prompts, and the guardrails in place prevent such images. Most people have never seen it generate such things, but apparently you have. I couldn't be more intellectually honest if I tried.

It started by school kids generating images of their peers.

Researchers and police investigated this -> experts.

People found out (like the researches and investigators) that its very easy to generate this type of content.

Elon Musk himself though absolutly knows about this and how easy is to generate porn. Its his thing, freedom before anything else. Why would anyone tell the richest person on the planet what he can or can't do.

Its disappointing to see that you don't think that the richest person on the planet doesn't need to do more to protect miss use but reframe it like this.

> "Elon Musk himself though absolutly knows about this and how easy is to generate porn."

Your claim directly contradicts Musk's statement on the issue:

https://x.com/elonmusk/status/2011432649353511350

Your claim also doesn't align with actual testing of the platform. This raises the question, why are you here posting disinformation?

You can't "generate" CSAM. CSAM definitionally had to be about abuse of real children. It's still bad and should be illegal but lumping them together is bad.
You do realise that saying "SpaceX are producing and distributing child porn, not CSAM" doesn't actually make things better for them, don't you?
Personally I think AI should be treated like photoshop. If someone is creating life-like imagery of child sexual abuse that you can't determine if its real or not, then that would be the criminal action of the person producing it. Software companies should not be acting like overlords on how their software can be used.

People get upset when states try to legislate that 3D printer manufacturers should detect and prevent people trying to make gun parts, but somehow this is different.

Sure. And SpaceX (or anyone else) were to run a child porn service using Photoshop, they should be held accountable for that too.

The tooling doesn't matter, it's the nature of the images they were generating and distributing that was the problem.

If I use a shovel to kill a man, the shovel maker did not engage in intentionally crafting a weapon of war.

How tools are used are a reflection of the people who use them, and I definitely sympathise that tools should have guardrails to not enable this, or at least detect it.

But if a pedophile uses Whatsapp to groom a child; I don't go after Whatsapp for being a neutral service... I go after the pedophile.

Just as well Grok isn’t a shovel then, hey?

If a shovel manufacturer was notified numerous times that their shovel was being used for murder and they had the capability to disable using the shovel for murder while retaining all legitimate uses wouldn’t people question why they didn’t do it?

> If a shovel manufacturer was notified numerous times that their shovel was being used for murder and they had the capability to disable using the shovel for murder while retaining all legitimate uses wouldn’t people question why they didn’t do it?

This is impossible-nobody can possibly block all illegitimate uses without also blocking some legitimate ones as collateral damage. Any moderation process (whether automated or human) inevitably has a non-zero false positive rate.

Now, you can argue that some misuse is so harmful, that the cost of false positives is worth it - but that’s a different claim.

I didn’t say block all illegitimate uses, though. We’re talking very specifically about disabling the production of CSAM. Which is something Grok seems to be able to do now! So I’m curious what legitimate uses had to be sacrificed in order to do so.
> I didn’t say block all illegitimate uses, though. We’re talking very specifically about disabling the production of CSAM

But what is “CSAM”? If by it you mean illegal material-different jurisdictions worldwide have different laws on that topic, so material which is illegal in one jurisdiction can be legal in another.

Ok, then let’s just say CSAM by definition of US law.

Twice now you’ve tried to expand the parameters of this so that it becomes something impossible to tackle. But there’s no actual reason to do that.

Grok is able to tackle CSAM, as demonstrated by the fact that they are currently doing it. The question is why they ignored the very public issue for as long as they did.

It was CSAM enough that the EU is investigating this.

Why are you muddling the topic?

If WhatsApp knew their platform was facilitating CSAM, and they were fully within their power to prevent this but chose not to - yes this would rightly draw criticism…
oh, we're just making shit up now because we don't like a company..

ok then.

Which part of that is made up?
the part where you claim they could “prevent this”, and imply it would be trivial.

Generative AI, famously, has difficult guardrails and there are constant “jailbreaks”.

I guess you mean that they could have been overzealous with the prevention of all “spicy” content to prevent this, but I don’t think thats even as true as you claim.

I’m no expert but it seems quite simple to detect the presence of children in an image and detect the presence of sexual content in a prompt. They expected their model to be used for porn, did they not expect pedophiles to use it? At best it’s an embarrassingly incompetent oversight.
Why is it so hard to generate with other models, but trivial with Grok?
Nope its a lot easier to generate anything with grok than with others.

Its not magic here and is well known.

People online regularly discuss if/when grok gets more or less flexible again as they do react based on media outcry.

And no its a LOT harder to generate anything realted to porn or naked or sexual on any other big well known image generator.

Ok, but what if all Whatsapp competitors explicitly banned the ability to groom children on their platform, but Whataspp didn't, and directly advertised it.
I find the premise of your comment completely incredulous.

I totally understand tribalism, and Elon and X aren't exactly well favoured. (not even by me)

But what you're saying right now is that they advertised the fact that they can create child pornography and deepfakes..

I simply don't believe it, unless you provide evidence.

Elon himself promoted Grok’s “spicy mode” that allowed generating NSFW content that the other AI vendors wouldn’t touch with a 20 foot pole.

Believe whatever you want. Elon’s beliefs and personality problems have been baked into the core of Grok, so it’s no surprise that it turned out to be a CSAM-generating MechaHitler that steals people’s data.

Anybody surprised when Grok turns out to be trash really should read up on the guy who made it.

Tumblr also permitted some more risqué content.

Yet we (rightly) condemned those that used this leniency to do nefarious things.

I'm really ready to get on the Elon hate train, and I will grant you that there was a problem that needs fixing, but I'm really not happy with the amount of censorship on these generative AI platforms.

Groks harness also clearly biases towards Elons views, Yet the washington post claims it's the most even handed and least likely to give politically biased answers: https://www.washingtonpost.com/technology/interactive/2026/0...

idk how to interpret all this, despite being genuinely anti-Elon, I don't think I'm personally willing to immolate a company forever because the guardrails were temporarily too loose.

I'm not trying to make an equivalency for facts vs deepfake porn, but there is one there unfortunately, and overall internet freedom has been curtailed a lot by advertising friendliness.

I also don’t think one mistake should define a company. But for me it’s just about trust.

Musk has proven time after time that he doesn’t deserve my trust. I will never trust Grok as long as he’s in charge of it.

I agree that the guardrails on the top models have gotten out of hand, though.

Fable for instance won’t answer even basic health questions. As if you are going to take nutrition advice and make a bioweapon with it.

Partly this is due to government interference. Hopefully we get to a better place as competition heats up with open and Chinese models.

Are you suggesting Grok hired some people on the dark web or some shit? An AI model can't generate CSAM what kind of bullshit are you spewing.
Might want to double-check that one, chief. xAI itself disagrees with you.

They admitted in court that Grok was used to generate CSAM: https://edition.cnn.com/2026/07/15/business/xai-sues-user-al...

Them attempting to claim something in a lawsuit doesn't make it a fact. Nothing mentioned in those claims is CSAM.
xAI literally says in their own lawsuit, "Defendant breached the xAI Terms of Service and Acceptable Use Policy by leveraging Grok to generate non-consensual sexually explicit images and CSAM".

"CSAM" is literally right there. If you think know better than xAI itself, take it up with them.

Twitter and Musk don't get to define USA law no matter what you think.
"CSAM" is literally right there. If you think know better than xAI itself, take it up with them.
https://arstechnica.com/tech-policy/2026/07/lawsuit-grok-use...

This is only the most recent example. Plenty of headlines like this one if you'd even bother to look.