[dijit]

If I use a shovel to kill a man, the shovel maker did not engage in intentionally crafting a weapon of war.

How tools are used are a reflection of the people who use them, and I definitely sympathise that tools should have guardrails to not enable this, or at least detect it.

But if a pedophile uses Whatsapp to groom a child; I don't go after Whatsapp for being a neutral service... I go after the pedophile.

[afavour]

Just as well Grok isn’t a shovel then, hey?

If a shovel manufacturer was notified numerous times that their shovel was being used for murder and they had the capability to disable using the shovel for murder while retaining all legitimate uses wouldn’t people question why they didn’t do it?

[skissane]

> If a shovel manufacturer was notified numerous times that their shovel was being used for murder and they had the capability to disable using the shovel for murder while retaining all legitimate uses wouldn’t people question why they didn’t do it?

This is impossible-nobody can possibly block all illegitimate uses without also blocking some legitimate ones as collateral damage. Any moderation process (whether automated or human) inevitably has a non-zero false positive rate.

Now, you can argue that some misuse is so harmful, that the cost of false positives is worth it - but that’s a different claim.

[afavour]

I didn’t say block all illegitimate uses, though. We’re talking very specifically about disabling the production of CSAM. Which is something Grok seems to be able to do now! So I’m curious what legitimate uses had to be sacrificed in order to do so.

[skissane]

> I didn’t say block all illegitimate uses, though. We’re talking very specifically about disabling the production of CSAM

But what is “CSAM”? If by it you mean illegal material-different jurisdictions worldwide have different laws on that topic, so material which is illegal in one jurisdiction can be legal in another.

[afavour]

Ok, then let’s just say CSAM by definition of US law.

Twice now you’ve tried to expand the parameters of this so that it becomes something impossible to tackle. But there’s no actual reason to do that.

Grok is able to tackle CSAM, as demonstrated by the fact that they are currently doing it. The question is why they ignored the very public issue for as long as they did.

[skissane]

> Ok, then let’s just say CSAM by definition of US law.

“CSAM” isn’t a legal category under US law.

“Child pornography” is a legal category under US law. But, according to the 2002 US Supreme Court case Ashcroft v. Free Speech Coalition (535 U.S. 234), so-called “virtual child pornography” (imagery produced by CGI or AI, not featuring the images of any identifiable real world minors), is (partially) protected [0] by the 1st Amendment, and excluded from the legal definition of “child pornography” in the US. So if “CSAM by definition of US law” you mean “child pornography”, then a lot of the material Grok was (reportedly) producing which people were labelling “CSAM” wasn’t actually CSAM by that definition.

[0] “partially” because it still might be unprotected due to the difficult-to-prosecute obscenity exception to the 1st Amendment, but it is excluded from the scope of the distinct and much easier-to-prosecute child pornography exception

[tick_tock_tick]

USA law would put everything created by Grok as legal.

[Lomlioto]

It was CSAM enough that the EU is investigating this.

Why are you muddling the topic?

[solumunus]

If WhatsApp knew their platform was facilitating CSAM, and they were fully within their power to prevent this but chose not to - yes this would rightly draw criticism…

[dijit]

oh, we're just making shit up now because we don't like a company..

ok then.

[solumunus]

Which part of that is made up?

[dijit]

the part where you claim they could “prevent this”, and imply it would be trivial.

Generative AI, famously, has difficult guardrails and there are constant “jailbreaks”.

I guess you mean that they could have been overzealous with the prevention of all “spicy” content to prevent this, but I don’t think thats even as true as you claim.

[solumunus]

I’m no expert but it seems quite simple to detect the presence of children in an image and detect the presence of sexual content in a prompt. They expected their model to be used for porn, did they not expect pedophiles to use it? At best it’s an embarrassingly incompetent oversight.

[bobsomers]

Why is it so hard to generate with other models, but trivial with Grok?

[dijit]

depends.

With Mango or Guava its even easier.

With the other models, they don’t generate pictures of Uranium because of the aggressive censorship.

If you don’t allow the colour pink, sure, you prevent a lot of porn, but you also prevent a lot of legitimate pictures of roses. Nobody knows how to properly censor these models so most are leaning towards caution, better a lot of false positives than the alternative.

This is most obvious with Fable. It barely functions.

FWIW Grok used to have aggressive controls too- and they loosened them after a lot of feedback because it was barely functional at showing humans, even if it was a popular figure in a non-risque situation, and this is the fallout.

[Lomlioto]

Nope its a lot easier to generate anything with grok than with others.

Its not magic here and is well known.

People online regularly discuss if/when grok gets more or less flexible again as they do react based on media outcry.

And no its a LOT harder to generate anything realted to porn or naked or sexual on any other big well known image generator.

[jazzpush2]

Ok, but what if all Whatsapp competitors explicitly banned the ability to groom children on their platform, but Whataspp didn't, and directly advertised it.

[dijit]

I find the premise of your comment completely incredulous.

I totally understand tribalism, and Elon and X aren't exactly well favoured. (not even by me)

But what you're saying right now is that they advertised the fact that they can create child pornography and deepfakes..

I simply don't believe it, unless you provide evidence.

[brokencode]

Elon himself promoted Grok’s “spicy mode” that allowed generating NSFW content that the other AI vendors wouldn’t touch with a 20 foot pole.

Believe whatever you want. Elon’s beliefs and personality problems have been baked into the core of Grok, so it’s no surprise that it turned out to be a CSAM-generating MechaHitler that steals people’s data.

Anybody surprised when Grok turns out to be trash really should read up on the guy who made it.

[dijit]

Tumblr also permitted some more risqué content.

Yet we (rightly) condemned those that used this leniency to do nefarious things.

I'm really ready to get on the Elon hate train, and I will grant you that there was a problem that needs fixing, but I'm really not happy with the amount of censorship on these generative AI platforms.

Groks harness also clearly biases towards Elons views, Yet the washington post claims it's the most even handed and least likely to give politically biased answers: https://www.washingtonpost.com/technology/interactive/2026/0...

idk how to interpret all this, despite being genuinely anti-Elon, I don't think I'm personally willing to immolate a company forever because the guardrails were temporarily too loose.

I'm not trying to make an equivalency for facts vs deepfake porn, but there is one there unfortunately, and overall internet freedom has been curtailed a lot by advertising friendliness.

[brokencode]

I also don’t think one mistake should define a company. But for me it’s just about trust.

Musk has proven time after time that he doesn’t deserve my trust. I will never trust Grok as long as he’s in charge of it.

I agree that the guardrails on the top models have gotten out of hand, though.

Fable for instance won’t answer even basic health questions. As if you are going to take nutrition advice and make a bioweapon with it.

Partly this is due to government interference. Hopefully we get to a better place as competition heats up with open and Chinese models.