[hn773746483]

It's just social engineering. No different than say, 2FA fatigue (blowing up someone's phone with 2FA "is this you? yes/no" prompts until user/child/wife/SO/etc clicks yes) or even just simply harassing IT helpdesk until they reset "your" password.

[terribleperson]

It's scalable, personalizable social engineering. I think that makes it a lot more dangerous.

[utopiah]

Yes but not free either. Spam works because it scales and even though 0.0000001% only might fall for it, it's still "worth" it. Here it might be 0.0001% instead but it's a lot more expensive, even with subsidized tokens, to do.

So it's interesting, feasible, but it's probably not as broad impact as the scariest scenario leads out to be.

Also I imagine that once exposed it becomes a well known pattern. Some will still fall from it but I imagine once it's been done few times it becomes even costlier.

The fact that Xz is mentioned and most of us know right away what it means show that we collectively learn.

[Forgeties79]

“Before LLM’s there was_____” I see this whenever an LLM’s impact is assessed. We know. The issue is scale and the ability for smaller and smaller groups (down to individuals) to execute at scale. LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging.

Fake news always existed. Now one dude in India can flood multiple sock puppet media accounts with right wing content/images (actual example) at a scale previously unimaginable. Same goes for social engineering tactics.

[RetroTechie]

> LLM’s are pouring massive amount of gasoline on existing issues and people just keep shrugging.

To use your analogy: this is much like a forest fire. Tinder-dry combustible stuff is piled up everywhere, there's no lack of ignition sources, and firefighters are thin on the ground.

Fun times ahead.

[coldtea]

Yes. It's as if some people can't understand anything becoming a new huge problem unless that problem didn't exist at all before.

[Forgeties79]

At this point I just assume half of them are not saying it in good faith or at least with any real consideration. They just want to hand wave away whoever is critiquing their tools.

[ezst]

This, and/or the tendency in tech circles to "think in absolutes” (like in code, seeing things binary, ...) which is especially annoying in security-related discussions.

[utopiah]

True but it's an arm race.

Only mentioning that it feasible or even has been done few times mean that people who care will act accordingly. It doesn't remove the problem but it makes it radically less effective already by just being aware of it.