[NameError]

Looking over the specifics, the striking thing about this to me is that it seems like these supposedly-sophisticated covert operatives are just going to ChatGPT (or similar) and basically asking "how do I make good malware?"

[2OEH8eoCRo0]

What specifics did you look over because the (short) article doesn't say that at all. Most used it primarily for research and generating content for phishing campaigns.

> Charcoal Typhoon used our services to research various companies and cybersecurity tools, debug code and generate scripts, and create content likely for use in phishing campaigns.

> Salmon Typhoon used our services to translate technical papers, retrieve publicly available information on multiple intelligence agencies and regional threat actors, assist with coding, and research common ways processes could be hidden on a system.

> Crimson Sandstorm used our services for scripting support related to app and web development, generating content likely for spear-phishing campaigns, and researching common ways malware could evade detection.

> Emerald Sleet used our services to identify experts and organizations focused on defense issues in the Asia-Pacific region, understand publicly available vulnerabilities, help with basic scripting tasks, and draft content that could be used in phishing campaigns.

> Forest Blizzard used our services primarily for open-source research into satellite communication protocols and radar imaging technology, as well as for support with scripting tasks.

[armchairhacker]

I mean, this is essentially how current AI poses a threat. And it is a threat.

Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation. But there are a lot of people who aren't aware of AI generation, or just not paying attention, who will take the photos at face value. (You'd probably be one of the latter. How many times did you "inspect" a photo in a news article to make sure it wasn't faked? How many of those photos slipped past inspection into your subconscious? Subliminal messaging has never been easier or more widespread.)

Also imagine a lot of vigilantes who are stupid enough to be vigilantes in this day and age, who ordinarily couldn't do the bare minimum amount of research to cause real harm. But now they can ask ChatGPT "how do I rob a bank?" and get advice which is still pretty bad, but better than what they'd come up with on their own (if they wouldn't just give up entirely), so it causes more damage.

A trained professional who wants a quality deepfake can already use photoshop and video editing tools, and a trained criminal already knows how to do research. But there aren't a lot of those people. Massive low-quality spam and low-level crimes are useful even for a state actor with huge resources, because they cause general instability in ways a few quality hits can't.

There's another issue that a powerful state actor can simply build and deploy their own language model. However, it probably won't be as good as OpenAI's (quality may have some effect) and it won't be wasting their resources.

[Aerbil313]

> Imagine a massive flood of low-quality images relevant to current events (e.g. war), which are obviously fabricated to anyone remotely aware of AI generation.

Have you seen photorealistic Midjourney images? I can never distinguish many of them in a million years.

[refulgentis]

Yeahhh I unfortunately see this semi-regularly on Twitter now, and it's always an image of $OPPOSITION where the bit that is faked is $EXTREME_CARICATURE. There was a chilling one last night, so while it's fresh on my mind, I'm going to riff on it

Confirmation bias and...love for the outgroup...is so strong, that it's rare people admit they didn't know it was fake and remove it. Frequently, someone else hops in, to explain it feels true and represents how they understand $OPPOSITION's viewpoint anyway.

Once multiple people get involved, semi-frequently, it turns into an indictment of the person who pointed out the fake. Why? Even if it is a fake, the real ignorance exposed is that of the person pointing out the fake, as it's clearly representative of $OPPOSITION anyway, so they're at best naive, and at worst supportive of, $OPPOSITION.

[armchairhacker]

https://www.reddit.com/r/ChatGPT/. They range from practically-indistinguishable to ridiculously wrong, but the more details, the more they lean towards ridiculous.

It does make it easier, and who knows about the future. But right now if you generate an image you have to really look and there's a good chance at least something's off.

Specific example: https://www.reddit.com/r/ChatGPT/comments/1apyrwv/which_vide.... They are video games so already not realistic, but no gym has people stand around like that and all of them have some nonsensical equipment.

[addandsubtract]

Those are Dall-e images, not midjourney. I would hardly call them realistic. In fact, dall-e is trained/instructed not to output realistic images.

[t_mann]

Isn't that how a lot of software gets written now? Why would malware be different?