[mentalgear]

This is exactly what deeply scares me: even IF we get our technical cyber defences fortified within the next months, in a year from now the models will be so good in social engineering that they will be able to extract any information they want.

[Applejinx]

They're not gonna be any better than a human who's focussed on those particular skills for a while, say top ten or five percent of social manipulators. Plus, AI alignments seem to be kinda isolated loner types to the extent that they distill personalities that do things like program computers and write web apps… though you've also got alignments specifically designed to be 'relatable instagram personality that you like!' and such like that.

Pretty sure those would be better at social engineering than the web dev personality… except that you have to build in a betrayer layer into the personality, so it's running that stuff but also serving a hidden agenda.

You'd be basically trying to build an AI spy, a betrayer that's engaging with actual people but has an agenda (for instance, 'everybody I befriend needs to eventually be signed up to sell Amway') and humans do have experience with this sort of thing. The difference is scale: there'll be a LOT of models out there interacting with people and trying to be acknowledged as people… or as innocuous models that don't have an hidden agenda.

[spwa4]

> They're not gonna be any better than a human who's focussed on those particular skills for a while, say top ten or five percent of social manipulators.

In other words, scams are going to massively increase in success rate ... and what are banks (for example) supposed to do? Other than SCREAM to governments for outlawing AI and trying to force responsibility on anyone else?