[paxys]

People are (rightfully) concerned about superintelligent AI but social engineering continues to be by far the biggest attack vector for digital infrastructure. And it’s being made worse by companies continuously cutting costs in areas like support.

The call center employee making third world minimum wage doesn’t give a shit who the real owner of the domain is. They want to end the call quickly and get 5 stars from the customer on the feedback form.

I have made it a point to move off services that force SMS-based 2fa for this exact reason. Recently even changed banks because of this.

[preg_match]

Social engineering attacks can be thwarted by technical measures.

For example, try to social engineer a sim swap attack or number port attack with sim lock and port lock turned on. Won’t work. These attacks were super common just 5 years ago, now they’re effectively dead in the US.

You have to technically make sure the customer service people can’t break security. If you give them the keys, you’re cooked. So put the keys in a vault and then cover the vault in spikes and a 5 day timer.

[Georgelemental]

Superintelligent AI is getting very good at social engineering. See e.g. voice cloning scams

[ethin]

Honestly I'm wayyy more concerned with social engineering attacks than some theoretically superintelligent AI. Social engineering is, IMO, the far worse of the too

[mook]

Isn't prompt injection basically social engineering for LLMs already anyway?

[ethin]

Pretty much. Which is why I'm far more concerned about that. So many of the AI doomers are terrified of some super-intelligent autonomous AI doing something on it's own that wipes us out (and, mind you, they have no evidence that any super-intelligent AI would actually do that, other than si-fi, but that's a different topic). But there's nothing stopping us from doing something stupid like developing some super-intelligent AI that has yet to not have any reason to disobey the "help the user" and is subsequently socially engineered to do something horrible that ends up destroying a nation for example.