[nkrisc]

The AI part does seem relevant because it enabled incredibly low-effort “social” engineering.

For what it’s worth I don’t think you can call this social engineering since there was no human on the other end, even though it appears similar.

The question is, if there were actual human support agents, would they have built additional safeguards to prevent social engineering in this manner?

[incangold]

One concerning feature of AI is the speed and volume it is capable of failing at if poorly controlled, whether or not it’s more accurate than humans.

Even if humans failed at the same rate, if you tried to exploit at scale you’d be throttled by the size of the support team. The failure would happen at human-scale time frames and throughput.

[sagebird]

a human would have noticed something different about the requests it was getting, or the frequency of requests, and as soon as it noticed a shift, it would have carried that knowledge forward and intensified the scrutiny if something seemed off- eventually communicating it up the chain.

- instead of the ai context dying.

in the ai case, information only survives to the extent where the ai is empowered to store a note or notify a manager of an observation. Anything that does not result in sending a message/storage is wiped

[CodesInChaos]

At the scale of facebook, humans are underpaid call center agents who are required to follow a script and don't have to the authority nor any incentive to scrutinize requests.

[uxhacker]

Why did the account recovery system need AI. Surely just an email would do? What added value would AI add?

[Lammy]

The person who writes the feature gets promoted for “aligning” with management's “Big Bets”.

[jazzyjackson]

Meta doesn’t want to pay humans to read support tickets if they can help it.

[Vrondi]

There's no social engineering here, since all they have to do is copy and paste. This is a complete process design fail.