Hacker News new | ask | show | jobs
by AnthonyMouse 188 days ago
I'm increasingly of the opinion that the modern practice of not telling people why they've been blocked -- or even that they've been blocked -- was devised by sadists to satisfy their proclivity.

The core of the flaw is that actual fraudsters and spammers are repeat players and ordinary people aren't. The bad guys expect to be blocked, so they test for it. They check if their messages are getting through and then notice immediately when they stop. Whereas real people expect their messages to go through, because why wouldn't they when they've done nothing wrong? And then become isolated and depressed because it seems like everyone they know is suddenly ignoring them.

The bad guys create thousands of accounts and play multi-armed bandit, so when some of them get blocked they can identify why by comparing them to the ones that didn't, or create new ones and try new things until something works, and thereby learn what not to do. Whereas real people have no idea what sort of thing is going to arouse the Dalek either before or after their primary account is exterminated.

So it's a practice that creates a large increase in the false positive rate (normal people have no way to know how to avoid it) in exchange for a small decrease in the false negative rate (bad guys figure it out quickly). In a context where false positives cost a zillion times more than false negatives because the bad guys treat accounts as a fungible commodity they acquire in bulk whereas innocent people often have their whole lives tied to one account.

And all of that is only disguising the real problem, which is that people get blocked having done nothing wrong. If you were expected to point them to the spam they sent or the fraud they attempted then you wouldn't be able to do it when they'd done no such thing, and then "we can't tell anyone because it would help the bad guys" is used to paper over the fact that you couldn't tell them regardless. When the decision was made by an opaque AI and then reviewed by no one, there isn't actually a reason, there's just a machine that turns you off.