[Houshalter]

No one cares about human judges that decide parole or to fire teachers. Humans are vastly worse. Attractive people are much more likely to get hired or get shorter prison sentences. Judges tend to give much harsher sentences just before lunch when they are hungry. Their predictions about what people are likely to reoffend are often worse than chance.

Additionally, the study about the "racist algorithms" was fraudulent. Their results were "almost statistically significant". I.e. not statistically significant. Compared to human judges which have well studied biases by race. There's nothing remotely fair about human judgement and you should always prefer an algorithm. In almost every domain they could find, researchers have found that even simple linear regression beats predictions of human experts.

And this has huge effects on our society. Humans making biased hiring decisions leads to mass discrimination against certain groups and very suboptimal employees. Having humans make loan decisions, means much higher interest rates, more people go bankrupt, and the economy grows much slower.

The EU banning the use of algorithms is just absurd.

[intended]

Yes, but Humans can go to jail.

You assume that your opponent is reality.

your opponent is other human beings. It takes precious little for a motivated person to learn how to hide malfeasance behind a black box.

Faith in un-corrupted black boxes should be considered the same way as faith in the un-coruptible internet

[Houshalter]

No one has any incentive to corrupt the black box. There are many, many cases where systems have terrible incentives that ruin everything. But I don't see how this is one of them. E.g. the bank has every incentive to make it's loan system as accurate as possible.

[xherberta]

This is a good point, and to her credit, Cathy O'Neil is careful to say that humans are worse. She's arguing for better algos, ones that are carefully thought out to avoid the worst of these pitfalls.

[p1esk]

I think he was complaining about the algos' decisions being hard to understand, not them being unfair. From your examples, it seems like we more or less can understand human decisions.