[rosser]

> The researchers Joy Buolamwini at Massachusetts Institute of Technology in Cambridge and Timnit Gebru, then at Microsoft Research in New York City, showed that some of the most advanced facial-recognition software failed to accurately identify dark-skinned women 35% of the time, compared to a 1% error rate for white men. Separate work showed that these technologies mismatched 28 US members of Congress to a database of mugshots, with a nearly 40% error rate for members of colour. Researchers at the University of Essex in Colchester, UK, tested a facial-recognition technology used by London’s Metropolitan Police, and found it made just 8 correct matches out of a series of 42, an error rate they suspect would not be found lawful in court.

[hubert1234]

So improve the software and fix the bugs? How is this a valid response?

[brylie]

I think the point is it shows systemic bias, which may be very difficult to "debug".

It is important to apply ethics when developing technology, as rationality alone is not sufficient.

[hubert1234]

I dont see why it should be difficult to debug unless you think this is all some kind of conspiratorial plot by white people or the "white system" who are all secretely racist against black people and that's why the detection rate was purposefully kept low by these evil people. And implying that we have to be irrational to be ethical is also pretty weird.

[rosser]

It's not a "conspiratorial plot", no. But time and again, research has shown that the implicit biases held by the people building these tools [0] are observable in the tool's results — and all people hold implicit biases.

It's not the tool we need to debug; it's us.

[0] Where, by "these tools", I mean machine learning writ large.

EDIT: I highly doubt your comment's parent was suggesting we had to be irrational in order to be ethical. Rather, I believe they meant we should be rational and ethical. Wouldn't you agree?