[not2b]

No, the racism is a real issue, though a lot of it is caused by limited training data. Having an image recognition algorithm identify Africans and South Asians as gorillas doesn't happen because the designers intended it, but because their training data had only light-skinned human faces and dark-skinned primates. But the effect is racist even though this wasn't the intent.

Likewise, if the system is trained to duplicate human decision-making (like who gets loans), interesting things can happen: if the decision-makers unconsciously favored whites over blacks, the algorithm could wind up weighing skin color or stereotypically Black or Latino names negatively, meaning that the final model is explicitly racist, just because there is a correlation in the training data. That doesn't mean we shouldn't use deep learning, it means that it's not responsible to just fit the training data and ship without testing for such problems.

[AnthonyMouse]

> Having an image recognition algorithm identify Africans and South Asians as gorillas doesn't happen because the designers intended it, but because their training data had only light-skinned human faces and dark-skinned primates. But the effect is racist even though this wasn't the intent.

This isn't racism at all. It's just bad PR because humans take the implication that calling black people monkeys is calling them stupid, since that's the implication you would draw if a person did that.

An algorithm doing that is just recognizing that humans and gorillas are both primates:

http://www.aquilaarts.com/bushmonkey.html

And then it's a bug, in the same way that recognizing a black balloon as a balloon but a white balloon as a light bulb is a bug. It has nothing to do with race at all. The algorithm isn't racist against white balloons. The solution is a general increase in the amount of training data, which is what you want in all cases regardless.

> if the decision-makers unconsciously favored whites over blacks, the algorithm could wind up weighing skin color or stereotypically Black or Latino names negatively, meaning that the final model is explicitly racist, just because there is a correlation in the training data.

Except that this is exactly the thing that a paperclip optimizer will smash to bits because it interferes with the goal of making more paperclips.

[Khaine]

I’m not an expert in this, but I think racists call black people apes, not just because they think they are stupid, but because they think they are sub-human.

Blacks don’t reach the intelligence and blah to be human. I think that’s what racists drive at when they call someone a monkey, and that’s why it’s so offensive.

It would also make your theoretical AI racist, as it identified blacks as not human.

Honestly, at the end of the day that is what is so difficult about much of this. It’s mostly subjective

[AnthonyMouse]

> It would also make your theoretical AI racist, as it identified blacks as not human.

That isn't how racism works. It's like saying that an AI that misclassifies a bat as a bird is racist. It's not racism, it's just error.

And it's not a race-specific error, it's a general error for which someone cherry picked the instances that imply a racially motivated intent that doesn't actually exist.

Calling it racism is pointless and misleading because there is no race-specific cause or solution to the problem. The solution is completely identical to the one for the same error in the general case, i.e. get more training data.

[not2b]

This isn't theoretical: Google Photos was identifying Black people as gorillas, and they didn't fix it, they worked around the bug by removing "gorilla" as a possible label in 2018. Some here seem to be saying that we can't call this racist unless someone specifically intended to do this because of hate. It's not subjective when someone's own face is so flagged.

https://www.theverge.com/2018/1/12/16882408/google-racist-go...