[danso]

"Racism" involves prejudice and discrimination predicated on the belief that one race is superior/inferior to another. If a face-detection algorithm fails to easily recognize black or Asian faces, such that people of that ethnicity/race have to go through a more manual/friction-laden process (think manual pat-downs and searches from TSA) to be verified as "human" or "citizen", then how is that algorithm not responsible for discriminatory treatment?

I do agree, though, that it is wrong to infer that the algorithms themselves are evil or malicious -- it's possible to be racist without having negative intentions.

But what does it say about a society if it continues to optimize algorithms (and related infrastructure) for one group over another?

[oneplane]

Of course those systems 'discriminate', that is what they are designed for. The problems aren't with the system, but with how they were set up, implemented or tested.

Say you make a sensor that is supposed to tell the difference between blue, green and purple, but you only test with one shade of blue and maybe two shades of green, you are going to have trouble actually matching your design goals.

In the case of the face detection system: they didn't specify and/or test is well enough, which can be due to a number of factors, but will most likely lie with the employees of the company that did the development. If they only have the classic 'pasty white guys' to work with, then it's going to be crap at actually doing face detections for all humans. On one hand you could setup a proper test protocol, on the other hand they shouldn't have taken broad or vague terms when developing/presenting the technology. If you don't have a broad selection of faces to test with, you shouldn't claim you have 'face detection', since you merely have 'detection of faces of the people that work on the project plus anyone who looks like them'.

This would be a completely different story if someone writing the face matching code specifically programmed code or wrote configuration data that targets skin tone or geometry of specific groups of people.

Some people would like to extend this type of technological issue into the area of HRM and race/gender-bias in society in general, but that is not a technology-only discussion and hardly something the people involved are qualified to argue about.

Also: >But what does it say about a society if it continues to optimize algorithms (and related infrastructure) for one group over another?

It says that society is imperfect, and that certain levels of xenophobia, bias and true racism exist. Doesn't say much about technology though.

[danso]

> If you don't have a broad selection of faces to test with, you shouldn't claim you have 'face detection', since you merely have 'detection of faces of the people that work on the project plus anyone who looks like them'.

Yeah, but that's what happened here with HP in 2009. I'm not a huge fan of their products these days but I don't think they would intentionally be deceptive here, i.e. I think there's a lot of room to blame incompetence before malice. If HP is a company with very few black employees, this kind of consideration may be completely off their radar. It's super unfortunate, but I don't see the company as evil or maliciously racist, per se (I think we can skip retreading the hiring for diversity debates for now).

> This would be a completely different story if someone writing the face matching code specifically programmed code or wrote configuration data that targets skin tone or geometry of specific groups of people.

Why does it matter? What's the difference between an algorithm that fails to perform because of programmer incompetence, or programmer malice? What's the difference to the end-user if the programmer was plain ignorant of good testing coverage, vs. a programmer who thought "Fuck it, minorities are a minor part of our user base. Not worth the extra engineering effort!"?

Technology is an unavoidable part of the problem. Because it is the technology that allows us the power and freedom to create and apply scalable algorithms to machinery and computers. This automation allows for efficient and reliable decision-making, and we as a society decide where that automation is appropriate and worthwhile, i.e. where human agency is no longer needed.

But technology and its fundamentals are still a key factor. Creating a multi-racial face classifier is fundamentally more work and difficulty than one trained for just one race. The math and physics are unavoidable. And every engineered system and product has to make tradeoffs between production cost and feature set.

In the case of the light-skin-optimized HP web cam, I think it's important, and fine, to call it "racist" -- a black HP customer will have an inferior experience fundamentally because he is racially black. But this isn't just a way to quickly assign blame. Recognizing that tech is fundamentally limited is the first step in understanding that systemic racism (e.g., all the decisions that led to the "racist" camera) could be a contributing factor to the camera's substandard performance.

Much harder to get to that thinking if we have a mentality of, "how could the computer be wrong/flawed?"