[danso]

They likely didn't purposefully "do" anything wrong -- at least in the sense of some racist engineer tampering with the system to have it come out with those results.

But this is the nature of machine learning algorithms, including both the process of supervision and ability to view the impact feedbacks have on the algorithms, and also, the impact the quality of the training set given to the algorithms. At a lesser company, the problem could be as simple as very few black people represented in the training set, so that when the algorithm sees a dark-colored human-like shape, it is more "likely" that that shape is a gorilla (which is human like and pretty much always has dark fur) than it is a human, because the algorithm was trained mostly on light-colored humans. The Google Photos algorithm obviously takes in more kinds of input and factors besides visual composition so there was probably more to it than this.

Or maybe not...who knows? I'm not interested in reviving a discussion about importance of diversity in the engineering workforce, but this is one kind of problem that can slip by the most competent and well-intentioned of engineers simply because they're less aware of how disenfranchisement can propagate into technical problems, no matter how correct and powerful the math behind the algorithm.

Another example from a few years back was when HP released a auto-tracking webcam that became infamous after a black retail employee uploaded a YouTube video of how the camera ignored him but not his white co-worker:

http://www.cnn.com/2009/TECH/12/22/hp.webcams/

I'm in 100% agreement that this was likely not HP's intentional fault, and also that face detection of darker complexions is computationally more complex than it is for lighter complexions because of how contrast is used by the algorithm...but I most definitely know that if I were an HP engineer, and if the CEO and/or my direct boss were black and tried out a prototype that behaved as it does in the aforementioned YouTube video, there is almost no fucking way that the product would be released as-is, with my excuse being "Well, accurately detecting black faces requires a much more complicated training set -- that's just how math works!"

[anindyabd]

I have no doubt that neither Google nor HP made those error maliciously. I was just curious as to whether or not it's possible to incorporate some sort of... tact?... into these recognition algorithms to avoid labeling people (or other things) offensively. Is it just a matter of a larger training set? It would be hard to cover all sorts of people in all sorts of poses, with all sorts of lighting conditions, etc.

[tyrust]

It's not about tact, it's just the algorithm doing its best. In order for the algorithm to be capable of "tact", it needs to recognize that it's looking at a person (or whatever). And if it recognized a person, then there wouldn't have been this problem because it would just label it correctly.

[IanCal]

You can certainly include tact. The algorithm thinks it's more likely it's 51%-49% gorilla/person split, but a level above that chooses person as the answer as even though it'll be wrong more often, the impact of the error is lower.

This is why you shouldn't just train your system to hit higher accuracy figures but also investigate the type of errors it's making. This needs to be done while thinking about your specific use case and domain.