[bryan_w]

I agree, I also think that it's possible everyone involved could have been working with diversity in mind and still were blind to this bug.

Imagine someone makes a diverse facial recognition model where the confidence for detecting black faces are usually 99.96% even in rough lighting and 99.99% for white faces. They may have an acceptance bar at 95% so it's well within tolerance.

Combine that with an auto-cropping algorithm that takes a image, does computer vision on it, and selects the object that has the highest confidence and fits within the crop window.

When tested on both, portraits of white people and black people, it would pass, but in the examples, it falls down.

I say all of this not to excuse Twitter -- I still think they need to rethink how their autocrop works and fix it, but I don't think people these days do any type of people recognition without thinking of diversity.

[claudiawerner]

I find it hard to see why they're using computer vision for cropping at all. Years ago there was a post on Reddit which showed how Reddit crops images for thumbnails; as far as I remember, it was to do with colour density, not facial recognition, and written with a simple image library in Python. With some images, you could predict what the thumbnail would contain. Perhaps that also had a racial bias, but at least it wouldn't be contributing to a long list of sociological criticisms of machine learning.

[eggsnbacon1]

I think some bias may be inherent.

To blue eyes again, you could make a facial recognition system that just looked for 2 blue iris shapes and probably score 90%+ accuracy on most datasets (of exclusively blue eyed people and non-human objects) because blue is so rare in animal eyes, and so rare in nature. With such a strong face signal it may be impossible to match accuracy for non-blue-eyes humans without purposely making match quality for blue eyes worse.

We may be seeing a similar but weaker bias for darker skin. Light skin is fairly rare among mammals. The only solution to eliminating recognition accuracy bias may be to nerf accuracy for people with light skin. There have been many reports of facial recognition erroneously classifying dark skinned humans as animals. If part of model accuracy classifies lighter skin animal = more likely human, its going to be very hard to remove that bias because its empirically true. If the classifier is unsure if a shape is a human or animal face, but its very lightly colored, it may make the model statistically more accurate to report higher confidence that a lighter colored face is more likely to be human.

this may be more of a "most animals are brown" than a "brown people look more like animals" problem with the models. A possible path to fixing the bias is to crop all faces detected, disregarding model confidence of whether the face is human