[root_axis]

I think the negative reaction is reasonable. Clearly, if a human did this it would a problem, so why should it be acceptable for an automated system to do the same thing? The fact that it is unintentional doesn't negate the fact that it's an embarrassing mistake.

On the other hand, imagine a world where these labels were applied by a massive team of humans instead of a deep learning algorithm. At Facebook's scale, would the photos end up with more or less racist labels on average over time? My guess is that the model does a better job, but this is just another example of why we should be wary about trusting ML systems with important work.

[jessaustin]

Clearly, if a human did this it would a problem, so why should it be acceptable for an automated system to do the same thing?

One worries that the corporate overlords are preparing the legal system for completely impune manufacturers of self-driving cars. "Sorry your child is dead; the car did it so there's no one to sue or convict."

[ipaddr]

That raises the question, is it embarrassing or an expected mistake to be learned from. Many things are mislabel many things are labelled properly but we never say AI must feel pride at the good labeling job why would we give emotions to an emotionless system?

[root_axis]

> it embarrassing or an expected mistake to be learned from.

I would say it's both. It's embarrassing for Facebook because it looks racist even though it really isn't. The system might be emotionless but the people who interact with it aren't, and we don't expect them to be.

[crooked-v]

> it looks racist even though it really isn't

It absolutely is racist. Racist outcomes are still racist regardless of whether there's a guy in Klansmen robes at the steering wheel or not.

[saltyfamiliar]

How is it a racist outcome? This has nothing to do with the belief that one "race" is inherently better than another. It's a simple categorization error due to insufficient training data.

[crooked-v]

It's a racist outcome because racists have a long history of comparing black people to primates, and because this results in a service that's actively worse and less useful for black people than those of other ethnicities.

Also, a 'simple error' performed by company with absurd amounts of money and several extremely public examples from its peer companies as to what not to do is, at that point, more negligence than anything.

[root_axis]

> Racist outcomes are still racist regardless of whether there's a guy in Klansmen robes at the steering wheel or not.

Yes, I understand what systemic racism and implicit bias are, your condescending snark is appreciated.

Anyway, it's not racist because the result is not the product of implicit bias or systemic racism, it's a software bug that would have been possible no matter who was working on this software. As I wrote in another comment: the whole point of ML is to adapt to what is effectively an unbounded set of inputs, pretty much by definition there will be cases where even a team of 100% black people will train a model that, given the correct input, will fail in ways that particularly affect black people.