[ggreer]

> Reminds me of how Google Photos couldn't differentiate between a black person & a monkey, so they've excluded that term from search altogether.

Technically that is what happened, but it paints an incorrect picture in people's minds. Out of the billions of images that Google Photos had auto-tagged, it tagged one picture of two black people as "gorillas".[1] This was probably the first time this had ever happened. (If it had happened before, it surely would have been spread far and wide by social media & the press.)

So Google's classifier was inaccurate 0.0000001% of the time, but the PR was so bad that Google "fixed" the issue by blacklisting certain tags (monkey, gorilla, etc). If you take photos of monkeys, you'll have to tag them yourself.

I'm sure Google could do better, but the standard required to avoid a PR disaster is impossible to meet. If the classifier isn't perfect forever, they're guaranteed to draw outrage.

1. https://twitter.com/jackyalcine/status/615329515909156865

[mediumdeviation]

Our expectation of our algorithms are based on human performance. A human would never tag a black person as a gorilla, or vice versa, and if someone did it even once we could pretty safely conclude they're either extraordinarily incompetent, or racist, and in either case we wouldn't trust any tagging done by such a human.

[forapurpose]

> This was probably the first time this had ever happened. (If it had happened before, it surely would have been spread far and wide by social media & the press.)

That is a very big leap. Social media might be widespread, but almost everything in the world goes unremarked upon. Think of all the news stories that turn up an old tweet or Facebook post that, if anyone had paid attention at the time, would have stopped events from progressing.