[pen2l]

Something grotesque was put forth by a technology made by an entity that is comically monied. A mistake was similarly made by another monied entity only months ago, so it should have dedicated considerable effort to prevent such things. This is the way this works, we hold higher expectations out of the ones who have resources.

Please do not trivialize acts that have the potential to cut humans so deep with handwavy substantiations. Facebook should have known better, and done better.

[Cycl0ps]

By the sound of it this is a problem that neither entities nor their moniedness can solve. I'm sure these companies watch each other, and when one steps on a metaphorical rake the others are likely taking notes on how to avoid it on their future attempt. And yet rakes are still being stepped on.

When you have an automated system that has irregular behavior to a given input, we call that a bug. Bugs exist in all software, not always unique, but always present. This software is no different than any other. It will have errors. Because the software is categorizing faces, its errors will result in miscategorizing them. The only relevant questions to this are how frequent these errors are and how disparate they are across racial lines.

Another reference: this one is a Tool-Assisted Speedrun of a game that relies on basic image recognition software. While not entirely related, it does show how error-prone these algorithms can be. It's also fun to watch. https://youtu.be/mSFHKAvTGNk