[Dominisi]

I worked in a Law Enforcment office that used Facial Recognition. Every single time we would give officers a potential match, it came with a disclaimer that it was a POTENTIAL match. It was up to the detectives and officers to actually do their job and find other pillars of information.

This isn't a failure of facial recognition, its a failure of the stupidest people I went to high school with thinking its some sort of perfect system.

[teachrdan]

If the software vendor knows the end user will not use their product as intended, at what point is the vendor complicit in the impacts of that misuse?

[michaelmrose]

You can't paint a proper moral picture of a situation by imagining your responsibility is compartmentalized and that you operate in an environment where others will or at least could in theory operate ideally.

Such an approximation must as part of the moral calculus be incrementally replaced with a more accurate picture that takes into account the actual foibles of your fellow humans actually behave.

If given how people actually behave evil or injustice is done that is your fault. Not partially your fault, your fault guilt doesn't divide it just multiplies. You are I think bound not to do the least you can do but rather the most you can do. The best idea I've heard in this thread is to always show multiple results with no indication of which was higher priority. Force a human into the loop.

[LargeWu]

Given the well-documented penchant of police to use “fits the description” as a pretext to harass the public, especially people of color, I would say it is borderline negligent to allow the use of algorithmically generated probable cause justifications.