[wyqydsyq]

> in this case, facial recognition tech was unreliable entirely because of the people running it.

Wrong.

In this case the facial recognition tech itself was arguably not unreliable. The human operators were the unreliable factor.

The recognition tech reliably tagged the impersonator as Ousmane exactly as it was instructed to do. The system worked exactly as intended. It is the human operator whose intention was wrong.

This has nothing to do with AI being unreliable and everything to do with the employees of this SIS company going "yep kid's black, he's the one who did it" without half a thought.

[2pEXgD0fZ5cF]

> The human operators were the unreliable factor.

Separating the two things (operator and technology) is merely a technicality, in reality they are not transparent to the general public and should be treated as parts of the same.

If AI™ only happens in a black box behind closed doors with people getting the opportunity to make inconvenient results disappear and that is basically the way it is trying to be established then no, it is not reliable.

The operator is part of this whole system, if it can't be used without the unreliable operator, the tech is not reliable.

[josefx]

> In this case the facial recognition tech itself was arguably not unreliable.

You could train a fake money detection AI with cat videos from youtube, it would probably do something with images of cats. However I hope no one would try to argue in front of a court that the resulting AI would be reliable at detecting fake money. In this case a stolen "do not use as id" slip was apparently good enough to serve as input validation, I would be surprised if their database wasn't overflowing with bad data.

[jjk166]

It wasn't validation, it was labelling.

The AI correctly identified that the faces of people in two videos were the same person. It did it's job. The person responsible for correctly identifying the person in both these videos failed to do their job correctly.

[sb8244]

IMO, the operators of the AI are part of the production execution of that AI. There were not proper constraints in place to prevent this from happening, and it wasn't fixed after the first time(s) that it occurred.

Sure you can argue that the pure tech worked as designed, but the system includes the human element as much as the tech element.

[zo1]

The operators made it worse of course, whether enabled by AI or not. We've been dealing with this exact same issue, without AI, for decades when it comes to identity theft.

These "operators" that you say are part of the production execution of "AI" are also part of every single institution that led up to this person being here and existing. Everything from registering their birth, capturing their DMV details, capturing their details when enrolling at a school and opening a bank account, taking their photo for the driver's license, etc. All gooey, imperfect, corruptible and fallible people that taint the "chain of custody" about the person's identity.

[harry8]

>This has nothing to do with AI being unreliable

This has everything to do with AI being unreliable. AI is unreliable. It was unreliable yet again here. Anyone relying on AI as the evidence in a prosecution needs to have their faces rubbed in the ignorance. AI is as unreliable as any human being. In fact, it is more unreliable because people understand the limits of the reliability of humans and look for corroboration and examine the evidence with a mind on its potential shortcomings. Yet there is still this pervasive and pernicious belief that "Computers don't lie" which is completely and utterly false and worth repeating often, with emphasis until it is part of human collective understanding of the world to the same degree that water is wet.

"Computer says x" by itself counts for nothing. Anyone presenting evidence like that by alone should be presenting a huge red flag that something nasty is going on, via either malice or sheer incompetence.

[jjk166]

The AI was reliable. It correctly identified that the individuals in two videos were the same person.

The problem was that one human recorded a name that they should have known was not real, and another person read that name in a report and arrested someone with that name. The story would have been no different if a human had correctly recognized the same person in both videos.