| This line caught my attention: > The FBI said its system is 86 percent accurate at finding the right person if a search is able to generate a list of 50 possible matches, according to the GAO. But the FBI has not tested its system’s accuracy under conditions that are closer to normal, such as when a facial search returns only a few possible matches. What the GAO study[1] actually said: > However, we found that the tests were limited because they did not include all possible candidate list sizes and did not specify how often incorrect matches were returned. ... The FBI’s detection rate requirement for face recognition searches at the time stated that when the person exists in the database, NGI-IPS shall return a match of this person at least 85 percent of the time. However, we found that the FBI only tested this requirement with a candidate list of 50 potential matches. In these tests, 86 percent of the time, a match to a person in the database was correctly returned. The FBI had not assessed accuracy when users requested a list of 2 to 49 matches. According to FBI, a smaller list would likely lower the accuracy of the searches as the smaller list may not contain the likely match that would be present in the larger list. In other words, their acceptance test procedure was gamed from the beginning. [1] https://www.gao.gov/assets/700/699489.pdf |
So much of forensic science is a sham, we claim as a country to uphold a system of justice whereby you're innocent unless proven guilty beyond a reasonable doubt, yet so many prisoners on death row have been exonerated, some posthumously even when there's no real incentive to look for evidence of innocence at that point.
Who wants to bet that prosecutors are going to start using flawed facial recognition results as if they are equivalent to a victim picking out a suspect from a lineup of 10 people? "There was a 99% chance of a match"