[JohnTClark]

Every time I read this kind of message i feel like it's propaganda that is trying to make the west weaker by convincing people to not work on military technology. What you make can hurt people, yes, some people deserve to be hurt.

[jsinai]

Just throwing this out there since we’re talking about morals: who deserves to be hurt and how do we decide that?

One of the big issues with tech involvement in security and defence is the shift in onus of responsibility from person to algorithm. Just this week we’ve seen a story top HN about a black man who was falsely identified as a terrorist by a facial recognition algorithm [1]. Closer scrutiny showed that the terrorist score was 52%, barely more than a coin flip. Even closer scrutiny showed that the image was grainy. Not once was a human called into the loop to assess the algorithm.

Maybe the intention was good (let’s use facial recognition technology to catch terrorists), but the tech itself was flawed and there was a failure to imagine scenarios where an innocent person is denied their liberty because the tech didn’t work. Add to this the racial inequality, and the lack of empathy becomes deeper. Would anyone feel comfortable deploying facial recognition technology if there was a 50% chance (a coin flip) that they themselves would be hit? Finally, this goes beyond race. Who (in the West) decides who (not the west) should get hurt (be killed really) because they are collateral? Again the empathy gap decides that life is cheap on the other side.

The problem with tech culture is that we don’t like to imagine scenarios where it just simply isn’t up to the task. This can become a matter of life and death (or liberty) when security and defence are involved. At no point am I saying that countries should not invest in technology for security and defence, but that rather such systems are weak if they’re merely brute force (even a conv net can be trained to brute force it’s test accuracy) and not highly accurate. And right now the technology isn’t ready for deployment.

[1]: https://news.ycombinator.com/item?id=23628394

[unishark]

The application of AI in security is primarily needed to due to the enormity of the problem, not due to its ability to replace people. There simply isn't enough manpower or time to train experts and have them examine all the data. You can argue some of the data should not be examined at all for privacy reasons. But there are plenty of scenarios where privacy concerns won't hold. Such as comparing the security camera photo of a bank robber to mugshots.

The question of when it is reliable is an big one people have been working on a long time. For example in medicine the stakes are even higher. The problem is simply that recent technologies (deep learning) have taken a huge leap forward in performance, but a huge leap backwards when it comes to being able to assess confidence.