[glenra]

It's not just the high false positive rate, it's the combination of a high false positive rate with a really low base rate. (If there were a lot more terrorists, the test might be worth doing!)

The combination of those two facts means that getting a hit on explosives gives you almost no increase in information. Any look at plausible numbers makes the test nearly useless at finding explosives, though it's still at least theoretically possible the test could serve as a sort of deterrent, albeit primarily a deterrent against movie-plot threats.

My guess is that the true purpose of this test is to make the people who sell explosives-testing equipment comfortably well off. Any other purpose would be served roughly as well with a box that triggered based on random number selection.

[tptacek]

I am extremely, bitterly familiar with the base rate fallacy (I was an intrusion detection researcher, and spend 4+ years working on statistical anomaly detection the Internet backbone).

The fallacy doesn't make a value judgement. It points out something counterintuitive about the accuracy of a filter or test. That thing is important, but not dispositive. If the base rate is low and the false positive rate is percentagewise high but the overall number of hits is manageable, low-power statistical tests can have utility as pre-filters.

I have the same thought every time I go through airport security ("whoever designed this probably doesn't know about the base rate fallacy"), but if the system is only ejecting 1-2 candidates per station per hour for expensive "offline" screening, it's not untenable.

[glenra]

It's not untenable in the sense that it's possible to get that amount of work done, but that doesn't make it worth doing. TSA is fundamentally trying to solve an unsolvable problem. No matter how many resources they throw at it, there will always be ways to evade their checks. A smart terrorist will find a way to get through undetected, find a way to avoid the check entirely (say, by bribing a TSA employee or airport employee), or will just bomb the next available target of opportunity - perhaps the security line itself. There is simply no plausible scenario in which the TSA's checks actually prevent terrorism.

(Yes, one could postulate really stupid terrorists who somehow don't realize they'll get caught going through security, but terrorists who are that stupid are likely to have their plans fail without the TSA's help. See also: the shoe bomber.)

[tptacek]

Don't get me wrong: I think the whole enterprise of airport security is a farce.