[SannTek]

You raise a good point. Interestingly, your comments run contrary to what a lot of law enforcement officers have said to us.

In general, the police are hyper vigilant about buying only devices that are independently validated to be very accurate. Every conversation we have had has eventually lead to "is it NHTSA approved?". The reason for this need for third party validation is that the police are incredibly court room sensitive. If there is any chance a defense attorney would be able to pull out a study showing low specificity or sensitivity for a device, the police will simply not buy it. Third party validation gives them that guarantee.

You are right that the police want more convictions (or less time consuming convictions), but the way they do that is by having very accurate devices that are defensible in court, not inaccurate devices that risk cases being thrown out. That is why making sure our device has low false positive is very important.

[neuralzen]

I admire what you're doing, and hope you can adhere to scientific integrity while developing and selling your product, but that simply isn't true. At least in Washington state. A few years ago Washington State purchased a whole bunch of Draeger 9510 (alcohol breathalyzer) units without ever having tested them. After two years of sitting unused, the warranties were going to expire and they finally got around to testing them, and purchasing extended warranties at that point. However their initial tests failed multiple accuracy tests to the point that the state's lab halted testing, since it was so off. A few months later Draeger patched the firmware and OS application (the breath/alcohol measurement firmware being written by a single contractor in Texas, and versioned as 0.7) and while testing looked better, it still wasn't accurate and they just pushed it out the door for deployment...they had already bought them, after all. I'll link the records below...they didn't even have the correct versions of the software and firmware accounted for and signed off, which led to other problems (and Draegar has had other problems of accuracy in other court cases, such as Florida, but that's a side tangent).

My point is, a lot of state governments (at least in the US) spend money first and verify later, and 'verify' can be a damnably loose term. Loose enough to violate WAC 448-15-020 (in Washington state's case) which requires a 'reasonable degree of scientific accuracy'.

EDIT: Apparently the original link (below) is no longer available in WSP's records, which is interesting. You can view the document via archive.org, here

https://web.archive.org/web/20170211114622/http://www.wsp.wa...

original (defunct) http://www.wsp.wa.gov/breathtest/docs/webdms/Draeger/Valid_D...

[SannTek]

Thanks for the info. Definitely something we have not run into or considered yet. I guess we just naively assumed that all police precincts were as scrupulous as those we talked to :(.

There is some stuff we can do, however. For instance, we can directly build the QC schedule into the device so it does not work unless you get it calibrated. We can also have QC cartridges that have a barcode that the device reads to make sure you actually ran a calibration check properly and the device passed.

We were so focused on internal methods for making the device accurate, but maybe we should put some thought into how we can make sure the police use it properly.

[jjulius]

>I guess we just naively assumed that all police precincts were as scrupulous as those we talked to :(.

Bless your heart.

[neuralzen]

Really, if you're producing something that is reliably accurate, I would expect that would be all anyone would expect, along with proper training, verified updates, and audited software/firmware, etc. In the case I cited above it was more an issue with the lack of accuracy of the product, and sloppy follow through from the vendor, with money and political pressure thrown at the problem - something which the police departments themselves likely have less influence on anyway.

[Daniel3]

If your product is at all worthwhile, it will constantly be tossed back and forth between lawyers, trying to uphold or disprove its validity. ("Well here's an independent study done on how arresting officers with shaky hands get inconsistent results when administering the SannTek Cannibus test" et al et al ad nauseum) While yes, this is a problem in any society that applies capitalism principles to the practice of law.... I don't think there's necessarily a better system, sadly. Don't feel responsible to fix it all yourself. Anyway, pursue the technology as well as possible, by all means work in as many checks and balances as possible for correct use, and constant calibration, BUT ALSO talk and document openly during your design phase and testing the ideology you develop to govern false positives versus false negatives. This is an extremely engaging technology ethics discussion. If you drive the error to be false negatives, at a rate of 5%, will you not be able to sell it, because your customer is most interested in an effective tool? What is the societal benefit of getting this device to be accepted and used by the police? If you can achieve FN=4% is it worth it, or is the societal benefit actually much higher, possibly at FN =20%. Can you just run the test 5 times in a row, or with 2 stand alone devices to reach the dependability rate you desire? If you drive the error to be false positives, at a rate of 1%, can you sleep at night with the lives your product will effect, or do you need a fall back like your device determining with 100% accuracy that cannibus is present in someone's system (just not a definitive, exact amount), PLUS a statement from an arresting officer stating erratic or dangerous driving or behavior in conjunction with your device's input. I'm very excited about what you're doing. I didn't know (still not yet convinced) that the science is solvent, but I've been asking about this device for years.

[michaelmrose]

Everyone says they are ethical including the unethical I'm not sure how you possibly took this at face value.

[mbrameld]

> In general, the police are hyper vigilant about buying only devices that are independently validated to be very accurate.

How do you reconcile that with the reality of the NIK tests which are notoriously unreliable and ubiquitous among police departments?

[lowdose]

Like the lie detector?