[starbeast]

Given this bit -

>"Since this was a task more suited to Cylance Protect, they rolled out that tool in a free trial mode, and it "lit up like a Christmas tree." At this point, OPM began using Protect extensively in its diagnostic process, despite not committing to license it from Cylance; they eventually agreed to do so on June 30th, a day before the trial period was set to elapse. Cylance did not actually receive payment for months."

- it seems that the takeaway is even more devastating. Don't start work unless they have already paid.

[_8j50]

Just a note,cylance is infamous for false positives.

[stonogo]

What does that matter? The current standard is millions of false negatives.

[_8j50]

Because they said it "lit up like a christmas tree". Couldn't find the virustotal stats page comparing vendors but Cylance had ~5x higher than the next false positive leader. It's not bad if you can filter them out and have contextual awareness but lighting up like a Christmas tree means little.

[starbeast]

On the other hand, if you know something is bad for false positives then unless it is so bad as to be unusable, you would expect that, on average, getting a few results is dubious, but lighting up like a christmas tree probably means something is actually there.

[acdha]

That's really not a safe assumption — an incorrect result repeated thousands of times does not become correct — and it definitely means that you now have a big problem of reviewing and validating tons of noise which will delay the time before you find whatever valid results are present.

I've seen multiple tools in this class — code scanners, IDSes, or web app scanners — which caused security problems by training everyone to assume that the results are always false-positives until they missed something real or soaking up so much human time that nobody made progress on the major improvements which would have prevented a breach.