[milkshakes]

Fundamentally, a security analyst authors detections, reviews surfaced alerts, or identifies hypotheses to investigate. In the case of reviewing surfaced alerts (the firing of a detection which may or may not be authored by the security team), differentiating true positives from false positives is subtle and often requires context or further digging. Of course, this requires time, which costs money, so you can imagine the tension there.

This process can often be subtle, and difficult to automate. In many cases, the issue is automating the economical delivery of enough context to the deciding function that a clean choice can be made. However, even with enough context, and enough documentation, escalating vs suppressing an alert can often be a judgment call. Humans are meat based pattern matchers, and a decade's worth of "ML" and "AI" advancements still not sufficiently precise (as in vs recall) enough to filter out "things that look bad" from "things that are bad, for our specific environment", that knowledge still lies with the security team.

[muraiki]

Only the slides are available, but the presentation "AI is Not Magic: Machine Learning for Network Security" at CMU's FloCon in 2020 was about this: https://resources.sei.cmu.edu/library/asset-view.cfm?assetid...