And the way they verify you are doing what you say you are doing is by asking you to provide evidence, which is usually pretty easy to demonstrate that a policy was followed once or twice, a lot harder for them to pick up consistency issues or exceptions.
I've had SOC2 auditors choose a random commit from our GitHub history, then ask to see the associated Jira ticket, logs from the build and deployment, etc. Hard to reliably pass an audit if you don't know which changes they'll drill down into.
They also asked for proof of system-enforced processes (e.g. GitHub branch protection rules and the setting for enforcing peer review for each change) which were basically proof of consistency.
They do that because in the DRL process you specified a change management process involving Github and Jira. If you specified a different process (for instance, Post-It notes applied to the bathroom wall), they would randomly ask for evidence about those Post-It notes.
That's what we're talking about when we say virtually any tool you can come up with will satisfy "vulnerability scans". For Cloudflare, it was nmap. I think they're right about this.
A SOC auditor who tells you that you can’t use an nmap scan to meet SOC2 obligations is a bad SOC auditor, because they’re attempting to enforce a constraint on you that SOC2 does not.
But the far more likely thing is that a medium SOC auditor, upon being told “we do our vulnerability scanning with nmap”, would say “I haven’t heard of nmap. You should use Tenable,” and if you’re letting SOC auditor drive your engineering you’d make a mistake and accidentally think that meant you needed to change your answer for SOC2 and go buy Tenable licenses.
The whole thread drifted way too far from a very mild push back I had regarding the claim « any automated process that can plausibly be described as instrumental in finding some kind of vulnerability is a "vulnerability scan" ».
My experience is that no, SOC2 auditors won’t consider literally any automated process of that sort as compliant. Which in no way implies the auditors are forcing you to use a licensed tool or driving your engineering.
I will stop that thread here, I don’t think that exchange is productive
It absolutely doesn't rely on competent auditors. The AICPA that fabricated SOC2, is the same AICPA that gives licenses to the auditors. At some point, they opened it up to getting it over the internet.
Indian companies open up shell businesses in Wyoming and elsewhere, get "certified", and offer rubber stamp auditing services. Few ever check if you actually have SOC2, or what auditor you used (since, by definition, they need to be "legit").
By the way, the AICPA website was recently throwing https expired cert errors. Their solution after weeks of me pointing it out on twitter, was to take down the entire website.