Hacker News new | ask | show | jobs
by baby 3261 days ago
"passes security audit". Is security audit an exam? What does passing mean?
3 comments

I doesn't mean anything at all, and people should stop writing headlines and paragraphs implying that it does.
Purely naively I would guess that it means during whatever audit they ran, no signs of insecurity were observed. Maybe it would be better to say that it didn't "fail" the audit?
You can't really fail an audit though. The point of an audit is to make your application more secure. Using terms like pass/fail just reinforces a sense of fear where there shouldn't be any.

A pentest consists of an analysis period, typically about a week. Then any flaws in your app are communicated to you, along with steps to reproduce them. When you feel you've fixed the issues, a retest is scheduled and the pentesters verify that each flaw has been fixed.

A healthy application is one that's pentested on a regular basis. Ideally after every release, though only big companies can afford that.

>You can't really fail an audit though. The point of an audit is to make your application more secure. Using terms like pass/fail just reinforces a sense of fear where there shouldn't be any.

I see, that's a good point I hadn't considered.

Want to guess how many audits Microsoft Windows "passed" before the SMB bug exploited by WannaCry became public? :)

That was one of the most heavily audited components too.

Yes, a security audit is an examination of an application and the processes around it. In this case passing means the application "is able to offer a good level of privacy and security. In other words, the Briar secure messenger can be recommended for use."
All it means to "pass" an audit is that at the conclusion of the audit there were no outstanding vulnerabilities that the "auditor" had found.

"Audits" and "passing" make some sense for network security, where you can run a checklist of best practices and known vulnerabilities. But you can't really "audit" source code in the same sense, any more than you can contract someone to spend 2 weeks finding all the sev:hi crashers or data loss bugs in your database.

It would be good if organizations could stop pretending that "passing" a software security assessment was meaningful.

What you really want to know is how many person/days Cure53 spent on Briar, who Cure53 had staffing the engagement, what the scope of the engagement was (what components were off limits), and whether they found anything that was subsequently fixed (it's an industry secret that one of the reasons you do an audit is so you don't have to publish the "real" findings).

From the report, it looks like they spent 13 calendar days testing (it's not clear how many person/days were spent), of which only 3 were dedicated to cryptography, and the audit was constrained to the Android clientside code.

For perspective: the "industry standard" software pentest of a reasonably complicated web application is 2 people, 2 weeks.