Hacker News new | ask | show | jobs
by lawnchair_larry 3261 days ago
As someone who does professional security audits, I would just like to say that there is no such thing as "passing" a security audit. In fact, most pen testing shops will carefully dance around actually making that claim in writing for a customer, because they know they are going to look bad when a bug is inevitably found in code they reviewed (and it's probably a dumb idea for liability reasons too).

There are certain certifications with falsifiable conditions that can be marked pass/fail. But, as I'm sure many folks here are aware, these are incomplete and often completely dubious. They don't purport to be "security audits".

What a real security audit tells you is that of the (probably 2-4) consultants that looked at a product for a few weeks (probably 2-6), these were the security bugs they found.

That alone contains little information, because the skill level and domain expertise varies greatly among consultants and companies. I can guarantee that if these results were withheld, and they gave the same codebase to another reputable outfit, the set of findings would be very different. There would likely be some overlap, particularly in the most obvious types of bugs, but bug hunting is way closer to art than science.

I know nothing about this project, and my intent is not to create doubt, but users of secure messaging apps should understand what an audit is and what it isn't.

Like other commenters, I was surprised to see 3 days of looking at crypto. It could be that the crypto is extremely simple and uses a few well understood APIs in a straightforward way, so this isn't a guaranteed red flag by any means, but it's a bit unusual.

And like any software, this is a 1 line patch away from being blown wide open. With every commit, an audit becomes increasingly meaningless. Just ask cperciva!

And perhaps I'm being cynical, but I always felt like the "conclusions" section of the audit report has an unspoken purpose of walking back from calling their baby ugly and keeping a decent rapport to ensure the possibility of future business. Not that I think what Cure53 wrote was not genuine, but there are natural incentives to be a little generous there. Again, I'm speaking from experience writing those sections as well.

Edit: Basically what tptacek said.

2 comments

I was not aware of the tarsnap issue, but for anyone else wondering, I think it's this: http://www.daemonology.net/blog/2011-01-18-tarsnap-critical-...
I haven't looked at the audit yet (and agree with your comments), but I can say a bit about what Briar is doing with crypto.

The focus is on a time window based hash derivation of keys for symmetric cryptography and tags to recognize streams. It currently uses blake2s and XSalsa20/Poly1305. Bouncy Castle is used for the core algorithm implementations when possible.

Connections are made via QR code and use ECDH with cofactor multiplication. There is also a simple bittorrent-inspiried synchronization level that is new and an encrypted storage layer for data storage (I'm not sure but I think this may use pre-existing code).

So there is some amount of crypto to look at but it is fairly basic and not doing anything exotic. The layering and heavy use of symmetric crypto makes the crypto simpler than might be expected based on the features (and battery use heavier).

Version 1 of anything is likely to have issues and hopefully even the release will have a disclaimer to that effect, but there is always a tradeoff between needing some amount of support for further development and trying to make the best app possible before releasing. Briar has been in development for years and they are aware of that tradeoff and trying to both be cautious and not allow the project to die from lack of usable result.

The transport layer spec is at: https://code.briarproject.org/akwizgran/briar-spec/blob/mast...

QR code based key exchange spec is at: https://code.briarproject.org/akwizgran/briar-spec/blob/mast...

Sychronization layer spec is at: https://code.briarproject.org/akwizgran/briar-spec/blob/mast...