[sodality2]

Declaring it secure after an audit is like writing 100% coverage tests and saying it's bug-free. You can't prove absence, only presence.

This title is the definition of sensationalism and only by reading the article do you find the truth: "Their tests uncovered no major issues or security vulnerabilities". This is a bad look for them and I'm wary of their company now...

[capableweb]

I agree with you that the title is a bit sensationalist. But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure? Or are you of the opinion that nothing ever can be claimed to be secure as there can always be holes that could be uncovered in the future?

Using openssh as an example, would you say it's secure when you're using public keys for the authentication? Their track record seems pretty good for the last years, but there might still be uncovered vulnerabilities, could it still claim to be secure?

[tailspin2019]

I think it's partly the phrase "Security experts declare" which is likely to be what's rubbing people up the wrong way.

The security auditors themselves would never actually word it like that in their reports, because the statement implies a degree of certainty that cannot really exist.

Here's an example of what the auditor's actually said:

"Auditors identified two low-severity vulnerabilities. Additionally, five general recommendations were reported. At the same time, we confirm that no important security issues were identified during the pentest."

There's a reason that audit reports will never say outright that something is "secure". They may say something like "strong and effective security measures are in place", but that's a very different kind of statement.

I think the article itself is great but the headline just falls on the wrong side of being a bit hyperbolic and seems to be optimised for marketing impact over accuracy.

[sodality2]

> But if independent security audits with no major issues uncovered cannot make you claim something is secure, when can you claim something as secure?

Nothing at all; it's a broken model. The server can at any time start serving malicious payloads [0]. The server hosts your mail but they also serve the webapp. The clientside decrypts the mail, but the server hosts the client code...

It's a fundamentally flawed idea, trying to retrofit encryption into email in this way, when the server essentially has to hold all of your mail. In this case, the only thing that would make me feel secure in using it, is a third-party OSS client that downloads the mail without using the webapp, using only client-side code. And even then, of course, the mail can simply just be not encrypted when being ingested by Proton. So even then I wouldn't really trust it without external encryption like PGP. In which case, why bother?

To be clear I do use private email services (protonmail, tutanota) but I am simply not going to fall for the illusion of guaranteed privacy; I just trust that they are what they say they are. They are still a better option IMO than something like Gmail, but I don't think they're a silver bullet.

[0]: If you think this is unlikely, see this: https://news.ycombinator.com/item?id=25337507

[prophesi]

This is why I'm excited for 9elements and Mullvad's System Transparency[0][1] project. The goal is to let the end users know that your server is indeed running the code it says it's running. Of course, open firmware and hardware also plays a role in this as a server running binary blobs still has the potential for malicious code to be unknowingly run.

[0] https://mullvad.net/en/blog/2019/6/3/system-transparency-fut...

[1] https://www.system-transparency.org/

[dane-pgp]

> The server can at any time start serving malicious payloads

True, and I call this threat model "Beware Each and Every Fetch" (BEEF) in contrast to the more common TOFU model (although if you trust a desktop app to auto-update itself then these two models might not be all that different).

In any case, I think you're being a little quick to dismiss the idea of server-hosted applications. It's true that browsers don't natively have a nice way of pinning specific versions of a web app, but there is the clever hack of SecureBookmarks[0] (if you're prepared to sacrifice the UX), or, more realistically, you can pin the web app version using some sort of browser extension.

Examples of the latter include the Signed Pages extension[1], and Code Verify[2], which is the result of a collaboration between Meta and Cloudflare (for securing the WhatsApp Web code, currently, but should eventually support other sites like Proton's too). Of course, it would be much better if this capability was natively included in browsers themselves, but hopefully adoption of this technology will pressure browsers and standards bodies to take ownership of this.

[0] https://coins.github.io/secure-bookmark/

[1] https://github.com/tasn/webext-signed-pages

[2] https://github.com/facebookincubator/meta-code-verify

[uselpa]

Secure email is snake oil; no amount of cruft can make it both reasonable secure and useful (as in federated). Other protocols better fill that space because they were designed for security needs.

[chrismorgan]

I wouldn’t call secure email snake oil: it meets some of the characteristics, but not all. “Snake oil” implies at least in part inefficacy and deceptive marketing; yet secure email is possible, though there are typically rather severe caveats (mostly around the question of which parts are being encrypted, and usability).

What is actually snake oil, and distressingly rarely realised as such, is first-party end-to-end encryption. That’s what sodality2 is actually talking about. And when you stop and consider it in this light, you realise that the significant majority of stuff that’s advertised as having E2EE is first-party and thus, to put it mildly, not robust.

In the context of email, here’s Fastmail’s take on it: https://fastmail.blog/advanced/why-we-dont-offer-pgp/.

[gruez]

I agree that I wouldn't use Protonmail if Signal was an option, but there are many situations where Signal isn't an option (eg. transactional email, people who don't use signal). In those contexts it's still better to use encrypted mail rather than ad-supported mail (eg. gmail, outlook), or even commercial mail (eg. fastmail), which are both unencrypted.

[sodality2]

End to end secure email definitely is, I agree. No matter what you have to trust the server. (Maybe a Tor-based email system would be better, where each user is their own server? reminds me of `cables`).

If you do trust the server then it can be acceptably secure.

[upofadown]

It appears the audit was applied to the Android and iOS apps. So no comment is being made here about the security of the webapp.

[sodality2]

I thought that was just a wrapper around the website, to be honest. But regardless, the app still loads JS from the website and executes it, so it's still a backdoor possibility.

[Beldin]

> when can you claim something as secure?

Typically: don't do that.

Nevertheless, if you insist: you can claim a certain abstraction of a system guarantees certain mathematically expressed requirements cannot be violated by a certain attacker model once you've formally proved that.

Of course, all implementations have implementation details which violate the abstraction, your mathematically expressed requirements may not fully capture your intentions, and in practice, an attacker may have additional options that your model doesn't consider. But hey, now you can truthfully claim that "the system" is "secure" - for some values of "system" and "secure".

[cascom]

I think if it it more like having a fire Marshall inspect a building, doesn’t mean it can’t catch fire the next day, but at least you know that it’s not death trap (smoke alarms, fire doors, sprinklers etc all check out)

[yakak]

There is a framework for this:

https://en.wikipedia.org/wiki/Evaluation_Assurance_Level

Proton is claiming something similar to EAL4, which is not secure, there is an assurance that not all trained reviewers can find a vulnerability. Openssh is a little less secure than that formally, but has more trained reviewers informally, which probably cover some parts extremely well and other parts sparsely.

[josh2600]

Here’s how I feel about this:

You can write all of the tests, you can get all of the audits, but there’s nothing that’s going to stop a 13-year old polish kid from mucking about in the guts of your tech. Security isn’t a promise you can make in absolutes, at some point you have to ship and you hope that you did everything well enough that there’s no low-hanging fruit.

There’s no such thing as a secure system, only systems which are more expensive to compromise.

[MattPalmer1086]

You can't claim anything is secure just because problems haven't been found.

It does build confidence in the security to perform security audits.

[jsjohnst]

> when can you claim something as secure?

here’s a maybe wild take, uh, never?

[capableweb]

But you'd agree with me that some things are more secure than others right?

[jsjohnst]

I’d be willing to say “there are things with known exploits, there are things which employ techniques that are known not to be safe making them less secure, and the rest is more akin to Schrödinger's cat as far as secureness goes.”

[LinuxBender]

You beat me to it. I have sat with hundreds of software and network auditors. Never once have I heard them say something was secure. At best they might say they didn't find any high severity issues in this particular audit. I am curious who their statement was intended for.

[WhyNotHugo]

Considering we live in world where Google and Facebook publicly state "We care about your privacy", I think this title pretty close to reality.

Have they proved the non-existence of bugs? Nope. But the title is also not the complete opposite of reality, which is what their competition seems to doing.