[tav]

When you validate server certificates from HTTPS clients, please be sure to use the right set of root certs. Mozilla maintain a decent list of these [1], but it's not in the PEM format that most HTTPS client libraries expect, e.g. Python's ssl.wrap_socket(sock, ca_certs="certs.pem").

Mozilla's list also includes distrusted certificates, so you need to be careful to leave them out when generating the PEM-encoded format. In fact, I'd strongly recommend using Adam Langley's excellent extract-nss-root-certs tool [2] which takes care of the subtle details for you.

And, if you are willing to trust me, you can download my pre-generated PEM-encoded cacerts file from a month or so ago [3].

[1] https://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw...

[2] https://github.com/agl/extract-nss-root-certs

[3] https://github.com/downloads/tav/ampify/distfile.cacerts-201...

[va1en0k]

I'd actually recommend just get the exact certificates you're going to work with. For example, if I'm using GitHub's API, it makes sense not to check all the chain, but just keep the GitHub's original cert.

[tav]

Disclaimer: I don't particularly agree with the Certificate Authority mechanism that we currently use with TLS.

However, given that it's what we currently have, I'd strongly advice taking advantage of the security that it provides. Requiring API client library authors to ship certs will make for poor security. Not only do certificates expire, they also get compromised.

It would be easy to conduct MITM attacks using revoked certs and API client library users would be none-the-wiser. Instead, it should be the responsibility of HTTPS client libraries to use the latest cacerts data and support features like OCSP [1] for validating certificate revocations, etc.

[1] http://en.wikipedia.org/wiki/Online_Certificate_Status_Proto...

[tptacek]

Taking a validated known-good cert and, from that point on, simply verifying that the cert remains bit-for-bit identical is called "certificate pinning", and while it does create problems with revocations, your service is going to break as soon as Github revokes their certificate anyways, so it's not like you won't notice.

[zmb_]

The problem is that the CA mechanism (in particular, blindly trusting whatever list of root certs your vendor ships) does not provide security. I cannot go and validate the practices of those CAs, and all it takes is one of them to get compromised (which they regularly do, have you taken out the compromised CA's root certs that ship with OS X, for example?).

The only certificates I actually trust are the self signed ones from my organization which I can actually go validate in person. While I have ZERO trust in any of the certificates that my vendor ships.

[throwaway64]

the problem here is rather than worrying about one particular cert getting compromised, you now have to worry about every CA in the world getting compromised, a much more likely possibility.

It seems the best course of action would be to trust only an individual cert, and check for revocation.

Also OCSP is basicly a joke, it works every single time, except when it matters (an attacker controlling your view of the world)

[va1en0k]

Yeah. I'll edit my post to add your points, but I can't do it right now

[codeka]

The problem here is that if your app is deployed in a corporate environment, it's possible (likely) that the corporate firewall is intercepting your HTTPS traffic and returning a different certificate, issued by the IT department.

So if you try to validate that the certificate is the specific one that your API server is using, it's going to fail in that scenario.

Depending on your app, you could just ignore that possibility of course.

[gizmo686]

How does your scenario differ from a man in the middle attack?

[jerf]

The word "attack". In theory, being in a corporate environment, this is a desirable man-in-the-middle rather than a hostile one. It certainly is man-in-the-middle.

[zurn]

It's exactly what SSL/TLS is designed to defend against, and neutering your apps & applying newspeak doesn't make it preserve the security provided by SSL/TLS.

[jerf]

Conceptually, in a work environment you aren't accessing the website, your company is. If your company chooses to add an SSL proxy for its own purposes, there's nothing invalid, wrong, or unethical about that. Conceptually, you're all functioning as one entity.

You may note I'm using words like "theoretically" and "conceptually" in these replies, and that's basically because ctz's point is accurate. It isn't hard for someone on HN to be more competent at SSL usage than the administrator of the SSL inspector. But, well, welcome to the corporate world. Can't live with 'em, can't live without 'em. But I don't think it's wrong on any moral or technical level, it's just potentially wrong based on more mundane considerations, like competence.

[codeka]

Nobody is saying it is. However, this is the reality in a surprisingly large number of corporate environments. I have to support a lot of enterprise customers in my day job, and working around corporate firewalls is a large part of the issues that come up for us.

[gizmo686]

So you still need to provide some mechanism to prevent malisous attacks. The point of validating certificates is to prevent this class of attack.

[codeka]

Yes of course. In a corporate environment, you would usually install the proxy server's CA certificate in your certificate store and validate that all certificates were issued by the proxy server.

My original comment was just pointing out that validating that the certificate you get when you connect to https://www.github.com in a corporate network may not be the same as the one you get on the open internet.

It's up to you to decide whether that's something you care about though.

[voltagex_]

Yes, but you're then reliant on the proxy to do the validation for you.

[ctz]

And for the proxy's private key (which you are henceforth relying on for all transport security) to be kept secure. Given my recent experience with products like websense, this is a very poor bet to make.

[jlao]

If GitHub's cert is revoked or expired, you'll have to manually go grab the new one. You want to trust the issuer of the cert.

[rfugger]

That's great until it changes.

[Hoff]

If you trust the cURL folks, there are regular conversions of the Mozilla root certificates available[1].

There's a Perl-language converter tool available there, for those that don't have Go handy for the extract-nss-root-certs tool.

And there's also a shell script there that uses the certutil tool [2] that can be used to extract the Firefox certificate store.

[1] http://curl.haxx.se/docs/caextract.html

[2] http://www.mozilla.org/projects/security/pki/nss/release_not...

[toyg]

Why working so hard, when M2Crypto [1] will do everything for you [2] ?

If you're doing anything serious with Python and SSL, you're going to use M2Crypto - period. Because when it comes to security, the less you "roll your own", the better.

[1] http://chandlerproject.org/Projects/MeTooCrypto

[2] http://svn.osafoundation.org/m2crypto/trunk/demo/x509/certda...

[tav]

Sadly, that M2Crypto script doesn't check for certificates which are not trusted for issuing SSL server certs. So whilst it happens to skip a few, it will include over a dozen inappropriate certs in the final output!!

This is exactly the problem that https://github.com/agl/extract-nss-root-certs was written to solve. I'd strongly recommend using it.