[tptacek]

I was going to blog this after tormenting Sleevi on Twitter with more dumb questions, but I guess we're just going to do this here.

The issue† --- it's super complicated in its particulars but not in its outline --- is that CAs have been issuing constrained intermediate CAs (CA's that can sign only a subset of certificates) that, because of a misconfiguration, can sign any OCSP message for their root CA. So, for instance, Sleevi points out a HARICA (Greek university CA) cert that is constrained by dint of not having the serverAuth EKU (so it can't sign TLS certs), but because it has the OCSPSigning EKU, can be used to sign OCSP messages.

What happened was this: lots of CAs run by companies in the wild (remember: there are all sorts of constrained CAs running inside companies that you aren't supposed to have to care about, in part because they're constrained) run on Microsoft's CA software. Many of those CAs want to support OCSP. The way you generally set up OCSP is to sign a Delegated Responder certificate, which is an end-entity (non-CA) cert with the OCSPSigning EKU, which says "this certificate can be used to sign OCSP certs for this part of the PKI". But Microsoft's CA is broken: it won't let you sign a cert with the OCSPSigning EKU unless your CA cert also has that EKU (the EKUs must "chain"). So, that's what CAs did, despite the fact that chaining EKUs like that changed the semantics of what the certs were intended to express.

So, as Sleevi put it, basically a bunch of CAs accidentally signed the OCSP equivalent of a bunch of CA:TRUE certificates. They gave their customers the ability to essentially disable revocation for the whole CA.

We can go back and forth on how sound the WebPKI revocation infrastructure is with or without this mistake. But at a minimum we should be able to stipulate that strengthening revocation is an important project for browser vendors, and CAs that accidentally break the revocation infrastructure by misissuing certificates are an impediment to that project.

There's an interesting backstory to this, which is what I actually wanted to write about, and that's CA/B Forum SC31. The CA/B Forum is the standards body that coordinates between browsers and CAs; they maintain the BRs, which are the bylaws for operating a CA that browsers will trust. The politics of CA/B Forum are weird, because CAs and browsers are structurally adversarial: browsers want maximal security regardless of the commercial implications for CAs (as they should). As a result of that weirdness, the browser vendors rely not only on the BRs, but also on their own bylaws for their respective root certificate programs. SC31 is an attempt by Ryan Sleevi to align the CA-approved BRs with the browser-approved root programs.

Of course, the browser root programs, particularly Mozilla and Google's, are the only thing that really matters, because those rules determine whether the browsers themselves will honor a CA's certificates. But the CA/B Forum is dominated by CAs who sort of dispute this fact. So, SC31 asks that the BRs import that (now common) browser root program rule that certs can only live for just over a year. The "one year only" rule was considered previously by the CA/B Forum and failed in a vote (CA customers don't like that rule), so the CA's are unhappy to be relitigating that point. But then, the litigation itself is kind of theatrical, in that browsers already made this decision and mooted the debate. Standards bodies are weird.

Anyways, what makes this funny is that, while the one-year restriction is the real problem that seemed to piss off the CAs in SC31, they also challenged some of its OCSP language. In the course of debating with them about whether SC31 described reality or not, Sleevi started looking at their OCSP issuances. And here we are: the CAs aren't even following their own BRs, and are breaking their bylaws in ways that materially impact TLS security.

I understood literally none of this when I first read Ryan's message. I thought I knew some stuff about the WebPKI, but even with help on Twitter I had trouble with this eldritch stuff. I don't know how people like Sleevi do it, and, apparently, neither do many CA operators.

This probably would have been a boring blog post anyways, but to make Kurt happy, I'll conclude with: "use Fly.io!". :)

† Wrinkle: Sleevi flags these CA certs as missing the ocsp-nocheck attribute, which is presumably how he spotted them. ocsp-nocheck says "this certificate can't be trusted to OCSP revoke itself, because that's silly"; the idea is, certs flagged ocsp-nocheck rely on very short lifetimes rather than OCSP to manage revocation. Very short lifetimes are things you expect on end-entity certificates, which is what OCSP Delegated Responders are, but not so much on CA:TRUE certificates, which are what Sleevi actually found.

[tialaramex]

> The CA/B Forum is the standards body that coordinates between browsers and CAs

I would quibble with the description of CA/B as a standards body. It's a standing meeting. Any CA/B documents including the Baseline Requirements manage relationships only between (potential) parties to CA/B itself, the browsers and public CAs.

The reason the meeting exists anyway is that it sucks for the public CAs if the major root programmes have conflicting rules or interpretations of those rules. The idea of the BRs is to as much as possible agree rules with everybody to avoid such conflicts.

Also I would rate Microsoft and Apple as equally significant with Google and Mozilla and it's at least tempting to add Oracle (because Java has its own root trust programme) too.

In terms of whether they'll throw their weight around we're used to seeing that from Google and Mozilla (e.g. on SHA-1 and on the Blessed Methods) but Apple proves here (on the one year certificates thing) that sleeping giants might wake up and kick over everything you're doing if it displeases them.

If Microsoft were to, for example, have refused to trust ISRG (they took a very long time to actually make any decision) I'm sure that we'd moan about it, but we can't make them, and without being trusted in millions of Windows PCs when their cross signatures expire that would the end of the story for Let's Encrypt right?

[jcranmer]

The strength of Mozilla is that it runs its root certificate policy very publicly, with public review periods and chances to objection, as well as detailing the incident reports on its wiki in full view of the public. I know some of the Google engineers are also very present on Mozilla's lists.

The other non-Google, non-Mozilla companies don't have anywhere near this level of public disclosure from what I've seen, and I suspect that they may follow some of Mozilla's groundwork (especially in terms of dishing out punishments in response to incidents).

[tialaramex]

Without any doubt Mozilla's public oversight role (via m.d.s.policy) is extremely important.

Google actually doesn't operate transparently either, except in the sense that it chooses to participate in m.d.s.policy. You won't find a public process behind Google's decision to require CT for Symantec's roots before it was mandatory for other roots for example, they just announced the policy as a done deal.

You are not alone in concluding that Mozilla's distrust decisions (I wouldn't characterise them as "punishment") are in practice copied by the other root trust stores. It is entirely possible that Microsoft (for example) has a large team of dedicated experts independently investigating incidents and just coming to coincidentally similar conclusions. After all, the facts won't be different if a Microsoft team investigates them than they are when Mozilla and third parties do so for m.d.s.policy. But it's a hell of a coincidence...

I would note that for initial trust decisions Microsoft in particular does not follow m.d.s.policy. If you run Windows there's an excellent chance that your computer (and thus Internet Explorer, Edge and Chrome on that computer but not Firefox) trusts poorly run Certificate Authorities from a variety of organisations and countries which don't seem very trustworthy.

For example the governments of Sweden, Slovenia and Thailand.

[Edited: This used to mention Venezuela but the Venezuelan government CA was in fact distrusted by Microsoft]

Now maybe Microsoft's team carefully vetted all these dozens of Certificate Authorities that aren't trusted elsewhere and concluded they're doing a great job. In some cases we know they weren't able to satisfy Mozilla (or volunteers contributing to m.d.s.policy) but in other cases they never applied at all. Maybe they're just shy?

So far we can say this doesn't seem to have caused any serious reported problems. So maybe it's fine.

[sleevi]

If Apple is the sleeping giant of PKI, Microsoft is the come-back kid. The actual set of CAs trusted by Microsoft has massively shrunk under the leadership of their new Root Program manager, and their transparency greatly improved. https://aka.ms/rootupdates shows a regular cadence, particularly on even months, of removing trust in a large number of CAs. While they still add CAs faster than any other program, they also have strong contractual guarantees on CAs in a way unlike that of Mozilla, Apple, or Google. And Microsoft is notoriously not afraid of using lawyers for noble causes.

[tialaramex]

[[Ryan Sleevi wrote the m.d.s.policy post this HN item is about]]

That link says the even number month changes are CA led.

Now of course you certainly have much better insight than I do into what's behind those CA led changes because I'm just a Relying Party with their nose pressed against the window. Maybe that new Root Program manager is encouraging participants to clean stuff up with an implied threat that if they don't Microsoft will. But as an outsider it still looks a lot like the old Microsoft root programme to me. Also Microsoft's "revoke or else" rule still sits badly with me despite its purported use to prevent people scamming Microsoft's customers. But I guess I'm glad to hear you think they've "greatly improved".