[woodruffw]

I think this part is really worth engaging with:

> Later, moving public key parsing to our own Rust code made end-to-end X.509 path validation 60% faster — just improving key loading led to a 60% end-to-end improvement, that’s how extreme the overhead of key parsing in OpenSSL was.

> The fact that we are able to achieve better performance doing our own parsing makes clear that doing better is practical. And indeed, our performance is not a result of clever SIMD micro-optimizations, it’s the result of doing simple things that work: we avoid copies, allocations, hash tables, indirect calls, and locks — none of which should be required for parsing basic DER structures.

I was involved in the design/implementation of the X.509 path validation library that PyCA cryptography now uses, and it was nuts to see how much performance was left on the ground by OpenSSL. We went into the design prioritizing ergonomics and safety, and left with a path validation implementation that's both faster and more conformant[1] than what PyCA would have gotten had it bound to OpenSSL's APIs instead.

[1]: https://x509-limbo.com

[tialaramex]

It is extremely common that a correct implementation also has excellent performance.

Also, even if somebody else can go faster by not being correct, what use is the wrong answer? https://nitter.net/magdraws/status/1551612747569299458

[woodruffw]

> It is extremely common that a correct implementation also has excellent performance.

I think that's true in general, but in the case of X.509 path validation it's not a given: the path construction algorithm is non-trivial, and requires quadratic searches (e.g. of name constraints against subjects/SANs). An incorrect implementation could be faster by just not doing those things, which is often fine (for example, nothing really explodes if an EE doesn't have a SAN[1]). I think one of the things that's interesting in the PyCA case is that it commits to doing a lot of cross-checking/policy work that is "extra" on paper but stills comes out on top of OpenSSL.

[1]: https://x509-limbo.com/testcases/webpki/#webpkisanno-san

[jmspring]

I’d say correct common path. OpenSSL due to hand waving deals with a lot of edge cases the correct path doesn’t handle. Even libraries like libnss suffers from this.

[nine_k]

Are these edge cases correct to the spec, or not?

[woodruffw]

There are multiple overlapping specifications for things like X.509. There are the RFCs (3280 and 5280 are the "main" ones) which OpenSSL generally targets, while the Web PKI generally tries to conform to the CABF BRs (which are almost a perfect superset of RFC 5280).

RFC 5280 isn't huge, but it isn't small either. The CABF BRs are massive, and contain a lot of "policy" requirements that CAs can be dinged for violating at issuance time, but that validators (e.g. browsers) don't typically validate. So there's a lot of flexibility around what a validator should or shouldn't do.

[pseudohadamard]

Yes.

The spec is often such a confused mess that even the people who wrote it are surprised by what it requires. One example was when someone on the PKIX list spent some time explaining to X.509 standards people what it was that their own standard required, which they had been unaware of until then.

[rout39574]

Got any links to that conversation? Sounds fun.

[pseudohadamard]

I'm drawing a blank on it sorry, it's somewhere in an archive of messages but I can't find the appropriate search term to locate it. However it did turn up a reference to something else, namely this, https://www.cs.auckland.ac.nz/~pgut001/pubs/x509guide.txt. It hasn't been updated for a long time but it does document some of the crazy that's in those standards. The various Lovecraft references I think are quite appropriate.

[pseudohadamard]

Technically yes because I saved the messages, which I saw as a fine illustration of the state of the PKI standards mess. However I'd have to figure out which search term to use to locate them again ("X.509" probably won't cut it). I'll see what I can do.

[RiverCrochet]

Remember LibreSSL? That was borne of Heartbleed IIRC, and I remember presentation slides saying there was stuff in OpenSSL to support things like VAX, Amiga(?) and other ancient architectures. So I wonder if some of the things are there because of that.

[fanf2]

Most of the performance regressions are due to lots of dynamic reconfigurability at runtime, which isn’t needed for portability to ancient systems. (Although OpenSSL is written in C it has a severe case of dynamic language envy, so it’s ironic that the pyca team want a less dynamic crypto library.)

[tialaramex]

The Amigans really like their system. So they kept using them long after mainstream users didn't care. By now there probably aren't many left, but certainly when LibreSSL began there are still enough Amigans, actually using an Amiga to do stuff like browse web pages at least sometimes, that OpenSSL for Amiga kinda makes sense.

I mean, it still doesn't make sense, the Amigans should sort out their own thing, but if you're as into stamp collecting as OpenSSL is I can see why you'd be attracted to Amiga support.

Twenty years ago, there are Amigans with this weird "AmigaOne" PowerPC board that they've been told will some day hook to their legitimate 20th century Commodore A1200 Amiga. Obviously a few hundred megahertz of PowerPC is enough to attempt modern TLS 1.0 (TLS 1.1 won't be out for a while yet) and in this era although some web sites won't work without some fancy PC web browser many look fine on the various rather elderly options for Amigans and OpenSSL means that includes many login pages, banking, etc.

By ten years ago which is about peak LibreSSL, the Amigans are buying the (by their standards) cheaper AmigaOne 500, and the (even by their standards) expensive AmigaOne X5000. I'd guess there are maybe a thousand of them? So not loads, but that's an actual audience. The X5000 has decent perf by the standards of the day, although of course that's not actually available to an Amiga user, you've bought a dual-core 64-bit CPU but you can only use 32 bit addressing and one core because that's Amiga.

[some_furry]

Now I wonder how much performance is being left on the table elsewhere in the OpenSSL codebase...

[Avamander]

Given the massive regression with 3.x alone, you'll probably be happier if you don't know :/

[Xraider72]

haproxy has an article on the subject

https://www.haproxy.com/blog/state-of-ssl-stacks

TLDR - on the TLS parts, quite a lot, up to 2x slower on certain paths. Amusingly, openssl 1.1 was much faster.

libcrypto tends to be quite solid though, though over the years, other libraries have collected weird SIMD optimizations that enable them to beat openssl by healthy margins.