[toast0]

> There was the logic that generates the SSL key pair, and there is the faulty logic that consumes it. Based on the description, it seems it's an indication of missing test coverage around the faulty code. If, when the faulty code was written, more time were spent on understanding the assumptions the code has made, then maybe the test wouldn't appear in the first place.

Based on the description of the bug, the bug was in Microsoft's SChannel TLS stack (I know this, because I found it too, and got a workaround into OpenSSL). I don't know about you, but I haven't written a whole lot of comprehensive tests for any TLS stack that I've used, unless I wrote the stack. I'm assuming jooster didn't work for Microsoft, he just worked for a company that released software for some flavor of Windows and used their TLS stack, because it was there.

This definitely fits into the category of nobody is going to test this, because it's not going to occur to anyone to test how the third party TLS library they're using handle public keys encoded without leading zeros, until they've ran into it before. Having a random key generated in a test suite means you've got a chance to see it; if you're lucky (and if you don't just retry everything without knowing why).

[joosters]

Nope, this was on a variety of unix systems, but no Windows code at that point. We used some RSA libraries for crypto primitives and hardware acceleration, with a lot of custom in-house code (BigInts and other math routines). There was no OpenSSL usage. I think the bug turned out to be in our own code, because it was a case of lots of frustration tracking it down, an ‘ah-ha!’ moment, and a quick code check-in fixing the problem there and then. (Yes, we also wrote a new test case!)

But it’s interesting to hear about an identical sounding bug in similar code/routines. I’d say ‘what are the chances?’ but crypto code is always painfully hard.