> Needless to say, OpenSSL is looking for contributors. Beyond contributing patches, interested developers can test the pre-releases, report bugs, and help to close bugs. The presenters concluded by saying that they would like users to get in touch, especially those who are distributing OpenSSL further downstream.
... and just a few days later they closed ~ 150 bugs, some with patches attached, "because they were too old":
That was something else that irked me in the article, where they decry the small $2,000 a year they receive in donations; but they don't address how much their commercial support contracts[1] bring in. OpenSSL seems to be both an open source project and a cottage industry all wrapped in one.
Whenever there's an article about OpenSSL, I get on my soapbox to talk about how shitty it is. The design is terrible, both internally and in the public API. Building it is a PITA. It is almost completely unaware of this thing called multithreading. It still is hard-coded to be able to read certificates only via disk files with fopen, despite having a screwy BIO system which is a half baked attempt at sometimes abstracting IO.
If it was anything other than a security library, it would have died long ago.
I contribute to some software that's relatively widely deployed and needs crypto primitives. Sadly on Linux there are few choices that support common algorithms and is usually available. The alternatives sometimes exhibit performance issues due to missing optimizations as well, or simply don't support the needed algorithms. On the other hand OpenSSL took it's sweet time to implement eg. 25519 based crypto or chacha20 which is also a pain - today Chacha20-Poly1305 is the only viable AEAD crypto system for embedded/low-computing-capability devices, including those with AES acceleration.
Another thing I'd like to ramble about a bit is their API/release management. In 1.1 they made a bunch of structures opaque - which is good IMHO - but due to the very inconsistent APIs before they needed to add some utility APIs to handle opaque structs in some places; instead of backporting those to 1.0.x, so that you, as an application developer, don't need to add conditional compilation and shims to every application, they chose not to. Additionally shims posted in the wiki were (maybe still are) wrong.
Forcing application developers to figure out API compatibility code for a crypto library is bad. That code will be tested very little or not at all, additionally it's hard to test.
Cryptography on Linux (or - on most platforms) is still in a bad shape.
> Cryptography on Linux (or - on most platforms) is still in a bad shape.
libsodium addresses the non-TLS part of this pretty well, doesn't it? I know it doesn't have any native password-based KDF that won't DOS your device under anything but simple load, but otherwise it uses good algorithms.
> today Chacha20-Poly1305 is the only viable AEAD crypto system for embedded/low-computing-capability devices, including those with AES acceleration.
True, but there are some nice-looking AEAD candidates in the CAESAR competition, some of which out-perform AES-GCM and CHACHA20 by a good margin. We'd probably all be using OCB mode if Rogaway hadn't used such a bizarre initial license (which took a few iterations to get in a sane state and still require you to pay him something like $70,000 USD for use in commercial embedded systems).
NaCl/libsodium doesn't really work for a lot of cases, partly since it only supports AES-(GCM) if the hardware supports it, which just ain't going to work for anything that uses encryption for storage ("You can't open this file on this computer, please go to a computer with AES-NI, thank you"), and partly because it's opinionated approach doesn't work for some software (AES-GCM is actually an instance of this).
Otherwise it's a nice library. So if it works for a project I highly recommend it.
--
I really hope that CAESAR moves the state of AEAD forwards. By all accounts it already has.
> Cryptography on Linux (or - on most platforms) is still in a bad shape.
Having worked with Windows and OS X crypto layers, OpenSSL, CryptoPP and libtomcrypt, I think that generally cryptography is in bad shape almost everywhere.
The libraries that are trying to improve the cryptography aspects tend to run into the issue that algorithms in heavy use are unsupported. The libraries that do support algorithms in heavy use tend to have poorly written code, convoluted APIs, or bad documentation.
I have a huge amount of respect for everyone doing work in this space. When things are working, most people ignore it, but when sometimes goes wrong, everyone is mad at you. I would imagine there isn’t a whole lot of motivation for most.
I don't know about CryptoPP, or your application of encryption, but wasn't libtomcrypyt a well-intentioned but wholly poor attempt at re-presenting crypto to developers? Is it even widely used? I came to it by way of libtommath, but recall feeling pretty quickly tomcrypto was not a crypto solution.
phk had a talk (at fosdem I believe) a few years ago in which he stressed that the easiest way for the NSA to "create" vulnerable code is to push out bad, unnatural APIs with horrendeous defaults into OpenSSL.
Gosh, to fix such a problem we'd almost need some kind of machine that can run a different program than its current one. Perhaps with evidence of that as a possibility, we could use a good crypto library instead, eg. https://github.com/jedisct1/libsodium
libsodium is not an OpenSSL replacement and doesn't want to be. It's a cryptographic primitive library and generally doesn't implement cryptographic protocols (eg. TLS, X.509, CMS, CRLs, OCSP, ...).
I recently experienced a bug with OpenSSL 1.1 and the team was pretty reactive!
I'm using OpenSSL 1.1 because it's the only DTLS implementation featuring all the extensions (OCSP stappling) and ciphersuites (AES-CCM-8) I need.
Interesting, I had no idea the changes that were happening to OpenSSL. It's sad that it took Heartbleed to kick it in the butt, but it's heartening that they appear to be heading in a better direction now.
I'm also curious how LibreSSL is doing—I haven't heard much about it lately. Did it fizzle, or is it still making good progress?
LibreSSL is part of the OpenBSD base system, and that is their primary target AIUI. They also maintain a portable version with support for a wider range of operating systems.
How come the biggest names in IT don't work towards creating a modern, better security library?
It's pennies for them, they wouldn't even see it on the balance. Having security issues slows down business significantly. Non-geeks keep hearing about major bugs in basic security and it surely has a negative effect on the spread of internet and services based on it.
LibreSSL is OpenSSL with some of the ugly bits stripped out. It's still aiming for compatibility, and so it retains many of the warts and will continue to do so. It's not a modern library though it may be a bit better than OpenSSL.
BoringSSL is similar, except there you additionally have the problem that google strongly discourages its use by third parties, as they consider it an internal library and semantics can change.
A better example of a modern library would be NaCL or libsodium.
That's not accurate - the LibreSSL developers have overhauled the internals fairly substantially in a lot of ways, and they are also making a sensible API on top of it (libtls).
I haven’t paid much attention to LibreSSL since the project started but I recently downloaded the source code for OpenSSH and was very impressed with how clean and readable the code for ssh-keygen is. After seeing that, I’d have a more confidence in the OpenBSD team; they already have a great reputation for security but looking at their actual output impressed me more. (Maybe, some day I’ll explore free Unix-like operating systems other than GNU/Linux.)
There are. They just ask for significant licensing fees to cover the work they did. Licensing fees most users of cryptography have no interest in paying. ;) Others are dual-licensed with little uptake.
An early example was Guttman's cryptlib which is dual-licensed:
Note: An interesting feature of it is that it embeds a security kernel that tries to ensure the algorithms are composed correctly.
Another dual-licensed one that seemed to at least have better code quality was PolarSSL used in Fox-IT's OpenVPN-NL project and partially verified by TrustinSoft's analyzer:
Save the best for last in Amazon's s2. It has much potential coming from the combo of simplicity, design-for-verification, and actual reviews by pentesters:
Note: There's also quite a few designed for robustness in smartcard industry. The EAL5+ one from IBM's Caernarvon team is likely to be high quality. The thing is we can't review them but we can review above code. So better to go with above. Also recall this list is of commercial replacements for SSL in addition to others' recommendations on LibreSSL, BoringSSL, etc.
It's 5-10 person-years of work, depending on where you draw the finish line, "just" to do something like BoringSSL. miTLS is at least five times that, and even there I'm assuming you start with people like Bhargavan and Fournet---both world class, people who've spent decades getting to the point where they can do this with 25 person-years.
That shows up on the balance sheet of the security-infrastructure team even of Microsoft or Google.
It's a coordination problem. If one company started trying to do it, they'd end up having to fund development while everyone else free-rode. And fundamentally there simply isn't that much money lost to security issues yet.
I would think they don't want to do it because they don't want to support it. Even though they are capable of creating it, they are then responsible for keeping it updated.
For me, Heartbleed was much much more than:
->At its core, Heartbleed was a simple bug, a missing buffer-length check
It was a catastrophic failure of a critical internet component that highlighted the truth that these supposed crypto security experts didnt have a clue.
We're talking about the kind of "simple bug" analogous to a television with bare wires as a power switch.
Imho it will take a lot more than a couple of years and a team change to reinspire faith in that brand of security. If we should at all.
I think that's somewhat unfair. People make mistakes, that's unavoidable and shouldn't be seen as an issue. Instead of blaming people for making mistakes we should consider what we can learn. We need to identify which mistakes happened, why they happened and what measures we can take to make such mistakes impossible or unlikely in the future.
It turns out that the industry is really bad that problem. Many libraries and critical parts of the infrastructure are developed using languages and tools that don't just allow mistakes to happen that can be prevented but make it easy to make such mistakes. Crypto is affected by this but there are many other areas which are affected by this problem to.
There needs to be a move towards more constrained languages like Rust that limit the potential mistakes, better development processes that prevent bugs from passing through that can be caught by humans and tools for testing that make it possible to test for a wider range of problems and not just those developers anticipate to occur.
Well, why do you think openssl is so pervasive? It's everywhere and it's not due to marketing or ease of use or code quality.
It's because nobody in their right mind wants to implement a TLS library. Openssl itself was only conceived because the original author was teaching himself C.
Another facet is that full compatibility to OpenSSL is required on the internet.
Ten years ago plenty of projects were using alternative SSL implementations (eg Peter Gutmann's cryptlib), many of which had slight interop problems with OpenSSL (which one to blame is impossible to say). In turn many projects switched away from these libraries - they had to, interop issues making software unreliable are vexing for every user - and moved to OpenSSL.
OpenSSL will also often be the first implementation that has a new TLS feature, it's the internet's demo implementation. New optimizations in algorithms tend to land first in OpenSSL as well etc.
Today there are like ~four or five relevant TLS implementations, and only one is in widespread use on servers.
I'm mentioning Rust to make clear the direction I'd like things to go and because there isn't really an alternative that allows for similar reusability and is practical for developing cryptographic libraries. That being said there are also implementations of TLS in other languages like Haskell or OCaml, that also wouldn't be vulnerable to these kinds of issues.
I agree that this is a difficult problem but I don't think that's really the barrier here. AI is also difficult and there is no shortage of improvement in that area. The real problem isn't the difficulty of implementing a TLS library, the real problem is inertia and an unwillingness to change.
SaferCPlusPlus[1] is an easier option when starting from a C/C++ code base. It's also one of the faster options, but if people are willing to accept some performance cost, I think the easiest solution would probably be to "harden" the LLVM/gcc sanitizers so that they can be used in deployed builds, not just debug builds. The sanitizers don't catch everything, but pretty close it seems.
I'll add that SAFEcode and Softbound+CETS automatically make C programs memory-safe against lots of problems. There was also the Clay language that could've been updated. C0 is a subset that was easier to verify used in Verisoft's microkernel and apps. Rod Chapman found an error in reference implementation recoding it in SPARK Ada. So that could be used.
All kinds of things could solve the C-level problems or substitute with a C wrapper but nobody cares enough to do it.
There are options now though. My production services terminate SSL with a non-C implementation. (ocaml-ssl may be beta and unaudited but my judgement call is that I trust it more than any C implementation).
Rust guarantees safety under certain conditions. That's it. It doesn't guarantee good design, bug free code, or code that never behaves badly. This is the kind of gratuitous maligning of code written in other languages that some Rust fans engage in that I find off-putting. Rust isn't The Solution To All Our Problems.
You are right, but any systems programming language that comes from the Algol or ML family doesn't suffer from "every line of code is a possible memory corruption" that C and its derivatives suffer from.
Rust or any other memory safe systems programming languages don't make all type of programming errors go away, but at least they make the memory corruption ones less likely to happen.
Also most of the C tools that help reduce memory corruption, if not hardware assisted like on SPARC v9 and Skylake, are mostly limited to having all source code available.
It feels like moving from Assembly to an higher level programming language.
I mostly agree. However I have this prediction: if Rust gains popularity, then in 5-10 years (or perhaps fewer!) someone will be complaining about memory issues in a Rust library that is peppered with "unsafe" blocks that don't properly respect the safe/unsafe boundary contract.
Even the core under-the-hood Rust std library is a human-vetted set of unsafe code that the compiler simply cannot prove as much about as Safe Rust. Human-vetted things are prone to error, no matter how good the humans doing the vetting are.
> Even the core under-the-hood Rust std library is a human-vetted set of unsafe code that the compiler simply cannot prove as much about as Safe Rust. Human-vetted things are prone to error, no matter how good the humans doing the vetting are.
At some point, it has to be human vetted. I don't see another way. The key advantage is the ability to build safe abstractions from unsafe components.
Will people abuse unsafe? Absolutely. Is it a lot better than the alternatives? IMO, absolutely. :-)
In my own experience, unsafe is rarely needed. I've only ever used it for ffi and lightly in very performance critical areas of code. The regex library, for example, has almost no unsafe. Its only use is to get rid of bound checks inside the DFA's inner loop.
The most (non-ffi) unsafe I've ever used is in a snappy compression library, where most of it was a means to more efficiently shuffle bytes around without paying for memcpy (using unaligned 8 byte or 16 byte loads/stores).
There is a pretty strong tendency in the Rust community to avoid unsafe. Some libraries advertise themselves as having "no unsafe code" as a selling point precisely for the dangers you've hinted at: humans make mistakes.
If you work on real objects made of metal and wood with dangerous tools like chain saws. There are a variety of protective measures you can take like guards, clothing and so on. These things don't provide guarantees safety under all conditions, they don't guarantee good design, objects that have no mistakes or that will never behave badly when used.
And yet, despite all of that not using these protective measures is considered by every reasonable person as incredibly stupid and worthy of a darwin award. This goes so far that there are laws that regulate this.
Nevertheless somehow when it comes to software all of that goes out the window. There is risk to other peoples money, privacy and potentially their lives? Who gives a shit? It doesn't make everything a 100% safe so let's use decades old tools that are difficult to learn are horrible to work with, allow for a huge number of completely preventable mistakes to happen, that we know happen often.
In any other context people would never come up with such a criticism. It would be considered ridiculously stupid to the point that some might think you're being sarcastic and making a joke and once they discover they not, they'd ask if there are some mental problems that might require attention.
Rust and languages that go into that direction don't solve all our problems and that's completely irrelevant.
So? Nobody disagrees on the fact that there is a problem. Putting blame on people won't solve the problem. It will piss people off. People you want on your side, enthusiastically solving the problem. Your attitude is actually dangerous and counterproductive to solving the problem.
This toxic culture is a big part why we are in this mess. Who in their right mind would want to try work on OpenSSL to improve it while being called an idiot? Who'd be willing to stand up for the project as a representative? Speak out for the project?
LibreSSL is only better because it does less and much more human attention is wasted on producing bug free C code. That's an improvement but it's quite limited.
The problem was bigger than that, basically no-one cares about their open source infrastructure and so it (including OpenSSL) is continually under-resourced.
It's yet another in a long line of bugs due to C. It is, of course, possible to have bugs in every language. Even in a theoretical bug-free language it'd be possible to flawlessly implement a buggy algorithm or standard. But some languages make some sorts of bug difficult, and others make them easy: C is in the latter group.
I suspect that assembler for the primitives and a good, sane high-level language (Haskell? Caml? Lisp?) would yield far more secure software than C. There are issues with relocatable memory and zeroing of keys, but those are, I think, addressable at the implementation level.
I don't have an impression that any of the mentioned languages can provide reasonable alternative to the library written in C or the equivalently compatible language, and the only language that it appears to me to have a chance to have such an implementation could be Rust, but somehow even there it appears to be too hard to do so and/or too little commercial interest to make something "good enough" for the most of the potential users. The Rust community doesn't have to implement all the functionality of Open SSL, but if it would produce at least just a proof of concept (e.g. just a minimal set of algorithms) which would prove both the capability to achieve the same speed and the provably higher security, that would be, I think, a huge start.
That would mean not wrapping the whole OpenSSL library in Rust calls but having some as minimal as possible, the tightest of the tightest loops implemented in ASM and everything else in clean and "secure" (not downgraded to the C freedom) Rust which would also pass the benchmarks and be capable to be called "on the high level" from the C code to which that library could be linked (probably the Rust code would not use GC or if it would it would have to be some so simple and minimal one that it would work by eventually calling the C malloc for bigger blocks and cause no different delays than the good written C). Not easy but I think theoretically possible for Rust. Once it would be achieved, implementing other algorithms would be possible to be done by different people and organizations.
I guess in order to achieve these goals the language designers would have to directly decide to support them, but I'd be glad if the status is already different. But I really believe that would be a huge argument for Rust: being provably more secure than C and achieving ability to implement effective drop-in replacements for C security libraries. Is it too high a goal?
I don't know how far we are from something like that, but maybe some Rust insider can estimate? What's still missing and how far we are from the point where what I described would be possible to implement?
It's performance was within 80% or so of regular TLS with Ocaml's safety and easier verification. Quite a few components were purely functional, too. I've seen SML embedded in LISP's and PreScheme compiled to efficient C or ASM. So, that means a static LISP could handle it piggybacking on C compilers. Ivory embeds a subset of C in Haskell to keep things easy to analyze and automatically safe. Parts of a SSL library could use it.
So, Haskell, LISP, and ML are already capable of replacing our SSL libraries if people choose. One is already done in Ocaml. COGENT, if people apply it to this, will take it further by having an easier-to-verify implementation in functional language with certified compilation to C or assembly.
I reposted your link on Lobste.rs. I noted that COGENT already was doing verified functional with certified translation to C or assembly. Also that F star to C seems to be Microsoft and INRIA's route. So, at least two projects are making it much easier to verify algorithms or protocols in functional way with C code resulting.
The other thing I noted is that most code meant for verification is designed to be easy for humans or machines. The F star and C is one of few that is easy for both. I rarely see that.
Thanks, but which of those can be simply taken as the library linked to C code and called from the C code, and so that it doesn't depend on GC? Because as long as that step doesn't exist, it can't be used as the possible substitution for the current libraries?
Can it be used from C, e.g. C that statically links the library? Does it depend on the GC? What is the current speed? Thanks!
Edit: Re "just know they exist" (meaning libraries) in the answer: That's why I started the topic, the people who use only C wouldn't have chance to try the library unless the people who know enough Rust give something explicit that works and can be actually tested in linkability, functionality and speed.
I am not sure about your other questions, as I haven't used them directly, I just know they exist. :) The former is possible, but I don't know if the authors have done any work on that.
> The code was hard to maintain and hard to contribute to, especially for developers in the US due to crypto export issues.
That's not really true: per https://www.bis.doc.gov/index.php/policy-guidance/encryption..., publicly available software (which includes free software) just requires a notification under exception TSU. All you have to do, per https://www.law.cornell.edu/cfr/text/15/740.13, is send an email to crypt@bis.doc.gov and to enc@nsa.gov with the location of the source code. As long as you don't change the location, you never have to worry about it again.
The regulation even notes that simply providing source code on a public website does not constitute export to a prohibited country.
It's really not a big deal.
(Note: I am not a lawyer; this does not constitute legal advice)
The original NaCl had assembly implementations, I believe generated with a Perl script (not unlike OpenSSL's asm generator), and looking around libsodium I cannot see assembly implementations of the loops. Does anyone know why that is? I believe the Linux kernel implementation of ChaCha20 carries asm versions.
>A recently added tool will modify the code to invert the sense of the conditions in if statements to see whether the test suite catches the resulting bug; if it doesn't, there is a coverage gap in the test suite.
Does anyone have any info on this tool? I'm intrigued.
... and just a few days later they closed ~ 150 bugs, some with patches attached, "because they were too old":
https://www.openssl.org/blog/blog/2016/10/12/f2f-rt-github/
https://mta.openssl.org/pipermail/openssl-dev/2016-October/0...