[rictic]

MD5 is also supposed to require a tremendous amount of computation, it's just that there are weaknesses in the algorithm that were eventually discovered, such that an attacker can short circuit much of that work. The reason that SHA-1 is being sunset is that some preliminary results by researchers suggest that it also has weaknesses that can enable it to be short circuited, and that researchers appear to be on a path to discovering those weaknesses.

[dperfect]

Hmmm... so the takeaway for me is that when designing critical systems that rely on hashes/fingerprints as identifiers, we should probably treat those as transient identifiers with a reasonable expectation of migrating them as newer algorithms replace older broken ones. Does that sound right?

[Diederich]

That's correct.

[dperfect]

I feel like this gets missed in the debate over the "best" hash algorithms. It seems like the message is always "use XYZ algorithm for everything - it's practically perfect and the only one you'll ever need". In reality, it should be more along the lines of "migrate to this one for now, but don't build things that depend on it being the best option next year since it's probably not perfect. Design your systems to easily accommodate changes to the algorithm."

[xorcist]

That is the common wisdom. However some people, most known perhaps djb, actually argue against it. Pluggable algorithms may allow for downgrade attacks, and in a world wide complex system there will always be holdouts for some reason.

They argue you would be better with hardcoding your system to known secure best practice. When the time comes to change it, you specify a new protocol version, as there will be new and better practices not only in algorithms but also in how they are used (mac-before-encryption being the canonical example of this which took far too long to change).

[VLM]

"Pluggable algorithms may allow for downgrade attacks"

This is a uniquely severe problem for hashing. MD5 is by far the fastest of the common hashes, at least 20% faster than SHA1 for example.

The Venn diagram is you've got popular fast hashes, of which md5 is arguably the best, and in fast hashes, you've got cryptographically secure fast hashes, of which md5 most certainly is no longer a member. Its awesome for everything non-crypto non-secure non-over the internet.

Last weekend I had to burn a legacy DVD and I compared the md5sum of the image to the md5sum of the burned DVD, thankfully they matched. As much as I distrust legacy media, there probably isn't a sentient opponent in the burner, although I've occasionally sworn otherwise, and the most likely failure mode would have been simple truncation or buffer dropouts, so sheer speed was the priority. The hashes obviously matched, it was a good burn.

I've also used them as checksums for engineering data files. Everything on the planet from AS400s to MySQL databases can calculate md5, which is convenient for cross platform interchange type stuff. Heres a data blob and its md5, does your hash calculation match? Oh it does, how nice to know you have perfect data integrity, here have the next one. On the internet I wouldn't trust the opfor with md5, but I can trust my own coworkers and my own engineering machinery not to try and attack me (well, probably).

That means until the end of time you'll unfortunately have people discover md5 is available for non-crypto use, then try to use it to hash passwords or DRM executables or something, which is not so wise. Or for a downgrade attack, they misdesign their crypto system to allow any hash installed on the machine as the hash type and not blacklist md5 and sha1 and maybe more.

What will die out is hashes like SHA1. Why use something almost as dangerous as md5 for crypto, thats slower than md5, and not as widespread as md5? Bye bye sha1!

[crylon]

BLAKE2b, an apparently well-regarded derivative of one of the SHA-3 finalists, is faster than even MD5 while retaining security. https://blake2.net/

[leni536]

> but I can trust my own coworkers

I'm sure that you trust them, but do you trust their computers? Do you trust all of them that they can keep their personal computers and their accounts secure?

Maybe it doesn't match your case perfectly, but in general trusting people and trusting people's accounts is not the same thing.

[bluefox]

Correct. http://valerieaurora.org/hash.html

[blutfink]

The "Reactions" table is hilarious – thanks for the link.

[userbinator]

The story of how the MD5 collision pathways were discovered is an interesting one - the first colliding pair is said to have been calculated by Xiaoyun Wang by hand, with little computer use. She has also figured out similar methods for SHA-1 and a few other hashes.

http://en.wikipedia.org/wiki/Wang_Xiaoyun