An obvious issue is that many protocols do two-sided negotiations of crypto algorithms during a handshake - so one of "most applications" which uses SHA-2 hashes by default could be vulnerable to a network connection saying "I don't support SHA-2, let's use SHA-3 pls?" and then exploiting the buffer overflow.
In a similar manner, apps that verify signatures/certificates often let the certificate specify which algorithm is used, so again effectively an attacker may be able to force the app to use SHA3 as long as it uses a library that has some support for it.
This is a common problem, and massively increases attack surface, since if a vulnerability can be found in any algorithm, it can be exploited. Downgrade attacks are a varient of the same.
Let me present an alternative:
Each protocol has one specific encryption algorithm and set of parameters. However, that set isn't fixed, but defined based on the current date and time.
For example, HTTPS could be defined as "SHA1 from 1996 till 2006", "SHA256 from 2005 till 2018", "SHA3 from 2017 till 2032", etc.
The dates can be written 10+ years into the future, so that software deployed today is still usable that many years into the future. As soon as the last date period expires, the software is not usable until it has been upgraded (unless both ends of the communication link choose to fake the date).
There is precedent for software 'expiring' - that happens whenever the root certificate of a trust store reaches an expiry date.
The downside is sometimes an algorithm might be broken before its date comes, and other times an algorithm may still be secure at that date. But that still seems better than our current mess of downgrade attacks.
As you say, it could be that an algorithm gets broken before its "expiry date" or remains secure well past that (look e.g. at AES). There is a similar but better alternative, usually called "opnionated cryptography". A protocol specifies a single cryptographic primitive for each type needed (hash function, block cipher etc) so no negotiation is needed. If one of the primitives gets broken, a new version of the protocol is released that uses a different primitive.
It seems to me that it may be better in many cases to support exactly two configurations (or some other small fixed number). This way there is no need to wait until a new version is released and dropping the vulnerable one won't cause any downtime. Ideally both codepaths would be regularly exercised (maybe clients should pick randomly when both choices are available). Then as soon as a vulnerability is discovered (maybe even before it is publicly revealed) all clients and servers could immediately drop support for the one that was discovered vulnerable.
Then the release of a new configuration can be done slowly with reasonable caution to get back to the desired redundancy level.
And then you still have a roll-out period during which those two protocols are available, so the two parties still need to negotiate which version they want. It's not clear to me what the advantage over the current situation is?
What if the server doesn't support version X? Then the client will try again with version X-1. This is a negotiation, it is just an inefficient one (client might have to try X-1, X-2, X-3 in turn if more versions are still co-existing; and contrarily, if client doesn't support any version the server does, you will not get a detailed error about the version mismatch, because they are entirely different protocols).
The point of NIST standardizing on SHA-3 is to gradually replace SHA-2 due to the rise of computing power and the likelihood it will become as weak as SHA-1 is now in the near future. Unfortunately, like American credit cards vs. European chip & pin, it's going to take forever to adopt.
No. The "rise in computing power" doesn't jeopardize SHA2. There are important design differences between SHA1 and SHA2 (here's where in my younger days I'd pretend that I could rattle off the implications of nonlinear message expansion off the top of my head). SHA2 is secure; don't take my word for it through, you can find one of the Blake2 designers saying SHA2 is unlikely ever to be broken, or Marc Stevens on a Twitter thread talking briefly about why his attacks on SHA1 don't apply at all to SHA2.
I agree, SHA-2 is secure as far as we know. But since it's based on Merkle-Damgard, it permits length-extension attacks - i.e. given H(x), one can derive H(pad(x) || y) without knowing x.
So we need to be careful not to use it in setting where that would be problematic. Or we can use it with workarounds, like double hashing (SHA-256d) or truncating its output.
SHA-3 is sponge based, so its output is always truncated, preventing length-extension attacks. So I think SHA-3 is a better default, though it's fine to use SHA-2 if you know what you're doing.
Truncated SHA512 hashes, such as SHA512/256, defeat length extension attacks by omitting part of the hash state from the output. They're also significantly faster than classic SHA256 for large inputs.
SMH. You're conflating "broken" by mathematical attack and having enough computing power to brute it (GPUs or quantum). Rise in computing power always jeopardizes the baseline brute cost of every algorithm, which is why standards shift over time, otherwise 3DES would still be recommended for new applications instead of AES.
In a similar manner, apps that verify signatures/certificates often let the certificate specify which algorithm is used, so again effectively an attacker may be able to force the app to use SHA3 as long as it uses a library that has some support for it.