Hacker News new | ask | show | jobs
by ethereal 4719 days ago
I'd just like to point out that `cryptography' is _not_ just about testing that `decrypt(encrypt(msg)) == msg`. memfrob'ing a region of memory does this, but would we consider that a 'secure' encryption scheme? I don't think so . . .

Even if it was -- how do you unit-test a PRNG to ensure the output is uniformly random? How do you ensure you don't have collisions in a hash function? How can you ensure that there's no replay attack on your protocol? How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?

These are all very real and valid cryptography concerns. Sorry if this sounds snarky, but in all seriousness -- I'm very curious as to how you would unit-test these.

2 comments

how do you unit-test a PRNG to ensure the output is uniformly random?

NIST standard test suite @ http://csrc.nist.gov/groups/ST/toolkit/rng/documentation_sof...

How do you ensure you don't have collisions in a hash function?

All hash functions have collisions, by definition, since they represent a unique checksum of a large amount of data in a smaller keyspace. What you are really asking is probably "how do you prove that you have difficult to predict collisions"? I don't know. (I assume that the mechanisms for doing that are based upon statistical analysis of the propagation breadth and speed of various single bit changes in the input stream over varying numbers of rounds?)

How can you ensure that there's no replay attack on your protocol?

A combination of session keys and message sequencing? (eg. Each message must include a hash of the last.)

How can you use unit-testing to demonstrate that there is no hole in your protocol like, say, when the Germans repeated the key twice at the beginning of the message while using Enigma?

There's an entire field of computer science around formal proofs; I don't know enough about it to really comment, however a partial answer may lay in there. Essentially that means adopting a design-time proof strategy in your testing rather than a post-facto unit-testing proof strategy, though.

Good points, but there is a domain of these claimed problems which are covered by automated testing, that's all I'm saying.

And no, you can't find security holes with tests (unless they are random-input style tests, I guess), but you can at least cover the hole afterwards with a test.