[zgotsch]

Is there a good way to find people who are qualified to do such a review? This paper was written by a Ph.D. student and professor of Computer Science at a respected university. The professor teaches a crypto course on Udacity (https://www.udacity.com/course/applied-cryptography--cs387). If they don't meet the criteria for being cryptographers, I wonder how many people in the world do?

[tptacek]

Not many. There's a Venn diagram to draw about academic cryptographers and practicing cryptographic engineers and people qualified to do cryptographic reviews and the overlap is less than you might think.

[dsacco]

I'm going to push back on this a bit.

Thomas responded alongside this comment to talk about how academic cryptographers are not necessarily qualified to implement original crypto, and I largely agree with that; however, I don't actually think that's the issue here. Rather I would pin this on a lack of peer review.

I could be wrong, but I don't believe the author of this paper has had it published or at least accepted in any journal or conference proceedings. Being an eprint format with endorsement rather than peer review, you can expect mistakes like this to happen often, even if the authors are ostensibly qualified. When you submit original research for publication you generally go back and forth a bit with adjustments as needed, and as long as there is nothing egregious you don't need to redo it all.

In this specific case, I believe the author fully understands the issue (or would, were it presented to them) and is fully capable of fixing it. A qualified peer review would (hopefully :) have caught this and other latent issues if an HN commenter did.

We see this in the broader mathematics and computer science communities, and we especially see it in sub-disciplines like machine learning as well. It's absolutely true that academic cryptographers should not be assumed capable of rolling their own crypto a priori, but in my (educated) opinion I would certainly place far more weight on crypto developed by an academic cryptographer than a software engineer without any particular training.

My platonic ideal for someone who is capable of developing original crypto is something like an academic with a PhD in math or computer science (focusing on crypto), who can develop software very well and who joins an applied lab for crypto engineering and development (like NCC's) or a top cryptanalysis firm like Riscure. Failing that, I'd probably place the most weight on someone who had a lot of training in crypto engineering or practical cryptanalysis over an academic with no implementation experience.

(I apologize if any of this is patronizing, I don't know what your background or familiarity with the academic process is w/r/t peer review, etc).

[forgotpwtomain]

> A qualified peer review would (hopefully :) have caught this and other latent issues if an HN commenter did.

Would the qualified peer-review necessarily be reading the NodeJS code, or just checking the theoretical soundness of the paper? I'm not so certain about the former...

[pbsd]

Almost certainly not the former, no. At best the code would be "supplementary material", which reviewers are not required to go over.

[tptacek]

You absolutely need both skills.