[j2kun]

Of course. But you still hire more skilled people over less skilled people, and if free time contributes to skill it's not necessarily unfair.

In fact, I think a quality CTF could level the playing field, since each candidate could be judged on their relative abilities (how far they can progress through a game that adapts to their prior knowledge). Moreover, one could offset the time commitment by paying candidates to do the CTF. Certainly paying people a decent wage for 6 hours is worth the benefit of hiring a better long-term candidate. There are all kinds of ways to fix the problem.

[tptacek]

That's definitely the ambition here, and it squares with our experience with the Crypto Challenges and Microcorruption: the "top tier" of people in both those events were, as a general rule, not very experienced in either of those problem domains.

[tom_b]

Am I wrong to be completely stunned at this statement? Did experienced candidates just not participate in the Crypto Challenges and Microcorruption?

I would have expected at least some experienced candidates to drop into that "top tier" of people. What's up with this?

[pbsd]

I finished 4th in Microcorruption. While exploiting vulnerabilities is not my day job, I have been playing CTFs for a long time, and am familiar with the process. It is certainly something one can overspecialize in.

Looking at the first page of the Hall of Fame, I see big names like Alex Sotirov, Russ Cox, Ricky Zhou, Ludvig Strigeus, and many other familiar names/handles usually seen at CTF events. There are also many unknowns, which I suppose was Thomas's point.

[tptacek]

There were "elite" participants, but they were numerically dominated by strong participants without the background. Which, if you're a hiring manager, is a very interesting and exploitable bit of data. Our odds of hiring Russ Cox or Alex Sotirov were not good. Our odds of being the first serious job for the next Alex Sotirov were better.