[j2kun]

I think I see what you mean: if this replaces interviews entirely then some people just won't be considered for the job because they have children. To me that sounds like a policy issue (it's illegal to discriminate based on irrelevant features to the job). I.e., not something that the designer of the CTF should be responsible for, but something a company could be very liable for if they aren't careful in how the apply the CTF.

[achompas]

Parent's point is that companies implicitly filter for this by asking candidates to participate in time-consuming multi-phase interview processes. It's already difficult to work contract-for-hire or take time off for several days of on-site interviews, and it gets much more complicated when you're not well-off or you also have to take care of your kids.

[j2kun]

Of course. But you still hire more skilled people over less skilled people, and if free time contributes to skill it's not necessarily unfair.

In fact, I think a quality CTF could level the playing field, since each candidate could be judged on their relative abilities (how far they can progress through a game that adapts to their prior knowledge). Moreover, one could offset the time commitment by paying candidates to do the CTF. Certainly paying people a decent wage for 6 hours is worth the benefit of hiring a better long-term candidate. There are all kinds of ways to fix the problem.

[tptacek]

That's definitely the ambition here, and it squares with our experience with the Crypto Challenges and Microcorruption: the "top tier" of people in both those events were, as a general rule, not very experienced in either of those problem domains.

[tom_b]

Am I wrong to be completely stunned at this statement? Did experienced candidates just not participate in the Crypto Challenges and Microcorruption?

I would have expected at least some experienced candidates to drop into that "top tier" of people. What's up with this?

[pbsd]

I finished 4th in Microcorruption. While exploiting vulnerabilities is not my day job, I have been playing CTFs for a long time, and am familiar with the process. It is certainly something one can overspecialize in.

Looking at the first page of the Hall of Fame, I see big names like Alex Sotirov, Russ Cox, Ricky Zhou, Ludvig Strigeus, and many other familiar names/handles usually seen at CTF events. There are also many unknowns, which I suppose was Thomas's point.

[tptacek]

There were "elite" participants, but they were numerically dominated by strong participants without the background. Which, if you're a hiring manager, is a very interesting and exploitable bit of data. Our odds of hiring Russ Cox or Alex Sotirov were not good. Our odds of being the first serious job for the next Alex Sotirov were better.