[karterk]

Practically speaking, someone looking for a job change does not have 10 hours to spend on every company's coding challenge. So, you will have only a few people who would go out-of-the-way to do something like that.

And, that's not very different from someone spending more than the expected 40 hours at work. One can't do much about that if that person genuinely loves coding all day and does not mind working 60 hours a week.

Finally, in the follow-up call, I do ask the candidate how long the assignment took and how she spent her time across multiple tasks. Obviously if someone spent 10x the expected time, then that's another data-point to the final decision.

[pmiller2]

Suppose a candidate spends 9 hours on your 3 hour challenge. What's the incentive for them to tell you the truth? I think the only way around that is to not tell them what the expected time commitment is, but that still doesn't completely remove the incentive to spend much longer than intended and to understate the amount of time spent.

As to the time commitment, I would say finding the time to do these coding challenges is no harder than finding the time to come in and interview. And not every company even does a take-home coding challenge. The last 2 companies I've worked at did not.

My point is not that there's no signal there. My point is that the incentives encourage candidates to spend arbitrarily large amounts of time on these challenges and to not mention the fact, which distorts the signal.