[karterk]

This has been an area of interest for me as well. I have spoken to a lot of engineers and hiring managers on this subject, and my takeaway is that no two engineers seem to agree on what's the best way to hire people.

Some think that code-while-on-the-phone interviews are bad (too much pressure for some candidates), while others think that take-home interviews bring all kinds of issues (e.g. candidates copy pasting code from Stackoverflow).

Some think that white-boarding is important (insights into the thinking process versus syntactic correctness), while others think that it's not a true evaluation of what the candidate will actually be doing at work.

Some think that asking pure algorithmic questions are bad (does anybody even implement their own hashmaps at work?) while others think that it's a reflection of how much a candidate explores the depth of a subject.

Some swear by HackerRank/Codility based automated tests to "filter" unqualified candidates, while others think that looking at Github for their code samples would be a better evaluation.

Some want generalists, while others want specialists.

Some people reject candidates on arbitrary "culture-fit" reasons.

Personally, here's my ideal interview process based on all these conversations and my own 100+ interviews in various places I have worked:

* Minimum reliance on resume - some people are just bad at writing about their achievements

* Have an initial 30-min call to learn about the projects that the candidate has worked on (especially if you are hiring a specialist). This should not be a filtering round unless you see something totally off target.

* Send the candidate a take-at-home coding assignment. This should ideally take only 2-3 hours and should be directly related to the kind of work she will be doing at work.

* Have another 30-60 min follow-up call to ask the candidate about the approach taken, alternate options, and possible extensions. This will help one ensure that the candidate had not merely copy pasted the code from the Internet.

* If everything looks good, bring the candidate onsite for face-to-face interviews.

* I prefer 2 rounds of face-to-face - the first round asking questions around the candidate's areas of strengths, and projects previously worked on by her. On the second round, I prefer asking questions that are from the domain I am working on. This will allow me to find out how the person attempts to understand a new domain, and whether the person listens/asks questions etc.

At the end of the day, interviews are subjective, and I have made peace with the fact that you're always going to miss out on people who might have just had a bad day.

[pmiller2]

I, too, have gotten interested in this area.

> Send the candidate a take-at-home coding assignment. This should ideally take only 2-3 hours and should be directly related to the kind of work she will be doing at work.

I used to be a fan of this, but I'm not so sure anymore. The reason is that if you give me a test that "should" take 2-3 hours to complete, I know I'm going to be competing with people who are spending large multiples of that amount of time on it.

Unfortunately, I don't have a solution to offer for this problem.

Edit: compete -> complete. Freudian slip?

[karterk]

Practically speaking, someone looking for a job change does not have 10 hours to spend on every company's coding challenge. So, you will have only a few people who would go out-of-the-way to do something like that.

And, that's not very different from someone spending more than the expected 40 hours at work. One can't do much about that if that person genuinely loves coding all day and does not mind working 60 hours a week.

Finally, in the follow-up call, I do ask the candidate how long the assignment took and how she spent her time across multiple tasks. Obviously if someone spent 10x the expected time, then that's another data-point to the final decision.

[pmiller2]

Suppose a candidate spends 9 hours on your 3 hour challenge. What's the incentive for them to tell you the truth? I think the only way around that is to not tell them what the expected time commitment is, but that still doesn't completely remove the incentive to spend much longer than intended and to understate the amount of time spent.

As to the time commitment, I would say finding the time to do these coding challenges is no harder than finding the time to come in and interview. And not every company even does a take-home coding challenge. The last 2 companies I've worked at did not.

My point is not that there's no signal there. My point is that the incentives encourage candidates to spend arbitrarily large amounts of time on these challenges and to not mention the fact, which distorts the signal.