[cpprototypes]

The primary issue with the current technical interview style is this:

  No one likes working on a problem with someone watching them constantly.

Most companies are underestimating the power of social presence. Just knowing that someone is watching you makes a huge difference. Imagine there are two people taking a high pressure test like the SAT. In one version, it's the normal setup, a room with a desk and silence. In another, there is someone constantly watching your paper and everything you do. Put yourself in that position and it's obvious that the second situation is much more stressful.

The effect is not limited to physical presence, it can also happen over a phone interview with screensharing. Consider all these thoughts that a candidate is probably thinking while trying to solve the problem:

What's this person thinking? Am I working too slowly? Should I type code faster? Is this solution what he/she is looking for? Should I talk more? I'm not sure what to say, ok stop panicking, just focus on the problem. Should I say I'm focusing on the problem?

The problem is not the whiteboard. The problem is that there is someone constantly looking at the whiteboard.

The solution is simple and I'm surprised that I've never seen a company try it.

Give the candidate a big whiteboard or a laptop with no network connection. Give a problem and leave the room. Come back later. When depends on the size and scale of the problem. Put the code on a projector and get other engineers in the room. Now discuss the code. Run the code through some sample data. Make some criticisms and see how the candidate defends their decisions. It's like a long code review session. Digressions are ok. Maybe talking some data persistence will lead to a discussion about caching. Maybe asking why the candidate did something will lead to talking about concurrency.

A bad candidate can't BS through this. They wrote the code, they have to either explain or defend it. And the way they talk about things also gives a chance to show their level of experience. The way a junior dev talks and a senior dev talks is very different.

The key thing is, let the candidate have some time alone to work. Stop pressuring them with having an engineer there constantly. The engineer doesn't like it because they would rather be working on something. The candidate doesn't like it because it's stressful. And there's no benefit to having someone there constantly. You don't need to hear their thought process in real time. You don't need to see the iterations they went through before the final solution. If you really want to know, you can find out all these things in the discussion session afterwards.

[arghnoname]

I worked somewhere where we did this, though there they even could search the Internet. It has the benefits you describe, but the one downside is that if someone gets onto a wrong track (solving the wrong problem, getting hung up on some immaterial detail, etc) then no one is there to help them along (most candidates, despite urging, seem not to ask), so you have to be more careful with the 'test' materials or you won't get a lot of signal variation for differing skill levels. Before one overhaul we had a very bimodal (got it easily or didn't get it at all) distribution.

It's a bit of an art to come up with tasks that let you accurately rank candidates.

[louthy]

What I do when interviewing candidates is do a standard face-to-face 'chat' to get their background and a sense of the person, and then give them a half-finished program (with some bugs). I say: "Go home, download the code, and finish it in your own time". They have all of the tools that they're comfortable with, internet access, etc. I also say, if you're not sure on anything to drop me an email and ask. Because in real life if I'm going to be their boss they'll have that opportunity too. I see that as a positive, not a negative.

It shouldn't take more than 30-45 mins for a competent programmer. There's also some instructions in the mini-design which are intentionally 'the wrong way' to do the task. This is to encourage the candidate to get in touch.

I think this shows their coding style, problem solving, but also how they can work with others. Junior devs tend to get in touch about the complexity of the problem, senior devs get in touch about the methods in the design.

This process hasn't yet let me down.

[OWaz]

The best technical question I've had so far was after the phone screen the hiring manager emailed me a question that I needed to solve in 2-3 days. During my investigation on how to solve the problem I discovered I would need to implement a K-D tree. I don't have a CS background so I just found some open source implementations of a K-D tree.

I sent the manager an email telling him that I couldn't write a K-D tree on my own, but I would use the implementations that I found online to solve the problem. He replied that what he was looking for was someone who would've relied on an existing solution to solve the problem instead of trying to code it all from scratch. So I passed his test and was called in for an in person interview which was just trying to see if I would fit in with the team. It all worked great and I was hired. I didn't have to do anything with a whiteboard.

[ltorresv]

To me this is a really good way to evaluate a candidate.

At the end of the day I care about whether or not you can solve a problem.

I really don't care if you solved it because you can use google or because you have a network of people to ask questions.

Technology changes too fast to expect people to always know it all. I'd rather have people that are resourceful and can dig their way out of a hole.