[icky]

Start with simple and fast screens, then progress to more fine-grained tests.

A simple programming question will filter out a huge chunk of the candidates (i.e., those not smart enough to code or to cheat), then email screen, phone screen, and then an in-person interview.

You should also figure out _exactly_ what qualities are most important in a candidate, and which ones you're willing to overlook.

Generally, I test for: (1) can they think? (2) can they code? and (3) can they learn?

You'd be amazed how much chaff you have to sift through to get even that much grain.

[LukeG]

We'd love to talk to companies that use coding tests in the hiring process. Do they work? How well? Any insight into the design of good filters screens?

[sokoloff]

10 years ago and for a different company (in the computer game industry), we used a two question test (in C): 1. Write hex2int (without using any stdlib functions, such as strtoul(s,null, 16) for example. :) ) 2. Delete a node from a singly-linked list.

Personally, I found a VERY strong correlation between what we wanted in a game developer and their answer to #1. #2 I didn't find all that valuable, other than to weed out people who'd never taken any algorithm/data structure coursework.

For that company, and those roles, it worked very well, because there's a correlation between bit twiddling, pointer operations, and knowing whether someone "gets" pointers and their job performance. That's not the case in most jobs, nor in most modern languages. (Who cares if you "get" pointers if your language doesn't have them?)

My current company asks on from a set of some fairly basic questions on the phone screen, and we pretty much grade the whole phone screen "pass/fail" (interview/thank you for your time). I can't say they work quite as well, but we're also looking significantly "up market" from what the game company needed.

My idealized criteria: For professional hires: keep it short, but not so basic as to be insulting. For college hires: keep it short, and probably basic. For all in-person/whiteboard exercises: ignore error checking, exceptional conditions, etc. It's just too much effort to go through in that format.

Best coding test I ever took was for Accolade, who sent a zip file of source code for a very simple game (fly an airplane across a 2D map with mountains) and asked you to write "AI" (in the game sense, not the compsci sense) for the player. You had an evening and mailed them back a zip file with your code included. That was fun to do, and I suspect gave them fairly fine-grained insight into how good a coder you were, as there's "working", "optimal" and "elegant" dimensions into which various solutions bucket themselves.

[plinkplonk]

Thoughtworks uses a (very simple) coding problem as the first step of the interview process. When I interviewed there, I found the problem too simple and was doubtful as to its value till I saw how it worked from the inside (after joining Thoughtworks).

Vast hordes of candidates with err "augmented" resumes get filtered out, as do "experienced" programmers (sometimes calling themselves "architects" and such) who haven't coded in (literally) years.

I would have thought more people would look up the solution online , but apparently even that effort is too much.

So, overall I would say a coding problem is an excellent idea. I am not sure how valid is the "tell me your program letter by letter over the phone" approach yahoo, google etc seem to take in their phone screens.

[nradov]

We use some simple coding problems during interviews. Subjectively they seem to work, but I have no objective way to quantify how well they work.

My favorite question to ask involves walking through a tree structure and summing up a particular property of all the leaf nodes. It can be done recursively or (preferably) iteratively. About half the candidates can't really do it. I often see answers that only process two levels of the tree, or aren't thread safe, or include lots of duplicate code, or simply don't work at all.

Joel Spolsky has some good suggestions on the subject. http://www.joelonsoftware.com/articles/GuerrillaInterviewing...

[icky]

Yes, they work wonderfully. Start with simple, quick-to-solve and quick-to-check problems, since the vast majority of applicants can't actually code.