[aaronsw]

Why don't you think this technique will scale? It asks three questions: Are they smart? Have they done good stuff in the past? Do you think you can work with them? These seem like really simple questions anyone can answer and in my experience competent people almost always answer them the same way.

[edw519]

Why don't you think this technique will scale?

Because you have provided no mechanism to insure consistent results no matter who is conducting the assessment.

If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.

All 3 of your questions are important. How do you know when you have the answer?

What is the definition of "smart"? "good stuff in the past"? "works well with others"? We all have different definitions.

You may think someone is smart because they can code Algorithm X 5 different ways in 7 minutes. But I may think they're not so smart because they piss off the user trying to determine what the algorithm should do.

You may love something they have done in the past while I hate it. For a thousand different reasons.

Can they code a bubble sort in 2 different languages in 15 minutes? Can they do an iteration with an early exit, based upon an easily determined condition? How much can they speed up a piece of code with multiple obvious opportunities? Whether or not you'd ever use these particular questions, they are much more likely to give questions like yours the consistent yes/no answer you want.

I applaud your desire to avoid the sins of the past. You lay out a sensible approach. What's also needed is the application of some standards and metrics to give a higher probability of consistent results. That's all.

[davidw]

Since I wrote something questioning Aaron above, I'll take the other side with this one:

> If you and I each assessed the same candidate using your methods, we could very easily come up with 2 very different assessments. And that's the trap you'll fall into as soon as you're not conducting every interview.

...

> What's also needed is the application of some standards and metrics to give a higher probability of consistent results.

I don't think that's necessarily true. If you're Google or IBM, it might be, but if you're hiring for a smaller firm, it may very well be best to get someone that is a good fit for that particular culture.

Which actually sort of ties into my other post: I think your point about scaling is important to consider, but it only kicks in at a certain size, and I have a sneaking suspicion that Aaron has not handled hiring for, say, Accenture (nor would he want to).

I don't think hiring is ever going to be a one-size-fits-all, cookie-cutter, "standards and metrics" sort of business as long as you're talking about hiring people.