[munin]

semi-serious devils advocacy. I don't know if the below is an argument that is correct but I hope for it to be proven wrong. I am pretty sure that accurately assessing individual capacity/potential is really, really, really hard.

"Google is a very successful technology company. While some of the problems they solve are very challenging and require innovative thinking and research, they have employees with decades of research experience and long publication track records to crack those.

New and seasoned developers will almost certainly have plug-n-chug everyday responsibilities, yet their interview process is still a fearsome battery of computer science questions. Their hiring also produces what is regarded as the most skilled CS workforce in the world and the market rewards them richly for it.

Why shouldn't we emulate this? Isn't it working? Aren't advocates of a disruption of this system essentially saying "Go ahead, relax your standards, what's the worst that could happen"? Is it taking your company 6 months to find a decent employee? Maybe that's about how long it should take because there just aren't that many decent employees."

[randometc]

The question is: what is your tolerance for (a) false negatives and (b) false positives?

Rephrasing (a): who are we passing on, are they in fact good potential employees, and what qualities do they have that our new hires might not?

Rephrasing (b): who have we hired that, despite whiteboard coding proficiency, isn't cutting it? And why?

In my experience whiteboard coding sessions feel adversarial, like a trap or a test, and you're basically waiting for the candidate to have an epiphany moment. Either they know the problem, or they don't but they get lucky and hit the "right" answer.

Doing reviews of code samples (or even pair programming against a new problem) puts interviewer and candidate both on the same side of the problem and feels more collaborative - I want to believe this gets you a better sense of what someone would be like to work with, and fewer false positives and negatives.

If anyone has evidence or studies either way I'd love to read them.

[munin]

I think this is the correct question to ask.

in my experience, false positives in early ventures (research projects, startups, project partnerships, whatever) are the kiss of death. If you need a group of people to do work at high capacity, bringing someone who does not 'fit' in when the group is young will probably destroy it. existing high-capacity people will become angry, there is less fault tolerance in the quality of the work and the rate at which it has to be done, etc.

it is usually the case that it is better to have 2 workers who click and are highly productive than 5 workers where 1 is not productive. so in the early days, failing closed seems very valuable.

when you are larger, there could be a higher tolerance. you could afford to hire someone and fire them a few months later if they do not work out, because the work that they are doing for you immediately is not as critical. however, there is also a certain culture that comes along with this and if you want to avoid that culture it seems like you still want to 'think lean' and part of that thinking could be to keep the amount of productive people high.

and it might also not go well with your current group members to say "yes, that hire was bad and we'll terminate them next week" for the fifth, sixth time in a quarter ...

[damoncali]

I agree this is the right question, but for the opposite argument. False positives in early stage ventures, in my experience, can be quickly and actively remedied. I've seen people fired less than two weeks after being hired without a substantial impact on the rest of the team or the product. If you can do this, you have a lot more freedom to take risks with new hires.

In a larger organization, if you hire someone, you're stuck with them in almost all cases. So be picky.

[danielweber]

Rephrasing (b): who have we hired that, despite whiteboard coding proficiency, isn't cutting it? And why?

Has someone suggested whiteboard proficiency is sufficient by itself?

The whiteboard is meant to filter out people who suck. "Add a cache to a JavaScript function" is a dumb question for reasons many others have pointed out. Joel's classic article[1] recommends these as examples:

• Write a function that determines if a string starts with an upper-case letter A-Z

• Write a function that determines the area of a circle given the radius

• Add up all the values in an array

Those are all trivial. You should be able to come up with another half-dozen if you sit and think for a bit. The language doesn't matter, and the syntax shouldn't matter (as compared to the semantics).

I hear lots of scare-stories of "great coders who choke on interviews," but I find it very hard to believe any kind of "great coder" can't add up the values in an array. I might ask "how do you find the median value in the array" just to see what they come up with. There isn't a right answer, but there are lots wrong answers.

[1] http://www.joelonsoftware.com/articles/GuerrillaInterviewing...

[randometc]

Has someone suggested whiteboard proficiency is sufficient by itself?

Fair point, that is a straw man. I haven't experienced an all-whiteboard interview - there's always been more to it.

The whiteboard is meant to filter out people who suck.

I think that's what some interviewers have missed. They jump straight in at the deep end with difficult algorithmic puzzles, and don't ever do an easy one. I have experienced this.

[danielweber]

They jump straight in at the deep end with difficult algorithmic puzzles, and don't ever do an easy one. I have experienced this

Yeah, I have hard algorithm questions I ask, but I don't open with them, and I know they are hard. I expect them to possibly fill the entire remaining time.

Also, for the hard question, it's not even about getting it "right": the one person I ever had get my hardest question right was flushed by other people in my process, and I've hired many people who never got close. I want to see how they try.

[groby_b]

It is adversarial, because it's a gating mechanism.

Sure, I'll be happy to discuss deep issues with you - after you've shown me that you're proficient in basic CS. It used to be that you could screen that over the phone, but now that you can look up the answers on your phone, that's kind of hard :)

[randometc]

Assuming your whiteboard session is a good test of basic CS proficiency, the real question is "does basic CS proficiency correlate with effectiveness at the job?" - if you're like Google the answer is probably yes, and if so you only have to have tolerance for false negatives. Given enough applicants that's clearly less of a problem than false positives.

My other point is that some companies might find there are other qualities they're looking for that occur in the pool of people they pass on. That's what's really worth looking at, I think - all the while being careful not to mistake "good cultural fit" for "promotes long term monoculture".

[bduerst]

But the question he is raising with the false positives is that it is a gating mechanism for what? The ability to talk shop on a white board?

[groby_b]

I'm assuming in my answer that the interview happens at a shop that cares about CS proficiency. (Mine certainly does ;)

If I have a false positive (i.e. you seem to understand CS but don't fit the company), it's not like the CS question is all that's asked. It's the launch point for a deeper discussion. That means a false positive hopefully gets caught in that later stage, but deep CS knowledge is sine qua non.

If that skill (knowledge of CS) doesn't matter to your shop, then yes, you shouldn't use it for gating.

[bduerst]

Anecdotal, but I interviewed for a position where they gave me a tab deliminated data set, and said, "Show us what you can do with this in X language". I had two days.

It was one of my favorite interviews.

[MisterBastahrd]

Google hires the best because they have the capacity to pay the best. If you're hiring a mid-level developer in Texas and can only pay 70K a year, don't be surprised if your candidates have a hard time working out esoteric brain teasers.

[j_baker]

I think this argument is simple, beautiful, and wrong. The assumption this seems to be built upon is that Google has developed a great hiring system that is doing well for us. I think that's a bit simplistic.

The reality is that (like everything at Google), the hiring process is constantly evolving and changing. Google puts considerable effort into improving its hiring process.

So, my counterargument is this: If Google is tweaking its hiring process and trying to make it better, why shouldn't you? Don't think of it as relaxing your standards, because it isn't. Think of it as changing your standards to better suit the kind of people you want to hire.