[timr]

"But if you're doing something where you need to people to excel beyond what they've done before, and perhaps beyond what your company has ever done before, then I think it's naive to think that additional testing for things like on-the-spot thinking, creativity, and diligence under pressure convey no useful signal. This is, in my opinion, an extraordinary claim and requires strong evidence before anyone should take it seriously."

The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

You can't whiteboard-test for excellence. Excellence is both contextual (i.e. it depends a lot on your company, team, culture, etc.), and based mostly on squishy, intangible factors that go beyond "code": picture the brilliant coder who dons his headphones, falls down a hole, and produces a pile of undocumented, complex code of zero business value. It's a cliche, but do we interview for it? No. We ask people to do a graph search on a matrix.

My contention is that we'd do far better with some simple, basic screens for technical competency, and then spending most of our time on communication skills, personality, clarity, organization, planning, business sense and team fit.

...but of course, these are questions with no single correct answer, so engineers are afraid of them.

[_dps]

> The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim.

Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work? If so then I suppose we'll have to agree to disagree.

Edit: I'll add that I personally "interviewed for excellence" as a professor for many years. There was, as far as I could tell, no doubt among professors that interviewing was a non-trivially useful part of evaluating a candidate for graduate school. Again, maybe we were all deluded. But that's a claim that demands some proof.

[timr]

"This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim."

Provide evidence that coding interviews does what you want it to do. Saying that "other people do it, and therefore it must work" is cargo-cult analysis.

I'm trying not to be a jerk here, but I already know the answer: there's no evidence. People do this stuff for exactly the same reason you're biased toward doing it -- because someone else with a big name did it, and nobody goes wrong by doing what Google does!

"Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work?"

I think it tells you something about that person's ability to do the skill you've tested. Sort of. Under extreme pressure.

Does asking people to code on a whiteboard tell you how they're going to work with their peers, communicate clearly and efficiently, document their code, focus on business goals, and generally not be an asshole (all of which are far more important skills for success in a group)? No.

Even as far as coding ability goes, I've many, many "brilliant programmers" who eat leetcode problems for breakfast but can't be trusted to write clean code on their own. It's a borderline useless signal.

[_dps]

I will fully agree with you that many interview practices are bad and "cargo cult". I'm just saying that I disagree on where the burden of proof is. The fact that large fractions of the industry do it and seem to believe that it has some value is , in my mind, what establishes this belief as the norm — in distinction to the belief that these practices have zero value, which is (again in my opinion) an extraordinary claim.

I'm saying "lots of people do it, I can give theoretical arguments for why it might be useful, and it seems common across industries, and this is the only industry I've seen that has a meme about it being totally useless ... so I'd like to see some evidence that it's useless". You're saying "prove that it's useful". We disagree on which direction bears the burden of proof.

Edit: to provide something a bit more explicit, the kind of thing that I would consider persuasive here (again just my opinion) would be several companies that have succeeded like Google, Amazon, Facebook, Microsoft but have employed hiring practices based just on FizzBuzz and communication skill evaluation.