[timr]

You're making an assumption that the tech screening process actually screens people on a criterion that matters. But nobody measures the false-negative rate, so they have no idea.

I've now worked at places that did CTCI interviews and "traditional" interviews, and I've noticed no difference in overall quality amongst the employees. I've known plenty of idiots who work at big, famous tech companies, and plenty of amazing people who never ran the whiteboard gauntlet at GooAmaFaceSoft.

My opinion has evolved: tech interviews are the result of generations of cargo-culting amongst a group of people who copied Microsoft, and never really questioned their assumptions. They're just as random and noisy as any other kind of interview, but far more arrogant. Spolsky was right that you should do a FizzBuzz test, but that's it. That's all you need. Everything else should be about communication, personality and the other intangibles that matter far more for every job that involves working with other people (which is all of them).

[_dps]

I half-agree, but Spolsky also makes an excellent argument in "Hitting the High Notes" [0] that there is disproportionate value delivered from excellence, as opposed to mere competence ("Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.").

FizzBuzz + short work sample + strong communication skills is probably not that far from optimal if you want to hire lots of people who are competent and work on problems that they have solved before. Note that I said "optimal" and not "good"; this is still a noisy process.

But if you're doing something where you need to people to excel beyond what they've done before, and perhaps beyond what your company has ever done before, then I think it's naive to think that additional testing for things like on-the-spot thinking, creativity, and diligence under pressure convey no useful signal. This, in my opinion, is an extraordinary claim and requires strong evidence before anyone should take it seriously.

[0] https://www.joelonsoftware.com/2005/07/25/hitting-the-high-n...

[timr]

"But if you're doing something where you need to people to excel beyond what they've done before, and perhaps beyond what your company has ever done before, then I think it's naive to think that additional testing for things like on-the-spot thinking, creativity, and diligence under pressure convey no useful signal. This is, in my opinion, an extraordinary claim and requires strong evidence before anyone should take it seriously."

The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

You can't whiteboard-test for excellence. Excellence is both contextual (i.e. it depends a lot on your company, team, culture, etc.), and based mostly on squishy, intangible factors that go beyond "code": picture the brilliant coder who dons his headphones, falls down a hole, and produces a pile of undocumented, complex code of zero business value. It's a cliche, but do we interview for it? No. We ask people to do a graph search on a matrix.

My contention is that we'd do far better with some simple, basic screens for technical competency, and then spending most of our time on communication skills, personality, clarity, organization, planning, business sense and team fit.

...but of course, these are questions with no single correct answer, so engineers are afraid of them.

[_dps]

> The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim.

Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work? If so then I suppose we'll have to agree to disagree.

Edit: I'll add that I personally "interviewed for excellence" as a professor for many years. There was, as far as I could tell, no doubt among professors that interviewing was a non-trivially useful part of evaluating a candidate for graduate school. Again, maybe we were all deluded. But that's a claim that demands some proof.

[timr]

"This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim."

Provide evidence that coding interviews does what you want it to do. Saying that "other people do it, and therefore it must work" is cargo-cult analysis.

I'm trying not to be a jerk here, but I already know the answer: there's no evidence. People do this stuff for exactly the same reason you're biased toward doing it -- because someone else with a big name did it, and nobody goes wrong by doing what Google does!

"Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work?"

I think it tells you something about that person's ability to do the skill you've tested. Sort of. Under extreme pressure.

Does asking people to code on a whiteboard tell you how they're going to work with their peers, communicate clearly and efficiently, document their code, focus on business goals, and generally not be an asshole (all of which are far more important skills for success in a group)? No.

Even as far as coding ability goes, I've many, many "brilliant programmers" who eat leetcode problems for breakfast but can't be trusted to write clean code on their own. It's a borderline useless signal.

[_dps]

I will fully agree with you that many interview practices are bad and "cargo cult". I'm just saying that I disagree on where the burden of proof is. The fact that large fractions of the industry do it and seem to believe that it has some value is , in my mind, what establishes this belief as the norm — in distinction to the belief that these practices have zero value, which is (again in my opinion) an extraordinary claim.

I'm saying "lots of people do it, I can give theoretical arguments for why it might be useful, and it seems common across industries, and this is the only industry I've seen that has a meme about it being totally useless ... so I'd like to see some evidence that it's useless". You're saying "prove that it's useful". We disagree on which direction bears the burden of proof.

Edit: to provide something a bit more explicit, the kind of thing that I would consider persuasive here (again just my opinion) would be several companies that have succeeded like Google, Amazon, Facebook, Microsoft but have employed hiring practices based just on FizzBuzz and communication skill evaluation.

[archagon]

“Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.”

Can I just point out how absurd this argument is? Even putting aside the issue of Salieri’s talent (which he had in abundance), the fact is that most of the Requiem wasn’t even written by Mozart, except for scattered bits here and there. And as far as the movement that everyone knows is concerned — “Lacrimosa” — Mozart only penned the first eight bars!

So yes, even people with a fraction of Mozart’s talent have and will continue to create things that are just as good as his own work. (Heck, maybe even better!) The rest is inspiration and marketing.

[_dps]

I am too ignorant of musical history to evaluate the analogy, but taking the myth-story as given in popular understanding (presumably based on "Amadeus") I think the content of the argument is clear, even if he picked terrible examples to highlight his case.

Having said that, I appreciate you pointing out these facts about Salieri and Mozart. Thank you for educating me a bit today!

[archagon]

I just think it’s especially ironic that the Mozart piece everyone knows and loves is in fact the one that’s a bit crowd-sourced!