[kfcm]

Here's food for thought.

Does this happen with experienced positions open in other departments?

Are experienced CPAs interviewing for the new position in Accounting required to balance an example set of books?

Are experienced Sales interviewees expected to go out and sell a sample product before being hired?

A lawyer for Legal expected to write a sample brief or appear before the court in a sample trial?

HR expected to do sample HR things?

Think about it.

[kemitche]

I won't say I agree with most interview processes for software, but here's more food for thought:

CPAs presumably have been externally certified, so perhaps you only need to check "team fit" things.

Sales interviews: if you can't sell me on yourself, why would I hire you? The interview is enough.

Lawyers I would assume have some number of public items for me to look at.

Software engineers: none of that applies. If they've contributed to open source, maybe you can look at that, but many of the great engineers I've worked with have not.

[timr]

You're making an assumption that the tech screening process actually screens people on a criterion that matters. But nobody measures the false-negative rate, so they have no idea.

I've now worked at places that did CTCI interviews and "traditional" interviews, and I've noticed no difference in overall quality amongst the employees. I've known plenty of idiots who work at big, famous tech companies, and plenty of amazing people who never ran the whiteboard gauntlet at GooAmaFaceSoft.

My opinion has evolved: tech interviews are the result of generations of cargo-culting amongst a group of people who copied Microsoft, and never really questioned their assumptions. They're just as random and noisy as any other kind of interview, but far more arrogant. Spolsky was right that you should do a FizzBuzz test, but that's it. That's all you need. Everything else should be about communication, personality and the other intangibles that matter far more for every job that involves working with other people (which is all of them).

[_dps]

I half-agree, but Spolsky also makes an excellent argument in "Hitting the High Notes" [0] that there is disproportionate value delivered from excellence, as opposed to mere competence ("Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.").

FizzBuzz + short work sample + strong communication skills is probably not that far from optimal if you want to hire lots of people who are competent and work on problems that they have solved before. Note that I said "optimal" and not "good"; this is still a noisy process.

But if you're doing something where you need to people to excel beyond what they've done before, and perhaps beyond what your company has ever done before, then I think it's naive to think that additional testing for things like on-the-spot thinking, creativity, and diligence under pressure convey no useful signal. This, in my opinion, is an extraordinary claim and requires strong evidence before anyone should take it seriously.

[0] https://www.joelonsoftware.com/2005/07/25/hitting-the-high-n...

[timr]

"But if you're doing something where you need to people to excel beyond what they've done before, and perhaps beyond what your company has ever done before, then I think it's naive to think that additional testing for things like on-the-spot thinking, creativity, and diligence under pressure convey no useful signal. This is, in my opinion, an extraordinary claim and requires strong evidence before anyone should take it seriously."

The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

You can't whiteboard-test for excellence. Excellence is both contextual (i.e. it depends a lot on your company, team, culture, etc.), and based mostly on squishy, intangible factors that go beyond "code": picture the brilliant coder who dons his headphones, falls down a hole, and produces a pile of undocumented, complex code of zero business value. It's a cliche, but do we interview for it? No. We ask people to do a graph search on a matrix.

My contention is that we'd do far better with some simple, basic screens for technical competency, and then spending most of our time on communication skills, personality, clarity, organization, planning, business sense and team fit.

...but of course, these are questions with no single correct answer, so engineers are afraid of them.

[_dps]

> The extraordinary claim is that whiteboard testing (or take-home projects, or...well, anything in the current tech interview) does any of those things.

This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim.

Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work? If so then I suppose we'll have to agree to disagree.

Edit: I'll add that I personally "interviewed for excellence" as a professor for many years. There was, as far as I could tell, no doubt among professors that interviewing was a non-trivially useful part of evaluating a candidate for graduate school. Again, maybe we were all deluded. But that's a claim that demands some proof.

[timr]

"This is just a naked assertion. This kind of testing happens in many places and industries, and many people seem to believe it's useful. Maybe they're all deluded, but that's precisely why I'm calling it an extraordinary claim."

Provide evidence that coding interviews does what you want it to do. Saying that "other people do it, and therefore it must work" is cargo-cult analysis.

I'm trying not to be a jerk here, but I already know the answer: there's no evidence. People do this stuff for exactly the same reason you're biased toward doing it -- because someone else with a big name did it, and nobody goes wrong by doing what Google does!

"Do you really think it is obvious, prima facie, that asking people to demonstrate some skills on their feet, or to do a short sample of work for you, tells you nothing about their ability to do good work?"

I think it tells you something about that person's ability to do the skill you've tested. Sort of. Under extreme pressure.

Does asking people to code on a whiteboard tell you how they're going to work with their peers, communicate clearly and efficiently, document their code, focus on business goals, and generally not be an asshole (all of which are far more important skills for success in a group)? No.

Even as far as coding ability goes, I've many, many "brilliant programmers" who eat leetcode problems for breakfast but can't be trusted to write clean code on their own. It's a borderline useless signal.

[archagon]

“Five Antonio Salieris won’t produce Mozart’s Requiem. Ever. Not if they work for 100 years.”

Can I just point out how absurd this argument is? Even putting aside the issue of Salieri’s talent (which he had in abundance), the fact is that most of the Requiem wasn’t even written by Mozart, except for scattered bits here and there. And as far as the movement that everyone knows is concerned — “Lacrimosa” — Mozart only penned the first eight bars!

So yes, even people with a fraction of Mozart’s talent have and will continue to create things that are just as good as his own work. (Heck, maybe even better!) The rest is inspiration and marketing.

[_dps]

I am too ignorant of musical history to evaluate the analogy, but taking the myth-story as given in popular understanding (presumably based on "Amadeus") I think the content of the argument is clear, even if he picked terrible examples to highlight his case.

Having said that, I appreciate you pointing out these facts about Salieri and Mozart. Thank you for educating me a bit today!

[archagon]

I just think it’s especially ironic that the Mozart piece everyone knows and loves is in fact the one that’s a bit crowd-sourced!

[cl0wnshoes]

Not that I don't agree... but for a CPA, lawyer, engineer, and many other professions you have to go through years of education and rigorous testing to acquire the titles. We pull down some of the highest salaries in the US, we have incredible responsibility, and our profession requires zero formal education or testing to prove our abilities. It shouldn't be too far fetched to understand why some companies feel the need to vet their candidates.