[tptacek]

Unfortunately, the idea behind coding challenges has been cargo culted into a lot of weak hiring cultures:

* The challenges often don't actually matter, and are simply part of the recruiting hazing ritual.

* They're totally subjective and just a proxy for the same bullshit friends-and-resumes process people were using already.

* No consideration is given to the amount of time the entire recruiting process is taking.

* The challenges themselves are totally divorced from the day-to-day expectations of the job, so people are doing CS102 algorithms problems for jobs that will mostly be about moving bits from one place to another.

I understand why people don't like them. But that doesn't really matter, because code challenges are so much more effective at qualifying people that it would be crazy for any competent shop to stop using them.

For a couple years (and, of course, for many years before at Matasano) we've been hiring resume-blind and with almost no interviewing (we do a final on-site, but it's a formality with no tech-out, which is something we tell candidates). We calibrate our challenges so that they'd ideally, for a qualified candidate, take less time than interviews and phone screens would, and have the advantage of being scheduled at the candidate's convenience (all in one night, spread out over a couple weekends, whatever).

We have candidates coming out of our ears (volume has been the single biggest problem with our recruiting practice), and the quality of those candidates is just as good as anyone else's comparable funnel.

This is the right way to hire technical staff.

[cdavid]

Since I have been in a position to do so, I tried to push for coding challenges, and base tech IV on the assignment itself. Your blog post about hiring has been my biggest inspiration on doing hiring. I hired 10+ people in 1.5 years this way, and I would rehire 90+% of them for most roles. I do see difficulties about practicing it though:

1. When you don't control all the HR process, you can't guarantee that the tech assignment is the main filter.

2. Because resume are generally so useless, if you look for a lot of people, you need to be able to review assignments very quickly. Also, to convince people to do the assignment, I make sure to have a phone call before the assignment, which is time consuming since there is no filter.

3. Writing good assignments is hard. You want something that does not take much time (half a day max), is quick to review, interesting enough for the candidate.

I have also not been able to make sure the assignment is the sole filter. I still filter at the (single, < 90 mins) tech IV. But that is at least partially because of my domain (ML engineering), where profiles are more diverse than pure software engineering.

[tptacek]

We did a company based on this approach, and problem (1) is the main reason it failed.

[SavageBeast]

Have an up vote for calling out this practice as what it is: "recruiting hazing ritual".

My personal favorite is when interviewing for a standard full stack web development role I'm asked to more or less re-implement some feature of an existing standard library rather than, for example, build a minimal slice of functionality that requires touching each layer of the stack in question.

My finding on this is that companies in general tend to hire for traits and skills that will never be used in the role they're hiring for. In fact, if I'm a dev lead and a dev on my team reimplements a hand rolled version of java.util.concurrent.LinkedBlockingQueue rather than use the one that Java provides then we're going to have a problem.

I'm sympathetic to the hiring manager here and I do understand that some people cant code a lick despite being able to talk reasonably well about it in an interview. I want to test drive a car before I buy it. I like to try on pants before buying them. Believe me I get it.

Its been my experience that the coding challenge companies want to use is often a self aggrandizing, chest pounding kind of virtue signal about How Smart We Are Here. I suppose that if I had ever been given a reasonable assignment related to the actual job problem domain I might feel differently but this has never been the case.

I also feel that coding challenges are just another manifestation of Analysis Paralysis where the ole "if I just had A LITTLE MORE INFORMATION" kicks in. An experienced interviewer (note I didn't say experienced developer) can smoke out 90% of this stuff in the space of a 2 hour interview.

To be fair to the author here I found his write up to be an excellent effort to solve the ongoing problem that is hiring in general. An honest attempt to improve the situation.

But consider the following:

Hello Candidate, we need a [your problem domain here]. Verbally tell me how to build it while I ask you open ended questions about each step in the process. Once you get through telling me about each major component, responding to my questions along the way, we'll review and discuss the workability/pros/cons/compromises of the solution you describe. You may resolve any issues that arise in this process here.

They either know the answers from the experience of having done it or something like it before or they don't. Call me names if you must but personally I've never understood whats so hard about this. Every Good Job I've ever had has hired me by this method.

[busterarm]

I mean, sure, but also, of course you would say this: You run a recruiting practice.

Everyone has secret sauce that works. Whether it's you or leeny or any of the other well-known industry blogger-veterans offering some fix for this problem. What a great way to cash in on name recognition!

Maybe the reality is that any process that has sufficient buy-in from both sides of the hiring pipeline works. That all that matters is aligned expectations and the means for all parties to deliver on them. Forgive my cynicism but maybe all we have to do is get everyone involved to be honest with themselves and each other.

Everyone is saying something different and has the data to back it up, but maybe it's all just fluff.

[tptacek]

How could it be otherwise? Would my opinion be more credible if I hadn't put my advice into action?

[hyperpape]

Do you think it matters that you're unusually outspoken and/or prominent?

The cargo-culting actually seems like a problem for both employers and applicants, because it poisons the market. You have to know whether the company is treating coding challenges seriously. Because a company that will ghost you or decline you for unclear reasons after giving you a coding challenge has wasted a lot of time.

Or put another way, if it's a hoop to jump through for candidates, coding challenges are awful. If it really means a good chance at that $10k+ raise, coding challenges are a great investment.

[tptacek]

I'm sure it matters a lot that I'm loud. But that's not a superpower; in fact, my friends and many of my coworkers would say it's the opposite. Nothing stops other hiring managers from doing the same thing.

[be1pdub]

" the quality of those candidates is just as good as anyone else's comparable funnel."

This is an irrational statement and empty claim. You can't analyze something you can't see nor have data on.

- Do you have a view into "anyone else's comparable funnel?"

I doubt it. Therefore your claim is dubious and empty.