[dahart]

I'd love to hear examples of how to do it right, without taking all my time. I'm genuinely interested in how to hire people without wasting their time or mine. I'd love to screen people with a day of pair programming or a 2 week paid internship, or whatever other good ideas are floating around. I can't.

You have just dismissed most tech companies. All the large ones (Google, Apple, Amazon, Microsoft, etc.) give coding puzzles as part of their interview process.

If you don't want to work for one of those companies, that's absolutely your choice, but you'd be wrong to say they have an entitlement complex or aren't the smartest.

> In my case because I have a bunch of open source available that they can just read.

I do read people's OS projects, but I simply can't do that for every candidate before I screen them, I don't have enough time. If the coding tests are easy, and you want me to see your open source project, then just ace the coding test and move on.

If you don't want the job, then don't do the coding test. It is your choice.

[crdoconnor]

In the past I've given programming tasks that mimic real life work as closely as possible and fit them into a reasonable time frame - for example, 1 hour of pairing/programming during an arranged face to face interview. If the test is well designed, touches on a number of areas and self contained, I think this was sufficient for me to assess their skills.

"Mimic real life" means no puzzles (unless I've literally experienced them in real life), no binary tree reversals and no big O notation questions.

>You have just dismissed most tech companies. All the large ones (Google, Apple, Amazon, Microsoft, etc.) give coding puzzles as part of their interview process.

I think that type of thinking leads to embarrassments like this:

https://twitter.com/mxcl/status/608682016205344768

I think he experienced exactly the same problem the OP is talking about and in this case it's not him who was dumb, it was Google.

I wouldn't rule out any of those companies but I think I'd rule out joining through the standard interview process - I'd look for specific people who looked to be doing exciting work on specific teams and try to befriend them.

>you'd be wrong to say they have an entitlement complex or aren't the smartest.

I think it would depend upon the team. I think they're not all geniuses, and they do have a tendency to drink their own kool aid. For sure some teams are great though.

[dahart]

> 1 hour of pairing/programming during an arranged face to face interview. If the test is well designed, touches on a number of areas and self contained, I think this was sufficient for me to assess their skills.

Sounds great. I've done this too, but I've never done it before screening candidates. I don't have enough time to screen candidates by programming with them.

> "Mimic real life" means no puzzles (unless I've literally experienced them in real life), no binary tree reversals and no big O notation questions.

Understood. My goal with my coding quizzes and knowledge questions is not to mimic real life. It's to asses the boundaries of the candidate's education and experience, without regard to skill.

> I think that type of thinking leads to embarrassments like this

There always have been and always will be false negatives, even with lengthy face-to-face interviews, even with pair programming, and even with paid internships.

One high profile false negative anecdote, while unfortunate, doesn't imply there's a widespread or unexpected problem. Applying for jobs always comes with a risk of not getting the job for a wide variety of factors that are outside the candidate's control. If it's a job you really want, all you can do is try hard.

I love homebrew, and I'm sure @mxcl is a fabulous coder, but the tweet you shared does make it sound like he expected to get the job without an interview, and might have come across that way, or might have not prepared at all.

[crdoconnor]

>I've never done it before screening candidates

No me neither. I screened with a 5 minute fizz buzz like task and CV fits. Bad programmers slipped through the net (and were caught by the test) but I never got anybody who literally couldnt code. I was happy with that balance.

I was afraid if I made it longer than 5 mins we'd filter out good candidates who couldn't be bothered with our bullshit.

>One high profile false negative anecdote, while unfortunate, doesn't imply there's a widespread or unexpected problem.

It does imply that it just doesn't prove it.

A wanton disregard for realism in interviewing is, in my experience, very clearly systemic and industry-wide.

>tweet you shared does make it sound like he expected to get the job without an interview

Seriously wtf? I think you're starstruck by Google and that is affecting your judgement.

Asking him about binary trees was dumb. Unless Google place him somewhere patently unsuitable for his skill set (kernel hacking/low level database code), he won't be using binary trees.

[dahart]

> It does imply that it just doesn't prove it.

@mxcl's experience doesn't imply it either. The only evidence for a widespread problem is the number of people who have that problem. I have never had an interview where I felt rejected for what I thought was a single dumb question, ever. And I've never seen it happen to someone I know personally, or at a company I've worked for.

> A wanton disregard for realism in interviewing is, in my experience, very clearly systemic and industry-wide.

I don't necessarily disagree with this, but could you elaborate more on what bad things are actually happening that affect people? Are good coders, by and large, not able to get jobs? Are good coders having statistically significant problems getting paid or finding enjoyable work? I don't think so. Please elaborate on what actual damage is being done, I'm not seeing any.

> Seriously wtf?

Seriously. I don't know what happened, but I'm not automatically on the side of @mxcl because he was turned down or because I'm a fan of Homebrew. He might be exaggerating what happened. Do you know for a fact that it was specifically the binary tree question and nothing else in his interview that lost his chances there?

For all I know, the binary tree question was put there just to see if he would scoff at actual programming questions given his high profile status, and he failed because he scoffed and not because he got it wrong.

> I think you're starstruck by Google and that is affecting your judgement.

Why? What have I said that suggests I'm a fan of Google at all? It seems to me like you're making wild assumptions here.

> Asking him about binary trees was dumb.

That's an opinion. One that is based on not knowing why the question was there, or what the other questions were. I do ask questions on topics that I don't expect the candidate to use in their job. I'm interested in whether they paid attention in school. I'm interested in what they know, regardless of their skill. I'm interested in what they don't know, and where their limits are. I'm interested to know if people are curious about software. I'm interested to know how people react to questions they don't know the answer to. And I have a rule to specifically reject candidates that get upset about being asked technical questions. Those are people I don't want to work with.

None of that precludes asking some realistic questions about things people will use on the job, in addition to any unrealistic ones.

[crdoconnor]

>I have never had an interview where I felt rejected for what I thought was a single dumb question, ever.

This is moving the goalposts. That is not the same thing as a wanton disregard for realism.

Have you been asked interview questions and set tasks which were not related to what you actually do day in day out? I have. LOTS.

>I don't necessarily disagree with this, but could you elaborate more on what bad things are actually happening that affect people? Are good coders, by and large, not able to get jobs? Are good coders having statistically significant problems getting paid or finding enjoyable work? I don't think so. Please elaborate on what actual damage is being done, I'm not seeing any.

Um, more false positives and more false negatives. It honestly feels weird having to justify why realism in testing is important. It feels so damned obvious to me. Would you design a purposefully unrealistic scientific experiment? Create a deliberately unrealistic automated test? What's special about interviewing that realism is of secondary or tertiary concern? What is more important than realism?

>I'm not automatically on the side of @mxcl because he was turned down

It feels kind of like you're automatically on Google's side.

The part that made me go what the fuck was when you said "he makes it sound like he expected to get the job without an interview" when he neither said nor implied anything of the sort.

And, earlier you took it as a kind of article of faith that everybody at google was obviously super smart, because Google.

There's definitely some bias there.

>I'm interested in whether they paid attention in school.

That's cool. I'm interested in whether they can do their job and I think it's kind of weird how people are seemingly so keen on setting tasks that test anything but that.

>None of that precludes asking some realistic questions

Strictly speaking asking what their favorite kind of chocolate is doesn't preclude that either, but there's a limited time available to interview and a limited amount that can be learned from asking them that - or indeed - anything else of tangential relevance.

[dahart]

> This is moving the goalposts.

What goalposts? You said it was an embarrassment that @mxcl didn't get hired by Google because of the "dumb" question. I've responded directly to your claims.

> What is more important than realism?

For me, questions that assess honesty, optimism, curiosity, potential, and communication skills all rank higher than technical questions that are "realistic". I care more about attitude and potential than I do about whether they can perform specific job duties already.

> There's definitely some bias there.

If there is, I wouldn't know it, but I think you're wrong. Do you know more than @mxcl tweeted? Do you know for a fact what happened? Have you heard Google's side of that story? Are you biased against Google? Do you believe one tweet is true and tells the whole story?

The fact that @mxcl tweeted with indignance about his interview experience is what implied he expected to get the job without an interview. That's a fact, not an interpretation. He didn't so much imply it as say it directly, that he shouldn't have to invert a binary tree because 90% of Google uses his software.

> I'm interested in whether they can do their job and I think it's kind of weird how people are seemingly so keen on setting tasks that test anything but that.

I don't really understand why you just got so snarky, it's probably my fault for arguing, but it was calm a couple of messages back. I'm honestly sorry if something I said ticked you off.

You're taking my comment out of context and trying to make it sound like something it's not. I can be interested in whether someone learned in school and still be interested in whether they'll do a good job, right? In fact, I'd go way out on a limb to suggest that doing well in school is a reasonable (but not perfect) proxy for how well someone might do in a job. Especially, but not limited, to new college grads.

[bitL]

Getting to Google is more difficult than getting to Harvard. They built their process that way risking rejecting great candidates in order to minimize false positives, bad hires, according to their definition of bad hire. So the Homebrew author could have expected it, it happens all the time. I object to using the same criteria throughout the industry on most jobs these days though, instead of only on top-end ones. I have nothing against top companies having top interviews, they are usually fun if you are good.

[crdoconnor]

Explain to me the justification behind using an interview process that deliberately placed no emphasis on realism.

This guy wouldn't be using binary trees at Google. He hasn't used them before. They are of minimal relevance in his area of expertise.

[bitL]

I guess you need to talk to Norvig or whoever designed their data-driven process and the definition of good hire they wanted to achieve. It used to be that when you joined Google you had no clue what your project will be before your first day, so I guess they wanted to maximize success rate on blind assignments to teams/ideas. For that certain abstract skills are more important than your past accomplishments you might not be able to reproduce in different environment with different rules. It's their money after all, they are desirable, they can select for whatever they wish.

[crdoconnor]

So, to be clear, you thought good reasons for an unrealistic testing process were:

* Argument from authority (because Norvig)

* "We might want to put a front ender in a back end job and vice versa so we need an interview process that accounts for that"

* Tests of "abstract skills" - as in, skills you won't actually use - are more important than tests of non-abstract skills which you will.

* "It's their money"

[bitL]

Nope.

> * Argument from authority (because Norvig)

No. I suggested you might want to ask Norvig why did they decide so

> front ender in a back end job

I think it's a bit different. It's like you are creating completely new stuff like Big Data 15 years ago where frontend/backend separation didn't exist yet. Likely the same holds for various machine learning roles right now. So those categories we will be using in the future have to be invented first, and for that you need slightly different approach

[bitL]

> All the large ones (Google, Apple, Amazon, Microsoft, etc.) give coding puzzles as part of their interview process.

They are considered the best and give above average compensation. Is your company considered the best, properly paying top talent, to employ the same schema? Those tests were done to distill the top end performers with the accepted risk of huge number of false negatives. Now every mom-and-dad shop is trying to use it. That's why I call it insanity, getting extreme practices into mainstream in our industry.

[dahart]

> Is your company considered the best, properly paying top talent, to employ the same schema?

Yes, this was true of every job I've ever had. Maybe I've been lucky.

I don't really understand why asking someone to do a small amount of programming for a programming job interview, when they claim to be an experienced programmer, is any sort of "extreme" practice.

You've expressed a lot of complaints about coding quizzes, some that many people here share, and some that I agree with.

I would urge you to spend your energy making a specific alternative proposal that companies can actually use. The OP's proposal isn't something companies can actually use for all candidates, even if they could use it for him.

I'm very interested in how to better screen and interview devs. I want to improve my process. But no process I've ever been exposed to is even close to perfect.

Make sure your proposal considers the company's perspective. The ideal process will benefit both the candidate and the company, but something that's pleasant for the candidate and unpleasant for the company will never get adopted no matter how good it sounds.

[bitL]

Because if you have in your CV verifiable statements that you worked on some super tough engineering challenges, including advanced snippets of code/projects on GitHub demonstrating your dominance in that area, doing silly low-end coding quizzes seems like a total waste of time. Can't you really see that? It's literally like if I asked you if you ever ran a company and if you could handle preparing a budget, despite knowing you are a serial entrepreneur that sold multiple companies already. It's just super arrogant.

[dahart]

> Because if you have in your CV verifiable statements that you worked on some super tough engineering challenges

If it takes longer to verify than it does to test your coding ability directly, why shouldn't I just ask you to do some coding? The challenges you took might be less impressive or well known than you think. The challenges you took might not say much about you if they were team challenges.

> including advanced snippets of code/projects on GitHub demonstrating your dominance in that area

That doesn't help me compare one candidate to another at all. Nor does your github demonstrate dominance in anything, unless your project is React or something like that. Github is a vast wasteland of barely used code.

> doing silly low-end coding quizzes seems like a total waste of time

It's going to take about as long to get through job interviews no matter how the interview is conducted. You can spend it programming, or you can spend it talking. The time spent is an investment in getting the job. If you don't actually want the job, then you're right, it's a waste of your time.

What's not going to happen, ever, is someone will take the initiative to read through all your work, verify the things on your CV, and offer you a great high paying job without going through the interview process.

> Can't you really see that?

To be very frank and honest, given all the reasoning and experience I've shared with you, this question gives me the impression that you might be very inexperienced.

> It's just super arrogant.

What, precisely, is arrogant? What are you talking about specifically? Using Hackerrank in an interview? Having an interview at all? Not noticing that you're a rockstar before talking to you?

Please take some time to articulate what the right interview process is, rather than spend any more time passing blanket judgements.

[bitL]

I think you are completely missing the point I am trying to make.

Imagine you have strong interest from Google, FB, Uber, Amazon etc. Google wants to waive their interview process as you are an open source contributor (they actually read your CV and clicked on the links). FB wants you to lead some ML team they have doing cool things etc. Other companies beg you to work for them even if you don't consider them interesting, willing to overpay you and pamper you.

Now comes your unknown company/startup. In order to even talk, you require passing some HackerRank coding test. I look at your Glassdoor reviews, you either have none, or few, or your compensation seems to be low etc. You might be working in interesting area, maybe I should give you a shot? Or I just want to see what current crop of interviews in your industry looks like, maybe I agree on going through the process? Maybe I even visit some interesting city you are located in and scratch it off my bucket list?

In the end I won't work for you. I won't consider anything you offer. I've gotten from you what I wanted - glimpse of the area you are working on for more ideas, keeping my brain up to date to interview requirements, visiting city I've never been before. You wasted time and money on me. You didn't get anything. I puzzled your head because you thought you'd have a shot at getting me. And that is the best outcome you'd get from this; most likely I wouldn't even talk to you after your initial requirements and go with other choices available to me.

[dahart]

Please propose a better process!

> Google wants to waive their interview process as you are an open source contributor. Other companies beg you to work for them even if you don't consider them interesting, willing to overpay you and pamper you.

That's not a realistic scenario, your point seems contrived. Google doesn't waive their interview process, as the @mxcl example demonstrates. Companies only beg to throw money at you and overpay you if you're famous or have a niche skill. If that's true for you, this conversation is irrelevant to you.

> Now comes your unknown company/startup. In order to even talk, you require passing some HackerRank coding test.

As I said above, I'm assuming this process only starts when you express interest in the company. If you're complaining about having to respond to recruiter spam, I can't help you. Nobody is forcing you to take any tests. You should only do it when it's for a job you want.

> In the end I won't work for you. I won't consider anything you offer.

What you're doing is avoiding false positives by screening for something you care about. The same thing Google does. Except Google has statistics on how well their screens work.

> You wasted time and money on me.

Or, more accurately, they saved time and money by not doing lengthy and involved interviews or researching you heavily before discovering it's not a good fit.

[crdoconnor]

>I don't really understand why asking someone to do a small amount of programming for a programming job interview, when they claim to be an experienced programmer, is any sort of "extreme" practice.

It isn't and that's exactly what you should expect.

Nonetheless it shouldn't be too much to ask that the test and interview ask relevant questions and that the company puts skin in the game that is commensurate with the sacrifice being asked of the candidate.

In other words, 5 minute screener tests are cool but if you make me do a weekend project you put me up in a 5 star hotel.

[mywittyname]

Those "mom-and-dad shops" are the ones who have the most to lose by hiring the wrong person. A sufficiently poor performing person could cripple them.

[WDCDev]

They also have a lot to lose by leaving the position unfilled for months on end. If you aren't Googlefaceapplezon, then false negatives could be as costly as false positives given time is money.