[crdoconnor]

In the past I've given programming tasks that mimic real life work as closely as possible and fit them into a reasonable time frame - for example, 1 hour of pairing/programming during an arranged face to face interview. If the test is well designed, touches on a number of areas and self contained, I think this was sufficient for me to assess their skills.

"Mimic real life" means no puzzles (unless I've literally experienced them in real life), no binary tree reversals and no big O notation questions.

>You have just dismissed most tech companies. All the large ones (Google, Apple, Amazon, Microsoft, etc.) give coding puzzles as part of their interview process.

I think that type of thinking leads to embarrassments like this:

https://twitter.com/mxcl/status/608682016205344768

I think he experienced exactly the same problem the OP is talking about and in this case it's not him who was dumb, it was Google.

I wouldn't rule out any of those companies but I think I'd rule out joining through the standard interview process - I'd look for specific people who looked to be doing exciting work on specific teams and try to befriend them.

>you'd be wrong to say they have an entitlement complex or aren't the smartest.

I think it would depend upon the team. I think they're not all geniuses, and they do have a tendency to drink their own kool aid. For sure some teams are great though.

[dahart]

> 1 hour of pairing/programming during an arranged face to face interview. If the test is well designed, touches on a number of areas and self contained, I think this was sufficient for me to assess their skills.

Sounds great. I've done this too, but I've never done it before screening candidates. I don't have enough time to screen candidates by programming with them.

> "Mimic real life" means no puzzles (unless I've literally experienced them in real life), no binary tree reversals and no big O notation questions.

Understood. My goal with my coding quizzes and knowledge questions is not to mimic real life. It's to asses the boundaries of the candidate's education and experience, without regard to skill.

> I think that type of thinking leads to embarrassments like this

There always have been and always will be false negatives, even with lengthy face-to-face interviews, even with pair programming, and even with paid internships.

One high profile false negative anecdote, while unfortunate, doesn't imply there's a widespread or unexpected problem. Applying for jobs always comes with a risk of not getting the job for a wide variety of factors that are outside the candidate's control. If it's a job you really want, all you can do is try hard.

I love homebrew, and I'm sure @mxcl is a fabulous coder, but the tweet you shared does make it sound like he expected to get the job without an interview, and might have come across that way, or might have not prepared at all.

[crdoconnor]

>I've never done it before screening candidates

No me neither. I screened with a 5 minute fizz buzz like task and CV fits. Bad programmers slipped through the net (and were caught by the test) but I never got anybody who literally couldnt code. I was happy with that balance.

I was afraid if I made it longer than 5 mins we'd filter out good candidates who couldn't be bothered with our bullshit.

>One high profile false negative anecdote, while unfortunate, doesn't imply there's a widespread or unexpected problem.

It does imply that it just doesn't prove it.

A wanton disregard for realism in interviewing is, in my experience, very clearly systemic and industry-wide.

>tweet you shared does make it sound like he expected to get the job without an interview

Seriously wtf? I think you're starstruck by Google and that is affecting your judgement.

Asking him about binary trees was dumb. Unless Google place him somewhere patently unsuitable for his skill set (kernel hacking/low level database code), he won't be using binary trees.

[dahart]

> It does imply that it just doesn't prove it.

@mxcl's experience doesn't imply it either. The only evidence for a widespread problem is the number of people who have that problem. I have never had an interview where I felt rejected for what I thought was a single dumb question, ever. And I've never seen it happen to someone I know personally, or at a company I've worked for.

> A wanton disregard for realism in interviewing is, in my experience, very clearly systemic and industry-wide.

I don't necessarily disagree with this, but could you elaborate more on what bad things are actually happening that affect people? Are good coders, by and large, not able to get jobs? Are good coders having statistically significant problems getting paid or finding enjoyable work? I don't think so. Please elaborate on what actual damage is being done, I'm not seeing any.

> Seriously wtf?

Seriously. I don't know what happened, but I'm not automatically on the side of @mxcl because he was turned down or because I'm a fan of Homebrew. He might be exaggerating what happened. Do you know for a fact that it was specifically the binary tree question and nothing else in his interview that lost his chances there?

For all I know, the binary tree question was put there just to see if he would scoff at actual programming questions given his high profile status, and he failed because he scoffed and not because he got it wrong.

> I think you're starstruck by Google and that is affecting your judgement.

Why? What have I said that suggests I'm a fan of Google at all? It seems to me like you're making wild assumptions here.

> Asking him about binary trees was dumb.

That's an opinion. One that is based on not knowing why the question was there, or what the other questions were. I do ask questions on topics that I don't expect the candidate to use in their job. I'm interested in whether they paid attention in school. I'm interested in what they know, regardless of their skill. I'm interested in what they don't know, and where their limits are. I'm interested to know if people are curious about software. I'm interested to know how people react to questions they don't know the answer to. And I have a rule to specifically reject candidates that get upset about being asked technical questions. Those are people I don't want to work with.

None of that precludes asking some realistic questions about things people will use on the job, in addition to any unrealistic ones.

[crdoconnor]

>I have never had an interview where I felt rejected for what I thought was a single dumb question, ever.

This is moving the goalposts. That is not the same thing as a wanton disregard for realism.

Have you been asked interview questions and set tasks which were not related to what you actually do day in day out? I have. LOTS.

>I don't necessarily disagree with this, but could you elaborate more on what bad things are actually happening that affect people? Are good coders, by and large, not able to get jobs? Are good coders having statistically significant problems getting paid or finding enjoyable work? I don't think so. Please elaborate on what actual damage is being done, I'm not seeing any.

Um, more false positives and more false negatives. It honestly feels weird having to justify why realism in testing is important. It feels so damned obvious to me. Would you design a purposefully unrealistic scientific experiment? Create a deliberately unrealistic automated test? What's special about interviewing that realism is of secondary or tertiary concern? What is more important than realism?

>I'm not automatically on the side of @mxcl because he was turned down

It feels kind of like you're automatically on Google's side.

The part that made me go what the fuck was when you said "he makes it sound like he expected to get the job without an interview" when he neither said nor implied anything of the sort.

And, earlier you took it as a kind of article of faith that everybody at google was obviously super smart, because Google.

There's definitely some bias there.

>I'm interested in whether they paid attention in school.

That's cool. I'm interested in whether they can do their job and I think it's kind of weird how people are seemingly so keen on setting tasks that test anything but that.

>None of that precludes asking some realistic questions

Strictly speaking asking what their favorite kind of chocolate is doesn't preclude that either, but there's a limited time available to interview and a limited amount that can be learned from asking them that - or indeed - anything else of tangential relevance.

[dahart]

> This is moving the goalposts.

What goalposts? You said it was an embarrassment that @mxcl didn't get hired by Google because of the "dumb" question. I've responded directly to your claims.

> What is more important than realism?

For me, questions that assess honesty, optimism, curiosity, potential, and communication skills all rank higher than technical questions that are "realistic". I care more about attitude and potential than I do about whether they can perform specific job duties already.

> There's definitely some bias there.

If there is, I wouldn't know it, but I think you're wrong. Do you know more than @mxcl tweeted? Do you know for a fact what happened? Have you heard Google's side of that story? Are you biased against Google? Do you believe one tweet is true and tells the whole story?

The fact that @mxcl tweeted with indignance about his interview experience is what implied he expected to get the job without an interview. That's a fact, not an interpretation. He didn't so much imply it as say it directly, that he shouldn't have to invert a binary tree because 90% of Google uses his software.

> I'm interested in whether they can do their job and I think it's kind of weird how people are seemingly so keen on setting tasks that test anything but that.

I don't really understand why you just got so snarky, it's probably my fault for arguing, but it was calm a couple of messages back. I'm honestly sorry if something I said ticked you off.

You're taking my comment out of context and trying to make it sound like something it's not. I can be interested in whether someone learned in school and still be interested in whether they'll do a good job, right? In fact, I'd go way out on a limb to suggest that doing well in school is a reasonable (but not perfect) proxy for how well someone might do in a job. Especially, but not limited, to new college grads.

[crdoconnor]

>You said it was an embarrassment that @mxcl didn't get hired by Google because of the "dumb" question.

I only said it was an embarrassment that he said that and that they asked such a patently irrelevant question and then rejected him. I think you read a lot more in to that tweet (and possibly the situation) than was actually there.

It's entirely possible that they rejected him for a perfectly reasonable reason, though there's nothing to really imply that that is so.

>For me, questions that assess honesty, optimism, curiosity, potential, and communication skills all rank higher than technical questions that are "realistic".

Ok, so if that's what you're really looking for, do your job adverts actually state at the top "we seek honest, optimistic, curious candidates with potential and communication skills?"

I've got to be honest I don't see many job adverts that state that and I think "able to do the job" ranks higher up the list of concerns for most employers than "has a sunny outlook on life". Still, if that's what you want...

Realistic tests will, if realistic, implicitly test honesty to a degree (the candidate will at some point have to admit that they don't know) and communication skills and many other important traits you might not even realize that you needed to select for - in proportion to their relevance to the role at hand.

>If there is, I wouldn't know it, but I think you're wrong. Do you know more than @mxcl tweeted? Do you know for a fact what happened?

A little, yes, but that's not the point. You don't know more than what he tweeted and yet you inferred something quite insulting that he did not say or mean.

>The fact that @mxcl tweeted with indignance about his interview experience is what implied he expected to get the job without an interview.

That's 100% your spin. To me, it implied that he was pissed at being asked irrelevant questions. A quick google can actually confirm that (there's a quora thread about him).

That's pretty much what this whole thread is about. The OP's rant was basically "recruiter, your test is bad, here's a more realistic one you should try". My rant was about irrelevant questions and unrealistic questions.

>he shouldn't have to invert a binary tree because 90% of Google uses his software.

...and neither should most people who interview at Google because asking that question is irrelevant for ~90% at software engineers even at google (never mind elsewhere).

Frankly, even if you are an engineer that does use binary trees it's a bad question to ask.

>I don't really understand why you just got so snarky

Honestly? Because, in a situation where you didn't really have enough information to form a judgment you instinctively sided with the large, powerful, faceless corporation and showed distrust of the little guy who made a great piece of software for free that people love.

I think that's a pretty unhealthy kind of bias to exhibit.

>I'm honestly sorry if something I said ticked you off.

Yeah, sorry I probably shouldn't have snarked.

[bitL]

Getting to Google is more difficult than getting to Harvard. They built their process that way risking rejecting great candidates in order to minimize false positives, bad hires, according to their definition of bad hire. So the Homebrew author could have expected it, it happens all the time. I object to using the same criteria throughout the industry on most jobs these days though, instead of only on top-end ones. I have nothing against top companies having top interviews, they are usually fun if you are good.

[crdoconnor]

Explain to me the justification behind using an interview process that deliberately placed no emphasis on realism.

This guy wouldn't be using binary trees at Google. He hasn't used them before. They are of minimal relevance in his area of expertise.

[bitL]

I guess you need to talk to Norvig or whoever designed their data-driven process and the definition of good hire they wanted to achieve. It used to be that when you joined Google you had no clue what your project will be before your first day, so I guess they wanted to maximize success rate on blind assignments to teams/ideas. For that certain abstract skills are more important than your past accomplishments you might not be able to reproduce in different environment with different rules. It's their money after all, they are desirable, they can select for whatever they wish.

[crdoconnor]

So, to be clear, you thought good reasons for an unrealistic testing process were:

* Argument from authority (because Norvig)

* "We might want to put a front ender in a back end job and vice versa so we need an interview process that accounts for that"

* Tests of "abstract skills" - as in, skills you won't actually use - are more important than tests of non-abstract skills which you will.

* "It's their money"

[bitL]

Nope.

> * Argument from authority (because Norvig)

No. I suggested you might want to ask Norvig why did they decide so

> front ender in a back end job

I think it's a bit different. It's like you are creating completely new stuff like Big Data 15 years ago where frontend/backend separation didn't exist yet. Likely the same holds for various machine learning roles right now. So those categories we will be using in the future have to be invented first, and for that you need slightly different approach