[nmalaguti]

Each time I read a post like this I say to myself “they’re not wrong, but something is also missing.”

If there was a way to evaluate a candidate based on how well they would do at their actual job, in an hour or less, that couldn’t be games or cheated, we’d all be using it.

That isn’t to say we shouldn’t work on improving the hiring process, but how much time, both from the perspective of the candidate and the company, is too much time to invest in a potential hire?

Would we get better signal if you came to work for us for a week and paired with everyone on the team? Sure! But how many candidates can take a week off from their current job? The author talked to 11 companies. Could they invest 11+ weeks in their job search full time in order to find their next role? What about the loss in productivity? Can you have more than one candidate in the office in a given week? What if your ramp up time for full time employees is actually longer than a week? Are you biasing for people who ramp up fast instead of people who will be ultimately more productive and impactful?

I often feel that engineers write these posts because they know that they themselves are good at their job, but feel it is silly to have to develop an alternative set of skills in order to signal that they are a good hire. Or they get rejected and blame the process or those alternative skills. I’ve felt the same way and wanted the system to be better tailored for my skill set, but balanced against the time investment for some alternatives, I see why we have the process we do.

[grue2]

I use a work sample (short paid contract), but it's not perfect and boy is it labor intensive.

As a hirer, you really can't win. There's not one hiring process nor guiding principle that doesn't seem to bring out the pitchforks of those who were frustrated, disrespected, or rejected by said process:

- Show us your open source work. (You're excluding all but a lucky few who have the privilege of writing open source code!)

- Okay then, show us a personal project or some work you've done in your free time. (What, so I'm expected to live eat and breathe code 24/7 to get hired?!)

- Well, how about a short contract/work sample? (How am I supposed to find the time to do that? I have a day job and a life!)

- Shall we try whiteboarding/coding tests then? (This is so insulting! Solving CS puzzles isn't what the job is about!)

One cannot, sadly, rely on the résumé. I have interviewed multiple self-deemed "experts" in such-and-such language, only to find that they could not even write a basic for-loop on the whiteboard.

[bonestamp2]

I mostly agree, but he still makes some good points. For example, "Give your interview to current employees".

One of my coworkers and I often joke about how we'd never pass our current hiring test, and yet we're among the highest paid and most productive people at the company. I've expressed this to the hiring manager and he still thinks it's a good approach. They have literally said they want more people like me... so they brought me into the hiring process, but I disagreed with them on almost every candidate, so they brought me back out of the hiring process. Some people don't know what they want out of interviewing a candidate, they just go with the status quo interview and assume they're going to feel good or bad about someone based on their results.

[seanmcdirmid]

Most of these interview tests are game-able, there are a few resources you need to study hard for and you’ll be able to pass the interview questions without many problems. This has the perverse effect of raising the bar since everyone is doing so great on the problems and filtering is needed! But in the end, you only select for who is better at studying for the interview, not who is a better developer. Fun stuff.

If you onboarded in a time before this, or you aren’t practicing for an interview, it is very likely you wouldn’t just pass the interview, like you wouldn’t likely pass a physics college exam right now even though you did when you were in school.

[gwbas1c]

I've found that it's usually obvious when someone is gaming my interview. Their answers are too correct, and too smoothly given. Those candidates are very quickly rejected and the recruiting agency basically told to scram.

[RJIb8RBYxzAMX9u]

Perhaps, but here's a personal anecdote when I was interviewed years ago. The interviewer asked me one of the trick problems popular at the time, which I knew the answer to. Ironically, it was not a problem I'd studied for; I just happened to have seen in elsewhere years prior, found it interesting, so I remembered it and its solution. To be clear, I didn't solve the problem the first time I'd seen it, either.

Nevertheless, I gave the interviewer a detailed, step-by-step, golden answer she was looking for, and she's obliviously impressed. After a few minutes, however, youthful stupidity^W^Wmy integrity got the better of me and I told her that I'd seen the problem before. So I got a 2nd trick problem, one that I didn't know nor studied for, and naturally didn't arrive at the golden solution.

I didn't get called back.

If you were to ask me to solve a problem that's on my study list today, believe me that I've got the selling part down pat now.

[seanmcdirmid]

If I study a bunch of interviewing problems really hard, I’m bound to do good on your test. I can study hard, I proved that in unveristy and grad school. Your interview process probably isn’t that unique, and for bigger companies like Google there is plenty of study material out there.

And everyone is doing this now, it just isn’t a certain few bad recruiting agencies and applicants. Heck, even the good applicants (the ones that you want to hire anyways) have to do it to get through the loop.

[gspetr]

It's almost as if the people who want the job the most would study for it the hardest.

[custos]

Best company I've been hired at did a sample project.

They had the project all setup and ready to go. All I had to do was fill in some of the more complicated bits (hooking up to the database using LINQ to SQL, and implement some queries to search by name/album and two or three other fields).

Then display them in a certain order.

I was given an hour to do it. Apparently, after I was hired, I was told I was the only one they interviewed that could do it. Ironic part was I'd never used LINQ to SQL before and had studied it the night prior. (It was on the job posting)

[sleazebreeze]

At my last job, I set up something like this. We were specifically interviewing people who claimed to have frontend and Angular experience - I confirmed this on every resume before the interview and again with the candidate before explaining the test. To test it, I gave the test to 2 coworkers who each completed it in under 15 minutes. Candidates were given an hour for it.

The test was a JSFiddle pre-filled out with Angular boilerplate. It had comments in places where things needed to be added and filled out. Just to be clear, there was no gotchas or tricks involved. It was very simple: "get JSON from this public HTTP endpoint, display it on the page in a table and then add a filter field. If there's time, add some CSS to make it look pretty."

Candidates were encouraged to use docs and Google if they couldn't remember off the top of their head how to do those. More than one candidate copy/pasted from Stack Overflow during the interview and while I wasn't impressed, I didn't count it against them.

Several candidates completely failed at it and I suspect it was a combination of nerves and lack of experience (one candidate it became clear that he had never used Angular before and he never got past the "get JSON from an HTTP endpoint"). During the test, I was more than willing to talk through the code with them and help them figure out the best way to do it. The one candidate who passed it was hired and they were very successful at the company.

[taco_emoji]

Was it marked as a requirement by the job posting? Because if not, that's just a test of knowledge, not ability.

[xigency]

The test is the ability to apply knowledge, which is probably the only thing they need.

The commenter did say it was mentioned in the job listing.

[taco_emoji]

Yes, I understand that it was mentioned, my question was whether it was mentioned as a requirement.

If the listing just said "LINQ-to-SQL knowledge a plus" but then gave you a test that actually required foreknowledge of it, then that's kinda shitty.

[alistairSH]

Shall we try whiteboarding?

I've used this in the past. But, I don't ask for actual code. I'm using it to see the candidates thought process.

More important than any psuedo-code is how well they communicate what they are thinking (whether that be trough writing on the wall or conversation). Can they break the problem down into workable pieces? Do they ask questions about the requirements? Do they try a simple solution and iterate on it? They they get hung up on small details and miss the bigger picture?

[DarronWyke]

Sysadmin/infosec guy here. We do the same thing. I was shown this a while ago and I use it on other candidates I've interviewed. Propose a problem, then as they start working through it, throw other wrenches into it or state that their solution didn't work. See what tangents they go off to.

Another neat one I was shown was, as a whiteboard test, was to write down all the letters, A-Z, then come up with the name of a command for each letter. It's a double-sided test -- it shows your ability to respond quickly, how you respond, and the ability to back up your responses.

Of course, joking questions like "vi or emacs?" can be good as well to see a candidate's character as to how they respond to technical inside jokes like that.

[viraptor]

Not sure about the alphabet. Got to K before I got stuck and thought... So in that moment, if I forget that there's kill that starts with a K, why does it matter? How does it relate to the fact that I know to use kill for sending signals? The recall for a situation where you'd use something and for a letter matching game seem to be very separate. (On the other hand I remembered join which I used maybe once in my life...)

[DarronWyke]

It's multi-faceted. It shows your ability to think quick and how you solve multiple problems. If you stop in the middle of the list and try over and over to come up with an answer, how will you react to a real problem? Or do you just give up when it gets hard?

It's testing more than just rote memorization. You also have to explain any commands you write down, what they do.

[kafkaesq]

You mean... "Hmm, my assumption, from looking at your resume and portfolio, is that you aren't quite ready to hold your own in a spontaneous, 1-1 discussion about an actual, real business or technical problem. And not only that -- I actually have severe doubts not only about your relevant experience, but about your basic intellectual capabilities. So let me give you this little made-up toy problem, with this little checklist in my mind about where I think you're going to screw up. And watch you perform. Being as I can tell you don't have a lot of options right now, and will put up with pretty much whatever I dish at you."

That might be OK for someone with, say, <2 years experience out of college. But for anyone mid-career or higher, it comes off as silly and condescending, and as not exactly a good use of their precious, irreplaceable time. And more to the point, conveys the exact opposite of the message you should be sending: that you know they're smart and intellectually self-sufficient, already. And that it's up to you to win them over, and convince them that you're worthy and interesting, and that they should jump over, and join your mission.

[alistairSH]

WTF? Maybe I didn't communicate it well, but the whole point is the discussion, not the answer...

More important than any psuedo-code is how well they communicate what they are thinking

It's not a test of technical skills. And you'd be amazed at the number of resumes that signal somebody can have a quality discussion about technical problems, but the candidate can't communicate for shit.

[kafkaesq]

But the candidate can't communicate for shit.

You know, it just might be... the interviewer who's not so hot at communicating.

Or is "communicating" okay enough, but following a template that's broken to begin with - and pretty much designed to make candidates feel alienated - or at best, highly unenthused at the prospect of playing along with.

[xigency]

I generally agree, but I was involved in an internal hiring interview where a candidate was extremely qualified. In that situation, I was able to ask a hard and open-ended problem about graph traversal on the whiteboard. And it was fun to see the interviewee's approach. "Wow, I haven't done anything like this in years," was the reaction.

Obviously that is an example of a low-stress interview. When the candidate was stuck I gave hints, and the whiteboard portion was only intended to last 10-minutes. It was also only 5% of the feedback I gave. The majority was comments on candidate's knowledge and experience.

---

That's definitely how I'd like my whiteboard interviews to go. I've experienced the back-to-back grinding interviews of softball problems to write in syntactically correct code. Getting dinged for errors and even getting a close but wrong answer is a pain in the butt. When you write a program and realize it's wrong at the end, the question, "What do you think the right answer is?" is frustrating. I can't say, well, I wish I had 30 minutes to start over. If you tell me the answer, we can talk about why I was wrong.

Those interviewers have been engineers peeled off of their daily projects though, so I can't say I fault them individually.

[gwbas1c]

Interviews are naturally awkward.

If you had that attitude during one of my interviews I'd show you the door.

I can't begin to count the number of impressive resumes that lead to unqualified candidates.

[kafkaesq]

If you had that attitude during one of my interviews I'd show you the door.

And with that attitude on your side -- "if this isn't going well, then by definition, it's the candidate's fault" --- let's just say we're clearly not match.

I can't begin to count the number of impressive resumes that lead to unqualified candidates.

Ditto for job ads leading to unimpressive companies with obtuse interview processes.

Interviews are naturally awkward.

As currently conceived. So maybe it's time to start thinking about them differently.

[alistairSH]

You're awfully full of criticism today. In your perfect world, what would the interview process be? How do I, the hiring manager, ensure you are what you claim to be, without wasting either of our time or hurting your feelings?

[jaredklewis]

Yikes. So which of the other 3 listed methods do you prefer? Or something else?

Also, I think you’re somewhat confusing whiteboarding and being interviewed by an egomaniac. If someone is a total jerk, I imagine any interviewing methodology will suck.

[jhgaylor]

Disclaimer: I've written code outside of work for my own satisfaction.

> - Okay then, show us a personal project or some work you've done in your free time. (What, so I'm expected to live eat and breathe code 24/7 to get hired?!)

I have software I can show off to potential employers but I didn't write it for them. I'll admit that I got really lucky to both enjoy coding and to have the privilege to so, but I didn't have to give up my life in order to get it done. I play video games many more hours per week than I code outside of work.

If I had to choose between otherwise equally qualified mechanics, I would tend to prefer the one with a hobby car at home.

[jschwartzi]

Whereas I would choose the one with the cleanest shop.

[logfromblammo]

I think I'd prefer the one who tells me the potential consequences of not doing a repair, without me having to specifically ask about it (or pry it out of them) and then verify the answer from another source.

Dealer service departments always fail this test. They will never tell you that it is not necessary to do a repair. Some will even say "I don't feel comfortable letting you drive it off the lot like this." Then you take it to another mechanic that says, "Yeah, this might affect your fuel economy by a tiny amount. I wouldn't worry about it in a car this old, unless you plan on moving to California."

[mark-r]

Great story, but can you make it relevant somehow to the discussion about interviews?

[logfromblammo]

Rather than hire the person who continues to do his day job at night, or who maintains an organized workstation, I'd hire the person that actually provides value to the company.

[munin]

What if you accept applications for 30 days, then write the name of each applicant on a card, put the cards into a deck, shuffle the deck, then draw cards off the top of the deck and make offers until you run out of open positions?

It isn't merit based, but it's hard to argue that it favors one candidate over another! And maybe it's mimicking the effectiveness of your current system!

[sraf]

This has the added benefit that you only hire lucky people. Noone wants to hire an unlucky person.

[grue2]

This has a certain genius to it. The same genius as sleep sort.

[8ytecoder]

This! We have tried all the points listed there and at least one person has found fault with that. That said, we follow pretty much the order you listed. The people invested in their job search do respond.

[bootsz]

I agree that there is no "perfect" interview process, but I have seen from experience that there are ways to go about it that reduce the crappiness for both sides, beyond what a lot of "top" companies are currently doing.

How about sticking with standard phone-screen/on-site process, but emphasizing writing real code, on a computer, solving realistic software engineering problems? I've interviewed at several companies (including one of the "big four") that took this approach and found it was the best overall balance IMO. You get evaluated in a more realistic setting than a whiteboarding interview, but takes the same amount of time an also doesn't depend on "side-projects" or "take-home" assignments.

Some general examples from my own experience:

- Here's a poorly-written / buggy program: Find the bugs and do some refactoring

- Here's a loose spec for a small program: Spend an hour designing & coding it up (on your own, no one looking over your shoulder the whole time), and then walk through it with the interviewer AFTERWARD and explain your design / answer their questions about it.

[jordanpg]

I've learned that a decent proxy for this mythical silver bullet of hiring is basic questions about tooling and networking.

Gauging someone's vast, claimed experience in some language or technology can be tricky, but if they can't explain what a rebase is, or how maven dependency management works, or how SSL works in general terms, it tells me a lot about what kind of technologist they are.

[jbattle]

An alternate way to do this (which avoids false negatives cause they don't know the same trivia as you) is to ask them to tell you about a tool/library they are excited about. Or ask them to explain some part of their last project in detail. And then keep asking probing questions to see how deep they can go. You get the same sense of "how deeply does this person actually understand the tools they use" but you dramatically reduce the risk of hitting one of their (rare?) blank spots.

Side benefit #1 is you get a sense of what they think is important

Side benefit #2 is you find out if they can accurately gauge how well they know what they think they know

Side-benefit #3 is you might learn something new!

[jpindar]

This is the kind of thing people in electrical engineering do.

[scryder]

As a preface, I’m not saying that those are inherently bad choices per se, because the underlying ideas are obscure enough that anyone who knows them at this moment probably knows other things you care about.

That said, never forget that ultimately your filtering criteria leak, and candidates will begin coming in with knowledge JUST about that, because you care.

Furthermore, you’re currently optimizing for trivia obtained in CS education that skilled programmers from other STEM fields and non-traditional backgrounds will lack. Besides people directly working on SSL, very few people need to know about symmetric encryption schemes at any practical level, and at best remember it at the same level I just stated it at, as a factoid. So you’re going to mostly find people who’d know that factoid and who’d remember that factoid; put another way, canned-answer spouters.

[blfr]

If you set up a web server for anything in the last few years you would have picked up something about SSL. Even if you just skim the rationale before copying the configs from Mozilla.

https://wiki.mozilla.org/Security/Server_Side_TLS#Modern_com...

If you used git, you probably even did a rebase. Or at least tried to. Or at least saw it as an option.

These aren't obscure university topics.

[jordanpg]

Right. They were chosen because experienced Java devs should be extremely comfortable with them. Not just familiar, but basically experts. And if they're not, it's a big red flag that casts a shadow over really anything else they have to say.

I'm sure each language/technology has a small list of universals that experienced practitioners should be quite comfortable with. Of course there are exceptions, it's not perfect, but my point is that these things are a good proxy for estimating overall competence level.

[rhizome]

By what measure are you prioritizing Java+git over e.g. Java+Perforce in terms of universals?

[hackinthebochs]

And I bet those are exactly the kinds of things you're interested in and could answer. What about people who just aren't interested in maven dependency management (seems awfully uninteresting to me)? If your process is set up to find people who think like you and are interested in the exact same things you are, your process is broken.

And just to be constructive: you have a nugget of a good idea, that a technologist should have fairly detailed knowledge in some domain of interest. Finding what those areas are and getting them to speak about it beyond a superficial level of detail might be a good hiring criteria.

[jpindar]

And that's one of the things done by companies in other businesses, who don't seem to have the same sorts of hiring problems that software companies do.

I'm an electrical engineer, and I've never been asked to design something from scratch in an interview. I have been shown existing design drawings or products and asked questions about them, which leads to various discussions. What we do in interviews is talk, both about what the company works on and about what I've worked on.

[dceddia]

I agree with your idea of a proxy. Others seem to be interpreting it as asking trivia questions about your favorite pet projects and technologies. I don't think that's what you're doing.

In any given specialty, I think good developers will pick up the stuff around that specialty. Java devs will learn about Maven and git and maybe some JVM customizations to tweak memory usage. Rails devs will learn about which gems are widely-used to solve specific problems, and git, and maybe some devops/deployment stuff with SSH keys and Capistrano.

The particulars matter less than that there's some amount of holistic knowledge of their specialty. Of course one can argue the specifics (what if they've used svn but never touched git, etc)... but if there are no specifics -- if the candidate only knows their specific language of choice but nothing outside of it -- I would argue that they're probably not that great.

[p-funk]

As a candidate, I'm the most excited when there are short contract/work sample coding challenges that are problems directly related to the job I'm applying for, and I wish more companies did them than whiteboard stuff. It tells both parties the most information out of all of those options. To the employer: "Can this person do the job, or not?" Pretty simple. To the candidate, I think a coding challenge can tell you a lot about the company and what type of work you're going to be expected to do. I think it weeds people out much more effectively than whiteboarding. Making time for this part of thing is part of the job search, the same as scheduling an onsite.

[yladiz]

On the third point, I think that of the four counter arguments, that one is the least convincing. If a candidate is applying for a new job and are asked to do a work sample or an exercise, but they can't find the time (generally exercises I've seen are between 3-5 hours long, and usually you have a week to finish them) then why are they applying for a new job? You could easily find 3-5 hours in a week to complete some exercise, even if it's broken up into multiple sessions of coding, if your goal is to work at a new company.

[marcosdumay]

You are missing tests. Either short-timed coding or written tests.

The best way is probably a mix of some of those. Yet, I have no idea what mix :)

[Headlight]

For what it's worth, we also think work sample assessments are the most accurate representation of whether a candidate will succeed (and there's a good deal of industrial psychology research that backs this up).

If you or anyone else is interested in a platform for technical work sample assessments, come check us out! www.headlightlabs.com

[bootsz]

> I often feel that engineers write these posts because they know that they themselves are good at their job, but feel it is silly to have to develop an alternative set of skills in order to signal that they are a good hire.

It does feel silly. But sadly it's often the way the world works. Standardized tests carry many of the same problems and benefits. Getting a perfect SAT score doesn't really speak to your level of intelligence as much as it speaks to your ability to dedicate lots of time and focused energy toward preparing for an arbitrary system of evaluation... which turns out to correlate somewhat well with being successful in school. There are few false positives but many false-negatives, just like with whiteboard-style coding interviews.

Is this fair? Maybe, maybe not.

What bothers me more than anything is that people either don't realize this distinction, or aren't honest about it. I was once rejected from a "big four" company after a whiteboarding on-site, and I was advised by the recruiter to "spend a year working somewhere with a very large codebase" and try again... As if that would contribute anything toward my ability to pass a whiteboarding interview. We all know the real way to pass whiteboarding interviews is to practice whiteboarding interview problems. A lot. I've been working as a software engineer on "very large codebases" for a while now and can confirm that I am no better prepared to pass that interview.

[rf15]

I remember reading another story on HN this morning from someone who applied to the big four (well, five) and got accepted by all of them. How they did it? By optimising solely for the hiring process and little else.

And, to be honest, I wouldn't hire this person - they are so focused on optimising towards the METRICS that they ignore what abstract goals the metrics are trying to measure. And that can easily come back to bite you as an employer; you can even see this behaviour if you look at what weirdness ML systems can optimise towards without the right constraints. It's a natural and easy way to do things, but it probably doesn't make good developers.

[rafiki6]

I quite disagree. This to me is a rather capable individual I'd hire simply because they are a good adaptive problem solver. Problem: Stupid interviews. Solution: game the system to get what that person wants, a job.

[JamesBarney]

Every large dysfunctional project I've been on has been run by "good adaptive problem solvers" who prioritized metrics over the success of the project.

Honestly all the best managers I knew who ran the most successful projects were all bad "adaptive problem solvers". Who never really played the classic game so instead focus on project success over politics.

[jbattle]

In the back of my head I wonder though - Once you hire them, will their goal be to optimize the same metrics you also care about?

[rafiki6]

As with all things in life, what you reward is generally what will be optimized for. So make sure your corporate culture doesn't reward dumb metrics.

[caconym_]

> And, to be honest, I wouldn't hire this person

So... you wouldn't hire a person who came in prepared and performed well according to your metrics and standards? Or you have no metrics or standards, and just throw the chicken bones to make a decision for each candidate? Or, I guess a third option might be that you do not tell the candidate anything about your interview process before they come in, all but guaranteeing that they cannot do any meaningful preparation?

I'm having trouble seeing why a candidate working hard to prepare for an interview could be considered a negative signal, which seems to be what you're saying. Correct me if I'm wrong, by all means.

[seanmcdirmid]

You wouldn’t want to hire them, but you probably would as they optimized themselves to your interviewing process. You’d have to add something to your process to weed them out, but then how could you tell? As hiring processes are all necessarily heuristics, they can all be gamed in some way.

This is why referrals or at least reference checks are so important.

[chrisbennet]

I agree with rafiki16, the candidate "hacked" the interviewing problem - great! The stuff you bone up for a technical interview isn't going to hurt your work performance, it just an adaptive response to a broken interviewing process.

[whatyoucantsay]

Don't hate the player. Hate the game.

[crdoconnor]

There is. Take a set of realistic, representative tasks from your day to day work, decouple it from the context as much as possible to limit the need company specific services, software or knowledge while performing the task.

There's a few reasons this is rare:

* Setting up a test like this requires prep work, whereas a chat, recycling old tests and downloading questions from the internet does not.

* It's rare for hirers to evaluate or be evaluated on their own hiring process.

* Lots of developers want to cargo cult Google's hiring process so they ask you to whiteboard algorithms.

[brango]

This is what we do. We created a small programming test that's very representative of 80% of our work. Candidates are asked to only spend 2 hours on it. If they can follow simple instructions and write a reasonable implementation we'll have a face-to-face.

This saves us a huge amount of time and I don't feel it's overly burdensome on candidates.

[crdoconnor]

>I don't feel it's overly burdensome on candidates.

I do. After doing a couple of these "2 hour" projects from companies I have a suspicion were not even really serious about hiring and not even receiving a courtesy rejection email I swore off them forever.

These days I just point the company to my github and am explicit that if that isn't enough to grant me a face to face I probably don't want to work there.

[tn_]

Someone sounds pretty entitled. You know how burdensome it'd be to verify that your repos were fully yours and how much energy an employer would have to exert to make sure everything was connected/architected well vs using a template I'd be already familiar with / created, that could be slightly altered to avoid it circulating the internet.

2 hours is not unreasonable for a take home project with the entire internet at your disposal. People need to be motivated to join the company they're applying to! (Anything more is definitely unfair for the candidate and doesn't scale well when reviewing code)

This notion that 2 hours is a lot of time, well plenty of engineers would rather have that than waste a few months memorizing algorithms they'll rarely use. It's pretty common for algorithm tests to be an hour long anyway.

So many software engineers have it backwards. Companies don't work for you, you're not even in the door yet. Unless you're a vp or principal engineer with a stellar bg, your tasks are replicable and most employers aren't going to be drooling over to get you on board as entitled / piss-poor attitude is going to cause more friction than it's worth.

[_jal]

Eh. There is this thing called a market. Sometimes producers have the whip-hand, sometimes it is the consumers. We have markets in labor, and anyone who (for instance) weathered the 2000 dot.com bust knows, employers in tech don't hesitate to use it when the wheel turns.

I refused a homework project for one firm[1]. It was a highly specific problem that only made sense for one specific version of an app server, and was really a lot of work with some tricky edge cases. Call me entitled, but after I made sure I understood what he was asking, told the interviewer where he could put his homework.

I think it is important to keep a sense of the power dynamic. It is really easy to start moralizing to the powerless (here, employees) when they do find themselves with bit of leverage for a change. Not only is it a bad look - punching down is for insecure assholes - systems break down without feedback and (sometimes) pushback.

[1] Well known, I'm not naming names because it probably was a fluke.

[crdoconnor]

>Someone sounds pretty entitled.

Maybe the guy who wants me to volunteer 2 hours of my time before deigning to have a conversation with me?

>You know how burdensome it'd be to verify that your repos were fully yours

I've never interviewed anybody who lied about the provenance of their repos. I'm pretty sure if I did, it would come out in a 5 minute conversation during the interview.

If one or two gifted liars slipped through the hiring funnel into the interview stage before being found out I wouldn't view it as a tragedy.

>This notion that 2 hours is a lot of time, well plenty of engineers would rather have that than waste a few months memorizing algorithms they'll rarely use.

2 hours is fine for an interview, but it's way too long to spend on a speculative application for one company.

[jasonlotito]

> 2 hours is fine for an interview, but it's way too long to spend on a speculative application for one company.

So... you limit yourself to no more than 4 hours effort applying to any company? Doesn't that limit your choices greatly?

[dceddia]

I think "2 hours" was in quotes because maybe many of these assignments are nominally 2 hours but realistically require 4-6. Just speculating though.

[ryandrake]

Consider that you're interviewing at 20 companies: What if they all did this? That's 40 hours, or an entire work week of unpaid work, for the chance at getting to the next round. Seems unreasonable to me.

[tn_]

I never hand out assignments like this (or have in-person interviews w/ a similar task while having the internet at their disposal too) unless the person's already sent in their CV/Resume, had a phone-screen and they seem like a good prospect.

Regardless, you're going to have to put in time unless you're already trusted by one of the seniors/leads/managers.. if that's the case then there's no coding assignment.

If you're interviewing at 20 companies concurrently you should probably not interview at so many companies at one time and narrow down who you would most likely want to work for.

[brango]

> These days I just point the company to my github and am explicit that if that isn't enough to grant me a face to face I probably don't want to work there.

I'd be happy with that if the projects looked they were yours. I can count on one hand the number of candidates who have had GH profiles, and even fewer with anything worth looking at. All I really want to see is a few examples of coding style and basic thinking.

To me, being an engaged participant in the open source community is a massively positive signal, even if it's just reporting issues and the very occasional PR.

[osi]

agreed. if it is a "2 hour" project, have that be part of the on-site interview (with the expectation that people might not complete it)

[crankylinuxuser]

I applied to one position here on a HN job thread a bit back. Looked right up my alley. Doing IoT, 3d modeling, circuit design, and glue code. So, I did the initial phone screen. Then they gave me the 'zinger': we want you to make a webapp for us. We expect this will take 8-12 hours (?!).

I responded back, that's billable time there. That would require me to take PTO at my current job, and do your work unpaid. And that, I will not do.

I kindly told them where they could put their job. (They still advertise jobs on the monthly, but they seem legit aside that onerous 1.5 days of work. And no, I won't mention whom. Hopefully they'll rethink their policies, but alas..)

[ryandrake]

On behalf of many, many people: Thank you! An 8-12 hour unpaid work assignment is an entirely unreasonable request.

[brango]

The thing is, as gilleain says, the majority of promising-sounding candidates aren't worth progressing with. This is a combination of:

* Failing outright to do anything even close to the requested task

* Not even basic error handling

* No tests

* Using generated code where it's not appropriate, or not bothering to implement e.g. catch blocks they've generated

The instructions explicitly tell candidates what we're looking for, so it's not worth chatting to someone without doing this filter.

[expertentipp]

Take home assignment is basically an inquisition-style "accuse yourself" situation. Implemented the requirements? Sorry, there are no tests and we took them as granted. Skipped non-trivial part of the task? Sorry, the implementation is incomplete. Used XHR? Sorry, Fetch API is the thing. Used Fetch API? Sorry, async/await are the thing. You sent URL to a repository? Sorry, we wanted an attachment. You sent the code as an attachment? Sorry, our spam filter threw it away. ENOUGH.

I'm applying simultaneously to tens of companies, if you insist I can send you a link to my repositories with all take home assignments I've done so far.

[rhizome]

Just a month ago I had a company task me with implementing an xSV parser with four cases, the fourth being "arbitrary delimiter." I implemented all cases except this last one (only because of time) and included the tests and benchmarks offered as "extra credit."

Ghosted.

[logfromblammo]

The best jobs I have ever had spent less than 2 hours total in on-site interviews. The worst companies that I never heard from again burned a whole day running me through the gauntlet.

[hinkley]

Last interview cycle I got 10 hours of homework total from three of my prospects.

It’s not that it’s 2 hours, or four. It’s that you’ve given two hours to twenty people and so have five other companies.

[gilleain]

We do the same. It's amazing to me how many people with superficially impressive CVs do one or more of:

* Cheat

* Fail to follow simple instructions

* Write no tests for there code (even the given examples!)

* Just horribly fail to implement an algorithm

All in all, it's just a filter to be used before proper face-to-face interviews.

[crankylinuxuser]

> Write no tests for their[sp] code (even the given examples!)

Well, I've been on the fence on this one. With imperative languages, testing is pretty much a requirement because of side effects and scope-creep (nahh, throw it into global).

Tests can be a good sanity check for simple checks like "I fixed a bug where X is supposed to give 22. Lets verify that"... But the problem is the tests are then code that can update and rot as main code changes occur.

I've preferred a more functional method. I don't want to test - I want to prove a function handles its inputs and provides the correct outputs. Unless there's a good reason for state , I prefer keeping functions clean. And even then, with state (say, a tally function for bandwidth), I prefer to have the variable updated with its own function. Keeps clean/nonclean separate.

Now.. one thing that's absolutely not acceptable is not commenting code. "Magic sequence in perl doing 6 things unintuitively" is not code comments. :(

[kbenson]

One person's "Magic sequence in perl doing 6 things unintuitively" is another person's "I actually know Perl, and the standard functions being used here, so this is nothing special".

Comments are good, but what's aceptable as a comment is entirely subjective to someone's competence in the language in question.

For example, if you can't look at a Schwartzian Transform[1] in Perl and understand what it's doing, if not immediately than after a moment or two of looking at the steps, then you do not actually know how to program in Perl, you're just muddling your way through. Nothing in the transform is special, and if you are confused by anything in it that's the equivalent of being confused by arrays and simple array syntax in C. A comment of "optimized sort on computed X attribute" should be more than enough, but to anyone slightly unfamiliar with Perl it will seem woefully inadequate.

1: https://en.wikipedia.org/wiki/Schwartzian_transform

[gilleain]

it is java, so yeah. of course production code could have all sorts of levels of tests and qa and all that jazz.

really, though, if i'm implementing an algorithm that i've never coded before it would seem strange not to write at least some kind of verification of its correctness.

comments can be language dependent - perl yes, java maybe (on an API, definitely) - and situation dependent. i wouldn't care about comments on small amounts of code with good variable names and sensible structure, for example.

edit : also apologies for the formatting, i hate typing on a phone!

[ryandrake]

> Write no tests for there code (even the given examples!)

Just curious, do the instructions ask for tests? If one of these "programming quizzes" asked me to implement algorithm XYZ, I would follow it to the letter and implement only the algorithm.

[gilleain]

we don't ask them to actually provide their tests, but implementing a non-trivial algorithm without tests is optimistic at best.

we give them a couple of examples (input, output) pairs - surely you would at least run your implementation past these, right?

[crdoconnor]

I find that the quality of candidates is a function of the quality of the hiring funnel and the money offered.

If your funnel spits out 'surprisingly' shitty candidates then it's probably not the best funnel.

[gilleain]

there are also surprisingly good candidates, as well. this is just the top of the funnel, but agreed that it could be better.

[richmarr]

Absolutely.

I'm not going to name or link, but this is a service we've started providing and have a growing library of assessment techniques (and analytics on the effectiveness of those techniques). This reduces the prep-work required.

The aim is to get to the point where you can say, e.g. "I need a Finance Director for an SME" and we can pre-populate a set of sensible defaults for you to tweak if desired. Baby steps though..

Once we build out those reports we'll also be able to report back to senior management on hiring manager performance. Metrics like typical time-to-hire & volumes etc. but also whether there's unusual demographic bias, the satisfaction of rejected candidates, and some others. I'd also like to make those metrics public (companies would have to opt-in).

[lr4444lr]

I agree, and I'm not sure what these kinds of posts are trying to accomplish, because they're speaking from hindsight and ignore with many other counterfactual possibilities that don't happen to support their point. Every alternative I've seen proposed to FizzBuzz and Whiteboard style tactics is flawed or self-selecting in its own way, and often come at a higher cost to one or both of the interviewee or the company as the candidate pool grows. Cultural fit getting a lot of flack these days for reinforcing unconscious bias is hardly proven fact, but even if it were, it would not be too hard to identify and reach consensus beforehand on what the team lacks, e.g. someone who is a testing fanatic, a security wizard, a deep dive experimentalist, etc. and to instruct all interviewers on how to spot those needed elements.

[richmarr]

> Cultural fit getting a lot of flack these days for reinforcing unconscious bias is hardly proven fact

Talking about things as a "proven fact" (or not) is not a helpful mental model. It's an impossibly high burden of proof that would mean we never learn anything. All we will ever have is a bundle of findings from studies with finite funding and various shortcomings. Right now we have a fair number of studies that found statistically significant increases in productivity from more diverse teams, and (to my knowledge) none showing the reverse. Perhaps there are some but they weren't published.

Is this a conclusive law of nature? No.

Does it seem probable on balance that the types of situations studied experienced the effects reported? Yes.

It's also not useful to talk generally about "cultural fit" because it's broad enough to be essentially meaningless. Companies that hire for "fit" well do so by unpacking exactly what behaviours they're looking for; entrepreneurialism, directness, risk taking, etc. and finding ways to assess those qualities.

If folks want to hire people they want to drink beer with, that's their choice, but failing to unpack the components of culture fit means that their choice is uninformed and unmeasured.

[hueving]

It would be nice if you could link to some of the key studies in this area. Just because there were a few flawed studies is hardly a reason to take it as actionable information.

Has it even been determined that diversity leads to productivity or is it just that diversity is a result of the same thing the productivity is?

[richmarr]

> Just because there were a few flawed studies is hardly a reason to take it as actionable information.

It's interesting that you (someone apparently unaware of the work in this area) is reducing my "a fair number" as "a few" and tainting them as "flawed" when all I did was to acknowledge "various shortcomings". Were you conscious of trying to minimise the weight of evidence without having actually seen it? I'm not judging you for it, it's normal, I'm just calling it out so that you're aware if you previously weren't.

As for whether or not this is actionable, I'm not aware of anyone acting on the basis of diversity carrying a productivity dividend. As far as I've seen (my company works in this space) it's used as a supporting argument to help justify measures to reduce the effect of unconscious bias (a separate area of study) which carries its own dividend, i.e. better employees.

> Has it even been determined that diversity leads to productivity or is it just that diversity is a result of the same thing the productivity is?

That's right, correlation and causation are not the same.

It's been correlated using real world company data, theorised under social capital models, observed under lab conditions relating to jury decision accuracy, and causally shown at a GDP level using computer models. There may be others. There may be publishing bias. There may be flaws. I've already spent enough time writing this so you can google them yourself. These studies each have various shortcomings, eg. self-reporting, scale, assumptions (in the case of the models) and relate to specific contexts rather than generally... so this is a case of the balance of evidence rather than "proven facts".

[wanderer2323]

Have you noticed how you dodged the request to link the related studies by bringing disproportial attention to the rest of grandparent's post in a condescending manner? I'm not judging you for it, it's normal. Just calling you out.

Also can you link what you consider to be representative studies on the topic?

[richmarr]

You're right, it was a little condescending, I was irritated by the way hueving minimised the evidence without apparently knowing anything about it.

(I'm sorry, hueving, that was immature of me.)

wanderer2323, I used to cite studies but no longer consider it a useful way to discuss topics like this on HN. Too often it descends into methodology theatre and too rarely does it result in useful dialogue.

So, with my apologies, no. I won't link to studies. If you're curious I've shared enough background in other comments for you to get started with your own research.

[lr4444lr]

There are two separate claims:

1) Ensuring that factors not relevant to job performance like age and gender are not excluded for in hiring or antagonized in workplace hostility under the vague protection of "cultural fit", because skills, knowledge, talent relevant to business success can be found across demographic subgroups. That's supported in studies.

2) Packing a company with people of those non-relevant characteristics with the assumption that diversity itself is the causal instead of the correlative factor. That claim I believe is not supported, and I challenge you to cite evidence for it, or else to qualify the statement "statistically significant increases in productivity from more diverse teams".

[richmarr]

I've addressed this in a bit more detail in other comments, if you need more you're welcome to look into it.

[lr4444lr]

I don't see citations in your other comments, nor sentiments that clarify controlling for factors identified as occupationally non-relevant. If you're advancing the idea (which I deduce from your other comments) that people who are diverse insofar as they have separately relevant attributes which the individuals themselves perceive as incompatible and deleterious, then I have no argument about it, and can admit I don't know anything about the research specifics in that sub-problem of management. But you should explicitly spell out that this is your claim. Furthermore, that's an internal problem, not the problem of rejecting people at the hiring phase for being a bad "cultural fit". For example, if you don't write tests, "move fast and break things", and feel strongly about that, but we have mission critical code and clients who will drop us for competitors if our software is buggy, you're flat out not going to work here. The "diversity" you bring will be objectively bad for the company, barring some proven god-like irreplaceable ability to deliver value on another front.

[rhizome]

If you're advancing the idea (which I deduce from your other comments) that people who are diverse insofar as they have separately relevant attributes which the individuals themselves perceive as incompatible and deleterious, then I have no argument about it...

This looks like an incomplete sentence to me; what is the "it?" "People who are diverse insofar as [condition] which they themselves [see negatively]" defines a certain (sub)set of people, but the idea, assertion, or conclusion that you're agreeing with appears to be missing.

[richmarr]

I have to confess I don't follow a lot of your comment, but I wouldn't describe whether writing tests (or not) as diversity.

[sheepmullet]

> Right now we have a fair number of studies that found statistically significant increases in productivity from more diverse teams

Are there?

Diversity defined how?

Repeatable experiments or statistical analysis of groups of companies?

It could be studied experimentally and so I am sure it has been.

The studies using statistical analysis to compare across companies are always weak and should be taken with a grain of salt.

[richmarr]

> Are there?

Yes.

> Diversity defined how?

Depends on the study. Usually gender or race.

> Repeatable experiments or statistical analysis of groups of companies?

Depends on the study. See my other comments in this thread.

> It could be studied experimentally and so I am sure it has been.

It has been. See the study into racially diverse juries making more accurate judgements.

> The studies using statistical analysis to compare across companies are always weak and should be taken with a grain of salt.

Everything sould be taken with salt because salt is awesome.

[notyourday]

> Right now we have a fair number of studies that found statistically significant increases in productivity from more diverse teams

People (manager/HR/VPs of Diversity/etc) do not understand the implication.

Here's an intellectual exercise:

Would a team of two people - one being a foaming at the mouth antifa and another being card carrying garb wearing KKK dude be very diverse? Certainly. Would this team be productive? Unlikely.

[richmarr]

I'm not sure your hypothetical is as clear cut as you think it is.

> ... be very diverse? Certainly.

Very diverse?

Okay, for the sake of your hypothetical let's assume that this antifa member has a different educational and socio-economic background to the KKK member. Otherwise they're just differ on one axis and may be identical on every other.

Let's also assume that this anti-KKK person isn't Daryl Davis, who has talked around 200 KKK members into hang up their hoods by engaging with and befriending them. He's an unusual case.

> Would this team be productive? Unlikely.

What's the timeframe? Can we assume that these people have rent to pay and need this job?

Social capital models indicate (and experimental evidence supports) that people in diverse teams feel like they've been less productive (despite actually being more productive). Reduced social capital makes people less comfortable collaborating and more willing to disagree, but the flip-side of that is reducing group-think.

Think how little group-think would be evident between these two individuals, they'd disagree on everything. Yes it would be uncomfortable, but they both need the job so there's a limit

But let's go national with this pairing hypothetical. Southern Poverty Law Center estimates KKK as 5-8000 members. Assuming 8000 and that antifa is similar and all are working-age, that means that in a US-sized of 320,000,000, maybe halved to focus on working age adults, you'd be lucky/unlucky to see many (if any) instances of this type of pairing. Even assuming that all 8000 were paired directly against a mortal enemy that still only represents 0.01% of the total pairs. Even if we assume that each organisation has 100x its official membership in sympathisers that's still only 1% of the pairings (and remember we cheated to make sure they were always matched up).

If we can't assume that these people need their job then that narrows these numbers (and hence the probabilities of occurrence) to include people in well-paid positions or with inherited wealth, or who would rather be homeless than work with someone they disagree with politically.

[JamesBarney]

> but feel it is silly to have to develop an alternative set of skills in order to signal that they are a good hire

The problem isn't that they have to develop a set of skills that combined with their true skills gives the interviewer a glimpse into their abilities.

It's that it's an entirely different set of skills entirely. We might as well be asking people to learn woodworking in order to showcase their software skills. The ability to sound knowledgeable and make someone like you in 50 minutes is pretty orthogonal to writing good code as a member of a team.

[bootsz]

> it's an entirely different set of skills entirely

I'm not sure whiteboarding-style interviews are entirely orthogonal. They are definitely not a measure of real software engineering ability, but I suspect there is decent correlation, enough to give companies a sufficiently low false-positive rate, which is what they care about. The flip side is you have a high amount of false-negatives, and it sucks to be one of the people who end up in that category, because you know you're good enough but you just didn't pass the arbitrary filter that requires that "entirely different set of skills". I've been there, as I suspect most of us have. It's not fun, but it's hard to see what would motivate these companies to change their practices as long as they feel their false-positive rate is low enough.

[JamesBarney]

A standardized white-boarding interview is probably not completely orthogonal. But too many white board interviews don't have a standardized set of questions, with a standardized set of criteria over which to measure them.

And the ad-hoc tests have very little validity, and rely more on what questions the interview asks, and how confident and charming the interviewee is. Not to mention lots of credentialism. "Candidate Joe didn't answer the question but he's a Googler so that I probably just because I did a bad job explaining that." "Jessica didn't answer the question because she didn't know anything about software".

[rlpb]

> I often feel that engineers write these posts because they know that they themselves are good at their job...

The Dunning-Kruger effect suggests that almost certainly, some non-zero proportion engineers who have written this type of posts are not.

https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect

[custos]

While what you're saying is true, I think the dunning kruger effect has an inverse bell curve based on level of confidence vs knowledge.

High confidence low knowledge. Moderate confidance moderate knowledge High confidence high knowledge.

Of course that only applies to the things the expert knows like the back of his hand. There are things I am 100% confident I know, because I've done it thousands of times. Then there are some advanced things I still question if I truly understand because I've only successfully done it a few times.

Of course when I was younger, the things I'm 100% confident now I went through the phase of "oh I can do this super easy" to "holy crap this is harder than I thought" to "I've done this 1000's of times, I got this"

[rlpb]

> There are things I am 100% confident I know, because I've done it thousands of times.

If you've done thousands of job interviews (for example), does that make you an expert at job interviews? Or a terrible candidate?

Now: how many people who pass every job interview immediately subsequently write an article about questions to ask at a job interview?

[dahart]

Dunning Kruger is only valid for simple tasks. The study tested, for example, whether participants get a joke. The participants, btw, were all Cornell undergrads, NOT the general population.

The "Dunning Kruger effect" actually reverses and there's a negative correlation for highly skilled and highly knowledge-based tasks like "engineering".

http://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-k....

"...some studies have shown that the asymmetry reported by Kruger and Dunning (i.e., the smaller discrepancy for high performers than for low performers) actually goes away, and even reverses, when the ability tests given to participants are very difficult."

So please don't treat Dunning Kruger as a way to claim anyone who's wrong or you disagree with is suffering from inflated confidence and lack of skill. Do note the potential for confirmation bias as well as inflated confidence in the very claim that others might be prone.

[ksk]

>The Dunning-Kruger effect suggests that almost certainly, some non-zero proportion engineers who have written this type of posts are not.

The effect can't suggest anything, you would have to test your own hypothesis on your own sample set of people writing those posts. Extrapolating from other studies on other sample sets used for completely different inquires is bad science.

[logicallee]

>If there was a way to evaluate a candidate based on how well they would do at their actual job, in an hour or less, that couldn’t be gamed or cheated, we’d all be using it.

Right, because there's currently a ten million dollar bounty up at way-to-evaluate-a-candidate-based-on-how-well-they-would-do-at-their-actual-job-in-an-hour-or-less-that-cant-be-gamed-or-cheated-bounty .com - which everyone knows about and has known about for decades, and checks regularly. The site can't be gamed and it's run as a public service. Literally within a day, and I mean a day, that anyone ever uses the winning method a single time, God himself will write up the method and submit it on the interviewer's behalf (invisible hand hypothesis), the site will judge it correctly and approve it (just world hypothesis), and the next day we'd all be using it. Oh also there's a genie that makes people use the winning entry I forgot to mention that, otherwise people might not all use it even once it's well-known.

I mean just consider the wildly implausible set of circumstances it would take for not everyone in the world to be using the best interview process in the world!

It's clear that it just doesn't exist. How could it?

/s

-

Edit: someone didn't like this. Just imagine what method you suppose by which everyone would learn the perfect interview method from each other. If that method exists, why would everyone know it?