Sorry, but they're crazy if they think halfway decent engineers are going to spend 7.5 hours of their time doing a coding quiz before they even have a personal interview with them.
They seem to be forgetting that potential engineers are interviewing them just as much as they're interviewing the engineers -- and such a crazy up-front time commitment is a sure way to weed out the engineers who have better things to do with their time.
This may be scalable for the company, but it certainly isn't scalable for engineers looking for jobs at multiple companies.
Here is an idea. Why not pay candidates for their time? e.g.
"We are serious about who we hire, so we're careful about who we do hire. However, we realise you're interviewing us as much as we are you, so we're equally as serious about showing respect for your time. We really appreciate it, so we'll pay you $m/hour while you complete this tests. We think these three tests should take a competent engineer around n hours to complete, so we'll pay you a max of $m*n for completing the process.
Good luck!"
As the OP highlighted hiring is an expensive process (expensive engineers interviewing candidates, double whammy of them not working on product while interviewing), it could be a lot cheaper to just automate some of the process, but make it more humane by compensating the interviewee.
There is a definite balance to strike between the humanity and scalability aspects, but I think it's an interesting idea.
We've actually done this before. Our process has evolved over time. We used to bring candidates in for a day to work with us and have them work on production code. We would pay them for their time.
Unfortunately, it wreaked havoc with the candidate's schedule, especially if they had another job, and took too long to get to a pass/fail for them.
The current process let's the candidate work on a production problem, but at a time of their choosing and on their own schedule. It also gets feedback to them much sooner, within a half hour or 3 hours.
We do that and works really well for us. Because it's not only the technical chops we want to test. We want to know how well integrates with our workflow, the quality of the questions they ask, how well they document what they are doing, etc.
We've automated the technical skills part of it. Hiring at Instacart isn't a completely automated, no personal interaction, process.
In between the candidate applying and the technical challenges is usually a phone call where we talk to the candidate to make sure we're on the same page. We find out what their goals are and if we can help. We explain how we work, and see if they'll like it. We explain the hiring process. We answer questions they have about us.
If we're both happy at the end of the call then we move them into the automated process.
We have an off-line coding test that probably takes about a day on average. This is after a three-question quiz that takes a couple of hours. I'm pretty surprised that so many people are willing to go through all of that.
There's no shared risk here, it's all on the candidate. Bad form.
Only something along the commitment of the 0.5-1hr "weed out" question can/should be done remotely.
If you're going to require people to do ~7 hrs of tests, that had better be at the office, pairing, with other folks (and at least give them a free breakfast/lunch/dinner!). 7 hrs while working a 40hr (or more!) week is a pretty hefty commitment, and I see no reason why that should be solely the burden of the candidate.
You should be able to trust one or two short, simple weed-out conversations (I tend to do one 30 minute "deal-breaker" discussion on non-tech stuff {make sure their and our deal-breakers are met}, then one about 1hr tech screen {fizzbuzz-sorts of things, but more diagnostic}).
After that, you should have a general sense of whether or not they are a) smart and b) can get stuff done. From there, you need to take a risk and bring them in. What is proposed is not fair to the candidate.
Yeah, 8 hours of essentially uninterrupted time is a lot to ask -- in fact it would basically be a deal-breaker for anyone with full-time commitments, or who otherwise values their time.
Also, (explicitly) timed coding tests imply that you don't really trust the candidate. Of course they could be lying when you casually ask how much time it took (or whether they got any help), but the few who go that route will probably quickly test out your other filters.
The trust signal, meanwhile, takes a lot of care and consideration to cultivate -- and shouldn't be discounted lightly.
You are right - 8 hours of uninterrupted time is excessive. And, our approach specifically does not ask for that.
If you see the constraints defined in the post - the goal is to have the interview be remote so that the candidate can do this wherever and whenever they want. And, the tests are specifically designed to be not longer than 4 hours long each. Perhaps what wasn't clear is that this is not in a single session.
Interviews can go for as long as a week - really depends on the candidate's schedule.
Our process 'weeds out' candidates as soon as we know that its not a good fit. This in reality saves the candidate ( who may be interviewing at other places as well as working like you mentioned ) as well as our time. The 7 hour number keeps coming up, and seems like a lot, but very few candidates actually spend that much time. And, most of those are actually hired.
We've tried conversations as well, but unfortunately, it is very difficult to objectively measure them into what will make the candidate successful at the company (interested to learn how you do it).
This. As a company, you miss out on the best candidates by implementing an interview process that serves your needs almost exclusively (at the expense of the candidate's desires).
Why didn't the engineers sit around the room and say, "Why do only 1 of 100 candidates that we interview meet our bar? Are we sourcing the right people?" or "Why does it take us 8 hours of someones engineering time to figure out if we want them to meet the team?".
Asking the best candidates–the ones you really want to hire–to do a full day of work just to be able to meet the team is insanity.
Full disclosure: Founder of Mighty Spring (https://www.mightyspring.com) — we're solving exactly these sorts of problems (for both companies and candidates) by turning a truly great recruiting interaction into a web service. Invites are sent out regularly.
EDIT
Sorry. If you can't implement linked list and think that first item is too much for an interview, HN is not the place to swing around. Keep downvoting lol.
TL; DR version:
It isn't fair to throw a company out of a windows because they are serious about their hiring process. It totally depends on their company's growth need! First bullet point is extremely fair but the rest aren't and I agree on that.
Longer version:
A lot of hackathon projects become startups and many of them don't require any algorithms. They can use libraries from Python and Ruby to do things they want to do. They can get away with a couple n^3 in their code and customers are okay as long as the service is stable.
And then there are startups out there building products using real algorithms that really require the knowledge of graph and sorting, dynamic programming and etc. If they need an engineer that can build on top of existing algorithms, then that's what they need.
I was a former intern at Mozilla and my interview was very simple and I really appreciate that. But what I do doesn't require me to know any sophisticated algorithm. I can survive with knowing Python and getting around with my brain. But interns on Firefox, servo and Rust probably had to face harder interviews, more tedious algorithmic types (although I heard most of those are basic too, like linked list and talk about C++).
I am also looking for new internship this winter and I fear these algorithmic interviews too, but some companies have serious need and they need people who can at least implement a linked list is important. I don't bother to read what their needs are.
I did roll my eyes when I looked at "3 hrs", "4hrs". That's a lot. But the first item: you don't even know how to write a linked list using your favorite language? Then that's the end. Linked list is not hard - you are not writing in C++ in this case.
The 7.5 hr is a different story, but I won't trash at the first point.
"Implementing a linked list" is essentially measuring 1 of 2 things:
1. How long it has been since you graduated from college. The further away the less likely you will remember how to do it, because you will never do it after college.
or
2. How often you have interviewed for other positions, because that is the exception to 1.
I'm over 10 years out of college. Last time I implemented a linked list was probably in my second year of a CS degree.
I still remember and will probably never forget how to at least psuedo-code a linked list. But you are right, I have never implemented it after college.
However knowledge of pointers, lists and b-trees have been very helpful multiple times throughout my career.
> However knowledge of pointers, lists and b-trees have been very helpful multiple times throughout my career.
Indeed. And such glorious moments they are when choosing the right data structure drastically improves performance.
However, if I were interviewing for a Rails developer position, the last thing I'd anticipate having to bone up on is algorithms and data-structures. I'd be expecting to be asked about... well, doing stuff in Ruby on Rails.
There is a reason why we have this thing called "time". You don't suddenly apply for a job do you?
If you were senior, 10+ years why would you be interview at this company? I supposed there are better positions out there for you.
Doesn't make sense with time. I didn't remember how to implement linked list after my class but I picked it up again over the summer as hobby to learn basic algorithm again.
I never said that. I said if people think the first bullet point is too much to handle, that's blatantly stupid. It isn't like the company is asking you to implement a red-black tree. I am not a fan of puzzle solver so I hate that kind of interview. But linked list? Come on.
it's like asking someone "Well, explain how list is implemented in Python" when you claim you have been programming with Python years. Well, if you can't even answer that question with a good estimate (it has a double space every time you create a list for example) then you probably shouldn't call yourself a Python guru.
"Not the place to swing around"? Firstly, there's plenty of entrepreneurs here who don't code at all. Secondly, that's a really silly thing to say. Do you honestly think that you can't be a great Rails engineer without having fluency with linked lists off the top of your head in an interview type situation? I can think of a thousand questions that would give me a much more accurate idea of how good a candidate would be as a web developer and the day to day issues they would face.
Sorry, that's not silly at all. You are taking the wrong context and you are fooling yourself thinking it is unimportant to question abililty.
Linked list is the simplest data structure one can pick it up in 5 minutes in using languages like Ruby and Python.
Do you honestly think any serious entrepreneurs would want a company growing hire someone who can't even tell you the basic of a linked list (can you imagine one cannot even tell you what a linked list is and the time complexity?)
You didn't even read my comment at all. A lot of things can be done by integrating with the ecosystem out there; but many of the libraries out there are horribly written and there are things you just can't build out of these libraries you find on PyPI.
People who build successful products cannot rely on an ecosystem forever. There are crucial components must be built by engineers using data structures and algorithms we know from textbook and then write custom algorithms that work for the use case.
Too bad. Some people just want to get away with ecosystem all the time; truth is you just cannot.
If a candidate never heard of linked list, wtf does that say about this candidate? Why would I want to hire him if he didn't even attempt to study a few one? I don't expect anyone to write a damn red-black tree algorithm; I can't and I doubt few people is capable of writing that algorithm from scratch (or even remember half of the properties).
Wow, what a douchebaggy post. I had good impression of Instacart as a product, but clearly Instacart as a company doesn't deserve that level of respect. They've completely failed to assess their potential hires on what actually matters.
Unfortunately, I've done a 5-7 hour "project", only to not hear back from the company for a month - and then it was from a recruiter, not a developer, who just told me that they had decided "not to pursue further".
I decided at the time that I would never do this again, and that's still more or less the case. I might be willing to make a deal - if someone from the company is willing to put a workday into one of my open source projects, even just trying it out and writing up the experience, I might be willing to spend a workday on their programming assignment.
Or, of course, if my family was starving and I was unemployed, I'd do it.
One huge difference here - they do say they'll give detailed feedback. If I had gotten detailed feedback on my programming assignment instead of crickets chirping for a month and then a recruiter call with a one-sentence brush off, that would have taken quite a bit of the edge off. I do give them credit for this.
But still, they are mainly looking for a way to use a candidate's time to their own at an 8:1 ratio. For me, this is a "no apply" condition.
Am I missing something here? The title says 1 hour hire but the problem listed says 7.5 (rails test: 0.5, backend: 3, frontend: 4). That seems anything but efficient and somewhat strange.
Why not provide the potential candidate with some actual work that you need done?
Why not 2 days, then? Maybe he should complete an entire project for free - why not, for a "several year investment?"
Personally, I think it's great when companies not only advertise these kind of practices with pride, but then send staff out onto the net to defend them against a veritable tide of criticism. It will certainly help you narrow your pool of applicants, and that is clearly what you are after - it says so in your "how to hire a human" flowchart! Haha.
I know of few non-entry level developers of any reasonable skill level that would seriously consider this one-sided of an interview process unless there was some clear potential reward. Getting a job at instacart hardly sounds like something to get excited about. It sounds like they've 'hacked' themselves out of consideration by many talented programmers.
Algorithms are easy, the solutions are all online. Cleaning up code is hard. Give the candidate a project with some failing tests and messed up code. Tell them to clean it up and add a feature or two. Congrats, you just tested someone on what they will actually be doing in their day job.
Fully agreed. We look at code cleanliness as part of the review. And, the algorithmic questions are not problems that you'd normally find online. They are custom problem that we've had to solve at Instacart
"A rejected candidate should know unambiguously why they didn’t get the job."
I'd have to say that's my favorite thing about this process. All too often applicants go away with no idea about what exactly why they weren't a good fit. If someone gets rejected, the least you can do for them is to tell them how, so they can improve themselves.
I know companies and HR have been trained to not answer these types of things based on potential legal ramifications.
I've had luck emailing after an unsuccessful interview and saying "Thank you for the opportunity. Informally, is there anything I can work on so that I could be a successful candidate in the future?"
Exactly. If the team just couldn't see me fitting in to their group. Fine. No problem at all. But please tell me so I don't lose sleep thinking I may have bombed the technical interview.
That statement is pretty much antithesis to everything anyone has ever said about hiring amazing engineers...What's ironic is, a Rails interview test is actually best for people who are willing to learn Rails and demonstrate that they can solve your problem with your tools. It's not really great (and potentially offensive) for people who will most likely be able to pass those tests in their sleep.
You shouldn't be hiring people because they have experience that matches your tech stack solely because they'll kick ass on the first week.
You should be hiring people that will kick ass 3 weeks in, and a year from now, and 5 years from now -- regardless of whether or not they currently have competence in the nitty gritty details of whatever libwidget you use at this exact moment.
"hacking" this, "hacking" that.
the word is being way overused. just because you came up with a solution you think is good for you doesn't mean you 'hacked' anything.
Always. About a year into the job you realize "solve challenging problems" part of the ad really just meant you're going to battle self-inflicted organizational and maintenance stupidity and office politics.
Everyone wants rockstars. But very few companies are worthy.
Maybe it works for them, but I don't find any motivation to work on test project that exist only for their own sake. I mean, if the test project is actually a part of the real project that I would be working, then that's okay. But to build something with made up requirements puts me off.
The challenges are all real projects that we've already built (and thus have a baseline for).
We don't agree with brainteaser interviews. Having someone build a made up project would be just as bad as a brainteaser. We wouldn't do that to a candidate.
It is a lot less time than you calculated. We stop the process any time they don't pass and give them specific feedback on why. Usually, this is during the initial phone call/basic test. Most candidates do 30 minutes at most, which seems very reasonable when applying for a position.
That's a good question. I hope this example gives some clarity to what I meant:
* Worthless Brainteaser: How many golfballs will fit in this Ford?
* Real-world Problem: Let's do cohort analysis on our users and see how our business is doing.
Part of the goal is also a consistent rubric for evaluating candidates.
In the past, we've tried giving every candidate a new problem to solve. It's not fair to them. When each problem is a brand new project it's difficult to evaluate a candidate objectively and consistently.
> The candidate will join us in the office towards the end of the day, where we’ll spend time getting to know each other over dinner or beers.
Really?
The technical portion of the interview process described here is receiving a lot of well-deserved criticism, but I feel this is worth pointing out as well because it's another example of how out-of-touch this company seems to be.
Any experienced candidate knows The Rule (don't drink during the interview process, even at a social event), so an employer that intentionally puts a prospective employee in the position where he or she is effectively asked to do so is incredibly unthoughtful. And foolish to boot, as this could very well become a legal liability.
On the contrary, we're very particular about hiring. We would rather go without an engineer than hire someone that will break the average team quality down.
The biggest waste of time is that candidates have to demonstrate general competency to every company he or she is considering. It's a waste of engineering resources checking every candidate and a waste of time for the candidates. Is there a way to unify that across companies (other than by creating a certification scheme)? Is there a way to standardize the code sample procedure?
Most, if not all, of these companies do not articulate what "the bar" is that candidates have to meet, or at least do not do so publicly or as part of the job req, so no, I don't think there is currently a way to unify that across companies. "The bar" is largely a euphemism for an information asymmetry in a company's hiring.
Basically "the bar" stands more as an internal psychological mechanism than for any set of objective criteria. More about providing frustrated to feel good about themselves -- "we're soooo selective, that's why we can't find engineers that don't suck" -- rather than actually sit down and think about what what they might be doing wrong in their hiring process... or (probing further) why their company may not be doing anything all that interest in the first place.
As if, you know, the candidates don't also have a "bar" that companies have to muster up to. Yet if a candidate where to write a blog post musing that "only 1 in 100 qualified meet my bar", you probably wouldn't want to work with them.
There's always at least one idiot who responds to a perceived imbalance in a particular domain by going to the opposite extreme. This is not a good solution to the problems with current tech interviews.
The problem with this approach is that it is unnecessary time consuming on the applicants side and that the tests aren't language agnostic. The goal to reduce the in-house effort is reasonable, however it can be achieved a lot easier. You can filter the 90% of applicants that are not qualified in tests that only take a couple of minutes. In my opinion the goal of automation in the hiring process should be limited to filtering out the obvious misfits, otherwise you risk to miss hidden talents among your applicants.
In the end you rather want to hire a brilliant engineer with little experience in the desired technology stack than a mediocre developer that is able to pass your automated tests because he has worked with the technology for years.
We (Heap) would use Manchuria if you were to open source it. We currently have a multi-tab spreadsheet. My previous employer had an awful Salesforce system which was about as fun as eating sand.
I wish more places had a policy of giving detailed feedback... it's bad enough getting rejected, but the stone-faced manner in which most places seem to do it makes it even worse.
The lack of respect for candidates is kind of frightening, given the current job market. What I'm hearing is that a successful candidate (one who goes through the whole process) is 1/8 as important as a current employee (based on the hours they're willing to put in). This sounds like it will do a great job of filtering candidates - assuming, of course, that you're trying to filter for employees willing to put up with abuse from their employer.
In Sydney and London it was common to hire a contractor for a short term, put them to work on your project, and if they are any good offer them a full time position.
As long as this is made clear in the interview, I see no problem with this process.
Attempts to get free labor out of interviewees have already been tried by other unethical companies in the past, and most seasoned developers know not to fall for this.
They seem to be forgetting that potential engineers are interviewing them just as much as they're interviewing the engineers -- and such a crazy up-front time commitment is a sure way to weed out the engineers who have better things to do with their time.
This may be scalable for the company, but it certainly isn't scalable for engineers looking for jobs at multiple companies.