[intelliot]

> Interviews are weird: the pressure of time, and not being able to look things up, distorts the code.

In the interviews I do, I tell the candidate that:

1. There is no time pressure. Work at a normal pace, as if you were working here. This is not a speed test. I don't expect you to finish. I mainly want to know how you think.

2. You should look things up. Behave the same way you would when coding at home. Use Google, Stack Overflow, documentation, etc.

This probably works better for the interviews I give because the problem is not implementing an existing algorithm. It's a realistic task, something that we've actually built on-the-job.

[PopeOfNope]

Does anybody believe you when you say those things in a job interview, though? I wouldn't. A job interview is fundamentally a competition against other candidates. If you have two candidates and one finishes the task in half the time he's more likely to get the job. Time is a factor in everything. Same with looking things up. It's considerably more impressive to be able to recall minute technical details without having to look them up.

The resulting cognitive dissonance adds more pressure to the situation instead of relieving it.

[shubhamjain]

> A job interview is fundamentally a competition against other candidates.

That's hardly the case. Hiring for programming is different from hiring for a cashier. The programming world, at least, if you hiring for good programmers, would rarely have a queue of candidates lined-up. You get tons of resumes daily, sift through BS, find someone good, get to know him more and after some rounds, you hire him if he is worth his salt.

Rarely you would hear the phrase, "We hired the other candidate because he was skin-of-teeth better than you." Judging from the lack of engineers, a company would be very lucky to have more than one really good candidate for a position.

[metaphorm]

this is exactly my experience as well. having very recently completed a round of interviews for new hires, I experienced this directly. we wanted to hire 2 engineers. we interviewed about 20. found 4 that we liked. 2 of those 4 took other offers at other places they interviewed. we hired the 2 (that we liked, they have been good hires so far!) that were left.

there was never anything close to anybody getting "cut" in favor of another interviewee that was fractionally better than they were. we fully intended to make offers to everyone that we thought was good.

[markburns]

> Does anybody believe you when you say those things in a job interview, though? I wouldn't. A job interview is fundamentally a competition against other candidates.

I think you are being a bit negative. Why not believe the interviewer? It seems pointless to trick candidates by suggesting they are safe to look things up, then secretly judging them worse than other candidates. I would judge more harshly someone who didn't ask if they could look something up, or assumed they couldn't. Particularly so if they had been told they could but then proceeded to struggle and not look something up or ask the interviewer. I'd rather someone I may work with have a relaxed attitude and inherent curiosity rather than a paralyzing fear of failure.

Also viewing it as fundamentally a competition is focusing a lot on the negative perspective from your side. An interview is about finding a good match between a company and an employee. It can be just as much about you finding somewhere you want to work as it can about them comparing you to other candidates.

There may not even be other candidates, and they could be trying to make you comfortable and give you an idea of what it would be like to work together. Maybe suggesting that they do understand the cognitive dissonance and they do understand that interviews are stressful and hence no need to add additional pressure.

[com2kid]

> Does anybody believe you when you say those things in a job interview, though? I wouldn't. A job interview is fundamentally a competition against other candidates. If you have two candidates and one finishes the task in half the time he's more likely to get the job.

Not at all.

I have one hour to decide if I want to work with a candidate.

How quickly someone works? Sure, that is important.

How good is their sense of humor? Can they joke around? Can they be involved in a technical deep dive on a problem? How about coming up with test cases?

I have 60 minutes and one programming problem. I've hired people who've taken from 5 minutes to 40.

[muddyrivers]

-- "I mainly want to know how you think." I totally agree with you on this. I hope my case could help contribute to the implementation details.

People have very different working styles. Since interview as a process can't be tailored to all of them, it is rationally refined to suit for the majorities. For the minorities left, I have to say one has to adjust, trying to go through the 5-hour drills.

I, unfortunately, is one of the minorities. Given a hard question, one has to think first. I am the opposite of thinking aloud. In real work, I either stare at walls, or look at the vast void far away, silently. I might have a piece of paper, drawing odd shapes and graphs, because I am primarily a visual thinker. In the interview, it would definitely look odd. It could be awkward as well if I completely fall into silence, ignoring the interviewers.

So I would face to the white board, wrote a few lines of pseudo-code, uttered some words, while trying to get my inner self into my real work routines. Sometimes it works relatively well, sometimes it failed completely, especially the interviewers kept talking that requires my interaction. In many cases, I understand the interviewers were trying to help, giving my hints, while the effect was the opposite of their intents.

[lukaslalinsky]

I think you are not really a minority. Most programmers I know work like that. I realized that this is why I can't really do job interviews. Not only are the problems there not real, so I don't care that much about solving them, the expected way of solving them is very unnatural for me. I'm more than happy to explain the problem in great detail to anybody after its solved, but explaining the things as I go usually does not lead anywhere.

[phamilton]

If you think exactly like I do, you may not be the ideal candidate.

However, if your approach is so radically different that I can't see how you got there, you may not be the ideal candidate.

Don't worry about being different in your approach. It is, however, very important that you are able to keep someone on the same page.

Edit: autocorrect

[agumonkey]

Repeatedly it's been written how the biggest minds found solutions at random times, and more often far from their workplace.

At my next interview, I'll bring a bed, ideas come when I lay in and stare at the ceiling.

[kevrom]

Mine always happen while in the shower. Me and that interviewer are going to come out of it a lot closer.

[danieltillett]

I have to say I also get some of my best ideas in the shower - walking is also good especially with overcoming block points.

[agumonkey]

It's quite weird how wandering/walking is such a common way to open your mind. I also found that when going abroad I would be filled with creativity. Maybe being removed from my home habits and games. Or maybe because the foreign context stimulates your brain. Somehow it seems like physically changing your perceptions triggers opportunity for your brain to change its view on abstract things too.

[danieltillett]

So much of your thinking and habits are tied to your immediate environment. For example, people find it much easier to overcome addiction when they move to a new location where all the triggers for their addictive behaviour are removed.

Problem solving is similar in that your current environment constrains your thinking. Getting up and changing your environment changes the constraints on your thought processes. The bigger the change in environment, the more constraints removed and the greater the opportunity for creativity.

[madeofpalk]

We use a variation: the scenario is that us, the interviewer, is a junior developer and we've submitted our code for code review. Then later we ask the candidate to add some trivial functionality.

We've found this works really well. There are unit tests, so its always a promising sign if they go for them first. Depending on what they pick up, change or ask questions about, lets us a lot about the candidate.

[julianz]

Yep we've used a page of code from one of our systems and asked the person to talk through what they can see, what it does, what they would ask further questions about and so on. It almost instantly separates people who know a bit about code from those who don't, as the latter will literally just stare at the page for a while before giving up.

[Corrado]

I really like this idea and think it would work wonderfully for me. Plant a couple of easy-to-spot errors in there as well as some more difficult optimizations and see who finds them. :) It takes the pressure away from writing new code, doesn't require a computer, and still allows you to see the thought process of the interviewee.

[mianos]

I wish, I did a google code interview once. The guy said we had only a short amount of time. When I solved the first one really quickly he told me, 'oh, ok, can we do another puzzle we don't have much time'. Suffice to say I went down in flames somewhat like the topic.

[voltagex_]

Can I interview with you? Normal tech interviews make me physically ill.

[eru]

You might also want to talk to Matasano. They are really keen on work sample tests.

[jerematasno]

Note that our work-sample tests are, not-insanely, done in the comfort of your own home, at your own pace, on your own schedule, and represent the work we actually do. As co-head of recruiting for NCC US (aka head of recruiting for Matasano), I think the answer to the in-person interview question of "write code to do this," is "thank you for your time, I'll see myself out."

[serve_yay]

I don't think that's good enough. It's unfair to ask candidates to write code on the spot.

[dataminer]

I have had my share of bad interviews but I don't agree with this sentiment, why is it unfair? Haven't you ever been in a situation where you have to create a hot fix and deploy it as fast as you can. Haven't you ever gone through a crunch period where the deadline is closing in and and you have to fix that elusive bug. There is limited time for everything, the sooner we realize that the better.

Having control over your nerves when s* hits the fan is a valuable quality, which we should all strive for.

Most of these interviews are tests to see your approach to solving a problem. The best practice is to come up with a strategy before you hit the first key, ask as many questions as you can and find as many corner cases as you can. All the sane interviewers will appreciate that even if you don't end up with the best solution. And please treat all interviews as a learning experience, you will be much better off.

[learc83]

I must've been working in a much different environment than you. I've never had to solve a difficult problem in 15 minutes while someone who knew the solution watched over my shoulder and did their best to answer my questions without answering them too well. Ohh yeah and the results of my performance had potentially life changing consequences.

The adversarial and extreme time boxed nature of a coding interview is something that is truly outside the day to day experience of the vast majority of programmers.

>Having control over your nerves when s* hits the fan is a valuable quality, which we should all strive for.

That may be true, but think about that for a minute. Go back to college and think of how many people in the class were comfortable going up to the board and solving problems in front of everyone.

When I was taking Automata, we could get an extra point on our final grade by going to the board and correctly working out a problem that we hadn't seen before. Only myself and 2 other people ever did it--in a class of 60.

Do you think that your company is solving problems so hard and paying so well that you can can only consider 3 out of every 60 developers (or whatever the real ratio is) who are otherwise qualified who have also mastered control of their nerves far beyond what is required 99% of the time on the job? Maybe if you're Google, for the rest of us we need to find a better solution.

By the way, even though I was able to solve problems at the board, I hated every minute of it, and I refuse to work for companies that require this kind of interview.

[nilkn]

> I've never had to solve a difficult problem in 15 minutes while someone who knew the solution watched over my shoulder and did their best to answer my questions without answering them too well. Ohh yeah and the results of my performance had potentially life changing consequences.

So you've never taken an exam in your entire life?

I don't mean to be snarky. I just don't understand this extreme disdain for coding interviews, even when they're very forgiving and flexible. Yes, exams aren't fun, but they're necessary.

Even if you can look up whatever you want, use whatever language you want, take as long as you want, do it right on a real desktop (not a whiteboard), and if you're asked a reasonable, real-world problem, not an algorithmic brainteaser, you'll still have people say it's a horrible process and it's unfair. And that's already way more forgiving than any exam I ever took in school.

I mean, are programmers expected to not be able to do anything at all in an interview setting? Like you can't be expected to produce any code of any kind, as if you don't know how to program at all? I think it is a bad sign if someone who is a talented programmer utterly buckles under a little bit of pressure. To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

So what, if I hire you and at some point need you to hotfix something in, is that unfair? Maybe I do know how to do it myself but don't have time because I'm busy with something else. That situation doesn't seem all that different from the interview, except it would actually be higher pressure because there's a real problem, not a fake one.

[learc83]

>So you've never taken an exam in your entire life?

You missed the context where we're talking about a professional setting.

>And that's already way more forgiving than any exam I ever took in school.

You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

>To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

And you'd be wrong. I've worked with very talented programmers who are absolutely terrible at interviewing. The problem is you're biased by your experience. You are probably good at this kind of interview and you are probably a good programmer, so you assume that any good programmer must be a good interviewer.

I know for a fact that these 2 processes are orthogonal. Books like cracking the coding interview exist because it is possible to practice and game the system. I've met just as many people who are good at interview questions, but are terrible software engineers, as I have people who are bad at interviews and are excellent software engineers.

The fact is that our current hiring processes are broken. Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this. Newspaper writers work on deadlines, but they don't have to write an article on the spot while someone watches them type.

Studies have constantly shown that work sample tests are really the only form of interview that shows real predictive power (that and general intelligence tests). The closer you can make the work sample to real work, the better. Finding the nth item of some arbitrary sequence in 15 minutes while someone watches your every move is not close enough to real work to have much predictive power.

>So what, if I hire you and at some point need you to hotfix something in, is that unfair?

If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there. I'm also probably very familiar with the domain, and it's not some artificial problem that I may have never seen before. But more than that, why don't you see the difference between a hotfix and an artificial situation where someone who already knows the answer is standing there watching your every keystroke, and judging your value as a programmer?

There are better ways to interview programmers. Here is one:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

[Fargren]

Going on a tangent, but:

>You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Yes. Is this not common? I've had a number of teachers who, as a graded form of evaluation, would ask us to present a topic to the class, and/or ask questions about it. This was extremely common on high school, and happened once or twice in the more advanced courses in my college. I also had to defend my thesis to graduate, which was basically a more lengthy version of this.

I live in Argentina. Maybe it's a cultural thing.

[nilkn]

> You missed the context where we're talking about a professional setting.

I didn't miss it. I just don't see why it's relevant.

Why does your ability to handle that situation differ so greatly depending on whether it's an academic or professional setting?

> You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Well, when we give technical interviews we let the candidate sit at a desktop and we don't particularly pay much attention to what they're doing. We definitely don't just stare at them. I know that at some popular companies like Google they may use a whiteboard and they might (but might not) pay more attention as you write your code. I don't really advocate that.

I know that's not always the case, though; I interviewed at Palantir once, for instance, and one of the interviewers ignored me completely until I was ready to present my solution.

That said, in school there were in fact a few occasions when I had effectively an oral exam. There also were occasions when I needed to do some work on a chalkboard in front of several people (if not the entire class). There were some occasions when I even needed to deliver a 45 minute highly technical presentation to the class and that doubled as an exam.

> Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this.

Well, my roommate's a chemical engineer at a major oil company. For his current job, he had to give a presentation to the entire interviewing team about his technical work at previous internships. It's not an exactly isomorphic scenario, but I'm sure they paid a great deal of attention to every single thing he said.

Lawyers have to take the bar exam, which I imagine is easily just as hard as an interview at Google. It might trigger social anxiety less, but that won't last long -- by the nature of the profession, you'll be in front of the court eventually.

Doctors have to take similar exams, equal in difficulty (if not greater), plus they are observed extremely rigorously during labs and clinical work.

> If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there.

It's not that regular of an occurrence, but it would be a sort of big deal if you were totally unable to act in that sort of situation. There would be a limit in what sorts of projects you would be given.

"Life and death" is an exaggeration, though. It's not any more life and death than a 15 minute quiz.

> Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Are you suggesting that you can perform well when you have someone helping you, but you can't when you don't? Or is it just an issue of social anxiety?

[Retric]

Context is vary important in problem solving. If your coding a quick fix for something and there is a code frease in 20 minutes your not grasping at straws wondering do they mean 100 numbers list or a 100 gigabyte list etc.

[jeff_marshall]

I sort-of disagree.

If you aren't judging whether they get a correct working solution (i.e. passes some unit tests after running it through the compiler/interpreter), and actively engaging them on their thought process while they attempt to find a solution, it can be a useful tool (esp. in early filtering with really trivial problems). Interpreting the results, and picking the coding problem, are not easy tasks.

That said, I've certainly found myself in the position described by the story, and all I can say is that I'm glad I've never actually needed to find a job in my career (new opportunities have always come from people who already know me and my track record). You may (though I wouldn't bet on it) get fewer bad "hire" decisions by doing a code interview like the one in the article, but you also get bad "no-hire" decisions. I'd love to see some empirical data on the subject from someone who knows how to design an experiment.

[eru]

Why? Isn't writing code exactly what you are interviewing for?

[learc83]

How often are you asked to solve a problem in 15 minutes while someone watches over your shoulder?

[eli]

Seems like you'd have a similar problem with almost any job interview for any type of position.

[learc83]

Ask your friends who aren't software engineers. How many of them solve problems they've never seen before while someone watches over them during an interview? The vast majority of other professions spend their time talking about past projects.

For instance all my designer friends show off their portfolios, then maybe they get a take home work sample. My writer friends submit samples of their work. And my engineer friends discuss past projects they've worked on.

[eli]

I take your point, but actually I work in media and a lot of my writer friends had to take writing or editing tests as part of their interview.

[fapjacks]

Maybe if you give the candidate four hours. But in the hour or so that is ubiquitous among individual interviews (even if repeated four or five times with different interviewers), you aren't going to be any meaningful information out of anything the interviewee will have time to code. An hour is enough time to write something trivial. Anything that really pushes a developer's skills is going to need a big part of that hour just for them to put pen to paper for design. The best interviews I ever had involved a week-long spare-time development project at my own pace. Those are the only interviews I ever was able to really show what I'm capable of. Incidentally, my interview at the Googleplex I thought was worthless.

[eru]

Interesting. They'd need to pay me quite a lot of money for a weeks worth of effort.

[fapjacks]

Well, I would say the total tasks were tasks taking about a handful of hours, but the deadlines were about a week out from when they were given. So essentially, they were spreading the "normal" five or six hours worth of interviewing with developers and doing trivial tasks (that I think don't really show the interviewer what kind of programmer I am), but compressed into one project, meanwhile time-wise spread out across a week. I've interviewed maybe five times like this (most recently twice last year). Definitely I felt like the code they got was a good representation of the kind of code I write normally. I would never agree to working for a week for a job interview without pay. Though most recently I had an interview where the guys didn't even ask for my resume once. They only ever looked over my Github, which for me was a nice touch, and made me feel like those guys know wtf they're doing. I should mention here that I interview for fun continually, otherwise I would have taken the one the guys offered who didn't even ask for my resume.

[jmilloy]

Fairness is definitely not relevant here.

But, an interview technique can be useful for a company or not. Asking candidates to write code on the spot isn't inherently useful or not useful. Assessing a candidates typos and compiler errors smugly is probably not very useful. But assessing their approach to understanding expectations, understanding problems, and solving problems is useful. If a candidate just assumes the expectations without asking in the interview, they are likely to do the same on the job.

[perdunov]

Having no time limit is actually more stressful than to have a reasonable time limit. No time limit is basically "as fast as you can".