[learc83]

>So you've never taken an exam in your entire life?

You missed the context where we're talking about a professional setting.

>And that's already way more forgiving than any exam I ever took in school.

You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

>To me, that means they either really aren't nearly as talented as they think they are, or they won't be able to deal with the pressure of the job anyway.

And you'd be wrong. I've worked with very talented programmers who are absolutely terrible at interviewing. The problem is you're biased by your experience. You are probably good at this kind of interview and you are probably a good programmer, so you assume that any good programmer must be a good interviewer.

I know for a fact that these 2 processes are orthogonal. Books like cracking the coding interview exist because it is possible to practice and game the system. I've met just as many people who are good at interview questions, but are terrible software engineers, as I have people who are bad at interviews and are excellent software engineers.

The fact is that our current hiring processes are broken. Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this. Newspaper writers work on deadlines, but they don't have to write an article on the spot while someone watches them type.

Studies have constantly shown that work sample tests are really the only form of interview that shows real predictive power (that and general intelligence tests). The closer you can make the work sample to real work, the better. Finding the nth item of some arbitrary sequence in 15 minutes while someone watches your every move is not close enough to real work to have much predictive power.

>So what, if I hire you and at some point need you to hotfix something in, is that unfair?

If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there. I'm also probably very familiar with the domain, and it's not some artificial problem that I may have never seen before. But more than that, why don't you see the difference between a hotfix and an artificial situation where someone who already knows the answer is standing there watching your every keystroke, and judging your value as a programmer?

There are better ways to interview programmers. Here is one:

Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Repeat this process if necessary.

Get everyone together and talk about past projects the candidate has worked on. Have the interviewers evaluate the candidate, and make your decision.

You know whether the candidate can code, and you've removed the adversarial nature and unnecessary stress from the interview process. Most importantly, you got to see them work in a much less artificial environment that's closer to what they'll be doing day to day.

[Fargren]

Going on a tangent, but:

>You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Yes. Is this not common? I've had a number of teachers who, as a graded form of evaluation, would ask us to present a topic to the class, and/or ask questions about it. This was extremely common on high school, and happened once or twice in the more advanced courses in my college. I also had to defend my thesis to graduate, which was basically a more lengthy version of this.

I live in Argentina. Maybe it's a cultural thing.

[learc83]

Were you asked to actually solve new problems you hadn't seen before in front of the class?

> would ask us to present a topic to the class, and/or ask questions about it.

This is common in the US as well, but presenting on a prepared topic and solving novel problems are 2 completely different things.

[Fargren]

I've had to do both, but presenting on a prepared topic was much more common. I don´t recall ever being graded on a novel exercise presented to the class, but I certainly have had to solve them.

[learc83]

I've had professors that like to require writing on the board as well. But as in your case the key difference is, they weren't graded.

That being said, a professor student relationship isn't analogous to an employer employee relationship. Hopefully an employer wants to make the interview process as enjoyable as possible to attract as many qualified applicants as possible.

[Fargren]

I agree, the relationship is not analogous at all. That's why I said I was going on a tangent: I was more interested in learning about the evaluation system in the US than it's possible parallels with job interviewing,

[nilkn]

> You missed the context where we're talking about a professional setting.

I didn't miss it. I just don't see why it's relevant.

Why does your ability to handle that situation differ so greatly depending on whether it's an academic or professional setting?

> You took exams in school were the professor was looking at what you were writing the entire time? Your exams were in front of several people on a whiteboard?

Well, when we give technical interviews we let the candidate sit at a desktop and we don't particularly pay much attention to what they're doing. We definitely don't just stare at them. I know that at some popular companies like Google they may use a whiteboard and they might (but might not) pay more attention as you write your code. I don't really advocate that.

I know that's not always the case, though; I interviewed at Palantir once, for instance, and one of the interviewers ignored me completely until I was ready to present my solution.

That said, in school there were in fact a few occasions when I had effectively an oral exam. There also were occasions when I needed to do some work on a chalkboard in front of several people (if not the entire class). There were some occasions when I even needed to deliver a 45 minute highly technical presentation to the class and that doubled as an exam.

> Other professions don't conduct interviews like this. Go talk to some mechanical engineers who've been working for 10 years and ask them if they interview like this.

Well, my roommate's a chemical engineer at a major oil company. For his current job, he had to give a presentation to the entire interviewing team about his technical work at previous internships. It's not an exactly isomorphic scenario, but I'm sure they paid a great deal of attention to every single thing he said.

Lawyers have to take the bar exam, which I imagine is easily just as hard as an interview at Google. It might trigger social anxiety less, but that won't last long -- by the nature of the profession, you'll be in front of the court eventually.

Doctors have to take similar exams, equal in difficulty (if not greater), plus they are observed extremely rigorously during labs and clinical work.

> If you're regularly giving me life or death tasks that I have 15 minutes to solve, I don't want to work there.

It's not that regular of an occurrence, but it would be a sort of big deal if you were totally unable to act in that sort of situation. There would be a limit in what sorts of projects you would be given.

"Life and death" is an exaggeration, though. It's not any more life and death than a 15 minute quiz.

> Pair the candidate with an interviewing engineer, give them an hour or two to solve a problem as a team. The interviewer isn't an adversary; their job is to assist the candidate like they would if they were working on a real problem.

Are you suggesting that you can perform well when you have someone helping you, but you can't when you don't? Or is it just an issue of social anxiety?

[learc83]

>Why does your ability to handle that situation differ so greatly depending on whether it's an academic or professional setting?

I didn't say it did. But just because something happens in a university doesn't make it the optimal way to conduct an interview.

>I don't really advocate that.

That goes a long way towards alleviating the problems that most people have with this type of interview.

> For his current job, he had to give a presentation to the entire interviewing team about his technical work at previous internships.

Talking about past experiences is completely different from solving problems under pressure. Almost all professions require this, and I'm at a loss as to why it's not good enough for software.

>There were some occasions when I even needed to deliver a 45 minute highly technical presentation to the class and that doubled as an exam.

Again not really the same thing at all. Delivering a prepared presentation is an entirely different issue. And if public speaking like this is required for the job, I see no problem in including it in an interview.

> by the nature of the profession, you'll be in front of the court eventually.

Yes, you said it, that kind of thing is an essential part of the job for lawyers, not so much for software engineers.

>plus they are observed extremely rigorously during labs and clinical work.

Yes this is true, but this happens when they are just starting out. No one is going to take a surgeon with 10 years of experience and force him to do an operation for an interview.

>It's not that regular of an occurrence, but it would be a sort of big deal if you were totally unable to act in that sort of situation. There would be a limit in what sorts of projects you would be given.

Again, I have never once encountered a situation where I had 15 minutes to solve a complex problem that had never seen before.

>Are you suggesting that you can perform well when you have someone helping you, but you can't when you don't? Or is it just an issue of social anxiety?

I'm not suggesting that. I do fine in technical interviews the same as I did extremely well on exams in school. However, even though I often made the highest grade on an exam, I hated every minute of it, and I feel the same way about technical interviews.

Here's the bottom line. I do fine in technical interviews, but they cause me more stress than they're worth to me. You may be fine with this kind of interview, but from the comments on hacker news a very large percentage of software developers are not. Many people hate them, and their predictive value is dubious.

The interview format I outlined is objectively more similar to the day to day work of an average developer. Studies have show that work sample tests are the best way to judge a potential employee. Why not strive for a better interview process?

It really comes down to this. If you can eliminate a section of the interview that many people dislike, which isn't critical to the performance of the job, and it will increase the tests predictive value, why wouldn't you do it? Do you want to hire people who are good at interviews or do you want to hire people who are good employees? If company A depends on an interview process for which 50% of otherwise qualified interviews will perform well on, and company B has an interview process for which 60% of otherwise qualified candidates will perform well on, which company will perform better in the long run?

Current technical interviewing techniques are demonstrably terrible. Their predictive power is terrible. The only argument is whether there is a better way to do it. I think there is.

[nilkn]

> I didn't say it did. But just because something happens in a university doesn't make it the optimal way to conduct an interview.

I honestly think you're sort of dodging the question here, maybe not intentionally.

> I'm at a loss as to why it's not good enough for software.

Probably because you can say anything you want to impress people, which means that as an interviewer you can't really trust anything they say and must probe them very deeply about any work they've claimed to have done. At that point it's already a sort of technical interview.

We actually use a mix of this. We don't require a formal presentation but one of the interviews involves having the candidate discuss past work in detail.

We definitely would not feel comfortable eliminating coding interviews on the basis of that one interview alone. In the past, we've hired some people on the basis of things like this, ignoring poor results on the technical screen, and those have been our worst hires.

> No one is going to take a surgeon with 10 years of experience and force him to do an operation for an interview.

Well, a surgeon's literal every move is being observed during every single operation. And to get to that point, the surgeon was basically subjected to excruciatingly rigorous daily oral examinations for 5-7 years during residency.

> they're predictive power is terrible.

Honestly, we haven't had many issues with it, so I don't know that I could agree it's terrible.

Actually, and I'm not exaggerating here, every single one of our worst hires has come from giving somebody the benefit of the doubt when they didn't do well in the technical interviews.

---

At the end of the day, I agree with you wholeheartedly that we need to find a way to improve the interview process. I just don't think the current process is that terrible, especially when you get away from large, lumbering places like Google and look at smaller, more forward-thinking companies. Google has absolutely zero ability to adjust to specific candidates.

I understand your frustration with being experienced and still being subjected to technical interviews on BFS or whatever. It's annoying and sometimes even insulting, and in the worst case it demands that you go spend an inordinate amount of free time brushing up on interview material that never actually shows up in the real world. So I'm very much with you on the inadequacies of the current procedure; I just don't think it needs a drastic alteration.

I think replacing whiteboards with actual computers, algorithmic brainteasers with real-world problems, and extreme timeboxes with more open-ended formats are the main changes that are needed. Just let someone sit at a computer and write a program to do something fairly common. And don't stare them down while they do it. I really, honestly do not think this is that unfair or crazy.

It seems a lot simpler than and just as effective as doing a pair programming event where you have to make sure that both people have not seen the particular task before, which basically means the interview will have to be completely customized for each candidate. And that's not necessarily a good thing -- it means it's pretty hard now to compare two different candidates.

[learc83]

>Honestly, we haven't had many issues with it, so I don't know that I could agree it's terrible.

If it works for you, then it works for you.

The one thing I know for sure is that peer reviewed studies over the last 50 years show that work sample tests are the absolute most predictive tests you can perform.

If what you're doing is as close to a real work sample as you can reasonably get, then you're probably doing better than 95% of companies. And from your description of your process it sounds like you are.

>It seems a lot simpler than and just as effective as doing a pair programming event where you have to make sure that both people have not seen the particular task before, which basically means the interview will have to be completely customized for each candidate.

The pair programming isn't necessarily essential in my opinion. The essential part is eliminating the artificial adversarial and timeboxed nature of the traditional whiteboard interview.

>it means it's pretty hard now to compare two different candidates.

That's actually a pretty key insight. The practice that many companies have of allowing interviewing engineers to use any problems they want is completely opposed to this.