Hacker News new | ask | show | jobs
by lettergram 3294 days ago
That's actually how I conduct interviews. It's been immensely successful at weeding out candidates within 5 minutes; although most interviews are 45 - 60 minutes.

Basically, give them less than ten lines of code, ask them what it does, where are a couple bugs, ask what would you name the function, etc. Then we talk about how to improve it. I'd say, less than 20% of people I interview pass. It's actually amazingly low how many people can find a bug and communicate it.

Half the people don't even tell me what they are thinking. And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down. They think in their head, don't work through the problem at all.

12 comments

"Half the people don't even tell me what they are thinking."

Maybe they are introverted or simply need to think before they talk. Around 50% of people are introverted (less in usa). Many people are like that and simultaneously quite skilled. Moreover, some environments punish errors, so people who worked/studied there tend to be conditioned to think before talking.

"And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down."

You are not buddies, you are interviewer about to decide whether they get hired. Many people shutting down might mean that they are not comfortable juggling "buddy" social role and expectations and "serious job interview" social expectations simultaneously.

"You are not buddies, you are interviewer about to decide whether they get hired. Many people shutting down might mean that they are not comfortable juggling "buddy" social role and expectations and "serious job interview" social expectations simultaneously."

Very good point.

Right -- you don't need the "act like a buddy" part for this technique to work.
I'm not sure about introverted. I am, but when I get excited about a topic, I can be as loud as the next person. As long as I feel I have something to contribute, that is. I can rant for two minutes straight and then suddenly and unexpectedly shut up when I've made my point. People are sometimes surprised and ask why I suddenly stopped talking. I'll answer: "I've made my point."... :3

So my bet is on error avoidance.

I'm pretty similar, but solving a bug in an interview question is hardly something I'd get excited about.
I agree. Still, the point I was trying to get across is this: While introverts do not strive to take center stage for its own sake, they are perfectly capable to do so for any good reason. Introversion is not shyness.
Being an introvert isn't an excuse for poor communication. Regardless of its cause, poor communication is bad in a work environment, and it makes sense for interviews to select against it.
This is significant, and can be more so depending on the company and environment.

I've had my own team members tell me they aren't engaged, don't speak up in meetings, etc. because they are an introvert. Like it's a condition we must accommodate. Bullshit.

Communication is a skill. We value clear communication in our company. If you join our company, you may not be an expert in communication, but you will grow that skill.

An interview is a different scenario, but still -- you need to be able to communicate with me. If you can't, we will pass on you as a candidate.

"don't speak up in meetings"

People should speak in meetings when they have something to say. Not just so they speak - that just waste everybody time.

The more important point is that if 80% of people shut down while they are talking to someone, maybe has more to do with that someone then 80% of population being uncommunicative.

We're not quite that naive and clumsy. I'm the first one to make sure we keep discussion on-point and actively shutdown the monologuers of our team.

So let's clarify -- "speak up in a meeting" means when a senior person doesn't provide input at a time that would provide better direction, identify issues earlier in a process, etc. If you're a senior engineer, I need your input to ensure we're doing the right thing(s).

The forum of a group meeting isn't always in the wheelhouse of someone who freezes up when speaking in front of others. It's understandable -- I used to be one of those people. It can be out of your comfort zone, and doing so feels super-risky as well as plain frightening. It is still just a skill to be learned, no different than building muscles through exercise.

In our group, we talk a lot about trust and support. Everything we do is in the spirit of making each other stronger. Everyone has strengths and weaknesses -- share your strengths with the group, and build your weaknesses from the strength of others. It sounds pie-in-the-sky, but as the head of my group, I make sure we take this seriously.

So we don't have 80% of our group shutting down -- nobody talks over the top of anyone else, and we don't allow it. We need the input of the best of those, and sometimes that involves speaking up. If you're on my team and that notion makes you nervous, I understand. But as I've always explained to my team, we're going to help you build skills, and communicating is one of them.

If you want diversity you try to accommodate different personaility types. Otherwise you end up with a mono culture.

There is value in silence and limited communication.

Communication is a behavior and act, not a personality type. The suggestion here is that it's ok for someone who doesn't communicate well to be accommodated in the name of diversity. The notion of "poor communicator" as a personality type doesn't jive with me.

Replace "communication" with "data modeling" or "project management" or "reading". There are plenty who don't necessarily possess those skills.

Should we expect that others shouldn't develop skills in those areas (assuming they are important for their role) in the name of "diversity"?

While I don't fully agree with the parent of your reply, I think they're right about communication being a skill. I am an introvert myself, but I've learned not to let that get in the way of me communicating with my colleagues or friends.
Not speaking up in meetings doesn't mean they don't have this skill though. Some people (like me) don't like to add on-the-spot opinions on something, but prefer to go and mull it over, look into it etc before providing feedback. Doing it on the spot often leads to a lot of wasted time and overly long meetings. Meetings and other synchronous communications are, in my opinion, very wasteful and expensive.
Thinking in your head before talking is not poor communication through. Not being able to explain yourself is, never talking is and shooting every half baked idea loud while others concentrate is.

The fact that someone takes time to put together ideas before opening their mouth is not bad sign. Bad sign would be if they don't share result once they have done thinking.

This is interviewing 101 though: establish a report with the interviewer if at all possible. If the interviewer tries to be a buddy that is in your favor to go along.
rapport
TIL Rapoport's rule, https://en.wikipedia.org/wiki/Rapoport%27s_rule

Oh I do love the interwebs.

I'm an introvert, hates social gatherings, etc. Yet I've found the ability to think aloud extremely helpful. Sometimes just vocalizing my thought makes the thinking clearer, almost as if hearing myself again makes the brain think over every thought once again. In a collaborative situation (such as pairing) this is even more helpful.
I believe it does exactly this. The rise of "rubber duck" debugging is a tool to extract this sort of thing, in fact.
Generally to get candidates who are like this to open up, I simply assert:

"Tell me what you are thinking about right now"

"What is jumping out at you for you to be stalling?"

"Did you see something?" --> "Why?" --> "You seem to be quiet and deep in thought, what are you thinking about?"

etc...

Yeah, that's great if you're NT and all that, but if you're interviewing someone who is anxious about the situation this might only add to the pressure, creating a melt down situation making it impossible for them to think.

In a normal work environment where he or she is left to think freely the the same candidate might excel.

I'm sorry, but this is an interview. It's entire point of existence is to allow communication.

If the candidate would like some quiet time to think about the solution, I expect them to respond with "I think I'm forming a solution, just give me a couple of minutes to think about it before I present my case".

If they can't even do that, then I'm sorry, but they are no good to anyone. You can be the smartest and best developer in the world, but if you're completely incapable of representing yourself and your ideas; to have a dialog about your work, you are effectively worthless as an employee.

Some companies might have a place for that special someone, who you can lock in a room for a month and he will later emerge with an amazing new piece of code that will solve your problems, shielded from all the problems of the outside world, but companies where that is possible are very rare.

Some companies might have a place for that special someone, who you can lock in a room for a month and he will later emerge with an amazing new piece of code that will solve your problems, shielded from all the problems of the outside world, but companies where that is possible are very rare.

Also, being able to develop like that is very rare because in the real world, requirements change or are vague and need to be clarified, or your software has to interoperate or integrate with other software. There is simply no way to avoid regular communication and still produce something that works how it needs to. Problems where you can shut yourself away for a long period of time certainly exist, but in the grand scheme of software jobs, they're few and far between.

And on a normal workday in normal surroundings that person might do just fine. An interview is a totally abnormal, high pressure situation.

The point is that with this kind of make or break style of candidate vetting you're probably leaving a lot of great talent by the wayside.

Thanks, yes, that's exactly my point.
Unfortunately interviews are brief, the point of this isn't to add pressure its to relieve it. Silence for too long in such a short period introduces pressure. Letting them use me as a sounding board we can create a connection that opens dialogue and allows ideas to flow.

The whole point is to make them comfortable and that takes the highest priority. Being able to read the individual and react to them is key, not everyone is the same, hence why there are a multitude of responses, and the ones given are just a starting point.

Ah, I read you now, thanks. I may have reflected on one interview that went particularly badly, being somewhat reminded of it. But with the extra information, that is a great approach if I understand you correctly - especially the silence adding pressure part, which is absolutely true.

I.e. if stuck at something, even if seeming trivial, it might pay (to go back or have a chat around it, even guide him/her a bit so as, etc) to get the candidate loosened up and feeling safe which should then yield better responses after.

You can bikeshed interviewing processes that leave out this or that potential candidate forever. And any candidate who bombs in any kind of interview could turn out to be the next Einstein.

There is no one-size-fits-all interview technique; if there was, it would be used all over the place. Good interviewing techniques are about playing the numbers, not leaving no programmer behind.

That sounds awful, you are actively sabotaging someone by ruining their concentration while they're trying to solve a bug in unfamiliar code under time pressure. This is like having a manager requesting status updates every 10 minutes when you're trying to fix a production failure.

Even if I solve the problem I'm walking out of that interview with a very negative opinion of your company.

If you completely shut down during the interview and don't communicate, you're probably getting walked out from the interview. If you cannot or will not communicate, then the interviewer is unable to evaluate you.

I also don't think he was advocating nagging either, simply asking questions to get the candidate talking. If you "go dark" for 10 minutes, you're pretty much definitely stuck. Talking to the interviewer might get you unstuck. Staring at the whiteboard quietly probably will not.

Yep, totally not nagging, giving them room to think and then jumping in to get them talking is definitely what you want to do here.

This also isn't your day to day job so you have to keep in mind that these interviews are generally held to an hour length and you can't really give them 30 minutes for a single problem that wouldn't give you much insight into the person.

It also heavily depends on the culture of the company as well. So being able to communicate effectively and act under pressure is something that goes with pair-programming and code reviews.

To note, I have never had anyone ever leave an interview feeling bad about the company or our process, the interview is about the individual.

In one interview, I was left in a room alone with coding exercises. The involved frameworks I did not knew and I had internet available. Then they came in and we talked about solutions.

I still think it was pretty cool way of interviewing.

You might be over-estimating the complexity of the kinds of problems generally used for this.

You're not bothering people while they're taking an expected amount of time to look at it. You're stopping a several minute silence after handing someone a basic loop and if statement.

For most positions we're talking about, this should not be a major problem. I found when doing something similar people either looked at me like I was crazy for giving them such a basic problem and after being reassured there was no catch answered simply and clearly, or just couldn't do anything. Followup questions about the code then went terribly (e.g. a JS contractor asking for £500+/day who didn't know what the difference between global and local variables was, why you should put var in front of things, etc).

These things are often good to weed out the incredible number of people who seem to have little to no coding ability at all for jobs where that's an obvious pre-requisite. I've also had people fail to solve a basic problem (roman numeral generator or reader) at home in whatever language they wanted, not even getting vaguely close. One person even sent in their broken version pasted into a word document.

If someone can't express himself, most likely that person won't work well in the team.
Some people take a bit of time and familiarity to warm up.
True. I am not sure if I want to take the risk though. I don't want to be a friend or tell jokes but i want to be able to talk technical stuff.
There is an enormous difference between a person taking a minute to collect their thoughts without speaking and a person being unwilling or unable to communicate. I can't be sure, but it sounds like at least some people in this thread are talking about pushing back against the former as well as the latter.
I am fine if they think about it for minutes or need a while to sort thoughts out. But at some point I would like to get some response. What else can I go by in the interview? And even if that person can write great code I am not sure things will work in the long run if he can't explain himself.
> Half the people don't even tell me what they are thinking. And no matter how many times I try to work with them, act like their buddy, or w.e. they just kind of shut down. They think in their head, don't work through the problem at all.

As it happens most of my "aha!" code-related moments come when I'm either under the shower or when I'm washing the dishes. Very rarely did it happen for me to look at a piece code while in front of my computer and then immediately be stuck by what that code does or by some hidden bugs in it. It's all sort of mechanical, at least while I'm sitting at my desk. I'd say it would be very hard for people like me to reproduce that "aha!" moment during an interview, in front of some other people who expect me to have those moments of enlightenment right there and at that precise moment.

You can always find something to talk about in the code, though. You can start by just describing what each line does, in your opinion. Then you can go back and start to group the lines - e.g. "lines 1-3 are the initialization, lines 4-6 are probably intended to flurb the glorb", etc. Yes, it's possible that you can find an interviewee who is great technically but shuts down during an interview. The question is how much time and effort you want to invest digging for that person, especially in light of such studies.
You could look at a more complicated piece of code, let them think about it for a bit, and if they don't have that aha moment you can still discuss it with them. For example, pick a problem you someone on your team struggled a bit with, something that you wouldn't necessarily expect a candidate to solve during an interview, and discuss solutions with them. I think you could learn a lot about the interviewee this way.
omg, are you me. No ahas at interviews. Good to great ahas at random times. Fuck whiteboarding interviews.
> They think in their head

I'm left wondering why this would be a problem.

I don't know if the following is what the GP meant, but:

I've often had the pain of dealing with people who try to do it all in their head. Sure, one or two are good enough where it's not a problem. The rest of them? They keep getting confused and making mistakes. For things that are not very complex. If they had only written it out or drawn on paper/board, they would not get confused.

It's most obvious when I try explaining some code's algorithm to them. They'll begin with understanding, and then 70% of the way through they'll keep getting confused because they couldn't hold it all in their head.

I've often had to tell these people "Don't try to do it in your head!"

I have sort of the opposite problem with a coworker. This coworker only wants to talk through things and doesn't want to work them out on paper or a white board. The problem is that I can understand/remember almost everything I read, but have almost no ability to comprehend things that are being spoken (I suspect people are talking faster than my ability to think and parse what they are saying). I've talked to him about this to no avail. He just wants to talk things through, and then expects me to be following along and to provide insights when I have no idea what he is talking about. But if we can get it on paper or a whiteboard then I can reason about it quite easily.
I know the type of person. I usually can't follow them either (less about them being fast talkers, more about them being bad at explaining things). I found myself to usually only half-listen to what they say, and letting the other half of my mind to reconstruct the problem independently by itself. Sometimes I fail, and then I need to ask them to repeat what they just said. Other times my mental model is spot on, and I can find their problem quickly.
Try to maneuver him to where there's a whiteboard and write things down as he's talking through them: if he's reading cues, he'll slow down so that your visible note-taking keeps pace.

If that doesn't work, get him a rubber ducky [] and tell him to go talk to it if he's not willing to compromise on communication styles.

[] https://en.wikipedia.org/wiki/Rubber_duck_debugging

I worked with somebody like that once. He would get massively frustrated because he needed that back and forth to think through ideas and I was giving him nothing. I would get frustrated because I had to digest information before I could say anything about it and he wouldn't stop talking long enough for that to happen.

I had a lot of success with asking him if I could think about the problem and we could schedule a meeting in half an hour or so. He was much happier with that result because I had thought through the problem and was now able to give him the back and forth he needed. I was also much happier because I was no longer expected to come up with insight on problems I had no time to chew on.

I am often both requesting and receiving verbal problems related to the software I maintain (an Engineering simulation and analysis tool). When receiving a problem verbally is to slow the other person down by challenging any assumptions they may be making. By asking them to clarify it allows me to build a mental model of the problem before at a sustainable pace and also helps them identify a bad assumption they may be making.

When I am asking someone else to be the rubber duck I strive to highlight any assumptions I am making at each step of defining the problem. If I don't do this the other person will usually struggle to keep up.

Can you keep some paper or a whiteboard around?

You don't have to just accept doing your mutual conversation exactly as he likes it.

That is not the opposite problem. Talking through them and working them out on paper are the same thing. They are the opposite of someone doing it in their head.
Not being able to successfully and clearly communicate their thought process about a simple problem one-on-one is a bad signal for being able to communicate clearly about a difficult problem in a group setting or being a useful sounding board for technical ideas.
It's clearly important to be able to communicate about technical problems, but is it really necessary to be able to narrate your thoughts about a technical problem in real time as you're first having them?

In my work experience I don't remember ever having to do that. I'd look at code & investigate a problem independently, then talk to someone about it afterward. I think that should be considered at valid approach to the interview.

It sounded from the example like the code under discussion was quite simple, "ten lines of code". So it's plausible your "look at & investigate independently" is allowed to happen, and should take less than a few minutes, say, and then this interviewer is inviting them to walk through things in a shared manner.

In other words, from my perspective, you're not actually in disagreement with their method. They weren't saying "narrate in real time as you're first having [thoughts]". Practically nobody does that ever, or is expected to do it, except in other practices like mindfulness or therapy and what have you. They were saying they ask them to understand it, explain it, and then talk about improving it. OP's problem was with the candidates whom couldn't understand the code, and weren't even willing to talk about the state of their understanding so that the interviewer might help them walk through things further...

Yeah, but if you shoot for elaborate ones you will get politicians/lawyers not coders.

I don't have any source but from my experience mathematical/computer minded people are more likely to be introverted/talk less then arts people.

>I don't have any source but from my experience mathematical/computer minded people are more likely to be introverted/talk less then arts people.

I think you are right about that, but that just means that if they want to be effective members of a team in an organization, they need to work on their communication skills. Ability to communicate is something you can learn to do better.

I know this for a fact. I used to teach public speaking, and students came out far better at it than they came in. The same is true for many other communication skills. See for instance Marshall Rosenberg's book Nonviolent Communication, which is great for interpersonal conflict situations.

In fact, I think a lot of introversion is at least in part due to poor communication skills that could be improved with training and practice.

I studied engineering and went to an engineering school (EE, ME, CE, etc). I have had contact with many engineers on a daily basis, although I do not do this work anymore myself, and I have found that engineers can talk very well - actually I have never met one that can't and/or won't (the real problem is getting them to shut up). So in my experience, smart useful people are always able to communicate. There may be the occasional savant who is awkward or such but I have not met them.

As for programmers - same thing. If they are any good they are more than able to talk the talk.

The article explicitly notes, though, that most excellent programmers were also excellent communicators.

Effective communication doesn't necessarily mean you talk a lot or are extroverted. It means that you say things that matter when they matter, and that is a crucial skill to have in almost any work environment.

Having a high signal-to-noise ratio is (obviously) also critical to communicating successfully about code / technical issues. Talking for the sake of talking or waxing poetic about the understood 5% of the problem at the expense of the 95% remainder is a negative signal too.
No. Thinking out loud is not the same as communicating a thought process.
for one thing, you can't get partial credit for ideas/work that only exists in your head. (Similar to why it's a good idea to show work on homework assignments and tests)

If they sat there for a bit thinking about the problem and then provided a solution, I'm sure that would be fine. But that's not the situation he was describing.

Because you might need to work with the team at some point to help each other solve a problem.
At least in my case, there's a big difference between

(a) discussing an open problem with a colleague, and

(b) explaining what's going on in my mind as I'm attempting to solve the problem single-handed.

I'm very effective at (a), and it's a skill that I've used many times as a software developer and/or grad student.

But (b) is more typical in psychoanalysis sessions.

May I ask what you feel the difference in the two is?

I often have to explain a problem to my coworker and vice versa. It's also really helpful when pair programming. Just curious about your experience.

There's a weird dynamic in interview questions you rarely, if ever, get in the workplace: the person who actually knows about the problem is staying quiet while the one just starting to reason through it is doing all the talking about it.

There's no back and forth -- you're not actually trying to help them solve anything -- it's someone who already knows the answer judging how much of it I can reason out in half-an-hour. I can't take 5 minutes to think it through, I can't stop talking for any significant amount of time without being told to narrate what I'm thinking about, etc. That's nothing like a workplace situation, where they'd be expect to hold up half the conversation, be providing input and thoughts on the problem, and I could just tell them I'd be with them in 15/this afternoon/tomorrow/etc.

It would be more realistic if the interview didn't get to see the question before the interview, either. (And why I think design questions work better than coding questions -- you can do something they're not expecting in the design, ask more kinds of questions about the spec, etc which cause an honest back-and-forth.)

"They think in their head, don't work through the problem at all."

Those are independent clauses. Just because the individual is not talking out loud, should not be indicative that they aren't working through the problem. Personally, I am a very visual thinker and layout visual models in my head for problems with high (3-5d) dimensionality that would take longer to draw clearly in 2D space while explaining all the traversals I'm mentally making. I would then reduce a solution, and explain that, or I apologize for zoning out.

It became obvious to my office mates when I was working through and reducing a problem space, because I would "hang" on the thought. Sometimes I would be down, stuck in thought, for 15-30 seconds and according to them they could tell because my face loses emotion and my eyes flutter about.

That's an interesting approach, but I wonder if you're screening for people who can speak and code vs just people who can code.

I'm a product manager and I suspect I would do pretty well in this type of interview unless the snippet is especially complicated and/or esoteric.

In that case you might be screening for people who can talk but not actually code or people who can code but might not be the best communicator.

Interestingly enough, if youre optimizing for specific outcomes, you may not need to screen out the latter.

Engineers who struggle to communicate but are productive can be extremely successful given the right environment - including having someone on the team who knows who to work with various personalities and is technical enough.

Just some rambling, but it's worth pointing out given the scarcity of engineering talent. Not everyone is going to be Paul Bucheit, ace product manager and engineer all wrapped into one.

I'm not sure why you're being downvoted, but it's a valid comment. Even if someone disagrees, you clearly communicated a point that adds to the discussion.

Many are taking the article at face value, and are assuming there are no methodological flaws and that this study has been repeated enough times to invalidate any comments to contrary. It also didn't turn this topic into a "case closed" matter, as it's too big of an unsolved problem for that. They also just analyzed interviews, but did they monitor job performance after the interviews and for how long?

Additionally, differentiating between being a "good communicator" and a "good interviewee" is frequently overlooked. People too frequently rationalize their own interview approach as not the problem. Most candidates accept that they could improve and perform better at interviews, but a lot of people conducting the interviews don't think their process could do better.

> In that case you might be screening for people who can talk but not actually code or people who can code but might not be the best communicator.

From TFA and literally the thing that started this whole thread:

> Furthermore, no matter what, poor technical ability seems highly correlated with poor communication ability – regardless of language, it’s relatively rare for candidates to perform well technically but not effectively communicate what they’re doing (or vice versa).

Right but that would also be true and exactly what you'd expect if interviewer mainly judged technical competency on ability to communicate. Since the form is self reported it's impossible to separate out whether the correlation is causal (poor communicators are poor technically) or due to interviewer bias (poor communicators have a harder time convincing interviewers of their technical chops) or even other factors (being flustered hurts both a candidates communication and technical skills).

This is after all an advertorial piece of data spelunking not a study.

I'd suggest making it more than 10 lines of code. Similar to kids finding easter eggs -- you want a reasonable number of things out there to talk about. It sounds like you try to get there by keeping it simple, but I would posit that some people's brains may lock up on a single block of code and having it in context or more there gives them something else to look at. Our brains are very bad at having insights when focused on the thing you're trying to have an insight about.
I bet you could go farther and make a really bad piece of code and introduce it as such, and then have them talk about just how many things are bad about it.
Just curious, if you've weeded them out at around 5 mins, why does it take 45-60 mins?

Also, could you possibly just do an initial phone/Skype interview to save time in that case?

I've been in this situation. Often we know within a few minutes whether the candidate is a good fit for our team, but for legal reasons we have to stretch out the interview so that the candidate can't make a claim against us. It would be better for both of us if we could cut them loose quickly, but we have to cover our butts because that's the kind of litigious society we live in.

And their is an initial phone interview, but it is done by HR or our boss, neither of whom are technically proficient.

Isn't that why a lot of companies do tech phone screens?
If it's because of anti-discrimination laws, can a technical phone screen be short? You could discriminate based on voice too so I'd think that'd still be an issue.
I think the advantage is that the phone screening is much shorter than an hour, for everyone.

And "you haven't passed this interview" is less suspect than "I ended this interview early"

It's been immensely successful at weeding out candidates within 5 minutes

Indeed, it's amazing how quickly people reveal themselves in this way.

No whiteboards, no quiz shows, no grilling of any kind needed. Just: "tell me your story".

When you put people under the pressure of not having a job, a time limit and worst, the pressure of having someone looking over you, bugging you, and judging you, it can be incredibly hard to think straight or say anything at all.

You can act buddy all you want, but people aren't stupid. They know if they take a few extra seconds too long, you're already thinking "this guy sucks"

Each moment of silence makes the interviewee feel more tense. Instead of staring at them, why not leave the room while they read through the code and collect their thoughts?

You could even have HR do it. When they arrive, HR hands them the code and instructions. Then, they read it for 15 minutes, until you walk in.

I'm absolutely not an "introvert" of any sort, but if you give me something I've never seen and ask me to reason about it, you'd better take a very wide step back, stfu, and wait for 10m - if you're to get any results from my slow brain.
That's great! I really hope interviews like that become more common.