[WrtCdEvrydy]

I have personally given up on "whiteboarding people"... reminds me of waterboarding.

I skim the resume, underline anything we use internally and talk to them about experiences.

We've done over 300 interviews in the last expansion year (2017) because we had to build a brand new development office (+50 devs).

After hiring all of these people, we've quantified this as three things: Curiosity, Ability to Learn and Ability to Listen. 2 out of 3 is good, but 3 is best. Almost every person we've hired who has been a success has had some of these, they're willing to be curious about new technologies and how things work, they are capable of learning (desiring isn't the same thing, they are capable of it), and listening is a good trait for enabling communication.

It is still possible that we chose these traits yet rejected other traits which are far more successful, and there is a chance we also rejected good people who would have also succeeded.

[atleta]

I don't exactly whiteboard, because in the past several years I've only interviewed online and while I'm looking for the same qualities, I found that (a simple) coding challenge does help with determining these. (I'm also looking for the smarts, but not "google smart" scale.)

Just by struggling with a (really not that hard) challenge tells a lot about how well someone is able to listen. E.g. when you try to give hints to them. It also shows a bit about their demand for producing quality work. (The first solution they provide is usually not bug free. They miss edge cases. Then they add conditions to check for them but those are not needed because the algorithm can be written in a pretty simple form. BTW, I've been using the same challenge for years. Whether someone is willing to simplify shows a lot about how much they care about the quality of their work.)

But yeah, I found that usual chit chat about technology with a few directed questions also help (and I use them).

[pdimitar]

I'd completely fail your interviews and that's not a criticism to you.

I am fairly experienced and I can simulate stuff in my head quite quickly but this often has a detrimental effect -- I get a few ideas, see all their flaws and get an analysis paralysis. I flunked one otherwise promising interview by doing this.

Sometimes you just freeze, you know. I have confidence of having excellent problem-solving skills but they are not always summonable on demand right at this minute.

I have a fairly successful -- if very rich on employers -- career so I should probably view the above as a personal weakness and not as something inherently bad, but I am not sure.

How do you feel about such people?

(I usually prefer a take-home assignment even if it's more work. It gives me a chance to show excellent craftsmanship and attention to detail, which I really want to demonstrate during the interview process.)

[atleta]

Well, it's hard to tell. You don't know the difficulty of the task, I don't know your tendency to stress and freeze. But the puzzle I use is really just a bit more complex than fizz-buzz. So much so, that when I learned about fizz-buzz years after I started using this one, I was kind of relieved. Because they way I've gotten into using it is that I had to interview a guy to help me with a freelancing project (for a long time client) while we were working hard on finalizing a VC investment into our first startup. And since I was really busy with the startup, I just started figuring out and googling for a puzzle like 15 minutes before the interview started. This was meant to be the warm up task, I've just googled it to see if there is a trivial solution besides what I thought up. I've also took another one from topcoder (a medium hard from the level where I was at years ago when I stopped playing, which it self was something medium-ish, can't remember exactly).

Since in the past few years I've only been interviewing freelancers for temporary positions on the projects I've worked on, I kept going with this simpler task. Now even with that, some people struggle. And I see that as a very bad sign. But I'm always giving hints and helping. I always start by saying that if you have any questions or feel like discussing an idea, feel free. And if they keep thinking and don't tell anything I'll start the discussion. (But that's part of the test, because, you know, things like this happen during real work: you get stuck, you don't understand the requirements, etc. And then I will want you to reach out and ask/discuss.)

Another task I do during the interviews is code review. Even if you do badly on the first one. The piece of code (now actually two pieces) has a lot of room for improvement, while only about 20 lines long. That's a totally different kind of task, less about problem solving and I guess less prone to cause anyone to freeze.

[pdimitar]

> But the puzzle I use is really just a bit more complex than fizz-buzz.

Oh. We kind of talked past each other then. Sorry!

Just today I had ~15 minutes to craft a basic SQL update statement generator based on an (almost) arbitrary JSON input. Save for me misunderstanding the requirements initially, I aced it and the interviewer was impressed.

General problem-solving and chatting about real problems that the company has I found to be the best format of technical interviews. One experienced programmer can extremely easily call out BS if the candidate is faking their expertise.

---

As for me freezing, yeah, it can happen but I accepted it. I can't have all the IT problem space in my head all the time. I expect the interviewer to be flexible in these cases. If not, oh well, life goes on.

[vmception]

What I would actually do for your company on a day to day basis is very iterative with the IDE, API reference, unit tests and actual performance correcting me 20 times in ten minutes, allowing me to avoid all pitfalls and synthesize new pitfalls to avoid.

What I would do during your interview is performance theatre and it would tell you nothing. I would lie to you about whether I have seen the problem you presented before, because I would only be there because I was on an interview-circuit with every interviewer that thinks they are so clever and hasn't changed their question for years and I recycle and adapt answers, and for you, the answer to your question is probably listed verbatim on Blind, Glassdoor or Leetcode forums about your company and role.

[atleta]

I may not have been very obvious, but the puzzle I use in these interviews is very simple. So simple that I was a bit embarrassed until I've learned about fizz buzz. At first it was meant to be a warm up puzzle, but I stopped saying that basically on the first week of my interviewer career. When we interviewed a java architect guy (a friend of my then co-founder) who worked in that capacity for a big telco and completely failed at it.

It felt pretty bad, because he knew there would be another, harder task and we knew that he had no chance, so what do you do. But again, the task is simple, you don't need an IDE, you don't need to use any API. Some people do the very obvious solution (though that never occurred to me until I saw someone going down that path) which does need maybe 1-2 calls to the stdlib. But I always tell them that syntax details don't matter, they can use any language, and if they want to use some lib or API then they can just say that and I'll believe the thing is there and the call does what they say it does. (Unless it's obviously not true then I'll say how I think it works.) But it's mostly theoretical, because as I've said you don't need any APIs (though some people sometimes ask about the exact semantics of some less common language features and then we just agree that it works the way they say it works). It may sound cryptic, but that's just because I don't want to give away the puzzle :). E.g. think about what happens when an addition (using the plus operator) causes an overflow in their language of choice. It's just an analogy and even the thing it's an analogy for (that came up during one of the interviews) is not needed, but given how some candidates solve it, it may come up. And then I'll ask what could happen then, how do they think it works. If I know how that works I'll tell them if not, I'll just believe it works the way they say. Doesn't matter.

It's not a theatre. The idea is to somehow simulate a situation similar to what you'd experience during work from the perspective of problem solving . E.g. you'd be surprised, but a lot of people just don't think about testing their solution. So if you are really testing habitually, then I'm pretty sure I'd see (hear) you walk through the code, and run your test cases in your head. Maybe you'd start with stating test inputs and expected test outputs. A few people I've seen actually did that. And of course, these all register and tell a lot about how you work and how you think.

>every interviewer that thinks they are so clever and hasn't changed their question for years and I recycle and adapt answers, and for you, > the answer to your question is probably listed verbatim on Blind, Glassdoor or Leetcode forums about your company and role.

1. My company is not on Glassdoor or Leetcode. It doesn't apply to small companies 2. The task is simple, it's not worth memorizing it. Actually it's so simple that if someone needs to memorize this, then they will have a problem memorizing harder puzzles for sure. 3. It's pretty easy to identify if someone knows the quiz (though, of course they can pretend that they don't, but that's borderline sociopathic) 4. I also use a code review challenge which is harder to reproduce, though some guy could be unfair and post it there 5. If this becomes a real problem for me, I'll probably add pair programming to the mix

I understand the motivation for some people to want to circumvent interviews, thinking those don't do justice for them. It's hard to tell. I've never been afraid of interviews, though I've never really interviewed to any job where I though I had to get in. The problem with posting questions and trying to circumvent is that the response will probably be that interviewers just keep changing the questions which will result in less predictable outcomes. Or everyone will just require a long, take-home project (though you can game that one pretty easily too) or throw more automation at you. (e.g. a 1.5 hours long HackerRank quiz just to get through to a human.) Seems like and instance of the prisoners dilemma to me.

[vmception]

same, I stopped doing whiteboarding. I have people solve some domain knowledge exercise (like their specific platform or IDE or package manager, stuff that would actually spend the company time on a day to day basis). Most companies could get away with that, even "big tech" can because their employees leave after 2 years too, before they are even eligible to join some other teams at some companies.

[pdimitar]

I wish I met more interviewers like you.

[rrrrrrrrrrrryan]

A few years back, I built out a team of about 30-ish technical people and had a similar experience, but I phrased it a bit differently: Aptitude and Enthusiasm were the best predictors of success, and I phrased most of my questions in a way to get a gauge of those two metrics.

"Aptitude" correlates strongly to your "ability to learn," "enthusiasm" correlates loosely to your "curiosity." I think "ability to listen" is more of an obvious prerequisite to being a functional employee.

The real takeaway is that experience doesn't matter nearly as much as people think it does, especially in a company that has a culture of ramping up new hires very well.

[path411]

How do you easily test an interviewee's "Aptitude"? Especially in a developer sense. I struggled with how to do this when asked to help with interviews in the past.

[rrrrrrrrrrrryan]

Yeah - sadly, it's extremely subjective. In the beginning, I hired some lousy people and some great people, and eventually I got much better at quickly and accurately sizing up someone's competence, independent of their personality. The shy people need some softball questions to help them feel relaxed; the bigger personalities need to be poked and prodded a bit to see how they react under a little bit of pressure. A good team has a mix of different personalities, and if they're all bright and eager, they'll get along just fine.

Anyway, to answer your actual question: One thing I'd always ask at some point is, "Can you tell me about a time where you solved a hard problem?"

They'll think you're trying to gauge their problem solving abilities, but their answer will give you much more valuable insights regarding their aptitude and enthusiasm. While they're answering, you'll get to ponder:

"What does this person think is hard?"

"How well can they communicate a complex topic?"

"Does solving hard problems excite them?" etc

[d2ncal]

How do you probe for these skills? Are there any questions that help you to identify them?

[WrtCdEvrydy]

You can get a feel for it... ask about personal projects, then follow-up on why. The best candidates will cop to "I wanted to check out the new tech (curiosity)", "I wanted to learn the new programming language (learning)".

[mgkimsal]

How do you deal with "I don't want to learn new language X"?

I do investigate some time with new languages/tech to a degree, but to the extent I have time to learn, I tend to want to go deeper in the things I have competence in already. There's generally always more to learn (between the language itself evolving and the ecosystem developing) such that taking time away from those areas to 'explore' some hot new thing tends to not have very useful ROI.

I've done 'learn new language X' for... way too many years already, and prefer to get better at what I know. That doesn't mean I never learn new things or new tech, but it's generally in service of a defined goal as opposed to "oooh that looks neat...".

I've been doing this as a hobby for 40 years, and professionally for around 26. I have the feeling some places would look at 'the old guy' (like me) as 'the stick in the mud who never learns the new tech', yet in some cases I've been 'learning new tech' since before they were born.

Some of the original question was rhetorical - I already know how some places react to the "I don't spend time learning multiple new languages/stacks every few months" line. But... keep it in mind that there may be other ways to measure that curiosity besides just looking at 'time with new language'. ;)

[WrtCdEvrydy]

> I tend to want to go deeper in the things I have competence in already.

That's still learning, the issue is someone who says "Well, I don't have to learn anything new".... "at all".

[enraged_camel]

This comes up a lot on HN and the common objection is that the criteria you listed discriminates against talented people who have families (for example) and therefore no spare time to indulge their curiosities by learning new languages or whatever.

[hodgesrm]

It's perhaps better to say, look for people who like to learn how things work. Good tech people can explain the plumbing.

[WrtCdEvrydy]

Wanting to learn something is fine, being curious about a new technology is fine. It's the people who don't want to learn anything and have no curiosity about new technology. I spent 30 minutes talking to someone about their plan to build a machine learning machine and try to learn some machine learning in python.

[beforeolives]

You can't really quantify attributes like curiosity, ability to learn and ability to listen so it's difficult to interview for them without having all kinds of biases sneak in.

How do you deal with this in your process?

[onei]

Quantifying it is hard for sure, but some sample questions I've used in the past:

1. (Curiosity) How do you keep up to date with tech? Then explore something recent they started playing with and how they found it.

2. (Ability to learn) Ask about past failures, what they thought went wrong and what they'd do next time. If that's a bit harsh, ask what they'd change about their last major project.

3. (Ability to listen) For me this is about how they work with others, so probe for areas of friction they've had, particularly with people outside their immediate team.

You can't discount bias, but you get a baseline over a few interviews and can go from there. It's worth pointing out that all of these are soft skills, and quantifying those is always hard. Sometimes you've just got to trust your gut. It also helps if you've spent time working in a customer focused job, because you tend to be better at spotting egregious personality flaws.

[t0mbstone]

"Curiosity" is a weird trait to gauge, because the person might be completely exhausted and fatigued by the ever-shifting quicksand of the tech world, but still be extremely curious about things outside of their job, such as their hobbies.

For example, I have recently started doing a lot of cooking and bread baking, and I've watched hundreds of youtube videos about those topics (which I could argue demonstrates curiosity), but I still haven't bothered to learn React or Angular because I barely have to deal with javascript in my current job, and where I do, jQuery usually works just fine for my limited needs.

When employers say they want someone who is "curious", what they are really saying is they want someone who is willing to spend all their personal time self-educating on programming topics, effectively making themselves more valuable from an employment perspective. The employer, of course, is rarely eager to actually allocate "learning time" during normal business hours, and is rarely willing to give you a meaningful raise when you actually have self-educated on your own time.

I really wish more companies actually TRAINED their employees DURING BUSINESS HOURS instead of just trying to hire "curious" people.

[mgkimsal]

"Curiosity" has caused me to try to get more involved in business meetings/decisions, but that's often not the 'curiosity' tech managers want. As you say, they want people who are 'curious' about new tech and will learn in their personal time.

I've done this for decades, and there's a degree of regret about how much time I've spent 'learning' and 'being curious'. Not a huge regret, but... it's help reduce the need to dive in to some things altogether. But.. that comes across as "not interested in learning" to some people. No... I can usually just tell after a quick skim if there's going to be business value in tech X. We don't need to spend weeks prototyping something that is obviously not a fit... unless... we're doing this for show, or politics, or something else. If that's the case, just say "we're using X" not "let's investigate X to see if it can meet our needs".

Good on you for having non-tech hobbies. I doodle on guitar, and am 'curious' about that, but it's not going to land me any tech jobs. :)

[raspasov]

Companies should definitely train their employees during business hours but there’s people actually wanting to spend most waking free hours learning about programming topics. Those people are usually some of the best in their field. I am not advocating for extreme narrowness of interest. A “T” shaped skillset is possibly the best.

[WrtCdEvrydy]

> can't really quantify attributes like curiosity, ability to learn and ability to listen

You can get a feel for it... ask about personal projects, then follow-up on why. The best candidates will cop to "I wanted to check out the new tech (curiosity)", "I wanted to learn the new programming language (learning)".

Listening is a little bit more difficult to quantify, but we're mostly looking at people who know how to follow the flow of a conversation (a good friend once told me it's like a battle, their side shoots a little bit, we shoot a little bit back and back and forth)

Edit: Some people don't have personal projects and that's fine, you can ask what they would like to learn or if they want to learn something. Someone who just rolls in to write some code for a paycheck is perfectly fine but if someone has a desire to learn something, it's a good trailing indicator.

[Baeocystin]

Having been on both ends of the income spectrum over the years, I am convinced that curiosity/side-project questions are simple proxies for financial stability rather than anything deeper. The only people who will have the time & energy to do such things are ones whose more prosaic needs have been met. FWIW.

[WrtCdEvrydy]

True, it can be a proxy for it but even the desire to want a personal project (it's not a side project, it's not a hussle check) implies the person sees the development field as more than just a paycheck.

Someone seeing the field as just a paycheck will often cease to want to improve themselves, take new challenges and other desired traits.

Sadly, some of these conversations do devolve into some weird eugenics-like items....

[wutbrodo]

It seems like a fool's errand to me to stamp out 100% of bias. Any bar that requires subjective assessment creates an opening for bias, and not every job can be assessed entirely by a completely mechanical rubric.

This is what things like representation are for, as they attack the problem for a different angle.

[deathcalibur]

Usually the qualities that matter most cannot be quantified (un)fortunately... One way to combat this is to have the interviewing team consist of people with different backgrounds. For example, if you want to hire more women, it probably makes sense to have women involved as interviewers for all candidates. If you don't have the ability to do this, then I'm not sure what the solution is.

[dmos62]

Haha, usually, in the context of interviews, you don't think of gender as being a type of background.

[f6v]

Nice try, AI! We, humans, will never reveal our secrets. And you, my friend, won't replace us in this soon.

[flyinglizard]

You just speak with the person about their interests, hobbies, past projects they’ve done.