[manfredo]

First a note on how to read the relevant study. The link to the study in the WSJ article is paywalled. This is a non-paywalled link [1] is the results [2]. 22 applicants solved a problem alone, 26 with a proctor present. Without the proctor present about 2/3rds passed, with the proctor present 1/2 passed. Using score >= 2 as "passing", 12 out of 16 men passed in private and 4 out of 4 women. 11 our of 20 men passed with the proctor, and 0 out of 6 women. The methodology looks robust, but especially with the claims with respect to gender I'd want to see a sample size larger than the single digits before making any generalizations.

With that aside, my broader thoughts on tech interview processes: Companies want an interview process that are,

- Successfully distinguishes between people that have the knowledge and abilities required to perform the job.

- Has systems on accountability, consistency.

- Is relatively easy to train employees to administer the interview.

- Has a relatively low time-commitment for everyone involved, both interviewer and candidate.

In reality, though, there are tradeoffs between each of these points. For instance, using a set question bank improves consistency and accountability. Rubrics can be more explicitly defined, and bias limited. But it means candidates can google for questions beforehand. This was a salient issue when I worked at Dropbox, there were only about a dozen technical. interview questions for a 2,500+ person company. Having developers come up with their own unique question helps mitigate this, but reduces accountability.

Likewise, I've had some novel interview processes that more closely approximate real working conditions. One company's interview was conducted over git-hub. It was asynchronous, with tasks spread out over a week. After building the first solution, the interview came back with further feature requests and comments on the first iteration. This tested the candidate's ability to refactor existing solutions to meet changing tasks, and the ability to integrate feedback. These are things that are rarely captured by whiteboard interviews, but are arguably some of the more important skills in software development. But on the flip side, it was much more time-consuming in aggregate than 4 hour-long interviews.

No one interview process has all the advantages. I think a lot of companies settle into a pattern of 1 or 2 remote technical screens followed by a circuit 3-4 hour-long interviews because it's logistically robust. It's a format candidates are familiar with. It's easy to train new employees to conduct these interviews, and there's broad enough set of people participating that one biased signal isn't going to be decisive.

> Companies could also drop problem-solving tests as currently offered and instead ask candidates to spend five minutes explaining how they would perform a particular job-related task, Dr. Parnin says. Focusing on communication skills in this way, Dr. Parnin says, can reveal how a candidate thinks.

But do we want to hire the candidate that can talk about high level ideas for five minutes in a convincing manner, but can't code Fizz Buzz? Or a binary search? I get that some people might stumble due to pressure, but at the end of the day if you need some mechanism to determine if the candidate has the required skills or not. And administering a test is an effective way of doing this.

This isn't the first time articles like these have been written. X group is disadvantaged by problem-solving tests, so don't use said tests. But how then do you determine whether or not the candidate has the required skills? Usually whatever supplants the technical interviews are also subject to unfairness: referrals, recruiting alumni from specific universities or companies. While far from perfect, I still have trouble seeing what could replace problem-based tasks as a means of demonstrating skills.

1. http://chrisparnin.me/pdf/stress_FSE_20.pdf

2. https://imgur.com/EIiofkQ

[908B64B197]

> Likewise, I've had some novel interview processes that more closely approximate real working conditions. One company's interview was conducted over git-hub. It was asynchronous, with tasks spread out over a week. After building the first solution, the interview came back with further feature requests and comments on the first iteration. This tested the candidate's ability to refactor existing solutions to meet changing tasks, and the ability to integrate feedback. These are things that are rarely captured by whiteboard interviews, but are arguably some of the more important skills in software development. But on the flip side, it was much more time-consuming in aggregate than 4 hour-long interviews.

These you can afford to do if you are Google. If you aren't, candidate sort the places they want to work at in descending order. By the time they get to that take-home, they might already be further along the interview stages at better companies.

[manfredo]

Google certainly has more monetary resources. But in my experience large companies like Google are the ones that can't afford to do something like this. The logistical benefits of white-boarding interviews and the accountability gains are much more important to large companies. This interview process was for a smaller company.

But you're absolutely correct that the more time-consuming the interview process is for the candidate, the more likely they'll interview somewhere else. In fact, that's exactly what I did: I received an offer partway through this git-based interview process, and the offer was good enough that I didn't see value in completing it.

[flukus]

Google is a large well known and well paying company that can get candidates to make that sort of time investment, smaller companies can't because they're just generic companies that no one particular wants to work for, I'll skip the elaborate test and apply for the next generic company in the list.

Aside from that, I really don't have time for these elaborate interviews if I'm already in a job, companies that do this are limiting their potential candidates to the unemployed.

[908B64B197]

> companies that do this are limiting their potential candidates to the unemployed.

And sometimes there's a reason folks are unemployed...

[mechEpleb]

Yes, usually that reason is that once you are unemployed you fall to the bottom of any hiring manager's list of eligible applicants.

[Sohcahtoa82]

I'm always kind of surprised that the "take-home exam" type of job interview gets so much hate on HN.

Even if it takes me up to 8 hours to do the work at home, I think I'd rather do that. I feel like I'm pretty good at application security and programming, but I struggle to think on my feet when someone is grilling me with a job on the line. As soon as the interview is over, I get a huge case of staircase wit and think of all the better answers I could have given for each question. Even something as simple as "What's CSRF?", I could write pages about. But at a job interview? I have a hard time coming up with more than two-sentence answers.

Same thing with coding. I've written programs using breadth-first search, depth-first search, quicksort, binary searching, etc. Ask me to do it in a job interview and suddenly I'll forget the syntax for `if`.

Though FWIW, I haven't had a take-home interview test. Maybe I'd feel differently if I was actually given one.

[flosincapite]

These "exam"-style interviews are innately suspect for the same reason they are useful: because they approximate real work very closely.

Look at how many malicious entities use spurious job postings to farm resumes and other kinds of data. You think there aren't unscrupulous companies out there extracting free labor from their "interviews"?

If the employer is very reputable, you might be able to feel safe undertaking this kind of interview. But when you are applying to a shady tech start-up (and they are all shady, and they're where most of the jobs are), there is no way to know whether the job actually exists.

[emodendroket]

The thing is, if I spend 8 hours prepping for algorithmic interviews, I can use that prep for several interviews. If I spend 8 hours solving a toy problem, it is completely useless anywhere else.

[OldHand2018]

> determine if the candidate has the required skills or not

What are the required skills, and how long will they be the required skills?

I've yet to have a job where I was doing the exact things they hired me for 6-12 months later. Things change constantly, why isn't "dealing with change" a highly-ranked skill for job candidates?