[Aurornis]

Hiring based on gut feeling about how impressive the candidate feels can be misleading when you only do a little hiring. It can work for small samples sizes if you have a strong front end filter or you are primarily getting candidates through trusted referrals.

Then one day you encounter a candidate who is great at impressing people. They leave you feeling excited with the possibility of working with them. You feel delighted after each encounter. Then you hire them and they’re not good. At all. They don’t know basic things that you assumed they would based on how they spoke in conversation. They used all the right words and maybe even recited the precisely correct things to say for a system design question about past work, but when they have to do the same work they’re lost.

It’s a weird feeling to discover your intuition about someone was completely wrong, because we all think we’re better than average at separating the wheat from the chaff. I think it happens to everyone who does hiring at scale.

[sanderjd]

I'm certain that this is true. It certainly sounds true.

But honestly, does anyone here have experience with doing this kind of interviewing at scale, for experienced software engineering roles? I've been in the industry for coming up on a couple decades, and I have been involved in doing lots of interviews at times, and in that whole period of time, at every company where I've worked, we did the standard multi-round whiteboard / coderpad interviews. Do other folks here actually have recent experience "hiring at scale" in this industry, with a process focused on candidates' experience? Who is doing that?

And also, the question is not "are there ever false positives?" or even "is this biased toward a certain kind of false positive?". Nobody thinks any way of hiring is perfect. Even ignoring bullshit artists, sometimes very competent people just aren't a good fit for the actual demands of a particular job. The question is one of tradeoffs. Are the failure modes and biases of a particular process worse than those of another one?

To me, the current standard process comes at an enormous cost. At any job I've ever had, every time I think that maybe my time there has run its course, I immediately think, ugh, but I'll have to go through the f**ing interview process. I'm not a person who does research on this, so I don't have data or anything, but I must not be alone in this, and I think it is likely a meaningful friction in the job market. (Which, I guess is good for employers, so it probably makes sense that they like the status quo!)

[cmrdporcupine]

Yep.

When the money was raining from the sky and compensation bands were going up and you could land some job with, like, free food and microkitchens and massage points and whatever...

Then fine. I guess it's worth the drama in the interview. Those places had potential boatloads of false positives and could afford all the false negatives from potentially bogus interview processes. I trained to do interviews at Google twice and I thought the process was stupid, but it was the price of admission into their fish tank of privilege.

But now the whole process feels ridiculous. You'll grind through an interview and get the prize, and still feel in a state of total insecurity after you get the job.

[sanderjd]

Yep, it was totally worth it for me to subject myself to the big tech hiring process. I have no student loan debt and own a nice house now because I did that. (And I learned a lot and met a lot of great and unbelievably smart and talented people during my years at that job also.) But the day I did the interview panel was still one of the worst most stressful days of my life. And it gave no signal on how well I was going to do in that job. It gave a lot of signal on how well I could manage subjecting myself to a miserable ritual, which is not nothing, but it sure ain't much.

[cmrdporcupine]

Or there's potentially this: that the skills we think are universal and transferable ... are not.

That our industry is so bespoke and different and non-standard around tech stack organization and work culture between shops that in fact someone could have been performing top tier at their previous jobs and just completely not function in your workplace.

And because we have a recency and confirmation bias around your own positions, we treat their failure to thrive in our environment as evidence of incompetence.

When it's just really a bad fit -- one you can't really shake out in a verbal conversation or a whiteboard coding exercise.

Or there could even be something wrong with the way our own workplace is organized. One that we're used to working around, but the new hire is not.

There's so much broken everywhere I've worked. When I see a misfit I generally assume it's a combination of factors, not a person lying or being incompetent (though yes, I've seen that, too)

[roryirvine]

We see this occasionally in consulting: a strong performer will sometimes land in a gig that turns out to just not be a good fit for them. We know it happens, we make a point of watching out for it, and we can correct for it when it occurs - but we can't reliably predict who it'll happen to, and in what circumstances.

Mostly, they simply move off to another gig where they return to their previous strong performance. Occasionally they take a hit to their self-confidence and take a while to get back up to speed (we try hard to mitigate this, but it does happen). But sometimes they'll take a break, work out what was causing the poor fit, and come back much stronger than before.

It's not unreasonable to expect a similar pattern outside of consulting - it's just that, for perm staff, the psychological barrier to cutting your losses and moving on are much higher.

[sanderjd]

Yeah. This is why I really wish the job market were optimized to lower switching costs, rather than seemingly the exact opposite.