[tptacek]

Serious question: have you done much hiring? Because this was a huge component of my lived experience as a hiring manager since the late 1990s, and every hiring manager I've talked with directly since has had similar experiences. If you do a lot of hiring, and you're honestly surprised that there are people who do really well in interviews and turn out not to be good hires, I'm curious to hear more about your experience.

[shantly]

Yeah, both as just another part of larger hiring panels and in tech lead roles with the biggest say in who gets in. Maybe I've been rejecting people who'd be fine and am committing the same sin but it seems pretty easy to get a sense of "this person has been working 5 years but is incredibly junior and barely understands the stuff they're working with" or "this person has been working 5 years and really knows their shit" from like 20 minutes of conversation, and I've not seen a true bullshitter get through hiring despite a lack of heavy, time-intensive tech-challenge-based interviews anywhere I've worked. Closest we had one place was a really easy screening project, which we didn't make anyone not-obviously-junior bother with, and I bet a few minutes on the phone with the right person could have replaced that with no real harm to the process.

I have, however, on two occasions probably personally convinced someone they had dodged a bullet by passing on me when I bombed that sort of quizzy interview—they were, I am entirely confident given the job descriptions, my work history, and actual feedback I've received from managers and peers, quite wrong, but I bet they were very sure I was useless and nowhere near being fit for the job.

[EDIT] I guess I should add that I have a history of not realizing that something I'm good at is not actually easy or obvious to others, so possibly I'm just unusually awesome at assessing developer suitability through conversation—I really doubt it, but maybe that's what's going on. I've also not found a way to make this fit with any sort of "we ask everyone the same set of questions because we want spreadsheets at the end" process, as you've got to tailor the interview to both the position and to the candidate, as presented on their résumé and related material. So if you want spreadsheets and quantifiable-everything then I'm not sure how you match that up with my preferred style.