[majormajor]

I agree races to the bottom aren't always helpful - see also college admissions and how so many kids are loaded to the gills with "extracurriculars" to "stand out" - but ...

> If a company is aware that they are missing out on good applicants due to how they conduct interviews, they should improve their interview process.

if only it were that easy!

I've never worked at a company that hasn't wanted to improve their interview process.

Even that "tell people roughly what to expect" thing is a result of trying to improve the process - you can more easily tell the difference between "even with hints, they weren't able to hack it" and "they were able to figure it out, though we're not sure how much the hints had to do with it." That's a step better then "they passed the test" and "they didn't pass the test, regardless of if it's because it caught them by surprise or just that they aren't able to."

It's not perfect, but most of the common things I've seen pitched here to "just" make it a better process, like trial periods and take-home "real work sample" tasks, seem wildly impractical as at-scale replacements.

EDIT: my personal favorite is when the candidate has a system they've worked on in the past that they can explain in detail and describe challenges and solutions for... but... the pool of people who can do that has been even smaller than the pool of people who can at-least-pseudocode their way through some mild live problem solving and design questions.

[xboxnolifes]

> I've never worked at a company that hasn't wanted to improve their interview process.

My point isn't that fixing the process is easy. It's clearly not. And I absolutely sympathize with the issues of hiring. My point is that improving peoples' interview skills doesn't make them better employees, or make qualified candidates easier to find, since it's not a skill that is unique to only people who are actually qualified for the position. Just as qualified people will seem better, so will unqualified people learn how to seem qualified. It's applying a fix that does nothing but: (1) reduce signal to noise, and/or (2) satisfy the "gut feelings" of interviewers on what a good candidate look like.

If companies are goading applicants to improve interview skills, all else being equal, assuming this actually leads to increased job offers for the candidates, it means companies are filtering too hard for unimportant "interview skills".

----- EDIT

> EDIT: my personal favorite is when the candidate has a system they've worked on in the past that they can explain in detail and describe challenges and solutions for... but... the pool of people who can do that has been even smaller than the pool of people who can at-least-pseudocode their way through some mild live problem solving and design questions.

This lightly touches on part of what I mean. Companies frequently test for things outside of what they intend to. Its not possible to completely avoid, but, based on my own interviewing experience, it's doesn't seem like much work goes into adjusting for it. Just breaking down this example, I'll explain what I think you are looking for, and what you end up asking for.

Intentionally testing for:

- experience working with a complex system

- able to identify issues in a complex system

- able to figure out solutions to said problem

- able to identify tradeoffs to said problem

- able to communicate the problem, solution, and tradeoffs

Unintentionally testing for:

- ability to recall, on short notice, details of a complex system from (likely) months ago, including it's tradeoffs and outcomes, without resources on hand.

- the ability to recall challenges and learnings from (likely) months after they've been experienced and absorbed. (This is a big one for me, personally)

- knowing how complex is complex enough. An applicant may think bringing up a system they do not consider to be particularly complex may hinder their chances by not meeting expectations.

Every point from the unintentional section is something that could completely prevent someone from coming across as a good candidate, but have no bearing on how well they can actually do their work.

[majormajor]

I actually think all three of the things you list as "unintentionally testing for" are extremely important.

Systems often work well for months and don't need touching, then have an incident, or need a new change, etc. You can remember that stuff fairly quickly? Perfect. You can't? I think that's a potential flag, directly related to "able to identify issues in a complex system" and "able to figure out solutions to said problem."

But the other reason why not many candidates do well on that is because there's also just a lot of companies where the day to day work is not-complex problems on not-complex systems (even (especially?) at FAANG where the typical dev is a very tiny piece of the machine). So in that case we have to fall back to made-up approximations for "complex" problems that don't require a ton of extra context.

[ksaun]

I'm not sure I understand your point/perspective, but maybe it's due to differing assumptions about what constitutes "interview skills." I consider refreshing one's memory about complex systems or challenges that have been experienced and absorbed to be important parts of interview preparation (from your "unintentional" list). That is, "interview skills," in my opinion, includes practice at describing one's experience, and doing this well helps the hiring manager correctly assess the candidate's practical on-the-job skills.

I feel like I may be misunderstanding the point you are making.

[xboxnolifes]

> That is, "interview skills," in my opinion, includes practice at describing one's experience, and doing this well helps the hiring manager correctly assess the candidate's practical on-the-job skills.

I agree that doing these things is "interview skills". That's basically my original point. People spend more and more time practicing "interview skills" rather than practicing what the job actually is. This is the wasted productivity, and it means that every candidate needs to spend more and more time studying tactics to get a job, over spending time actually being good at, or even just doing, their job. If you urge all applicants to get better at interviewing, that's not going to make it easier to find qualified applicants. It's just going to make more people seem like good applicants, since bad applicants can also improve their ability to interview.

The more "interview skills" we require people to have, the more detached interviews become from actual jobs, which advantages people who practice "interview skills", which makes it harder for companies to decern good candidates from bad (since one doesn't need to be a qualified candidate to be charismatic or lie). It's at this stage that companies seem to come to the conclusion that the solution is to focus even more on "interview skills", even though it's those "interview skills" that start the chain reaction in the first place.

Back to your example interview question. After passing an interview, does one ever need to frequently spend time refreshing one's memory about complex system's they worked on in the past (and are no longer working on), on a moments notice, without references, and tell people stories about them? If so, I would yield it's probably an excellent interview question. Even without that caveat, it's probably still a good question. My main goal with that part of my comment was to point out that questions have unintended failure modes that, in my experience, tons of people aren't aware of. If one isn't aware of these failure modes, one will not be aware that they are unintentionally skipping over potentially good candidates.

[DiggyJohnson]

I’m not sure I agree with your analysis of the those unintentionals. They all seem loosely related to “ability to communicate about complex systems”, which I consider a valuable quality in an interview and colleague.