[q7xvh97o2pDhNrh]

> It's often OK to not solve the problem as long as you give an interviewer insight into how you think

It's not, though. Many people have no clue how to interview well, and way too many tech interviewers are obsessed with whether or not the candidate got the "right" answer.

Anyone who dropped multiple mediocre solutions before a good one in an interview with me would likely get a strong hire — I love to see this kind of iterative thinking, and finding people who can role model a healthy exchange of rough-draft ideas is always a great boon for the team's psychological safety (and by extension, their creativity and productivity).

But I think, industry-wide, it's not great. As a SWAG, probably ~50% of interviewers are more interested in the correctness of the answer rather than the caliber of the thought process and communication.

[Frost1x]

>As a SWAG, probably ~50% of interviewers are more interested in the correctness of the answer rather than the caliber of the thought process and communication.

And this is why most the interview processes as practiced today are a joke. It's far more this weird sort of cargo-cult hazing process than any actual sort of reasonable assessment. Few people give challenging problems that they expect won't get solved to step through thought processes, they have some predisposed ideal solution or perhaps probably optimal algorithmic solution on the spot. That lends itself well to a combination of assessing rote memorization and chance, I suppose.

[shepardrtc]

> It's far more this weird sort of cargo-cult hazing process than any actual sort of reasonable assessment

This is 100% what it is. A "We went through it so you do too." sort of thing, combined with the fact that management can't interview everyone and SWE's aren't good at reading people to determine if they're lying.

I have a friend who's a manager at a FAANG and while there's a lot of good, smart people there, there are many, many people who are terrible at what they do and are very difficult to deal with.

My company does not do whiteboarding and makes an effort to get to know the person. We rarely get bad hires. Maybe 1 out of 100? Give me 10 to 20 minutes just to talk to someone and I'll tell you if they'll be a good employee. One of the first things I do is assess if they're lying on their resume. If they're not, then I'm comfortable believing in their skill set.

[will5421]

Where do you work? Or do you know other places that do this? I’d be interested in applying.

[devonkim]

I've said before to other technical folks that the interview process in tech is more a measure of ability to handle anxiety and stress than whether one can solve problems competently and gracefully. Granted, in my current role I make it clear to candidates that they may have to perform under pressure at 2 am when there's an outage possibly due to their code or even others' and many people are relying upon you in particular for guidance. I also make it clear that we do a LOT of things to keep such a skill from being required but no process is perfect so being able to handle such a situation without panicking is an important factor in the hiring criteria because each of those situations I can count have been very important and high impact (it's part of why they're so stressful, right?).

Every other practice we have in our interview process is justified based upon historical precedent as well as a team decision for where the team wants to go next.

[listenallyall]

To effectively fix an emergency situation like an outage, you have people extremely familiar with most/all aspects of your systems walking through a checklist of checks, verifications and tests to identify the root cause of the issue and then go to work on that. The point is that people are "under pressure," sure, but they are extremely comfortable and familiar with the domain. They already know what is supposed to work or what the expected results are to certain basic tests, and that familiarity and knowledge is what maintains a level head and prevents someone from panicking.

This is pretty much the opposite in an interview situation, even when it's a subject people know well, there's way too much "unknown" about your particular setup that the interview doesn't/can't know.

[devonkim]

The test has a number of fairly loose factors that are explained to the candidate as more about approach and communication than a “correct” answer (although don’t keep sitting and wallowing about a particular problem like the Python GIL for what is meant to be about exploration together). 1. Many outages are due to systems not even known to us to even exist for a team, so being able to determine when one knows something v. does not is a skill in the sense of knowing when to stop wasting time and to escalate and when to keep trying to troubleshoot their responsible areas - response time isn’t the KPI in an interview and never should be. Part of why I ask this is to determine if a candidate will give up really, really easily and just escalate saying “this isn’t my problem” - this isn’t acceptable, has happened to us all at previous companies, and is basically one criteria for failure / dismissal. A more constructive attitude and approach is “I have done a lot of evaluation including A, B, C, D and do not have anything else in mind, I would appreciate another set of eyes” - that is basically all we can reasonably expect in an interview situation and of course nobody knows anything about a company’s systems as a new hire. There is usually no expected canonical answer as well which would bias interviewers for a “correct” solution or even approach.

2. I usually ask candidates questions and scenarios based upon their submitted code samples and will presume an OS or cloud provider they’ve listed as something they’re familiar with on their resume. If you have lied for any reason it will be very obvious very fast as I keep adapting the question to a narrower and narrower scope to being a purely toy problem that is now useless for measuring anything beyond whether they’ve seen the problem before. Usually we can spot some errors in code samples submitted and make it clear to candidates that we do not expect perfect or necessarily even working code! The test isn’t about necessarily being bug free but about the attitude one has about their work output and how they’ve thought about various failure modes. I’ve had candidates surprisingly often say “that can’t happen, I made sure of it in my test cases” and then I show with a quick test run I think of that their submitted code does indeed have some flaws that would have had issues in production causing, say, OOM issues, segfaults, etc. A successful candidate’s attitude is accepting / welcoming of criticism as a team effort, has strong but loosely held opinions when encountering data contradicting their position appropriately, and is able to accept a challenge from junior engineer with respect and sincerity. Red flags = arguing with the interviewer for asking an admittedly irrelevant question, getting very defensive about a technical decision, and saying adamantly “this should have been caught in a code review so it’s not relevant” and being dismissive about the question. Yes, I’ve seen all these responses before. Nowhere in this process does “is expert at X” enter the picture, and we are usually evaluating the quality of the questions asked by the candidate as well. Many of us on the team have respectfully asked the relevance and presumptions of a question and that’s a big plus IMO - they have courage, respect, and critical thinking skills under pressure. I make sure to tell them that during the interview because people are so used to taking orders and shutting up that it’s bad for the organization.

Another set of red flags I’ve seen is “I’ve never had to do X, it’s taken care of by Y” and being unable to reason or even conjecture about how Y could be designed and reassured it’s not about correctness when it’s on their resume. Like I had someone that was pretty clearly a competent, skilled programmer but was applying to be an SRE and didn’t know how to work on a system below the container level with any tools proprietary or OSS - they were recommended to a different team but not for the applied position.

I assure you that everyone I’ve interviewed for years now has said they’ve had a great experience in all the follow-ups with recruiters, felt that the interview was a technical conversation rather than a hazing / torture test, and that nobody felt the questions were not relevant to the job once hired. I’m not an asshole as an interviewer trying to push people to some limit or something that seems to be the result of most technical interviews in our industry.

[d0mine]

Here's a data point: anxiety from the interview (you depend on unfamiliar people judging you) is different from troubleshooting a technical incident (it is more on the exciting side).

[devonkim]

There are certainly unfamiliar people judging me and my coworkers when there’s an outage involving customers of various levels, yes. And in a previous company sadly yes we have had people let go for poorly handling incidents that resulted in lost business. It’s a lot of SRE in the ugly side

[d0mine]

Obviously, there can be many people who are affected by a technical incident. It may be wise to delegate handling upset customers and fixing a hard technical problem to different people.

Blaming people is a sure sign of broken processes.

[shepardrtc]

> they may have to perform under pressure at 2 am

When that happens, do you tell them they have 5 minutes and if they don't figure it out they're fired? Try that sometime and see how they perform.

[ericmcer]

Interviewees being able to instantly produce the right answer made me think of another problem Interviewers need to account for currently: people who are biding their time until the market heats up again.

There are definitely engineers hunting max salary/equity over everything and they are going to jump ship if they can get a FAANG role. Being able to spit out rote memorized leetcode and systems design questions is probably correlated with someone who wants to maximize their salary.

[epiccoleman]

A few years back I failed an interview because I gave the impression that I didn't know much about full stack web dev. (To be entirely fair, I didn't at the time).

I was a little irritated though because, though I was early in my career, I have no doubt that I'd have been a productive member of their team within a month. I had thought the interview went well - we discussed the problem they proposed in detail, we arrived at a reasonable solution by the end, I asked lots of questions and responded well when I was prompted about edge cases.

I was just fuzzy on implementation details and aspects of database design that I hadn't had direct experience with.

Anyway, I'm not bitter, not getting that job led to a fantastic gig that I still have today. But I did feel like they were focused on the wrong things in the interview.

[caskstrength]

> A few years back I failed an interview because I gave the impression that I didn't know much about full stack web dev. (To be entirely fair, I didn't at the time).

> I have no doubt that I'd have been a productive member of their team within a month.

Did you consider that they might have skipped you not because you were "fuzzy on implementation details" but because you projected that vibe "I don't know much about what you do but I bet I can be ask good as you at it in a month"?

[epiccoleman]

I don't think I projected that vibe. I knew ahead of time that the architecture interview was going to be the hardest part for me, and made an effort to show that while I had gaps in my knowledge, I wasn't going to be afraid to ask questions and have discussions to make it possible to get work done. I have a bit of an aversion to unearned overconfidence so I like to think that I did not project that in the interview, but obviously I can't experience myself from the interviewers perspective.

When I interview junior candidates, attitude, interest, and capacity for learning are more important to me than specific knowledge.

Anyway though, it was their choice, and I clearly didn't demonstrate what they wanted to see. Not getting that job left me in the right place at the right time for a job that I really love, so... ¯ \ _ ( ツ ) _ / ¯

[devonkim]

I look back at myself and the kind of cringe responses (and questions!) I had earlier in my career and for the most part the interviewers were right to reject me but not necessarily on the basis of technical merit (or lack thereof). On a lot of teams being able to communicate effectively and develop a strong rapport quickly is a bigger factor for an individual's effectiveness than how much quality code and designs can be produced. Ironically, this is an even bigger requirement for smaller organizations. For example, see a pathological case a start-up of two people where they must both be _very_ good at working together or the entire company will fall apart quite quickly. Also in a lot of interviews I was unable to demonstrate that I'm competent partly because I had already burned way out from months of being on-call and losing a great deal of sleep, so I was walking into interviews at basically my worst possible performance. It took me quitting for a while, working on my own terms, and recovering slowly to be able to demonstrate what everyone saw when I was actually operating at my general level of competence in various dimensions. Additionally, got some professional help and with some clinical diagnoses under my belt with solid medications everything got substantially better after that. Going through basically an entire career with severe handicaps has made everything seem much, much easier to handle.

It's ironic that the dynamics of targeting higher quality candidates are not that different from personal dating. While for larger organizations they seem to stress how much more damage a bad hire can be they have the capacity to absorb losses better than substantially smaller organizations on a purely mathematical basis. Likewise, individuals that have higher resources, emotional intelligence, capacity for grace, etc. can much better tolerate "bad" relationships yet are among the most stringent at avoiding "bad" relationships in the first place.

[jddj]

Capacity for grace is such an interesting framing. Is it commonly used in the sort of fields which measure people on those characteristics?

I put it into DDG but didn't find much.

[devonkim]

I came up with the term myself which is why you can't find it but I also half expected someone to have named it as well. Because "manners" or "nice" is a mostly culturally-loaded or very subjective term I wanted to say "be considerate to others above how even one would want to be treated." And it's a word not used daily except among the religious in the West, and that works well for those groups because "act how you would in church / template / mosque / synagogue" as a reminder of what civility can look like. Granted, some churches in the US are truly frightening to me and maybe I should refrain from calling on such emotions.

[nvarsj]

One of my more embarrassing interviews was early in my career, after I'd be developing HFT trading systems. I'd never worked on a web app in my life, and was asked the difference between 4xx and 5xx codes. I had no idea. Something I could have googled in about 10 seconds was the pass/fail criteria for an interview.

[mannykannot]

It is particularly galling when the interviewer either assumes it is an easy question now that they know the answer (especially if they did not figure it out themselves), or does not understand the subtleties that a more perceptive person realizes must be dealt with before reaching a justifiable answer.

[qumpis]

Can you elaborate on "and finding people who can role model a healthy exchange of rough-draft ideas is always a great boon for the team's psychological safety"? Would the opposite of this be flushing the idea maximally (and taking longer) before presenting it to the team?

[q7xvh97o2pDhNrh]

Yes, exactly. You want people to be able to say things like, "Hey, here's a crazy idea..." or "What if we..." without having to worry that they'll be torn apart instantly.

According to Jony Ive, Steve Jobs was exceptional at nurturing this kind of creative environment. He describes it perfectly in this WSJ article [1] from a couple years ago:

> As thoughts grew into ideas, however tentative, however fragile, he recognized that this was hallowed ground. He had such a deep understanding and reverence for the creative process. He understood creating should be afforded rare respect—not only when the ideas were good or the circumstances convenient.

> Ideas are fragile. If they were resolved, they would not be ideas, they would be products. It takes determined effort not to be consumed by the problems of a new idea. Problems are easy to articulate and understand, and they take the oxygen. Steve focused on the actual ideas, however partial and unlikely.

The other part of it is that ideas, as fragile as they are, can blossom and transform if you can get multiple people and multiple different perspectives involved in the early stages. That's why it's so powerful to have this kind of culture on a team; it lifts everyone up.

The problem with polishing the idea into a final draft before sharing it is that — whether or not it's good — it's almost guaranteed to not be as good as it could have been. And, by the time the "final draft" is ready, it's too late to get involved — all the interesting rabbit holes and tangents and possibilities have already been pruned away.

[1]: https://www.wsj.com/articles/jony-ive-steve-jobs-memories-10...

[OmarShehata]

Not the OP but I would say: yes exactly. A culture where you only see team members presenting ideas in their done state leads to overall worse solutions because people aren't collaborating & giving each other feedback on ideas-in-progress.

[temporallobe]

“psychological safety” is a newish buzz term that is basically an excuse for people who can’t handle criticism or debate, which I am seeing as increasingly common among younger devs. It is and should always be ok to freely exchange ideas without fear of being wrong, because everyone can be wrong. Even Einstein was occasionally wrong.

[wfme]

Your comment is a good example of a lack of psychological safety - you shoot the concept down with an intellectually dishonest argument, effectively ridiculing the idea.

The problem you describe is the part that is directly addressed by psychological safety. If you can provide a wrong answer and have people critique it but without ridicule, then you have a “safe” environment where people are comfortable to provide solutions without certainty on their optimality.

[ebiester]

It's the exact opposite. A team with strong psychological safety is exactly the type of group that knows how to present critique and debate, even between junior, senior, and management participants.

The point is that participants feel safe challenging an idea.

[buttscicles]

Psychological safety is what allows people to exchange ideas and be wrong without fear of it being used against them

[michaelcampbell]

> It's not, though. Many people have no clue how to interview well, and way too many tech interviewers are obsessed with whether or not the candidate got the "right" answer.

On the other hand, many interviewers are just normal people who have no idea how to gauge "how the candidate thinks", but like to think they are. I saw this a lot in the heyday of Google's brainteaser type questions. (And middle managers seem to revel in it.)

So the right answer is as good a proxy as they can hope for. It still sucks, but there you go.