[iamsb]

Having unscripted conversations is one of the best way to be swayed by unconscious bias in interviews.

Even though this advice sounds awesome, I will be cautious of putting it into practice without thinking through the bias problem.

I do remember reading multiple research papers on this, but unable to find them at the moment. From anecdote - In the last company I worked in London, only one team (DevOps) did not follow scripted interviews. It was the least diverse team, not just in terms of representation, but in terms of diversity of thought. Most of it was comprised of "tech-bros".

Scripted interviews do not mean you ask through a basket of questions. It just means that you stay within the guardrails of a set of topics and you go through all the topics. With in a topic, you have fair amount of flexibility. For example if you are hiring for a mid level Java programmer your topics may include - Java 8, Testing pyramid, Functional programming, type safety, developer safety(CI/CD/Rollbacks/Code reviews/Pair programming etc), some domain specific knowledge and so on.

[ryandrake]

> In the last company I worked in London, only one team (DevOps) did not follow scripted interviews. It was the least diverse team, not just in terms of representation, but in terms of diversity of thought.

Did this result in poorer job performance for the DevOps team, or any other negative business results that were specific to that team? If not, who’s to say which interviewing method was better or worse?

As an interviewer I’ve always felt very constrained by scripted interviews and “approved question lists”. I always struggle to really evaluate a candidate when I’m asking pre-selected questions without knowing why I’m asking those questions.

[iamsb]

> Did this result in poorer job performance for the DevOps team, or any other negative business results that were specific to that team? If not, who’s to say which interviewing method was better or worse?

It did. And even if it had not in this particular case, it will hurt the company in the long run. There is not even a shred of doubt in my mind that diversity (of thought) is the best investment that leads to success.

I have been using scripted interviews, the same method I mentioned in original comment, for more than 5 years now, hiring more than 200 engineers in three continent and I am super happy with my results.

[jldugger]

> It did. And even if it had not in this particular case, it will hurt the company in the long run.

Honestly, having been on plenty of interview panels for a decade now for DevOps and SRE roles, I'm not sure structured interviews, no matter how awesome they are, can solve the recruitment diversity problem. The war was already lost the moment the job description was published, IMO.

[ButtSpark69]

I think it's also important to keep in mind that at minimum it would take a generation to truly solve in a root-cause way (one that won't just come back the minute you focus on something else). Many of these biases get built during childhood when your brain is so so much more plastic.

[jldugger]

I'm not sure anyone in Kindergarten knows anything about having a job, let alone DevOps. =)

[pallavkaushish]

I have seen the same thing happening in recruitment for marketing roles. Totally unscripted interviews often leads to suboptimal results in terms of the quality of the candidate and their fit for the role. Made this mistake firsthand before devising my own process of hiring.

And the process I follow is based on having an exhaustive list of questions covering all areas of the role but during the interview if something else comes up, I don’t mind pursuing that and going unscripted. It often helps me add more questions to my list so my list of questions keeps improving.

I’ve been following this process for last couple of years and the structured process has helped me hire some of the best people I had the privilege of working with. Also, I feel more confident that I’m hiring the right candidate. But of course, there could be an element of bias in there.

Side point - I’ve just finished a SEO hiring guide that covers my whole process of hiring an SEO person (both junior and senior roles). Will be publishing that in the next 4-5 days. If anyone is interested in purchasing a copy, my email is in the bio.

[dclowd9901]

Research shows more diverse teams deliver better results. You’re asking something that can’t be proven though: is this thing that is occurring better than the thing that didn’t occur. We can’t know the answer to that.

[josephg]

> Research shows more diverse teams deliver better results.

FYI, there's plenty of nuance in the research that people like to gloss over. My understanding is that diversity of background / experience improves team performance. But having a diversity of values amongst your team decreases performance.

For example, if you form a diverse team where some people care about profits above all else, and other people care more about doing good in the world, the team will become less effective. Its really hard to use this research when hiring because a lot of values questions (like "who did you vote for?") are somewhere between creepy and illegal to ask.

Source: I used to work with someone who had a PhD in psychometric assessment. People saying "diversity=good" was one of her bug bears. I haven't read the research myself.

[dclowd9901]

I would ask if there was any published studies showing that "diversity of values" was bad, because searching Google for "diversity of values bad" does not yield any relevant results.

Then I would ask what her political leanings were next.

Studies seem to show that diversity initiatives fail mostly because of a lack of comprehensive diversity and inclusivity effort, e.g. companies talking big about it, but not backing it up with action, money, people, etc.

[dgb23]

Would it be illegal/creepy to ask then to sort a list of propositions or something like that?

[serial_dev]

So which one is it? Is it"already shown by research" or "something that cannot be proven". You can't have it both ways.

[dclowd9901]

What I said was:

> Research shows more diverse teams deliver better results. You’re asking something that can’t be proven though: is this thing that is occurring better than the thing that didn’t occur. We can’t know the answer to that.

Research is based on the cumulative results of many recorded events, in which we try to garner a pattern of relationship.

A singular event might have two inputs: we either do the thing or we do not. We can observe the results of what we do, but we cannot observe the results of what we do not. With the aforementioned research, we can _estimate_ the results, and, in some cases, we should make decisions with those estimates in mind, but these are two entirely different concepts. Macro and micro.

[commandlinefan]

> Research shows more diverse teams deliver better results

Seems like a bit of a sacred cow - how much better? Better enough to sacrifice, say, experience, contacts, or unique business knowledge for? How much, exactly?

[angarg12]

Odd comment. We hate the tech interview because it doesn't fairly evaluate on the job performance of candidates, but apparently it's ok to reject them because they don't fit the interviewer's tribe.

[rimliu]

I wonder when will diversity require to also be skill-wise: like it is required to have someone on your team who cannot code at all, just for the sake of bringing "different perspective".

[iamsb]

Aren't most teams already fairly multi-disciplinary comprised of product managers, UI/UX, and other skill sets?

[angarg12]

I'm going to go out on a limb here and say I think most candidates don't hate scripter interviews, they hate robotic interviewers that go mindlessly through a checklist.

When I train interviewers my advice is: have a script, but be happy to go off it. If interviewers keep an authentically engaged conversation, the questions will blend into the background.

Scripts helps in several fronts. They help to reduce bias, standardise calibration and practices between teams, and provide a fallback to get the interview on track. However good interviewers should keep a living conversation with candidates, and take interesting cues off script.

[dclowd9901]

It also means you have definitive qualifications for correctness. It’s harder to say “I’m not sure about this person,” when they quantitatively answered every question correctly. If you “have a conversation,” it’s a lot easier to have those itches, which might be primarily driven by bias, make your hiring immediately become problematic.

[gamzer]

How is diversity of thought measured?

[iamsb]

Lack of diversity of thought is easy to spot IMO. Bigger point is - you should ensure your processes are setup to at least try to achieve the kind of diversity you would like to achieve. Those processes may still fail, just like anything else.

[bryanrasmussen]

If someone answered every question the way the interviewer thought it should be answered that is probably an indicator of lack of diversity of thought, but I think it would actually be hard to spot because they would really look like the perfect candidate - unless the interviewer thought of themselves as a bad candidate / unoriginal thinker - which is unlikely.

On the other hand someone answers in a way that you have decided before hand is just the misguided answers of a particular tribe can be lack of diversity of thought, because you happen to be right about the preconceived ideas of said tribe, but pretty hard not to see that as the result of bias.

If on the other hand someone says some things you don't understand, espouses ideas completely alien - insane!

on edit: clarification

[gamzer]

I honestly don’t comprehend this answer. You mentioned unconscious bias in interviews:

> Having unscripted conversations is one of the best way to be swayed by unconscious bias in interviews.

Yet, lack of diversity of thought is “easy to spot”. Without being swayed by unconscious biases?

[iamsb]

You are taking diversity of thought comment, which I mentioned as an attribute of a team and trying to apply to an interview, which is not what I intended. Your interview processes need to ensure that you do not hire based on biases, so that you will have a diverse team.

Also interview processes are not just interviewer protecting against his/her bias, but also against bias of interviewee which seems to be somehow lost in these discussions.

[dgb23]

By asking questions and comparing the answers.

[oblio]

Standard precog manual, page 46.

[philwelch]

> I do remember reading multiple research papers on this, but unable to find them at the moment.

That’s too bad, because the obvious followup question is whether those papers survived the replication crisis.