[thenewarrakis]

I recently conducted an interview for a position on my team and noticed that whenever I asked a question, the person being interviewed would give me a decent reply. But then whenever I asked a followup, the person started every followup answer with something like "I apologize if my responses are not meeting your expectations".

About halfway through it clicked that this person was just typing whatever I asked them into ChatGPT and then reciting me the answer :(

[sublinear]

Interviews are more of an art than a science. I try to keep it conversational and open-ended.

If I sense something amiss, my opening is simply for them to tell me about themselves with no further context or direction.

Normal candidates will start asking me what I want to know as they trickle out the things they're most proud of. They'll ramble for a bit and then eventually I will pull on some threads they gave to get the ball rolling.

Bad candidates will either freeze, start reading their resume to me, not talk about work, or just start saying nonsense. Maybe all of the above. I give an extra shot to those who freeze by giving them a nudge, but I wrap it up immediately with the rest.

[kfor]

Do you worry that you risk introducing bias into your interview process with this sort of unstructured questioning? There is quite a bit of research [1] demonstrating that structured and standardized interviews across candidates are one of the most crucial ways of preventing various types of bias, conscious or not.

[1] Here's a useful summary article: https://hbr.org/2016/04/how-to-take-the-bias-out-of-intervie...

[vasco]

Good interviewers have the candidate feel like it's open ended by asking open ended questions that still tick the boxes they need to tick. Meaning the interviewer has a structure and a series of things to make sure to get information about, but to a casual listener of 3 different interviews, they might not even be able to piece out what those questions are, because you can make it fully contextual to the person.

With a bit of luck and skill you can get through a whole interview and take all your notes and the candidate doesn't even feel when you switch from one question to the other. Start open ended, make sure you can tick the boxes you have to tick from your questionnaire and dig in the threads you need to dig. Some people aren't good at doing this and they need to be more led by the interviewer, in those cases you can easily "adjust down" and be more explicit, but this way you get the best of both worlds.

Standardized notes with specific topics as well as the opportunity for people to tell you about what _they_ think was interesting about those situations, which is hard to predict from just a questionnaire.

[michaelt]

You wouldn't want the entire interview to be unstructured, but because every candidate is different, I think it's more than fair to give people an opportunity to highlight their own strongest areas.

I wouldn't want to overlook a great candidate because I omitted to ask whether they invented UDP, or whatever.

[cutemonster]

Talking about their strongest areas or inventions can be part of a structured interview.

[sublinear]

Not unless you ask. Resumes omit many things due to length constraints, and the candidate might prioritize recent work over quality work if it's too long ago.

Legacy skills are important too, but those details don't always make the cut in both the resume and job description.

[cutemonster]

> Not unless you ask

Yes well that's what the interviewer is doing: asking things,

And the interviewer can have a question in their list: Ask job applicant about their strongest skills? (And maybe inform the applicant beforehand if they want to think about what to say)

[NoMoreNicksLeft]

Doesn't that depend on what they're trying to accomplish?

If one is trying to determine the top-performing candidate (for the position being hired for), does bias actually interfere with that?

Especially considering the sorts of bias that are likely to be introduced (those adjacent to personality preferences), those who don't fit well are likely to poorly influence coworkers to a degree that negatively impacts total performance of a team.

[MattPalmer1086]

Is the definition of a bad candidate simply someone who doesn't interview well?

Just wondering if you have any data that supports the idea that those people would not perform well in a role, or just that it's very hard to tell anything about them in an interview!

[manquer]

Effective communication and collaborating with a team is important competency to almost every role, while a few of the candidates may have technical skills and it was hard to identify in the interview, if they severely lack basic communication skills then it is likely they will not succeed in the job.

The exceptions are when the role requires and candidate has extraordinary (i.e. 10x) skills then most other typical requirements such as communication or behavior can be relaxed, such roles are not common and handling ( and interviewing) rockstars as a manager is special skill of its own.

[sublinear]

"Interviewing well" isn't just a matter of getting along with me. I pick resumes similar enough to what we already do (as most places do). If you can describe the day-to-day of what we do without much prompting that's a very strong signal you're a good fit. You don't need data for this beyond the experience of making a few bad hires and lots of good hires.

Like I said, the interviewing process is not much of a science. I'm nowhere close to the only person who does this. I've been on the candidate side of the interview process too you know.

[MattPalmer1086]

Sure, just curious. Interviewing well is hard, on both sides of the process.

At the end of the day you have to make a decision and if someone doesn't make it easy for you to hire them, then they don't get the job mostly.

I have been overridden a couple of times and candidates were hired that I thought were poor. In both cases I was right about their weaknesses, but in one case they had strengths I did not acknowledge and they were excellent. The other one was a disaster!

[cmh89]

> If you can describe the day-to-day of what we do without much prompting that's a very strong signal you're a good fit

Why not just ask that then? "Tell me about yourself" is just a bad open-ended question. I've been in enough interviews to anticipate generic bad questions like that or "what's your biggest weakness" style questions, but not everyone has.

Hiring managers frequently can't differentiate between people who are knowledgeable and people who are just good at interviewing. I've coached many of my peers and done lots of coaching with success because at the end of the day, interviews are about taking advantage of the information asymmetry between your actual experience and what the hiring manager can actually know.

[sublinear]

> Why not just ask that then?

Because that's not all I want to know. I want to see the first things that surface in their mind and what they enjoy talking about. It's not a trick question and the only wrong answers are very clearly wrong sometimes even to those who don't work in software.

> Hiring managers frequently can't differentiate between people who are knowledgeable and people who are just good at interviewing.

I don't agree with this at all. We don't have "hiring managers". That's the problem right there. What the hell are they going to know about the position if they're not in it?

The developers have always done the interviewing of candidates at every place I've worked. We know exactly what to ask and what technical answers actually indicate they can get the job done. It's not hard to know what to ask when you actually do the job you're interviewing someone about.

[cmh89]

>I want to see the first things that surface in their mind and what they enjoy talking about. I

How does that question get to that? I'd guess most of them are thinking 'why did I get asked such a vague question and how should I answer it. Do they want me to talk about my personal life, or maybe they have some weird confirmation bias. If they wanted me to talk about my work life, they'd probably ask right? What a weird question'.

>I don't agree with this at all. We don't have "hiring managers". That's the problem right there. What the hell are they going to know about the position if they're not in it?

You are the hiring manager if you are a manager who is responsible for hiring someone.

I've worked with a lot of managers and sat on lots of hiring panels. Managers who think they are going to play some mind game to get a purer view of someone are always bad interviewers who place huge amounts of stock in some arbitrary reaction to their mind game.

[l33t233372]

I’m wondering how differentiate:

> “ start reading their resume to me […] or just start saying nonsense”

From:

> “ trickle out the things they're most proud of. They'll ramble for a bit”

Generally rambling and nonsense are similar, and resumes contain trickles of things that people are most proud of.

[sublinear]

"Reading their resume" means not giving me more info beyond what's literally in their resume.

Example: "Oh well I've worked with React before. It was an interesting project. It was for an ecommerce project. (endless silence after)"

"Start saying nonsense" means they're just making things up.

Example: "So yeah I've used git before. It's a cool programming language. It reminds me of C. I love programming."

[Aeolun]

I’m always mildly confused by the question. Makes me wonder if you read my resume.

Still, I ask the question in more or less the same way during interviews. How people respond is more interesting than what they respond with.

[sublinear]

If you're interviewing with me I definitely read the resume. Not all places are the same though.

Occasional mild confusion in a good candidate is worth stumping the bad ones often. I do feel slightly bad, and I admit it's slightly rude, but it works and the interview process is what it is.

By the end of the interview, the rough start is hopefully forgotten. If you ever get this question upfront again hopefully you understand why and roll with it.

[crystaln]

Please stop doing this. You’re making judgements on candidates for social skills that most likely have nothing to do with their job.

Sounds like you’re interviewing for a date or a roommate, not an employee.

[9dev]

I'm sorry Dave, I can't do that. (Not GP, but acting the same) I'm looking for candidates I actually want to work with. That includes people with social skills sophisticated enough to freely talk about their prior work.

If a candidate can't do that, they are not a good fit for my company.

[canadianwriter]

I guess it depends on the position - if you have to work with people as a regular part of your position, social skills would be a part of the role, an important one at that.

[callalex]

Sheesh, I’m glad you’ve never interviewed me. What an odd trap to set.

[groestl]

What trap?

[wkat4242]

You do interviews over chat??? I've never seen that.

Weird of the person to just pipe in ChatGPT but conducting an interview without video is also weird IMO.

[thenewarrakis]

No, it was over video. The person was verbally reciting whatever ChatGPT gave to them. Their answers were verbally "I apologize for any inaccuracies, blah blah blah".

[Tostino]

That is pretty depressing. It's funny how easy it would be to not give yourself away too. It's ridiculously easy to spot some of the general ChatGPT tells and simply not recite them.

[djur]

They were typing in your questions? That must have been pretty slow and obvious.

[shagie]

There's a button on ChatGPT in my iPhone app that allows me to switch it to microphone mode rather than needing to type it. An off screen tap and you can get the results.

I'm sure that some people have hooked up the whisper model so that it can tap a desktop audio out and feed ChatGPT too.

[thenewarrakis]

Typing during an interview is pretty common since many people (interviewers and interviewees alike) take notes during interviews, so the typing alone didn't stand out as unusual.

[jozvolskyef]

I once concluded an interview with a candidate who was rather conspicuously engaged in clandestine typing, subsequently rectifying their previously incorrect answers. I suspected they were resorting to Google, given this was before ChatGPT came out. Much to my surprise, they ended up landing the job.

[PUSH_AX]

Arguably if someone can google the “answers” to your interview questions in 20 seconds they aren’t good questions to ask anyway.

[notahacker]

And if someone's ability to answer questions using search tools in real time is sufficient to impress a series of interviewers, they're probably quite resourceful at finding solutions to problems they've been given a few hours to solve

[metadat]

They got hired even though you said "no"? That sounds extremely unusual, typically employers of SDE types are cautious this way.

[jozvolskyef]

I don't know what exactly happened there. The two of us conducting the interview advised against this candidate in our report. About a month later the candidate was introduced as a new hire on a different team and I never saw them again in a company of <100 people.

[isaacremuant]

Even without the answers, I can't imagine I wouldn't realize by the pace and format of the responses that they were simply read to me instead of said.

I've never had such experience and it seems very low effort.

In any case there's absolutely no risk of someone passing an interview with ChatGPT on our end because it's about seeing how people think and what they've experienced.

[derealized]

Automattic does this.

https://automattic.com/how-we-hire/

[LordDragonfang]

Y'know, that's surprisingly on-brand for the company that owns tumblr, a website full of socially anxious introverts who would rather perish than make a phone call.

[derealized]

That's me! https://imgur.com/a/Zbv5da8

[Kiro]

Anyone who does not choose death over picking up the phone is a true psychopath.

[goatlover]

Also weird that they just repeated back exactly ChatGPT. How lazy can you be?

[vidarh]

Many years ago I had someone copy and paste two pages from an Oracle manual, complete with formatting, in response to an e-mailed screening question about MySQL that required a one line answer. It was just an initial screening where we had relatively low expectations and halfway expected a lot of people to search - letting a few too many people through was ok, we just wanted to whittle down the pool.

But after that we realised the screening worked even better when we input random sentences from the answers into Google and used it to weed out the people who were not just dishonest enough to give a verbatim answer from somewhere without disclosing it, but who very often did so in ways that meant they got it horribly wrong. E.g. another example was the guy who copied and pasted a forum post. Not only was the answer wrong, but it was followed on the forum by a large number of responses from other people explaining why it was wrong.

[jojobas]

You'd be surprised. After all, if we were not lazy we'd be still digging the ground with our bare hands. "How do I get X while doing less work" is a kind of laze that defines human.

[lukifer]

The engineer’s three virtues: Laziness, Impatience, Hubris

https://wiki.c2.com/?LazinessImpatienceHubris

[hef19898]

laziness can be a good thing! Real lazy people put in a lot of effort to achieve the same outcome working less!

[WarOnPrivacy]

Procrastination too. I've had things go off the rails because procrastination wasn't allowed to manifest opportunities.

[robertlagrant]

Sounds as though you've read the page before.

[red75prime]

> "How do I get X while doing less work"

And we'd still be using neanderthal tools if everyone just copy-pasted previous work.

[sn41]

Reminds me of Ron Burgundy!

[cromulent]

Absolutely. Reminded me of Chris Pratt's famous P&R flu scene ad lib "Leslie, I typed your symptoms into the thing up here and it says you could have network connectivity problems".

[SgtBaker]

Did you not ask "Why are you sounding like chatGPT, are you actually reading straight off the prompt?"

Interviews go both ways

[dahhowl]

The point being?

As an interviewer you want to avoid being confrontational with your candidates. It's not helpful if they're good candidates, it's not helpful if they're bad-faith candidates like this one (they might be looking for some slip up on your side which makes the interview "illegal" and cause for them to sue your company), it's not helpful if they're simply not fit for the job (but in good-faith) as you don't want to make them feel bad they're not fit for the job.

If you already understood that they are not giving satisfactory answers, or like in this case they're just reading off the chatgpt answers, best to just keep gathering more proof to your decision not to hire them (by asking them more questions which can highlight your reasons) and finish the interview with the good ol' "we'll let you know."

[robertlagrant]

"If you were a Henry VIII-style character but in the film Trainspotting, how would you describe an n-tier architecture capable of serving 50 requests per second?"

Edit: I had to. Here's a small excerpt: Scalability: Allow the architecture to grow in strength and size.

[pharrington]

There are worse things than feeling bad about not being fit for the job. One of them is falling into perpetual confusion and self-doubt because you got ZERO feedback as to why you didn't get the job. Deliberately holding back information that would help the other person achieve their goal is deception, despite your good intentions. Be more honest.

[cutemonster]

I wonder if / think that that way of thinking, applies to many areas in life? At least that's how most people go about dating, is it not (no one says "oh you're just pretending to be such a successful man / good looking woman but you're just making things up)

[tayo42]

How are people like this getting interviews and my resume goes to some black hole...

[bee_rider]

I assume they just send out a million resumes. If you do thing correctly you might have a 10x chance, per resume sent, or getting an interview. So they send 100x as many. This is possible because they are putting in 1/1000’th of the effort.

[eru]

There probability of getting an interview might still be lower than your probability.

[lvncelot]

Heh, it's the Earpiece Conversation trope[1] in real life.

[1] https://tvtropes.org/pmwiki/pmwiki.php/Main/EarpieceConversa...