[mjohn]

I've never tried it, but in the age of AI why not ask candidates to interact with ChatGPT to solve a problem?

In my (limited) interviewing experience the good candidates were the ones who had a good mental model for solving novel problems, were inquisitive and asked good questions. I think these attributes are also a reasonable indicator of general intelligence.

The people who I hired that were less successful were the ones who solved the coding exercises but had a poor mental model of how to solve novel problems and did not ask good questions.

Trying to use ChatGPT could take the pressure off from regurgiating coding minutae. The interviewer would be able to observe how the candidate approaches the problem and whether they can spot where ChatGPT goes wrong. It might also help introverted candidates since it feels less confrontational and there is likely to be less anxiety about unimportant details (after all any syntax mistakes would be ChatGPT's mistakes).

[kweingar]

Could candidates opt out of using ChatGPT? I don’t think I’d be comfortable using it in an interview.

[mjohn]

Sure, why not, I'm just thinking out loud to be honest and am probably biased towards feeling quite comfortable using something like ChatGPT. I don't ask ChatGPT to write code for me but regularly use it to explore, a bit like a rubber duck that talks back.

Just curious - why would you not feel comfortable using it in an interview? Interesting to hear a different POV.

[kweingar]

It’s a little overwhelming given the time constraints. I don’t know how I would fare trying to juggle a conversation with the interviewer, a conversation with ChatGPT, and typing into a code editor all at the same time, especially when ChatGPT usage involves vetting the output for hallucinations or misunderstandings. A search engine would be much quicker and easier for me to use.