[Terr_]

Whoah, hold up: Why should we believe that success using an LLM to (possibly blindly) look up the answer to interview-questions will strongly correlate to success using an LLM to craft good code, properly tested, and their ability to debug it and fit it into an existing framework?

Heck, at that point you aren't even measuring whether the candidate understood the question, nor their ability to communicate about it with prospective coworkers.

If there are any questions where "repeat whatever ChatGPT says" seems like a fair and reasonable answer, that probably means it's a bad question that should be removed instead. Just like how "I'd just check the API docs" indicates you shouldn't be asking trivia about the order of parameters in a standard library method or whatever.

[jliptzin]

Nothing I hire for requires someone to do the World’s Most Challenging ™ life or death problems under pressure from memory. I think that’s true for the vast majority of tech companies. If I need someone to wire up a database to a react interface, or write some cron scripts, or refactor an old nodejs codebase, that is all stuff that chatgpt would be a great tool to use. I don’t care whether they’re doing it from memory or not.

[Terr_]

> Nothing I hire for requires someone to do the World’s Most Challenging [...] from memory

That's a bit of a strawman: I didn't say anything about the ease/difficulty of the role being filled, and I implied rote memorization was not meaningful.

To reiterate, interviews should measure good data for choosing between candidates.

That's not happening when the given problem is solve-able by an LLM using a human as a proxy, everybody's just burning man-hours of company/applicant time on interview-theater that isn't useful for making a decision. (Well, not unless the hiring goals include "willingness to jump through hoops".)

[jliptzin]

And what if every problem the position I am hiring for is solvable by an LLM?