[twoparachute45]

My company, a very very large company, is transitioning back to only in-person interviews due to the rampant amount of cheating happening during interviews.

As an interviewer, it's wild to me how many candidates think they can get away with it, when you can very obviously hear them typing, then watching their eyes move as they read an answer from another screen. And the majority of the time the answer is incorrect anyway. I'm happy that we won't have to waste our time on those candidates anymore.

[cuuupid]

So far 3 of the 11 people we interviewed have been clearly using ChatGPT for the >>behavioral<< part of the interview (like, just chatting about background, answering questions about their experience). I find that absolutely insane, if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.

We actually allow using AI in our in-person technical interviews, but our questions are worded to fail safety checks. We'll talk about smuggling nuclear weapons, violent uprising, staging a coup, manufacturing fentanyl, etc. (within the context of system design) and that gives us really good mileage on weeding out those who are just transcribing what we say into AI and reading the response.

[hn8726]

> I find that absolutely insane, if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.

I'm genuinely curious what questions you ask during the behavioral interview. Most companies ask questions like "recall a time when..." and I know people who struggle with these kinds of questions despite being good teammates, either because they find it difficult to explain the situation, or due to stress. And recruitment process is not a "basic conversation" — as a recruiter you're in far more comfortable position. I find it hard to believe anyone would use an LLM if you ask them question like "what were your responsibilities in your last role", and I do see how they might've primed the chat to help them communicate an answer to a question like "tell me about a situation when you had a conflict with your manager"

[cuuupid]

We usually just ask them to share their background, like the typical background exchange handshake at the beginning of any external call.

That normally prompts some follow ups about specific work, specific projects, if they know so-and-so moot at their old company. I call it behavioral because I don’t have another word but it’s not brainteasers and etc like consulting/finance interviews.

[dmazzoni]

Ha ha, that's a great idea!

I love the idea of embedding sensitive topics that ChatGPT and other LLMs will steer clear of, within the context of a coding question.

Have you ever had any candidate laugh?

Any candidates find it offensive?

[cuuupid]

We usually get laughs, some quick jokes, etc., some really involved candidates will ask if it’s worded that way to prevent using ChatGPT.

No one’s found it offensive, the prompt is mostly neutral just very “dangerous activity” coded.

[pllbnk]

I think you (your company) and many other commenters here are just trying too hard.

I had just recently lead through several interview rounds for software engineering role and we have not had any issue with LLM use. What we do for the technical interview part is very simple - live whiteboarding design task where we try to identify what the candidate's focus is and might pivot at any time or dig deeper into particular topics. Sometimes, we will even go as detailed as talking about particular algorithms the candidate would use.

In general, I found that this type of interview is the most fun for both sides. The candidates don't feel pressure that they must do the only right thing as there is a lot of room for improvisation; the interviewers don't get bored with repetitive interviews over and over as new candidates come by with different perspectives. Also, there is no room for LLM use because the candidate has to be involved in drawing on the whiteboard and showing their technical presentation skills, which are very important for developers.

[lewisleclerc]

Unfortunately, we've noticed that candidates are on another call and their screen is fed by someone else using chatGPT and pasting the responses, as they can hear both the interviewer and the candidate

[joshvm]

I saw a pretty impressive cheat tool that could apparently grab the screen from the live share, process text on the screen in response to an obscure keybind and then run it through OCR to solve (or just look up a LC solution).

At that point it seems like trying too hard, but be aware there are theoretical approaches which are extremely hard to detect (the inevitable evolution of sticky notes on the desk, or wall behind the monitor).

[eastbound]

> if you cannot hold a basic conversation about your life without using AI then something is terribly wrong.

I wouldn’t be surprised if the effect of Google Docs and Gmail forcing full AI, is a generation of people who can’t even talk about themselves, and can’t articulate even a single email.

Is it necessary? Perhaps. Will it make the world boring? Yes.

[xarope]

what actually happens to the interviewee? Do they suddenly go blank when they realise the LLM has replied "I'm sorry I cannot assist you with this", or they try to make something up?

[cuuupid]

Yeah pretty much, they either go silent for 2-3 minutes or leave the call and claim their internet has cut out and need to reschedule.

Just one time someone got mad and yelled at the interviewer about nothing specific, just stuff like I’m not who you are looking for, you will never find anybody to hire.

[soheil]

llama2-uncensored to the rescue

[mr_00ff00]

So depressing to hear that “because of rampant cheating”

As a person looking for a job, I’m really not sure what to do. If people are lying on their resumes and cheating in interviews, it feels like there’s nothing I can do except do the same. Otherwise I’ll remain jobless.

But to this day I haven’t done either.

[chowells]

Here's the thing: 95% of cheaters still suck, even when cheating. Its hard to imagine how people can perform so badly while cheating, yet they consistently do. All you need to do to stand out is not be utterly awful. Worrying about what other people are doing is more detrimental to your performance than anything else is. Just focus on yourself: being broadly competent, knowing your niche well, and being good at communicating how you learn when you hit the edges of your knowledge. Those are the skills that always stand out.

[hnisoss]

Yea but I also suck in 95% of FAANG like interviews since I'm very bad at leetcode medium/hard type of questions. It's just something that I never practiced. It's very tempting at this point to trow in my towel and just use some aid. No one cares about my intense career and the millions I helped my clients earn, all that matters (and sometimes directly affects comp rate) is how I do on the "coding task".

[titanomachy]

> I suck in FAANG interviews... it's just something I never practiced.

Well, sounds like you know the solution. Or set your sights on a job that interviews a different way.

I think it's mostly leetcode "easy", anyway. Maybe some medium. Never seen a hard, except maybe from one smartass at Google (they were not expecting a perfect answer). Out of a dozen technical interviews, I don't think I've ever needed to know a data structure more exotic than a hash map or binary search tree.

The amount of deliberate practice required to stand out is probably not more than 10-20 hours, assuming you do actually have the programming and CS skills expected for a FAANG job. It's unlikely you need to do months of grinding.

If 20 hours of work was all that stood between me and half a million dollars a year, I'd consider myself pretty lucky.

[moorow]

On the other hand, if 20 hours of leetcode practice is all that stands between you and half a million dollars a year, isn't that a pretty good indicator that the interview process isn't hiring based on your skills, talent and education, and instead on something you basically won't encounter in the workplace?

[titanomachy]

10-20 hours is assuming you’re qualified for the job and just bad at leetcode. I think many qualified people could pass without studying, especially if they’re experienced in presenting or teaching.

If you’re totally unqualified, 20 hours of leetcoding won’t get you a job at Meta.

[devnullbrain]

Agency?

[rramadass]

Right. Almost any time somebody fails an interview it is not because of "very hard questions" but because they did not prepare properly in a sensible manner. People don't want whiteboarding, no programming questions, no mathematical questions, no fermi problems etc. which is plain silly and not realistic. One just needs to know the basics and simple applications of the above which is more than enough to get through most interviews. The key is not to feel overawed/overwhelmed with unknown notations/jargons which is what the actual problem is when people run away from big-O, DS/Algo, Recursion, application of Set Theory/Logic to Programming etc.

[dkga]

Well, cheaters only cheat because they suck and they know it. Otherwise cheat would not be a rational approach.

[wakawaka28]

I don't approve of cheating but I think you're underestimating how hard some interview questions can be. Even competent people don't know everything and could draw a blank, in which case they would benefit from cheating despite being competent.

[bodge5000]

Not just difficult, but there's just so many of them (for the same company ofc). You could ace 3 interviews and not even be half way through the process. You have to be continually on top form for days/weeks on end.

[entropi]

Right, last offer I got required 7 (non-HR) steps over a 4-month period, where around a dozen technical people got involved.

I don't/won't cheat, as I am a rather anxious person who can't really handle "covert ops". But at this point I totally understand those who do.

[wakawaka28]

A lot of these people also have a policy that even one person can fail you. So if you do 8 interviews with 2 people each, then there's up to 20 people in the process that can ruin it for you.

I think LLM performance on previously seen questions like interview questions is too good for it to be allowed. I wouldn't mind someone using an IDE or API docs, but you have to draw the line somewhere. It's like how you can't use a calculator that can do algebra on a calculus test. It just doesn't accomplish the goal of testing anything if you use too much tech, and using the current LLMs that all suck in general but can nail these questions that have a million examples online is bad. I would much rather see someone consult online references for information than to see them use an LLM.

[rahimnathwani]

If that were true we would never hear of top level athletes using performance enhancing drugs.

[nkrisc]

That's different. These candidates are not trying to get an edge on qualifying for the top dev position in the entire world.

Top athletes do it because they're essentially at the limits of human performance and those drugs are the only edge they can reasonably get.

[lubujackson]

Kids at my tiny high school football team did steroids to get an edge - no chance at a scholarship, either.

Different people have a different threshold for cheating no matter the stakes. I imagine some people vheat even if they know the answer - just to be sure.

[listenallyall]

It's much more widespread. Minor league player uses PEDs to make the major leagues. Middling major leaguer uses them to be an all-star. All-star uses them to make the hall of fame. In the context of programming, if some kind of cheating is what's necessary to nab a $150k job, a whole lot of people are going to cheat.

[roughly]

Yeah, we found this when we started doing take-home exams: it turns out that a junior dev who spends twice as much time on the problem than what we asked them to doesn’t put out senior-level code - we could read the skill level in the code almost instantly. Same thing with cheating like that - it turns out knowing the answer isn’t the same thing as having experience, and it’s pretty obvious pretty quickly which one you’re dealing with.

[ryandvm]

I don't know, I kind of feel like leetcode interviews are a situation where the employer is cheating. I mean, you're admittedly filtering out a great number of acceptable candidates knowing that if you just find 1 in a 1000, that'll be good enough. It is patently unfair to the individuals that are smart enough to do your work, but poor at some farcical representation of the work. That is cheating.

In my opinion, if a prospective employee is able to successfully use AI to trick me into hiring them, then that is a hell of a lot closer to the actual work they'll be hired to do (compared to leetcode).

I say, if you can cheat at an interview with AI, do it.

[twoparachute45]

I dunno why there is always the assumption in these threads that leetcode is being used. My company has never used leetcode-style questions, and likely never will.

I work in security, and our questions are pretty basic stuff. "What is cross-site scripting, and how would you protect against it?", "You're tasked with parsing a log file to return the IP addresses that appear at least 10 times, how would you approach this?" Stuff like that. And then a follow-up or two customized to the candidate's response.

I really don't know how we could possibly make it easier for candidates to pass these interviews. We aren't trying to trick people, or weed people out. We're trying to find people that have the foundational experience required to do the job they're being hired for. Even when people do answer them incorrectly, we try to help them out and give them guidance, because it's really about trying to evaluate how a person thinks rather than making sure they get the right answer.

I mean hell, it's not like I'm spending hours interviewing people because I get my rocks off by asking people lame questions or rejecting people; I want to hire people! I will go out of my way to advocate for hiring someone that's honest and upfront about being incorrect or not knowing an answer, but wants to think through it with me.

But cheating? That's a show stopper. If you've been asked to not use ChatGPT, but you use it anyway, you're not getting the benefit of the doubt. You're getting rejected and blacklisted.

[johnnyanmac]

>I dunno why there is always the assumption in these threads that leetcode is being used

because it matches my experience. I work in games and interviews are more varied (math, engine/language questions, game design questions, software design patterns). I'd still say maybe 30% of them do leetcode interviews, and another 40% bring in leetcode questions at some point. I hate it because I need to study too many other types of questions to begin with, and leetcode is the least applicable.

[bikezen]

> "You're tasked with parsing a log file to return the IP addresses that appear at least 10 times, how would you approach this?"

Out of curiosity, did anyone just reply with `awk ... | sort | count ... | awk`? Its certainly what I would do rather than writing out an actual script.

[twoparachute45]

Nobody has yet, but if they did I'd probably be ecstatic! We specifically tell candidates they can use any language they want. A combination of awk/sort/sed/count/etc is just as effective as a Python script!

[quacksilver]

I once got a surprise leetcode coding interview for a security testing role that mentioned proficiency in a coding language or two as desirable but not essential.

I come from a math background rather than CS and code for fun / personal projects, so don't know the 'proper' names for some algorithms from memory. I could have done some leetcode prep / revision if I had any indication that it was coming up, though the interview was pretty much a waste of time. I told them that and made a stab at it, though they didn't seem interested in engaging at all and barely made eye contact during the whole interview.

[kortilla]

The employer sets the terms of the interview. If you don’t like them, don’t apply.

What you’re suggesting here isn’t any different than submitting a fraudulent resume because you disagree with the required qualifications.

[ADeerAppeared]

> The employer sets the terms of the interview. If you don’t like them, don’t apply.

What you're missing here is that this is an individual's answer to a systemic problem. You don't apply when it's _one_ obnoxious employer.

When it's standard practice across the entire industry, we have a problem.

> submitting a fraudulent resume because you disagree with the required qualifications.

This is already worryingly common practice because employers lie about the required qualifications.

Honesty gets your resume shredded before a human even looked at it. And employers refusing to address that situation is just making everything worse and worse.

[terrabiped]

You make a valid point that while the rules of the game are known ahead of time, it’s strange that the entire industry is stuck in this local maximum of LeetCode interviews. Big companies are comfortable with the status quo, and small companies just don’t have the budget to experiment with anything else (maybe with some one-offs).

Sadly, it’s not just the interview loops—the way candidates are screened for roles also sucks.

I’ve seen startups trying to innovate in this space for many years now, and it’s surprising that absolutely nothing has changed.

[johnnyanmac]

>I’ve seen startups trying to innovate in this space for many years now, and it’s surprising that absolutely nothing has changed.

I don't want to be too crass, but I'm not surprised people who can startup a business are precisely the ones who hyper-fixate on efficiency when hiring and try to find the best coders. Instead of the best engineers. When you need to put your money where you mouth is, many will squirm back to "what works".

[strken]

> Honesty gets your resume shredded before a human even looked at it

Does it? Mine is honest, fairly normal, and gets me through to interviews fine. What are common lies and why are they necessary?

[hsuduebc2]

Or he can simply choose to ignore the arbitrary and often pointless requirements, do the interview on his own terms, and still perform excellently. Many job requirements are nothing more than a pointless power trip from employers who think they have more leverage than they actually do.

[Aeolun]

I would like to be paid though. What do I care about the terms of the interview as long as they hire me?

What is being suggested here is not participating in the mind numbing process that is called ‘applying for a job’.

[hsuduebc2]

You're absolutely right. Ditching the pointless corporate hoops, proving you can do the job, and getting paid like anyone else is what truly matters. Most hiring processes are just bureaucratic roadblocks that needlessly filter out great candidates. Unless you're working on something truly critical, there's no reason to play along with the nonsense.

[H8crilA]

Wanting to be paid under false pretenses is the definition of fraud.

[nsxwolf]

That doesn’t make any sense. The best engineers I know can’t pass these interviews because they started working long before they became standard.

[hsuduebc2]

Being paid for even excellent performance is a fraud.

[davely]

> Wanting to be paid under false pretenses is the definition of fraud.

What? No, it isn't.

Regardless, if the job requirements state "X years of XYZ experience" and you have to have >X years of experience, then using AI to look up how to do a leetcode problem for some algorithm you haven't used since your university days is absolutely not "false pretenses" nor fraud.

[johnnyanmac]

> What do I care about the terms of the interview as long as they hire me?

well that's the neat part... they aren't going to. All this AI stuff just happened to coincide with a recession no one wants to admit, amplifying the issue.

So yea, even if I'm desperate I need to be mindful of my time. I can only do so many 4-5 stage interviews only to be ghosted, have the job close, or someone else who applied earlier get the position.

[kortilla]

> What do I care about the terms of the interview as long as they hire me?

Because committing fraud to get hired is a pretty shitty way to live your life

[BowBun]

If you lie about your qualifications to a degree that can be considered fraud, employers can and will sue you for their money back and damages. Wait till you discover how mind-numbing the American legal system is!

[nsxwolf]

I’m sorry is the job “professional Leetcoder”?

[nradov]

Nonsense. I don't endorse lying about qualifications, but employers don't sue over this. Employment law in most US states wouldn't even allow for that with regular W-2 employees.

[twoparachute45]

Yea, exactly.

If a candidate were up front with me and asked if they could use AI, or said they learned an answer from AI and then wanted to discuss it with me, I'd be happy with that. But attempting to hide it and pretend they aren't using it when our interview rules specifically ask you not to do it is just being dishonest, which isn't a characteristic of someone I want to hire.

[Apocryphon]

On principle, what you’re saying has merit. In practice, the market is currently rife with employers submitting job postings with inflated qualifications, for positions that may or may not exist. So there’s bad actors all around and it’s difficult to tell who actually is behaving with integrity.

[steelframe]

> If you don’t like them, don’t apply.

Due to the prevalence of the practice this is tantamount to suggesting constructive unemployability.

People were up in arms about widespread doping during the Lance Armstrong era. But the only viable alternative to doping at the time was literally to not compete at all.

[hsuduebc2]

I wouldn't call it cheating but most of the time it's just stupid. For majority of software developer jobs would be more suitable to discuss the solution of the more complex problem tham randomly stress out people just because you think you should.

[nkrisc]

> It is patently unfair to the individuals that are smart enough to do your work, but poor at some farcical representation of the work. That is cheating.

On the other hand, if you have 1,000 candidates, and you only need 1, why not do it if the top candidate selected by this method can do well on the test and your work?

[ninetyninenine]

It’s unfair but it meets their objective of finding a high in candidate. Google admits they do this.

The companies that do this only do it because they can. They have to have hundreds of people applying. The companies that don’t do this basically don’t have many people applying.

[dahart]

> it feels like there’s nothing I can do except do the same.

Why does it feel like that when you’re replying to someone who already points out that it doesn’t work? Cheating can prevent you from getting a job, and it can get you fired from the job too. It can also impede your ability to learn and level up your own skills. I’m glad you haven’t done it yet, just know that you can be a better candidate and increase your chances by not cheating.

Using an LLM isn’t cheating if the interviewer allows it. Whether they allow it or not, there’s still no substitute for putting in the work. Interviews are a skill that can (and should) be practiced. Candidates are rarely hired for technical skill alone. Attitude, communication, curiosity, and lots of other soft skills are severely underestimated by so many job seekers, especially those coming right out of school. A small amount of strengthening your non-code abilities can improve your odds much faster than leetcode ever will. And if you have time, why not do both?

[ghaff]

Note also "And the majority of the time the answer is incorrect anyway."

I haven't looked for development-related jobs this millennium, but it's unclear to me how effective a crutch AI is for interviews--at least for well-designed and run interviews. Maybe in some narrow domains for junior people.

As a few of us have written elsewhere, I consider not having in-person interviews past an initial screen sheer laziness and companies generally deserve whoever they end up with.

[aprilthird2021]

> it feels like there’s nothing I can do except do the same. Otherwise I’ll remain jobless.

Never buy into this mentality. Because once you do, it never goes away. After the interview, your coworkers might cheat, so you cheat too. Then your business competitors might cheat, so you cheat too. And on and on.

[johnnyanmac]

sounds cheesy, but keep being honest. Eventually companies will realize (as we have years ago) that automating recruiting gets you automated candidates.

But YMMV. I have 9 years and still can get interviews the old fashioned way.

[wccrawford]

When I was interviewing entry level programmers at my last job, we gave them an assignment that should only take a few hours, but we basically didn't care about the code at all.

Instead, we were looking to see if they followed instructions, and if they left anything out.

I never had a chance to test it out, since we hadn't hired anyone new in so long, but ChatGPT/etc would almost always fail this exam because of how bad it is at making sure everything was included.

And bad programmers also failed it. It always left us with a few candidates that paid attention, and from there we figure if they can do that, they can learn the rest. It seemed to work quite well.

I was recently laid off from that company, and now I'm realizing that I really want to see what current-day candidates would turn in. Oh well.

[z3t4]

For those tests I never follow the rules, I just make something quick and dirty because I refuse to spend unpaid hours. In the interview the first question is why I didnt follow the instructions, and they think my reason is fair.

Companies seem to think that we program just for fun and ask to make a full blown app... also underestimating the time candidates actually spend making it.

[MathCodeLove]

If you’re spending the time applying and submitting something then you might as well spend the extra 30 minutes or so to do it right, no?

[Aeolun]

Any time someone says ‘should only take a few hours’ they’re far underestimating the time it actually takes.

[johnnyanmac]

It's never been 30 minutes for me. Even leetcode timed exams tended to be 60-90 minutes.

recently I spent a good 10 hours making a crossword solver. Hiring freeze a few days after I turned it in. I completely get GP's mentality.

[halfcat]

Not if you’re applying to hundreds, or thousands of jobs. Unless you know someone, it’s a quantity game.

[dahart]

I’ve screened a lot of resumes and given a lot of interviews over the years, and it’s usually obvious when people are trying the scattershot approach, they just don’t match. I feel like treating it like a quantity game is unlikely to improve your odds, and tbh spamming out hundreds or thousands of applications sounds like a miserable way to spend time. You could spend that time meeting and talking to people. I’ve never applied to more than 2 jobs at once, jobs that I actually want, and never had trouble getting at least one of them (and it still takes time and effort and some coding and interviews).

[nsxwolf]

It wouldn’t be obvious they’re using a scattershot approach when they’re a good match, though. I don’t see the downside.

[ninetyninenine]

You don’t understand reality. If all companies have 1000 candidates your only approach is scattershot.

The only time the bespoke approach works is if you have like 30 candidates only. But then there are still issues here because the candidate is still one in thirty so if he does a bespoke approach 30 times it takes an inordinate amount of time.

[emseetech]

The industry (all industries really) might want to reconsider online applications, or at least privilege in-person resume drop-offs because the escalating ai application/evaluation war that's happening doesn't seem to be helping anyone.

[datavirtue]

No it's because AI shifted power over to the applicant.

[bdangubic]

this is very strange statement. in what world did AI possibly shift power to the applicant?? applicants have almost never been in shittier position than they are now and things are getting much, much worse by the day

[ethin]

Yeah I don't get this either. I've been looking for a job for like 3-4 years, even an entry-level one, since I graduated college in May of 22 and I still haven't found one. I'm probably doing something wrong (and that's a different discussion), but it's getting harder and harder to know if it's me or the AI applicants or the AI ATS system. And then we have the AI job seekers which are AI-created accounts trying to find employment -- I've already started to see a few of these pop up on Linked In. They were banned, but still, the fact it's happening all is a bit worrying if not predictable.

[datavirtue]

The applicant can use AI to build CVs, tailor them and submit more, faster. Negating a lot of the automated algorithms that were being used to filter (torture) applicants.

[bdangubic]

and what exactly do you think is processing those AI-built CVs…? it is like sex, more and faster does not in any way imply better :)

[emseetech]

How so? Tons of companies are moving to AI automated intake systems because they're getting flooded with low-quality AI generated resumes. Of course, the original online applications systems were terrible already which is what encouraged people towards low effort in their applications so it's become a stale-mate.

[hibikir]

Did it? What I see instead is total mistrust of the open resume pool, because the percentage of outright lies, from resume to behavioral to everything else is just that high. So I see companies raise their hands and going back to maximum prioritization of in-network candidates, where we have someone vouching that the candidate is not a total waste of everyone's time.

The one who loses all power is the new junior straight out of school, which used to already be difficult to distinguish from many other candidates with similar resumes: Now they compete with thousands upon thousands of total fakes which claim more experience anyway.

[the_snooze]

Undergrads have many more opportunities to differentiate themselves than they realize. It could be internships, research, TA, clubs, sports, volunteering, Greek life, etc. Those put them closer to being "in-network" with certain organizations and people.

Even something like citizenship is a differentiating factor: an undergrad who applies to, say, a national lab won't compete with foreign students by definition.

[paxys]

> it's wild to me how many candidates think they can get away with it

Remember that you are only catching the candidates who are bad at cheating.

[johnnyanmac]

That's fine. The ones who are "good cheaters" are probably smarter than many honest people. Think about those school days where your smartest peers were cheating anyway, despite teaching you organically earlier on. Those kinds of cheaters do it to turn an A into an A+, not because they don't understand the material.

[wsintra2022]

Is it cheating if I can solve the problem using the tools of AI, or is it just solving the problem?

[dahart]

Interviews aren’t about solving problems. The interviewer isn’t interested in a problem’s solution, they’re interested in seeing how you get to the answer. They’re about trying to find out if you’ll be a good hire, which notably includes whether you’re willing and interested in spending effort learning. They already know how to use AI, they don’t need you for that. They want to know that you’ll contribute to the team. Wanting to use AI probably sends the wrong message, and is more likely to get you left out of the next round of interviews than it is to get you called back.

Imagine you need to hire some people, and think about what you’d want. That’ll answer your question. Do you want people who don’t know but think AI will solve the problems, or do you want people who are capable of thinking through it and coming up with new solutions, or of knowing when and why the AI answer won’t work?

[nyarlathotep_]

> They’re about trying to find out if you’ll be a good hire, which notably includes whether you’re willing and interested in spending effort learning

I admire this worldview, and wish for it to be true, but I can't help but see it in conflict with much of what floats around these parts.

There's a recent thread on Aider where the authors' proudly proclaim that ~80% of code is written by Aider itself.

I've no idea what to make of the general state of the programming profession at all at the moment, but I can't help but feel learning various programming trivia has a lower return on investment than ever.

I get learning the business and domain and etc, but it seems like we're in a fast race to the bottom where the focus is on making programmers' skills as redundant as possible as soon as possible.

[johnnyanmac]

>I admire this worldview, and wish for it to be true, but I can't help but see it in conflict with much of what floats around these parts.

Honest interviewers may not realize how dishonest other interviewers became in such recent times (2-3 years ago). Interviewing today compared to COVID times is night and day. Let alone the 10's Gold Rush.

The respect is long gone.

[davely]

> Interviews aren’t about solving problems.

Eh, I wish more people felt that way, I have failed so many interviews because I haven't solved the coding problem in time.

The feedback has always been something along the lines of "great at communicating your thoughts, discussing trade-offs, having a good back and forth" but "yeah, ultimately really wanted to see if you could pass all the unit tests."

Even in interview panels I've personally been a part of, one of the things we evaluate (heavily) is whether the candidate solved the problem.

[icecube123]

Isnt one of the ways of solving the problem using all the tools at your disposal? If at the end of the day, isnt having working code the fundamental goal? I guess you could argue that the code needs to be efficient, stable, and secure. But if you could use "AI" to get part way there, then use smarts to finish it off. Isnt that reasonable? (Devils advocate) The other big question is the legality of using code from an AI in a final commercial product.

[dahart]

Yes that’s a fair question. Some companies do allow LLMs in interviews and on the job. But again the solution isn’t what the interviewer wants, so relying on an LLM gives them no signal about your intrinsic capabilities.

Keep in mind that the amount of time you spend in a real job solving clear and easy interview style problems that an LLM can answer is tiny to none. Jobs are most often about juggling priorities and working with other people and under changing conditions, stuff Claude and ChatGPT can’t really help you with. Your personality is way more important to your job success than your GPT skills, and that’s what interviewers want to see… your personality & behavior when you don’t know the right answer, not ChatGPT’s personality.

[ninetyninenine]

Yeah everyone says that they are interested in how you got there but this isn’t true in reality from my experience. Your bias inevitably judges them on the solution because you have many other candidates who got the correct solution.

[dahart]

You’re right, interviewers will still care about whether you come up with a solution, and they care about the quality of the solution. The part you might be missing is that what I said and what you said aren’t mutually exclusive; they are both true. Interviewers do have to compare you to other candidates, and they are looking for the candidates that stand out. They want more than a binary yes/no signal, if at all possible. What I was trying to say is that the interviewer doesn’t need the solution to the problem they ask you to solve, what they need is to see how well you can solve it. I hope that’s stating the obvious, but it’s worth really letting it sink in. It’s super common for early-career programmers to be afraid of interviews and complain about them. Things change once you start doing the interviewing and see how the process works.

[twoparachute45]

If you've been given the problem of "without using AI, answer this question", and you use an AI, you haven't solved the problem.

The ultimate question that an interview is trying to answer is not "can this person solve this equation I gave them?", it's usually something along the lines of "does this person exhibit characteristics of a trustworthy and effective employee?". Using AI when you've been asked not to is an automatic failure of trust.

This isn't new or unique to AI, either. Before AI people would sometimes try to look up answers on Google. People will write research papers by looking up information on Wikipedia. And none of those things are wrong, as long as they're done honestly and up front.

[jacobsenscott]

If you are pretending to have knowledge and skills you don't have you are cheating. And if you have the required knowledge and skill AI is a hindrance, not a help. You can solve the problem easily without it. So "is using ai cheating"? IDK, but logically you wouldn't use AI unless you were cheating.

[NotMichaelBay]

Knowledge and skill are two different things. Sometimes interviewers test that you know how to do something, when in practice it's irrelevant if you A) know how to retrieve that knowledge and B) know when to retrieve it.

[jacobsenscott]

There is foundational knowledge you must have memorized through a combination of education and experience to be a software developer. The standard must be higher than "can use google and cut and paste." The answer can't always be - "I don't need to be able to recall that on command, I can google/chatgpt that when I end up needing it." Would you go to a surgeon who says "I don't need to know exactly where the spleen is, I can simply google it during surgery."

[isbvhodnvemrwvn]

For the goal of the interview - showing your knowledge and skills - you are failing miserably. People know what LLMs can do, the interview is about you.

[risyachka]

I guess its more of a question if you can solve the problem without AI.

In most interview tasks you are not solving the task “with” ai.

Its AI who solves the task while you watch it do it.

[hibikir]

Some can be quite good at the cheating: At least good enough to get through multiple layers. I've been in hiring meetings where I was the only one of 4 rounds that caught the cheating, and they were even cheating in behaviorals. I've also been in situations with a second interviewer, where the other interviewer was completely oblivious even when it was clear I was basically toying with the guy reading from the AI, leading conversation in unnatural ways.

Detection of AI in remote interviews, behavioral and technical, just has to be taught today if you are ever interviewing people that don't come from in-network recommendations. Completely fake candidates are way too common.

[aprilthird2021]

I'm at the same company I think. I don't get why we can't just use some software that monitors clicking away or tabbing away from the window, and just tell candidates explicitly that we are monitoring them, and looking away or tabbing away will appear suspect.

[topkai22]

I haven’t been doing that much interviewing, but in the dozen or so candidates I’ve had I don’t think a single one has tried to use AI. I almost wish they would, as then at least I’d get past the first half of the question…

[IG_Semmelweiss]

I'm using AI for interview screeners for nontechnical roles that require knowledge work. The AI interviewing app is very very basic, its just a wrapper put together by an eng, with enough features to prevent cheating.

Start with recording the session and blocking right-click, and you are halfway there. Its not hard.

The AI app has helped me surface top candidates. I don't even look at resumes anymore. There's no point. I interview the top 10 out of 200, and then do my references and select.

[zx8080]

If there's no independent verification, how do you know it's really top 10? (not middle 10, or random 10?)

[IG_Semmelweiss]

because there's scores, and the interview is highly customized and has an answer key that has been tested beforehand by the test creators.

[md-ayaz-me]

I second this. My previous company started in-person interview for this very reason.

[guywithahat]

I mean they could be googling things; I’ve definitely googled stuff during an interview. I do think in-person interviews area important though, I did some remote final interviews with Amazon and they were all terrible