[domrdy]

As with everything, it depends. Live coding interviews work. They’re not the best candidate experience, but they work at Meta, Google scale, minimizing false positives better than most other formats. What makes them stressful is the lack of interviewer training and the abstract, puzzle-like nature of the problems, which you can really only solve if you’ve spent time studying (e.g., LeetCode) or you’re fresh out of college or academia.

I’ve worked in the assessment space for 6 years and have seen many hiring processes, from Fortune 10 companies to startups hiring their first engineers. The range of signal required, "how much time can my engineering team spend with a candidate", and how much candidate experience you can get away with, is huge. I’ve also been a candidate myself and failed many live coding interviews. It made me feel terrible about myself. The last time for a role at Ycombinator (the interviewer was super nice).

When I work on my product, I try to view it through the lens of empowering candidates to show their skills and potential. I encourage our customers to use assessments that somewhat resemble on-the-job skills. I don’t like the phrasing “real work” anymore. An assessment shouldn’t be unpaid labor, it should be a way for candidates to demonstrate that they can do the job and handle future work thrown at them, and for hiring managers to feel confident extending what are often very high salaries in tech.

With AI, unfortunately, short take-homes (what I prefer as a candidate, using my own tools and editor) are becoming harder to maintain as a fair signal due to AI assistance. I’ve seen companies move back to onsite, and competitors deploy all kinds of proctoring and invasive monitoring.

The perfect solution, in my view, would be an assessment where the candidate feels relaxed and able to perform at their best, with their own editor and configuration, knowing that every other candidate in the pool has the same constraints in terms of time and tooling. It’s a tough problem to solve. I think about it daily and have not come up with a solution.

[zwnow]

A brick layer isn't asked to build a wall prior to getting a job. A certificate confirming he did learn the job is enough for companies to employ them. Same goes for a lot of other jobs. Just hire devs and kick them out if they dont fit ur needs. Why go through these humiliating corpo processes?

[jasode]

>A brick layer isn't asked to build a wall prior to getting a job. A certificate confirming he did learn the job is enough for companies to employ them.

It depends. A welder looking for a job -- even with a certification -- will often have to demonstrate his welding skill to the jobsite supervisor before he is hired. An airline pilot -- even with a rank of "captain" with 5000+ hours -- when trying to find employment with another airline, will still have to demonstrate piloting skills with a "check ride" as part of the hiring approval process. Sometimes, experienced pilots fail the check ride for whatever reason.

The common theme is that "existing credentials" is sometimes not enough.

[sarchertech]

In both of those cases they are actual work sample tests. A pilot is expected to demonstrate exactly what they do on the job every day. They don’t need to practice for interviews.

Those kinds of demonstrations are also very for professional jobs outside of hiring new grads.

And even in the trades it isn’t common.

[jasode]

>A pilot is expected to demonstrate exactly what they do on the job every day. They don’t need to practice for interviews.

Airline pilots that were laid off and are trying to find employment with another airline do study airmanship in preparation for interviews with another airline. Especially if the pilot is not type rated for the particular airplanes the other airline flies. E.g. the experienced pilot currently is rated for Airbus A320 but the airline he's applying only flies Boeing 747. The pilot studies because he wants to stand out from other candidates so the prospective airline wants to hire him and willingly pays for ongoing 747 training. Yes, the airline interview and check rides with the flight instructor are stressful because they can still fail even though they have 1000s of hours of existing flight experience.

[sarchertech]

> Especially if the pilot is not type rated for the particular airplanes the other airline flies. E.g. the experienced pilot currently is rated for Airbus A320 but the airline he's applying only flies Boeing 747.

That makes sense then because they are applying for a slightly different job.

It would make sense for a Java programmer to spend time prepping for a Python interview as well.

[mock-possum]

Languages are not monoliths though - a Java programmer might spend ten years focusing on one area of development, invested in one particular library or framework, or specific to one particular device - and then might spend time prepping for something different, still solely using Java.

In short - unless you really are just moving to ‘same job different company,’ not just same title, but same actual work - you’re gonna need to brush up while you’re trying to find your next gig. It helps you stay engaged with your profession, anyway.

[MontyCarloHall]

>In both of those cases they are actual work sample tests. A pilot is expected to demonstrate exactly what they do on the job every day. They don’t need to practice for interviews.

The example in the post (sum all even numbers in a list) is a perfect example of something programers do on the job every day. It's not leetcode puzzle bullshit; filtering and aggregating lists is an extremely frequent task in software, and no competent programmer should have to study to perform it.

[sarchertech]

I didn’t read the twitter blurb in the article where is mentions the test.

I was basing this on the 30 minute live coding session the author talked about.

Yeah that screening is simpler than fizzbuzz. I also think it’s a worthless test and I believe the tweet is extremely misleading.

I’ve been doing this for a long time and if you exclude candidates that fail the most basic resume screen, I would be surprised if 10% failed that test.

If you filter for only senior candidates with verifiable experience, I’d be surprised if more than 1% failed it.

If 75% of the candidates that get through to your coding screen fail that something is extremely wrong with your hiring process.

[rockemsockem]

> And even in the trades it isn’t common.

Source???

I'm definitely aware of welders and machinists having to show the quality of their work for interviews.

You might not think that basic coding/algorithms problems are good tests of actual performance, but being able to solve a problem, talk about your approach, debug issues when they come up, and then discuss expanding the solution in prod including trade-offs is actually a really great indicator of future performance in my experience.

[rockemsockem]

So much this.

Some people so often say that credentialing helps all these industries that they know nothing about, when in fact the norm or high-performing end of those industries do the exact same thing.

[josephg]

> A certificate confirming he did learn the job is enough for companies to employ them.

This is hilariously out of touch with real world hiring.

If you put up a job ad, there are so many people who will apply with all the certifications you can name. And if you ask them to write code, even something quite simple, they will fail utterly and completely.

I've interviewed a bit over 400 people at this point. When I was doing it as a full time job, people only talked to me after they pass a screening test - which already screens out the majority of applicants. Even then, about 3/4 of the people I've interviewed were terrible. So bad they could barely write hello world in the half hour we give them. This is on their own computer, in their favorite programming language. They did not have to talk to me during the process at all. A lot of the people who fail have graduated from decent universities. Some said they had 30 years professional experience.

I'm sure some of that is due to nerves. But a lot of it is simply because programming is hard, and most people don't have the right kind of brain for it. Lots of people - probably the majority if we're honest - bluster their way through certification programs or degrees. Many of them learn the theory but struggle with the practical skills. Sometimes they gather together in low performing teams at large companies where management doesn't know any better.

If you graduate from a music conservatory, you can probably play an instrument. But that isn't true of most computer science degrees. Lots of people graduate without being any good at programming.

Its also a numbers thing. Good programmers don't stay on the job market for long. Great people will send 1-3 applications and be hired. Or more likely be hired through word of mouth. Bad applicants might send hundreds. As a result, most job applications are written by dedicated people who get turned down over and over again by employers.

There's a reason fizzbuzz has become a cliche in our industry. If you put up a job ad, most people who send in an application won't be skilled enough to program it up.

[zwnow]

Fizzbuzz would be completely fine. Companies out there ask devs to solve highly niche cryptic tasks like the knapsack problem for no reason.

[IshKebab]

Yeah I think the fundamental reason this gets debated so much is that "whiteboard interviewing" encompasses both fizzbuzz, and leetcode dynamic programming nonsense. Some people are saying "fizzbuzz is great!" and others are saying "no you're totally wrong, leetcode is terrible".

Even this article falls into this trap. The first thing he quotes is fizzbuzz-level, but then the research paper he uses to argue against fizzbuzz actually used a much harder leetcode style problem.

IMO fizzbuzz-level problems are totally fine. You can't really argue against them. They filter out tons of people who I wouldn't want to hire, and nobody who should be hired can fail them, even under ridiculous pressure.

It's more debatable when you get to actually difficult algorithm problems but that's another argument.

(Also fizzbuzz itself is a pretty terrible "simple" problem because it feels like there should be an elegant "trick" solution but there actually isn't. Don't actually use fizzbuzz. The example in this article - filter out odd numbers from a list - is totally fine though.)

[akkartik]

Meta phone screens currently involve 2 medium/hard leetcode problems AND 20 minutes of behavioral questions. The inmates (i.e. we programmers) are running the asylum.

[baq]

who can pass this except recent college grads?

I wouldn't have any time to prepare for this unless I was laid off unexpectedly, then grinding leetcode would be a full time job for a while.

[josephg]

> who can pass this except recent college grads?

I can. So can a lot of the better candidates I interviewed. Most googlers and fb engineers can pass this stuff too.

Theres a much better way to test for senior engineers though, and it’s this: prepare a small program (200 lines or so) with some bugs in it. Write a few test cases which highlight the bugs. Then give the candidate the code and the tests and ask them to fix the buggy code.

Good senior engineers - in my experience - aren’t better than smart young grads at programming. But they’re soooo much better at reading code and debugging it.

I interviewed this insanely good Ruby engineer once. He had the body of a bear, and a bushy, greying beard to match. During the interview he gave me a masterclass at debugging. Before he even looked at our tests, he read the code and named out loud assumptions he was making about it. Then he started writing his own test suite to validate those assumptions during the interview. He fixed a few of our bugs before he even looked at our tests, in the first 10 minutes of the assessment. And he fixed another bug we didn’t know about. I can’t remember how he did at the programming section of our interview. But I’d hire him in a heartbeat from watching him debug.

The one thing to keep in mind if you do this is that most people will overestimate how much debugging you can do in half an hour or an hour and overcomplicate the program you give candidates. If you make a test like this, make the code simpler than you think and actually calibrate its difficulty with your coworkers.

[devnullbrain]

Me. Because, for about 2 weeks during lay-off gardening leave, I did study it.

On one hand this demonstrates that it isn't useful for testing innate skill.

On the other hand, it was the best paid work I've ever done. It was an essential part in me getting a job (not at Meta) that paid multiples better than previous roles.

This work allowed me to place myself in a group of people who can pass the test. Some of the people who are outside this group would make fine employees but don't want to or think to put in this work, for understandable reasons. But all of the lackadaisical applicants will fall outside this group. I get to signal that I'm not one of them.

I spent 4 years in higher education for a non-essential positive signal to employers: 2 weeks is nothing.

In the past I hated Leetcode, now I've begun to think of it as an opportunity. While as an individual my opportunity to shift the consensus on this form of interview is low, my opportunity to benefit from the arbitrage is high. Don't hate the player, etc.

[ericbarrett]

I once got asked to write a Levenshtein edit distance calculator for a “15 minute” SRE phone screen

[Sohcahtoa82]

I'm not sure I could write that even with a full day.

Even if I did, the solution would probably end up running in O(n!) time.

[michaelt]

> Just hire devs and kick them out if they dont fit ur needs. Why go through these humiliating corpo processes?

People actually find being fired a greater inconvenience and humiliation than a 60 minute whiteboard interview.

[sarchertech]

Passing a whiteboard interview doesn’t mean you are going to be a great fit for the job. It just means you can code.

If you don’t lie about being able to code, you’re no more likely to be fired than if you had passed a whiteboard interview.

[wijwp]

> Passing a whiteboard interview doesn’t mean you are going to be a great fit for the job. It just means you can code.

Did anyone say to only do a whiteboard interview?

[sarchertech]

No they didn’t, but passing the white board interview doesn’t make you any less likely to be fired unless you lied about your ability to code.

I have never seen someone get fired for things that otherwise would have showed up in a whiteboard interview (at places that didn’t do them).

[seadan83]

According to the data in the article, the white board does not even show ability to code.

[pelcg]

The truth the employers won't admit is that there is just too much money in this industry for very little effort.

Why do you think that famous employees get straight offers without doing anything verses many engineers still getting leetcoded and get left with no offer despite being over-qualified?

Soham was able to pass most programming assessments so well, the folks at bookface were discussing to ban him from applying to their startups.

You can see that another system has been created to make sure that the role is reserved for friends of the founder, ex-collegues of another team over an extremely qualified engineer out of no where.

[piqufoh]

In many non-US countries once hired there are employment rights. You cannot simply "kick them out if they dont fit ur needs". Isn't it preferable and less stressful for everyone if you can find the right person without having to hire and fire others first?

[sarchertech]

Most countries have probationary periods before most of those rights kick in.

[pjmlp]

Depending on the European country, there is a probation period between 3 to 6 months, where any of the parties can cancel the relationship at any time, usually 1 week notice, unless it is really bad.

That should be more than enough to assess if someone is fit for the job.

[hollerith]

How does an employer distinguish a worker who is trying hard only because he is on probation from a worker who will continue to try hard after the probation period ends?

[zwnow]

If they try hard for 6 months during probation, then congrats on having a motivated dev for 6 months. If they fall off hard after, kick them out. It's only 3 months of salary. Compare that to thesalary and hiring process of finding a good dev, which is more expensive in many cases.

[hollerith]

>If they fall off hard after, kick them out. It's only 3 months of salary.

Thanks for the info.

[bee_rider]

That wouldn’t be caught in a live-coding interview either, right?

At some point of your society has decided it values job security, the jobs will have to become secured. It is a trade-off.

[hollerith]

OK, but that is not responsive to "Just hire devs and kick them out if they dont fit ur needs."

[quails8mydog]

Coding tests aren't filtering for people who work hard, they're filtering for people who know how to code. Whether they will work hard on the job seems like an orthogonal question?

[Ekaros]

They don't but usually wages are scaled to average. So So average output will still be what is expected. Really bad ones well, you start giving notices. And then with enough evidence you can terminate contract.

[const_cast]

You can't, in the same way you can't distinguish a romantic partner who is using you from one who genuinely likes you. Because clairvoyance is not real.

That's just a risk we all have to take.

[devnullbrain]

There is no 60 minute test for months of malingering

[piva00]

In many non-US countries there is also a probation period before full employment rights kick in, allowing employers to fire new hires without reason, so definitely you can "kick them out if they don't fit your needs" in many countries.

[bluedino]

To work for our local utility company, they make you dig a giant hole by hand and you can't hit the line etc that's under the ground, and you have to scale some poles.

[munificent]

> Just hire devs and kick them out if they dont fit ur needs.

A developer is likely quitting another job they are heavily invested in, possibly moving across the country, and taking a risk when changing health insurance.

If you ask them to do all of that and then immediately turn around and fire them because you didn't really hire them, you just did so probationally without telling them, then you are running an absolutely toxic business. Word will get out and no decent candidate will even consider working for you.

[amosslade]

Brick layers are easy to fire on day one.

I’ve hired many construction workers over the years. In the construction industry, if an interview goes longer than 15 minutes you’re doing it wrong.

Interview summary: Interviewer: “Can you do the work?” Interviewee: “Yes.” Interview over.

When they start working, if they can’t do the work, they’re fired.

[zwnow]

Yea, and why doesn't this work in software in your opinion?

[falcor84]

Brick laying is a job where the progress is immediate and easy to assess. The better analogy would be of hiring a civil engineer - would you hire one to work on your project based just on a certificate?

[sarchertech]

None of my civil engineer friends do white board interviews unless it was a new grad hire. They mostly go off of work experience, degree, and certifications.

[pjmlp]

Do they get to talk about bridges they build on their spare time?

[sgerenser]

Wait, you don't build bridges in your spare time? You don't even have anything on BridgeHub? Immediate reject.

[ryandrake]

I wonder if hospitals ask candidate doctors about the hobby surgeries they perform at home in their spare time.

[falcor84]

Well obviously most doctors don't have an OR at home, so they practice their going surgeries at a co-operating space.

[ozgung]

How do you hire a civil engineer? Do you ask them to solve a series of construction puzzles for few hours?

[parpfish]

the FE and PE licensing exams are essentially a one-time test of "hey, can you solve these puzzles / do you know these facts".

[dionian]

Yeah well its different too because often the coding interviews are more like 'we know you're used to building with real bricks but just do it blindfolded without mortar in 100x less time for the interview, thanks'

[SpicyLemonZest]

People who see it as humiliating are misunderstanding the dynamics. Precisely because software engineers have so much market power, it's not so simple to kick them out; a company that developed a reputation for outright firing people would have serious recruiting issues. If a manager at Google, Facebook, etc. decides that one of their reports isn't doing a good job and has got to go, the process is generally to laboriously help them realize over the next few months that they've chosen to leave and are excited to explore new opportunities.

[nathan_douglas]

It would seem to me that a company who fired subpar performers would have a better reputation than a company that makes record profits and then lays off a few thousand engineers a quarter, and yet...

[SpicyLemonZest]

Companies in this category generally give multiple months of severance pay for the same reputational reasons. Cold comfort, of course, to the specific developers who got laid off and would rather have the job. But it effectively signals to the next crop of hires that the company is genuinely committed to their compensation packages.

[domrdy]

I think certificates are interesting in theory, and there’s an entire industry built around them. AWS does a solid job with its certifications: if you earned one five years ago, it’s still more or less relevant. I'm not sure how certifications would work for proprietary technology.

Labor laws in many places are less “flexible” than those in the US, and they don’t support what you’re proposing, for good reason. I wouldn’t quit my job or uproot my family just to convince a manager I’m worth keeping.

[scarface_74]

I’ve had 9 of the then 10 AWS certifications at one point and right now I have 7.

There have always been “brain dumps” for certifications as far back as 2007 with Microsoft certs.

The AWS certificates are easy to cram and pass with no real world experience. I only got mine as a guided learning path with a goal at the end. In fact, I passed the first one in 2018 without ever logging into the console and got all three (at the time) associate level certs within 6 months after opening the console at my job at a startup and the two “Pro Certs”.

Even AWS Professional Services (AWS internal consulting department where the consultants work for AWS full time) doesn’t make certifications a requirement coming in. But you have to get a couple within 90 days (associate) and 6-9 months (pro cert) depending on your position. I’m no longer there.

[sarchertech]

Most countries with very strict worker protections have probationary periods during which it’s much easier to fire people.

[tayo42]

These physical jobs have alot of process in place to assure some one can't do a bad job and if they do you can hold them accountable.

There's no code to match for software like there is construction. Auditors don't know anything. Managers and abibey are often to disconnected from work to know what's good and bad.

[zwnow]

Maybe its time for standards then?

[sarchertech]

Outside of new grads, Capital E Engineers don’t do whiteboard interviews. Teachers don’t teach mock classes etc…

[gjm11]

I don't know whether it's different in the US, but here in the UK it's pretty common for teachers to teach part or all of an actual lesson as part of the application process. Experienced teachers as well as new graduates.

[sarchertech]

I’ve never heard of an experienced teacher doing that in the US and I have plenty of friends who are teachers.

[smokel]

If someone is hired, they will most likely continue for a long time.

They will make friends, spend time learning the domain, and a company will invest quite some hours.

During the interview phase, it is easier to swap candidates.

[tester756]

Isnt betting fired more humiliating? like 10x more?

[zwnow]

Not if you went through 100 of these unnecessary interview processes solely due to small companies copying big tech where half of their dev team couldn't complete their hiring process.

[seadan83]

The humiliation is the least of it IMO. When interviewing, let's say you've got to leads. You get hired and tell the test to pound sand. Companies (I've found) never like that. It's a burned bridge (despite the strong hire signal of getting hired).

Now, a month later - fired - it's worse than not having worked at all. Good luck reaching back out and starting over from scratch with your best options now off the table.

[blindriver]

A brick layer that can’t do her job properly will be fired instantly.

If we could fire bad hires instantly, tech hiring could be a lot easier.

[zwnow]

You can. Lots of European countries have a 6 month trial period in which you can just lay off without providing a reason for it. Im sure in the US its even more strict.

[const_cast]

The US is about the same, 3 months is typical.

However it's still rare and companies rarely terminate that fast. They require lots of documentation, even in probationary period.

Why? Liability. You can fire someone in the probationary period, but it's not free. You can't fire them for being black or being disabled. That's still illegal.

So, to protect yourself, companies get documentation.

[appease7727]

So you think only people wealthy enough for a college degree should be programmers?

[ozgung]

That's the real problem with the current process. It assumes all the candidates are just random people without any credentials. So candidates need to prove that they can code each and every time. Even a Phd degree doesn't count. Years of work experience doesn't count. And they test you with challenging competitive programming questions. Not even just normal programming. This process used to be only for Google etc, where demand is too high. It's just an elimination process. But today every company in every country blindly copies this insanity. Solving a hard dynamic programming question in 20 minutes after 2 months of studying means nothing from a real engineering perspective.

[ryandrake]

Exactly. There's no FizzBuzz for a doctor, because the medical degree, residency, and passing the medical board exam provide enough of a signal of basic competence.

[zwnow]

How did u get to this conclusion?

[darkwater]

You wrote

> A certificate confirming he did learn the job

What would that certificate be?

[pjmlp]

In Germany that would be a recomendation letter, pretty much valuable by most HR departments, and not showing up with one isn't really recommended, a shitty one is better than none at all.

[zwnow]

Yea apprenticeship certification is what I have (Germany). It's a lot different from other countries apprenticeships and shows ppl "Hey that dude worked this job for 3 years".

College degree should also work as you usually gather some job experience during university.

What i not meant were specific certificates like the Cisco ones.

Even a letter of recommendation from a previous employer would count for me.

[darkwater]

Aren't those the dreaded recommendation letters where "good" means "bad", "very good" means "average" and "extraordinarily good" means "good" ?

[grim_io]

It's probably ego and delusion. Every crud generator company thinks it's special and has some secret sauce it needs to guard. I'm sure some do, but overall it's a case of monkey see (google), monkey do (like google).

[ambicapter]

Even crud generator companies still benefits from getting the best programmer they can, so they're going to do that.

[timeinput]

> They’re not the best candidate experience, but they work at Meta, Google scale, minimizing false positives better than most other formats.

Do they? That is an assertion with out any data. I've worked with F tier developers who worked at Facebook, and Google. They're trying to minimize their false positive rate, but they don't realistically publish it other than laying off large portions of their work force. Presumably because they weren't great, but passed the interview process. If you lay off 3-5% of your staff in a year you had a 3-5% false positive rate. Maybe it's minimized by these in person interviews, but that seems like an unacceptably high false positive rate given how much time the interviewers spend on the process.

3% is probably about as accurate as 'solve fizz buzz in python' prior to AI, and that's where they were in 2022.

[lubujackson]

It sure is easy to cover up bad hires when you print money. And "false positives" come in lots of different flavors. Are they an asshole? Do they add unnecessary complexity in everything they touch? Do they interrupt at meetings and interject their own pet ideas?

If you hire specifically on ability to invert btrees you will get precisely that. The question is how relevant that is to the various jobs being done there.

[Apocryphon]

How'd they get hired, in your opinion?

[timeinput]

At a FAANG company? A 5% or higher false positive rate in the FAANG hiring process. I feel like I specifically said that.

At the company I worked in at the time? An ineffectual hiring process for different reasons that was more on "feels", these folks probably would have been hired whether or not they worked for FAANG, but "they worked for FAANG" was always in the out brief. They seemed talented (I'm sure that's why they were hired at FAANG companes). I'm not proposing a solution, or saying that the company I worked for was perfect, but <worked for FAANG> means they made it through a 95%cutoff that is unreliable rather than they're a good SWE.

Additionally emulating FAANG hiring processes is probably not a great idea unless you can hire 1000 engineers and fifty of them as that is what FAANG does. If you can only hire 10 engineers you can not realistically mimic FAANG hiring processes.

[gabrieledarrigo]

> As with everything, it depends. Live coding interviews work

You cannot start your reasoning with "it depends" and then continue with an absolute. I could do the same:

As with everything, it depends. Live coding interviews don't work.

What's the difference?

[domrdy]

Thank you. I'm not a native speaker, what I was trying to say was, that it depends on the "use-case".

[pelcg]

> As with everything, it depends. Live coding interviews work. They’re not the best candidate experience, but they work at Meta, Google scale, minimizing false positives better than most other formats. What makes them stressful is the lack of interviewer training and the abstract, puzzle-like nature of the problems, which you can really only solve if you’ve spent time studying (e.g., LeetCode) or you’re fresh out of college or academia.

They work (for some) and leet-code weeds out the frauds that really cannot problem solve and assesses those who have not built anything to show to justify not doing it and can be applied to companies that are joining from an acquisition.

> The perfect solution, in my view, would be an assessment where the candidate feels relaxed and able to perform at their best, knowing that every other candidate in the pool has the same constraints in terms of time and tooling. It’s a tough problem to solve.

And that would be the fairest one which is to do the leetcode interview in person on a projected whiteboard and pair programming with the interviewer.

Very relaxing.

[nailer]

> I encourage our customers to use assessments that somewhat resemble on-the-job skills.

On the job will I constantly have to craft a narrative for another person while I code?

Do you know how many autistic people who are great programmers you're throwing away by asking them to multitask rather than letting them deep focus?

[domrdy]

I agree with you. I’ve failed many live-coding interviews because I start focusing on how the interviewer perceives me instead of the task at hand haha. Hiring managers still want to hear the candidate talk through a technical problem. I built a way for candidates to screen-record their take-home solution using only their microphone and screen share, with as many takes as they want. This lets hiring managers hear how candidates communicate and explain their technical implementations, a skill that matters on teams. But it means managers have to watch those videos. Many of our customers use it and candidates like it, but it still takes extra effort on the candidate’s and hiring managers part, with no guarantee that it will "pay off".

[nailer]

> This lets hiring managers hear how candidates communicate and explain their technical implementations, a skill that matters on teams.

Yes, communicating and explaining technical implementations matters, but not while I am coding. Allow people to present separately from coding.

[domrdy]

Agreed, the way it is implemented is that candidates do the recording after they submit their work.

[belter]

Sounds you are looking to hire competitive coders not software engineers,

[nyantaro1]

I would love to see data about correlation between competitive programming competence and actual software engineering competence. I would naively assume there is a big positive signal, but it is pure speculation.

[belter]

Very low correlation...A bit like pro soccer player vs soccer acrobatics street performer...Drone-racing pilot vs aerospace engineer certifying a commercial airframe...

[crystal_revenge]

> minimizing false positives

This has not been my experience at all. Years ago being ex-FAANG was a pretty good signal, but as the filter has increasingly become "regurgitate a bunch of leetcode" the quality of FAANG engineers has plummeted.

The teams I've worked that have used live coding filters all had far worse devs than all the startups I've worked with that didn't require live coding.

In the old "algorithm whiteboarding" days, I think it was a decent signal, but now it's just a sad parody of this and the results show.

[quails8mydog]

Yeah, in my admittedly limited experience the grade I assigned in a live code test (slightly more than fizzbuzz, but no tricks or algorithms required) matched up very closely with real world performance.

I even have knowledge of some of the fails as people higher up the chain decided to hire them anyway. They didn't do well from the feedback I received.

[throwzasdf]

- What makes them stressful is the lack of interviewer training and the abstract, puzzle-like nature of the problems, which you can really only solve if you’ve spent time studying (e.g., LeetCode) or you’re fresh out of college or academia.

This allows ageism without repercussions.

[sotix]

Some of the best engineers I've ever met are always false positives. They get nervous in live interviews. I don't think one can say live coding interviews work so matter of factly when it's eliminating some top, top computer scientists.

[lesuorac]

> As with everything, it depends.

That's the issue though. If you're paying top 1% wages then you should get top 1% performers.

If you want to pay bottom 20% wages then how do you select them using a whiteboard test?

[lubesGordi]

Just make the take home assume that AI is going to be used.

[exabrial]

>Live coding interviews work

do they?