Hacker News new | ask | show | jobs
by manfre 94 days ago
Any take home test trivial enough to complete in under 20 minutes could be completed by an AI. The only signal you get from a take home test is whether or not they can submit answers. It doesn't let you know if the candidate is capable of passing the test unassisted.

Take home tests were never a worthwhile signal. Pre-AI, people would search for solutions or have another person complete it.
2 comments
hansvm 94 days ago
Cheating is possible in the abstract, but I found a tight correlation between interview and take-home performance. For whatever reason, candidates didn't seem to cheat much.

The AI point is worth diving into a little. This was a year ago, so SOTA was worse, but I didn't find it terribly hard to write questions AI couldn't solve, whose answers you couldn't search for, and which good candidates could solve. The test was a few of those questions and a few which were easier to cheat, and almost nobody had good scores on just the cheatable section.

I don't think that moat will exist indefinitely, but today's AI just isn't very good at a lot of incredibly basic tasks unless the operator has enough outside knowledge to guide it in the right direction (and if a candidate did that I mostly wouldn't care because, by definition, they had the knowledge I was looking for). I use AI a lot, it's great at a lot of things, some even quite complicated, but it was weaknesses, and those are pretty easy to exploit.

link
jonpurdy 94 days ago
Your description of the test and your replies to questions indicate you've come up with a pretty great assessment for the role(s) you hire for. Especially where you mentioned:

> The test was a few of those questions and a few which were easier to cheat, and almost nobody had good scores on just the cheatable section

I also like how you allow/encourage self-assessment, where if a candidate can't do the test in ~20 minutes under zero pressure, they probably won't be a good fit in the role itself.

link
Esophagus4 94 days ago
Major interview platforms all can identify AI usage, and record the interview for humans to review later.

Employers can, with a somewhat high (?) degree of certainty, weed out candidates using AI.

link