[hansvm]

Cheating is possible in the abstract, but I found a tight correlation between interview and take-home performance. For whatever reason, candidates didn't seem to cheat much.

The AI point is worth diving into a little. This was a year ago, so SOTA was worse, but I didn't find it terribly hard to write questions AI couldn't solve, whose answers you couldn't search for, and which good candidates could solve. The test was a few of those questions and a few which were easier to cheat, and almost nobody had good scores on just the cheatable section.

I don't think that moat will exist indefinitely, but today's AI just isn't very good at a lot of incredibly basic tasks unless the operator has enough outside knowledge to guide it in the right direction (and if a candidate did that I mostly wouldn't care because, by definition, they had the knowledge I was looking for). I use AI a lot, it's great at a lot of things, some even quite complicated, but it was weaknesses, and those are pretty easy to exploit.

[jonpurdy]

Your description of the test and your replies to questions indicate you've come up with a pretty great assessment for the role(s) you hire for. Especially where you mentioned:

> The test was a few of those questions and a few which were easier to cheat, and almost nobody had good scores on just the cheatable section

I also like how you allow/encourage self-assessment, where if a candidate can't do the test in ~20 minutes under zero pressure, they probably won't be a good fit in the role itself.