[arandr0x]

Woah, I've been a hiring manager long enough that it's been a while since I've done a take-home code exam myself but I don't even think I would grade a LLM that way (because I moved to AI now). Unless your code exam is super trivial or the boxes themselves are table stakes ("code runs without errors", "code includes more than one function"), coding is creative enough that it's hard to come up with 20 checkboxes that cover whether a sample is any "good" let alone "shows better decision making".

I've had a few bad experiences when sharing feedback with candidates myself and I would understand doing the checkbox approach for feedback and/or just never sending detailed feedback, but actually grading submissions pass/fail based on a subset of criteria you jealously guard from candidates essentially selects for lucky people. If I wanted to do that, I'd just shuffle the submissions by number of bytes and discard everything that's a multiple of 5 or something.