| Hey Bill! Rating aside, I'm a big fan of Ad Hoc's homework assignments. Here's what I love about them:
1. Your team has made them public for others to use. This is HUGE. 2. The library is organized and the prompts are generally clear and explicit about the requirements to complete. 3. You offer support for candidates with questions about the assignment. 4. Kathy Keating told me about the grading environment you all use with rubrics, blind grading, and rotating cohorts of graders. We actually created similar tooling for the teams we work with. I also want to explain why they're rated as 3 stars (and why I think this undersells how good they are). When we designed the rating criteria, it was really important to recognize tests that could extract hiring signal without requiring a ton of time from candidates, since that's one of the biggest issues that candidates face. So one of our criteria was "setting clear expectations for candidates (e.g. time expectations)" and another was that the time requested was "reasonable...(<4 hours)". So a test that stated upfront that it would take 8-10 hours would meet only one of those criteria. You can see the full rubric if you hover over the "5-star scale" text in the sub-header. Unfortunately, the Ad Hoc tests don't specify time anywhere (technically missing both of these criteria) while doing many other things well that aren't captured in our rubric (e.g. candidate chat tool, blind grading). I admit that our rubric isn't perfect and it feels like the Ad Hoc tests are being doubly penalized for something that's easy to fix. In fact, if you add this to the tests, I'd happily update these to 5-stars. Finally, if you're up for a chat sometime, I'd love to meet you. I really appreciate the work that your team has done to improve the hiring experience for candidates beyond those at Ad Hoc! You can reach me at alex@trytapioca.com |