|
Hey, thanks for insightful feedback (blog post author here)! I fully agree with you about temp-to-hire or work-for-a-day models. And I think if the company has such a process in place, it is the sign of a well-developed culture. It is exactly the point of the blogpost -- evaluate using the right metric for the job. However, this is rare in practice, and I would say for the same reason the well-established culture is rare: it requires a lot of work to be done by the very same great people in advance. It is a chicken and egg problem. I saw countless times in multiple companies the signs of immaturity that prevent them from giving people access to their production: secrets in git, shared accounts, lack of established onboarding and offboarding processes, no processes for credential rotation, absence of audit logs, legal issues, you name it. The majority of companies can not afford a work-for-a-day option. Maintaining the dedicated challenge is another option for them, but it has its downsides, too. In our experience, the main one is that you either invest in creating the grading infrastructure upfront or spend a lot of time manually grading the submissions. Checking the correctness of the code is a hard task unless one can actually run it and see how it performs under a barrage of tests. I'm not saying there should not be a code review, of course. I am saying that the automated assessment of the code should be done before you start spending human time. Machines are cheap, your engineers are not. And at this point, one doesn't need AutoIterative, but has effectively reinvented it :) We walked this path ourselves over the years. We did fizzbuzz style interviews, that was suboptimal. We graded challenges manually for a long time, which was tedious and error-prone. We made all the mistakes and tried to fix them. We were using the wrong metric. Qualitative improvement was when we started using the right metric, and automating its assessment was the next logical step. |