Hacker News new | ask | show | jobs
by NeverFade 1398 days ago
> Then the candidate doing it at home using their system and their tools gives you a more accurate picture of their actual capabilities than a whiteboard or doing it on your hardware in an unfamiliar setting with a stranger giving every appearance of critiquing your work.

Or, they don't even do the task, but pay someone else to do it for them, so eventually all these take-home tests just go to the same set of expert test-takers who already solved all these tasks. Very efficient, and also worse than useless as a screening tool: the dishonest candidates most willing to cheat will score the highest.

> If a candidate chooses to burn a whole week working on it that can be safely left up to them I think. But a take home test is not unfair because they decided to burn that much time on it.

Of course it is. Worse: it creates adverse selection, from the interviewer perspective.

Top candidates who are looking for their next job while working a demanding full time job will not be able to dedicate more a minimum of time to each of the many take-home tests they are asked to do.

Meanwhile, desperate poor candidates will dedicate x10+ the time (or simply cheat) to pass.

So as an interviewer, you'll end up passing the random desperate candidate who somehow got through your earlier screening, and any good candidate with multiple other options and commitments will submit work that looks far inferior.

1 comments

If I set a take-home test designed to take two hours, and then very clearly tell the candidate to take two hours, and then they clearly spent way longer than that, then I will consider that a signal that they don’t follow instructions well and don’t value their time appropriately.
How do you know they clearly took over two hours?

We love talking about productivity here. You're telling me employers will systematically fail the candidates who performed most productively in their process, because of a speculation they took too much time?

Well, it's pretty simple if you don't given them the task until they agree they're ready to start, and then record when you receive the submission.

Any candidate capable of actually inventing a time machine should certainly be hired.

Haha yeah we implement 'soft time-boxing' by tracking git commit timestamps. It's not as stressful as having a visible timer and we won't block a submission that exceeds the recommended time, but reviewers can clearly see who took longer and their last commit within the suggested time, which helps to create an apples-to-apples comparison.

No candidate wants to be entered in the Hunger Games for who has the most time to sink into a take-home.

Most companies that use take-home tests don't use such systems.

Also, such a system does not prevent cheating, which is really the worst issue here.

And a reasonable response in any case is "Yah, I know you said that but I don't like to do sloppy work and there were clearly a number of different edge cases that needed to be handled for this to work reliably."
I've seen a middle ground where a timebox is enforced, but there's a discussion question for what next steps would be
Yes, this is what we do at my company. It's the only way this sort of thing is valid IMO... it's not a race, but I want to see demonstration of reasonable output.
That would actually be a huge red flag to me for hiring.

I don't want an architecture astronaut who is never going to ship anything.

An 80% solution when I need it is infinitely more useful than a 100% solution that's months late.

And I'd rather somebody admit that they took longer, say 4 hours to get to the 80% because they were a bit rusty, their IDE on their private machine wasn't as expected (e.g. Intellij CE vs the Ultimate at work), they had to look up how to bootstrap a service from 0 because they hadn't done that in ages even at work etc. than to under deliver a 20% solution in the 2 hours asked.

Limits are fine if you ask me. Give me a couple days in which I can do this on my own time and if my kid needs me for something 10 minutes after I started I can deal with it and just come back to the test an hour later. Specific hours? Screw you, not even gonna start.

The limit is the limit. If you don't finish in the time given, I want to see what you did in that time.
And this would be the opposite for me. Someone properly thinking out the edge cases and handling them (or at least pointing them out in comments) would be much more useful in the roles I'd be hiring for.

I need a stable system that can tolerate the real world and handle messy inputs without shitting the bed. A new hire that shows they consider their inputs and handle edge cases is a better fit to me than someone who focused on that 80% solution and didn't bother sanitizing inputs and crashes because of it.