[kybernetikos]

I disagree about the take homes. But this may primarily be because I've been hiring with a heavily weighted take-home for the last six months. Here's my reasoning:

1. I don't want to work for a company that uses CVs as a major part of its job screen, since they are nearly useless from an informational point of view. Steps that improve the quality of candidates also improve the expected quality of your future colleagues.

2. The take home gives significantly more information than a phone screen. Candidates who are rejected because they find the take home too hard, or who would have passed a phone screen but not the take home are saving time because they don't have to take any time off to come into the office for an onsite.

3. Having a good take-home means we can make the rest of the process more lightweight. We have an extension interview where we pair with you while you extend what you took home, and then you're done. This is much more respectful of your time than multiple days of algorithmic questions.

4. It's easier to make marking take-homes blind - i.e. to guard against various biases compared to phone screens and even in-person interviews.

5. The take-home is simplified, but similar to the sort of thing you will have to do in your actual work, so doing it gives you extra information about the job itself.

6. The take-home is designed to result in something that's fun. Many of our candidates say that they enjoyed doing it. Even some of the ones who fail it. Some even say it's a reason that they choose us over some other company.

It is true that some good candidates simply refuse to do take homes on principle since they have lots of choice. To try to combat this, we make sure we try to explain why it's worth your time.

If I told candidates 'congratulations, you've got a 1 day onsite interview' (where in the morning you're on your own and we send you home at lunchtime if you've not made good progress) instead of a half-day take-home (in which we are very flexible over how much elapsed time you need) and a 1.5 hour onsite extension, do you think candidates would prefer that?

[pmiller2]

I'm assuming you're referring to an un-timed homework assignment. Timed homework assignments are absolutely the devil and I don't think I'd do one of these.

My biggest issue with (un-timed) take home assignment is that often, as a candidate, this ends up with me investing a nontrivial amount of time, only to not hear anything back. If I'm going to spend multiple hours on something, I would kind of like something in return, such as feedback on my work. Very, very few companies will give any feedback since they don't want to be sued and/or have the candidate try to rebut the reasons why they were turned down.

[kybernetikos]

Yeah, it's untimed in the sense that we don't specify an elapsed time it must be done in. People have different things going on in their lives, and you can't just assume they can drop everything for your test. We specify a suggested amount of time so that people know not to spend too much time on it.

On the feedback point: this is a perfectly reasonable stance. Perhaps I should give better feedback to our candidates.

Part of my concern is that I've seen candidates show me emails from their recruiters that have attached all the detailed feedback I've give to their previous candidates.

The other thing is that at some points in the hiring, you're really competing against the others in the pipeline, so you might have actually performed very well, just not quite as well as one of the others. I have also seen one candidate fail based on multiple independent personal disrecommendations rather than on performing badly in the test.

I have noticed in previous interview processes that a lot of people think they failed for a reason other than the reason they actually did fail.

I do think it'd be more fair for me to give better and more detailed feedback, but there are a lot of pitfalls with the most obvious version of that. Perhaps there's room for a bit of disruption with a good solution to this problem.

[pmiller2]

Triplebyte does a fair job on the feedback front. I've been through their process recently, and found it to be more useful for me than any other company's process (I chose the coding project over the standard process). I dislike that they still put candidates on the spot by writing code literally in front of them[0], but the feedback I got from them was actually useful.

I think your issue with candidates showing you emails from their recruiters is actually an issue with recruiters more than your process. Recruiters will do just about anything they can to make the placement.

If you're in a situation where a candidate excels on the test but gets passed over in favor of someone who did even better, then, good for you. :) "We decided to go with another candidate at this time, but here's a little feedback on your coding project," would go a long way with me. If you're really feeling generous, a referral into another company would be a great thing.

Someone who fails because of personal disrecommendations, you can just give them a generic "We reviewed your coding project and here's what we thought of it... but we decided to go in a different direction. Best of luck on your search." (I'm assuming these people fail because they're assholes, not because they're incompetent, right?)

---

[0]: Seriously, guys, if you're reading this... I chose the project interview because I fall apart when being put on the spot, and I told you that. I don't have someone watching over my shoulder at work... and it's a good thing, too, otherwise I'd end up having anxiety attacks.

[swang]

i don't think pure cvs are the answer. i don't think a take-home is either. especially since a lot of companies expect you to put a lot of sweat into these take-homes.

if your take-home is a "half-day" which means... 12 hours? half a work-day? what? that does not sound appealing to me. unless you are actually building something unique, offering more stock, or more money, or a better work environment i am not sure how you can explain it's worth it to someone to go through your process and essentially wasting an afternoon to a week of their life doing the take-home. _especially_ when most companies will reject you for a lot of silly reasons.

in a perfect world your scenario would be "fine" and all your points your company may actually do. but across the swath of companies there is no guarantee of this.

for example you cite #4 as an point for doing take-homes, except i seriously doubt many companies mask/hide who the person coding it is. they usually just hand it to one of their engineers and go, "evaluate this" and now you're at the mercy of what kind of engineering person he/she is and whether they're currently too busy to give you a fair shake or any other myriad reasons.

you can barely get companies in silicon valley to admit there is bias let alone set up some blindness against it. unless they were able to submit the initial job application anonymously, how do you know your process isn't bias at that point from your HR person? maybe you do account for that, but how many companies realistically do that?

and again without knowing your company or your process, _every_ company with a take home says, "take your time", "we're not in a rush" etc but never actually tell you that they want just <X> and if you do <X> that's enough. people literally have no idea how you're going to judge their take home. maybe the question is unclear, maybe your view of a "good enough" ui to them is a ui that takes longer than the time you've alloted them. maybe you don't like it when they write their javascript without semi-colons but you don't bother to tell either the interviewee or the grader any of that. then there is also the problem that even with "correct" code a lot of engineers who review take-home tests just _love_ to nitpick to disqualify possible candidates. and if you _do_ have a process in place to guard against that, you would literally be the first company i've heard who goes that far in their interviews.

i don't doubt the way you're handling it is getting the results you want and is the best way to hire. but i doubt that's how other companies are handling it. now you expect people looking for jobs to be able to discern between you and the other group? you want me to apply and assume good faith that your process will only take half a day and is "fair"?

edit: i tried editing this to make it not sound very antagonistic, but it is late here in sf so it may come off like that so sorry if it does. i don't disagree that take-homes work for you. i just don't think most companies in the bay area have thought it out very much other than, "use take-home test in interviews"

[kybernetikos]

half-day means four hours. I'm being generous there, because a skilled developer can create a good solution in 2 hours, but many find they want to put around 4 hours in.

In terms of what you have to do for the take home to be a success, we explain that fairly clearly these days. I'm now on the 6th version of it because I ask for feedback from the people who do it and try to improve the wording and make it more clear over time. I also provide contact details for people with questions.

It's also a fact though that simply achieving the task is not enough if the code you used to do it is sufficiently poorly factored or overcomplicated to a ridiculous extent. We don't mark people down based on superficial coding style. Fails get marked by more than one person, the marking team is small and regularly talks about the individual tests they are marking, but there'll definitely be situations where working code gets marked as fail.

The person marking the take home does so before they get access to the CV. They do usually know the name, we'd need a more complicated process to redact names from submissions, but it might be worth doing.

I am sure there are ways we can improve our process, and I am aware that there will be lots of companies out there with a process even worse than ours, but I wanted to talk about it, partly as a way of getting feedback with the aim of improving it more, and partly because I hope that others can learn from what we've done.

I have been pretty happy about the team we've built with this process compared to teams that I've been involved with building in the past, so there are definitely good parts to it. The two big concerns I have with it are 1. are we being equitable with peoples time and 2. can we reduce the number of skilled candidates who refuse to do it on principle.