[songzme]

Good article, but I personally hate Take-Home coding challenge:

Some people spend an entire week on the coding challenge, which really set the bar unrealistically high. Once I turned in a project 2 hours after receiving the assignment and got rejected because they think I didn't even try. Every functionality worked flawlessly, but they want tests and for me to treat it like a real production work. But how do you motivate yourself to treat such a trivial take-home (which will never be maintained because its a take-home assignment) with the perfection you would in production code? In production you have much better insights like how your component impact others, how users will be using your app, etc.

[PeptoKelpto]

This is my gripe with take homes. I have spent a week on a take home before and got glowing reviews on how awesome the tests were etc. only to be rejected down the line without any feedback whatsoever. If I apply for a job now and they give me a takehome I kindly ask for an in person interview or decline to proceed with the process.

[janephilipps]

This is tough - I've had similar experiences. I think the main issue here is that the company didn't give you any feedback, so it's hard to know what happened. I've submitted a take-home challenge, but had hiring needs change in the time it took me to complete it, so the company passed. Though I think there are a ton of things you can do to make your self stand out as a candidate with a take-home challenge vs. a whiteboarding interview, they are not for everyone.

[anonydsfsfs]

I'm pretty sure no feedback is the norm, not the exception. I've done two coding challenges. One was reasonable and lead to an interview, but I was rejected with no feedback.

Second was kinda ridiculous: implement a stripped-down version of their website, with tests (and production-ready, of course). They claimed it could be done in 4 hours, which is probably true for all the devs that work there, since they already know exactly what's involved. I spent ~8 hours on it and submitted something that worked but had problems, which I documented thoroughly. Again, rejected with no feedback.

[Shorn]

Were you interviewing in Brisbane, Australia? Sounds very similar to a recent experience of mine.

[cyberpunk0]

More and more it is becoming "standard policy" to give zero feedback. It's quite pathetic to be honest

[kasey_junk]

I have fought the “zero feedback” policies at companies going back ~12 years now. I think it’s terrible but I always lose.

But here is why. For every personal story you have of working with a dev that can’t program, your recruiting staff has 3 of bad/lawyer invoking instances of dealing with crazy candidates. They are being honest with you when they tell you any other policy is too expensive.

[on_and_off]

This is why we use a timer on our take home exercise.

You are only supposed to spend 6 hours on it, we give a 24 hours timer so you are not too pressured for time (you can take breaks, have diner, a walk, etc) .

So far it works pretty well.

I have not yet seen any applicant writing tests. Just delivering the project with the requested features and acceptable code (no memory leaks, crashes, unreadable mess, etc) is more than enough to go to the next step.

If you demonstrate enough proficiency during this step, we will probably skip most tech questions and focus on fit in the next step.

[janephilipps]

This seems like a good happy medium type of take-home challenge. Out of curiosity, how many applicants have you had?

[on_and_off]

Wow, I don't know, I would say 5 challenges / week.

That's our third step though, first our recruiters filter applications (I have no idea how many people are funneled out at this point)

Then we have a 30 minutes phone call to discuss experience and presell the company. We also try to answer all their questions at this point and explain the challenge. Something along the lines of : "so the next step will be a take home test. We just want to see how you code, don't put too much pressure on you, just do your best in the time given"

Historically we have only been looking for pretty senior engineers, although we are relaxing this a bit.

The company has also been sponsoring a lot of visas (even with very interesting salaries, senior mobile engineers are hard to recruit .. most of them are already happy where they are), I think that take home are a good way to handle this case.

The applicant can take the exercise whenever he wants (the only limitation is that we might not be available to answer their questions in the middle of the night) and shows us what they can do.

[stevenjohns]

> but they want tests and for me to treat it like a real production work

When hiring back-end devs, I explicitly make it clear I want at least some tests -- and the more the better. Of course I don't expect 100% test coverage but writing tests shows that 1) you understand best practices 2) you know how to write testable (readable, maintainable, clean and clear) code. If candidates chose just one module and tested that well it would be enough to get the point across.

Seeing how people write their tests is also quite important for me -- if instead of using factories (or Faker or some other strong dummy data app) they instead decide to hardcode usernames and passwords, it's a huge red flag.

[jhall1468]

> 1) you understand best practices

Tests prove no such thing. Best practices aren't about output, they are about inputs. Tests determine if the expected output is the real output.

> 2) you know how to write testable (readable, maintainable, clean and clear) code.

Again, no they don't. I can test unreadable, unmaintainable dirty and unclear code.

> if instead of using factories (or Faker or some other strong dummy data app) they instead decide to hardcode usernames and passwords, it's a huge red flag.

No it doesn't. It shows that it's an arbitrary take home assignment that already took a week and they don't want to use yet another library to generate data for it.

Honestly, you put so much emphasis on testing which is odd, because it's the easiest, most mundane thing to write. It provides nothing but "busy" work and fails to test the most important aspect of any candidate: can they problem solve.

[stevenjohns]

> Tests prove no such thing. Best practices aren't about output, they are about inputs. Tests determine if the expected output is the real output

Testing in and of itself is best practices. Having familiarity with and a strong understanding of how and what to test is extremely important for any modern development cycle.

> Again, no they don't. I can test unreadable, unmaintainable dirty and unclear code.

Except it's significantly more difficult.

> No it doesn't. It shows that it's an arbitrary take home assignment that already took a week and they don't want to use yet another library to generate data for it

If you don't have a strong dummy data suite to test against, then you're not testing properly. This actually justifies exactly what I'm talking about.

> Honestly, you put so much emphasis on testing which is odd, because it's the easiest, most mundane thing to write. It provides nothing but "busy" work and fails to test the most important aspect of any candidate: can they problem solve.

If it's mundane and easy then it should just be done. What I've found is that most can't.

If the candidate pool was small, then I'd take less of an issue to it. But if 50 people come in and they can all problem solve, then I'm going to become more interested in who can solve problems elegantly.

[sitkack]

Directed at you and the parent. A better way to structure this is ask for at least one integration test, then tell the candidate that testing strategy and methodology will be discussed in the review.

I had a coding test, not take home, where I was able to code on my own on a simple problem for about 45 minutes for each session then have a code review. We did this three times over three hours on increasingly harder problems.

This exact same format could be done remotely.

[nmjohn]

> they instead decide to hardcode usernames and passwords, it's a huge red flag.

Why? That seems like an oddly specific thing to be a _huge_ red flag. Are you expecting random data to actually surface bugs - ie: making your tests non-deterministic? Or is there some other benefit you are looking for to make it such an important criteria?

[stevenjohns]

It shows a lack of familiarity with the tooling. It's actually quicker to write something that prototypes the model you're testing -- as little as a few lines of code -- that eventually saves a lot of repetition.

That prototyping later becomes extremely useful for building local databases -- a strong factory suite means I can rebuild my local database and have it completely populated with randomized dummy data. Changing locales on some packages means I can have a local environment populated with data tailored for a completely different country and language, for example.

Re: surfacing bugs -- in a lot of cases yes. And a lot of bugs have been found this way.

[simplify]

It looks like you're mistaking "good programming" for "programming the same way I do".

If a developer can test, that's a good fundamental skill. If they don't use a factory, that says nothing about their fundamentals – it's something you can easily tell someone who knows how to test to go learn & do once they're on the job.

[Clubber]

Especially for something as throw away as a take home coding challenge.

[stevenjohns]

It may be a throwaway for the candidate, for the employer it's $XXX,000 a year and potentially ending the search for better, more suited candidates.

If taking an extra an hour of time to sell your work is "throwaway" then that person is either shotgun applying to jobs (and not being hired for a reason) or they're not taking their application seriously.

[Clubber]

That's fine, just understand that is a technique for finding "growing" developers, and still not a great one for that. If that's what you are looking for, that's fine. If you want tops, this isn't the way to go. Unless you've already hired someone that knows a top, you probably won't get them anyway.

Think about it this way. It takes about 2 hours to learn how to write unit tests for someone who has never done it. Why make something as trivial so important in your interview process? I understand unit tests are important, but they aren't difficult.

[ThrowawayR2]

> It takes about 2 hours to learn how to write unit tests for someone who has never done it.

If it's really that trivial, the developer should be doing it so routinely that it should take less than 15 minutes to write unit tests for the take home exercise. Your statement literally demolishes any excuse for not doing it.

[pkaye]

Then maybe communicate that salary amount to the candidate before so they can also decide if it is worth their time or not.

[janephilipps]

Examples like this are why I think take-home challenges are a better way to assess a candidate's coding skills vs. their problem solving skills/how they think.

[drewbuschhorn]

Our strategy is to save the take-home for after the in person interview, and to pay you for the time (up to the limit we expect), then a follow up to talk about the work.

I'm about to start applying to new positions and it sounds like I won't get the same courtesy, but only you can decide if your enthusiasm for the position matches the opportunity cost of the interview process.

[milesvp]

Can you really justify my contractor rates though? My time isn't cheap, and I don't work until the down payment has cleared.

All a take home test does is eliminate your strongest candidates, since they generally won't bother. It has to be a truly extraordinary position before anyone with significant experience will waste their time trying to compete in this market. Keep in mind my opportunity cost while job seeking is networking. Every minute I waste on someone's takehome is time I could be tayloring my resume to another position or emailing with prospective employers, which makes them all the more expensive to waste time on.

[drewbuschhorn]

shrug offering pay isn't meant as an insult, but to show we value your time. If we can't afford your hourly rates, we probably can't afford to offer you a salary you'd be happy with.

[busterarm]

Really?

Considering I'll have to pay all taxes, medicare, ssi, etc. on all self-employment income and that the time I spend working on your challenge is time I'm not spending chasing other business, the contractor rate that I'm going to charge you is likely going to be around 300-400% of what I expect my salary to be for that time.

If you can't afford that, I'll probably think you're either exploitative or delusional or both.

But then again, your 6 hour programming assignment is worth about $1500+ of my time.

We take a salary because we want to offload risks and time spent doing sales to source our income, but we shouldn't have to do throwaway work for free to get that. Sounds too much like 'unpaid internship' to me.

[kasey_junk]

Note that in some jurisdictions, the candidates cannot take that pay without reporting it to their existing employers.

[davidbanham]

I completely agree with your criticism but I disagree with your conclusion.

The fact that some applicants will spend ridiculous amounts of time on a challenge is a critical flaw. It doesn't mean that take-home challenges are worthless, though. It just means they need a functional mechanism to enforce time limits equally on all candidates.

[qudat]

> Some people spend an entire week on the coding challenge, which really set the bar unrealistically high.

Simple, cater the challenge so that's not possible.

Some companies are just bad at conducting interviews, that's the real tell and not take-home challenges.

I've had terrible in-person technical interviews, but I'm not going to reject the entire concept of in-person technical interviews because I've had a couple bad experiences.

If someone doesn't want to participate in a take-home coding challenge then that's their prerogative, but not everyone thinks they are a bad idea. If I were requesting a take-home challenge and a candidate I liked said they will not proceed if they have to do one, then I would probably make an exception. But that's me working at a startup where I have full interview control.

[janephilipps]

Thanks! I definitely understand that many people are opposed to take-home challenges because they can be very time consuming. For me, I use them as a learning opportunity, so even though they are not necessarily going to be used or maintained in the future, they can be a great way to practice or get better at a specific language or tech stack.

[PeptoKelpto]

While this might be true, most developers (in my locale at least) fill their GitHub with little projects meant to fit this exact purpose. I would 100% consider someone with some semblance of a GitHub over someone who submitted a well done takehome with no GitHub. That's why I don't bother with them. I would rather see that you are at least trying to work on yourself as a developer outside of interviews and work. It shows initiative, dedication and a host of other great attributes that I just don't see from a takehome. I'm not saying there aren't great coders without GitHub accounts, but I would for sure be asking why not. I think a better use of time is to do "production" code on a whiteboard. FizzBuzz in person, if done properly, can tell you way more than any take home. Start with basic FizzBuzz or any other basic interview question and make them write it functionally and tesably and mock out some sample tests. In person, this will tell you a ton about the candidate and isn't super stressful for anyone because it's a well known baseline interview question. It also makes the interviewee feel like the company is investing as much time in the interview process as they are.

[busterarm]

Many of us signed contracts where 100% of the development work we do belongs to our company even in our off time. Myself included (though it's only if the work is in the same industry -- at the discretion of my employer of course). Some of the best companies hiring the best engineers do this. You are seriously limiting your pool if you favor candidates who spend their off time contributing to their GitHub. It's a bad bias to have.

Worse still, I've interviewed people who have put their employers' work on their personal GitHub, publicly and without permission, solely because of this reason. I have serious ethics/intelligence questions about those people and they become a strong no-hire every time I see it.

[PeptoKelpto]

I guess we will just have to disagree. What works for me may not work for others. I would never sign a contract that sells all my personal work to the company I only get paid 9 to 5 for, and I don't know anyone who has. Also, it's pretty evident when someone posts their companies code to their GitHub. Simple questions and looking at the code can dispel that. If they do manage to squeak by and pass all the interviews, I don't see how a takehome would have made it any harder anyway.

[BurningFrog]

The motivation is that you're showing off your skills with this code. You want to deliver the code that, in your opinion, is the best possible for solving the given task.

> which will never be maintained because its a take-home assignment

You will most likely be asked to add a feature or two to it in the followup in person interview.

[LandR]

I've did a few of these take home challenges during the interviewing process.

All of mines have had a time limit to them of an hour. The process has always been

1) arrange a time with the recruiter when I will be free for an hour, e.g.lets say 1PM.

2) recruiter emails me the challenge at around 12:58. I get the challenge and have to code the solution and have it back to the recruiter by 2PM.

Then at the face to face interview we've went through a code review of my code and talked about why I chose to do somethigns the way I did, if I had to more time what would I do differently, was there anything in my solution that I wasn't happy with etc.

Seems to work reasonably well.

There is no way on earth I would be dedicating a weekend or more to an interview task.

[HaloZero]

That's pretty good reason if that's true but setting that expectation would be useful. I've also never had that happen to me.

[BurningFrog]

I think basing the followup interview on the homework task is pretty standard.

You can always have someone else do the "homework" for you, but when you keep working on the same code base you have to show a real understanding of both the problem and your code, and it's quite close to a real work situation.

In hindsight I agree that telling people that will happen would level the playing field a bit.

[shill]

There are several possible reasons why you didn't include any tests in a take home exercise, and they are all bad.

1) You ran out of time to write tests.

This might be the least bad reason. You can probably talk your way out of this by explaining what tests you would have written if you had the time.

2) You don't know how to write tests.

This shows that you have only worked at places the don't do testing and that you haven't taken the initiative to learn about testing on your own.

3) You know how to write tests, but you don't think they are needed.

You could probably tell the interviewer why tests were not important for the particular exercise. But I have never seen a take home exercise that doesn't require a basic algorithm and some tricky conditional business logic. Both of which would benefit from unit tests.

4) The code you wrote isn't testable.

This is probably the worst reason. Everything is in one or two giant functions and you can't inject dependencies for testing without refactoring everything. This will frighten your interviewer.

[songzme]

Thanks for sharing, perhaps my workflow could be improved. Here's how I usually work:

1. Build a prototype of the feature I need to build

2. After I have a prototype, I start to understand the complexities of the project and I start thinking about how to modularize my code (the functions I would need, how the frontend-backend interacts together).

3. I rebuild the feature with TDD

For take-home assignments, I stop after step 2 because step 3 is the most time consuming part.

I'm pretty open minded about this, but with 5-6 years of development most take home assignments are so trivial and so self-contained.

Perhaps its really the take-home assignment that needs to change. For example, heres the sort of prompt that would motivate me to write good tests:

"Here's a private repository that every candidate we have hired has been building. We would like you to add a feature. Please build feature X on top of our existing code -> Link to repository that has an application with simple testing infrastructure"

Now the take-home assignment has more value and actually gauges how well the candidate would do in an actual working environment. When the candidate gets hired, his feature actually gets merged into the assignment and a new feature is assigned to new candidates.

[janephilipps]

This is a really cool idea - because working with code written by other people is usually a large part of any job, this type of assignment could also test for that. For newer devs, I've seen anything from simple Tic Tac Toe to being given a starter app with a general set of requirements to fulfill.

[timr]

Or most likely:

5) Your interview process is ridiculous, and I refuse to do more than the bare minimum when I am not even guaranteed an interview in exchange for my time investment.

I am an experienced programmer. I refuse to do take-home tests as a pre-screen for an actual interview, unless I know for certain that a position is one that I would want to take (e.g. I know someone who works there, and I know that it's a dream position. This basically never happens.) Moreover, if I have done a full interview loop and you insist on screening me based on inanity such as "he didn't write tests for this time-limited toy assignment", I will gladly move on to the next company that isn't ridiculous.

Take-home tests are screening devices. If you draw inferences other than "this candidate can/cannot write code" from a take-home test, you are using them incorrectly.

[toast0]

I probably wouldn't do a take home test for an interview, but if you're only willing to do the bare minimum, it would probably be a better use of your time (as well as whoever is going to grade it) if you simply refused to do it.

[timr]

Before I accept a take-home project, I explicitly lay out what I will do and the maximum amount of time I will spend doing it. I've walked away from companies that demanded more than I was willing to give.

I dislike take-home tests less than I dislike whiteboard interviews, so I try to be flexible, without being a pushover.

[janephilipps]

As an experienced programmer, what does your ideal interview process look like? Do you think your opinion holds true for junior developers as well?

[songzme]

The ideal hiring process imo is actually a training process that is less focused on 'finding the best' but rather focused on helping their local community become great engineers. Companies fund their own bootcamps and pay students $15/hr (with no engineering experience). Students learn and help each other how to code (following a structured curriculum) with mentorship by engineers from the company. When students have finished the basics, they work on open source projects or internal projects following proper engineering process / tools that the company use. When a student has proved that he/she learned how to work well with other engineers and has built a few features, they get hired by the company as a junior engineer and this junior dev would be immediately productive.

This might seem like a crazy idea, but its actually not. The previous companies that I have worked for all offer paid volunteer hours which could go directly to this initiative. The cost of hiring is around 30k per engineer (conservatively), which could help train someone living paycheck to paycheck for an entire year.

I spent the past year and a half starting a non-profit experimenting with this idea and invested all 200k of my annual salary to paying students. The coding tools and environment is set up like a baby version of Google's engineering infrastructure. Students progress are tracked by their merge requests and the complexity of the features that they have built. Students build tools that they use on a daily basis so they are heavily eating their own dogfood. Hopefully, one day I'll be able to build my engineering team by directly hiring from this non-profit. https://garagescript.org

[janephilipps]

Love the idea of investing in the local community. Training and mentorship is so important, and so many companies sacrifice it for other things.

[whatshisface]

5) You were interested in limiting the time cost to yourself related to being interviewed, expecting that the wider sqrt(t) uncertainty band in your abilities would not mess up the company's hiring procedure too badly.