[gravypod]

I haven't been able to find a better screening method than take home projects. A major issue that I find when watching hiring take place is people don't separate learnable/domain specific skills from the "inherent" skills you need as an engineer. Hiring someone who has a 10 year track record as a software engineer in systems software for a web-stack backend role should be an easy "Yes" if they are a skilled software engineer. Because of this I usually encourage people to make take home problems that are limited in scope and follow the restrictions:

  - Should not take longer than 3 hours for an inexperienced developer.
  - No learnable skills tested like tooling, language, and domain experience.
  - Simple to understand problem 
  - Problem should be obviously mappable to a solution
  - Provide sample input and sample output data (only 1 case). 

I then score using the following questions

  - Did the code work on the sample I provided?
  - Were edge cases in the spec accounted for?
  - Has the engineer followed a language standard (PEP/PSR)?
  - Were docs (install, compile, comments) provided?
  - Were unit tests provided?
  - Were sample cases the engineer ran manually provided?
  - Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.

I think the actual project should have low bearing on completion and the state and cleanliness of the code has a higher mapping to quality.

[Stratoscope]

Mobile-readable copy of your lists...

Restrictions:

- Should not take longer than 3 hours for an inexperienced developer.

- No learnable skills tested like tooling, language, and domain experience.

- Simple to understand problem

- Problem should be obviously mappable to a solution

- Provide sample input and sample output data (only 1 case).

Scoring:

- Did the code work on the sample I provided?

- Were edge cases in the spec accounted for?

- Has the engineer followed a language standard (PEP/PSR)?

- Were docs (install, compile, comments) provided?

- Were unit tests provided?

- Were sample cases the engineer ran manually provided?

- Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.

[hysan]

Would you be able to give an example of how limited in scope you're talking about?

Is it just a single "write a method that does this" type problem? I ask because I did a take home very recently which I know I could knock out quickly, but it took much long because everything you listed ballooned the amount of time it took me to complete it. Specifically:

  - Has the engineer followed a language standard (PEP/PSR)?

For JavaScript, it takes some time to setup a new project from scratch with linting and formatting configurations. Then documenting it in your README.

   - Were unit tests provided?
   - Were sample cases the engineer ran manually provided?

Once you go beyond a few "features" to write, this balloons in time due to both writing it with good descriptions and documenting it for the reviewer.

   - Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.

If what you ask is of sufficient complexity, it would just be normal to take time to think about what the interviewer is expecting here. Like what future features could there be? How might this be used in a larger codebase? etc. Then once you've decided on this, you'll want to spend time commenting and/or writing up an explanation in your README about your design choices.

Lastly, this isn't mentioned but I did this on the take home I recently completed - using version control and properly branching & merging on each deliverable. Doesn't take time, but if you are making atomic commits with good comments, that's just another way for you to lose time. Of course, I don't think the fact that I did this is going to be noticed as it wasn't written out as a requirement.

If you notice what I'm nitpicking, it's all the setup and writing that becomes a time sink for most. In isolation, it wouldn't break the bank, but with all those requirements combined, you've lost much more than 3 hrs. When I'm given take homes for interviews, this is where I've burnt the most time - taking the time to document well.

[mburger]

What about having a discussion on how the candidate would solve a given take home?

You could check if she would think of the required parts. For instance, would she think of unit tests? If she wouldn’t mention tests, you could ask „How would you ensure that your program works as expected?“

Thus, solving the take home theoretically.

[gravypod]

This is a great use of the take home. It's a good interview-starter to get conversation moving.

[gravypod]

> Would you be able to give an example of how limited in scope you're talking about?

At a previous company we gave a problem that boiled down to the following:

  1. Read a config file that contains a list of "Rules"
  2. Each rule contains a name, an ID, a comparison, and a value. 
  3. Read a json array of numbers from a file. 
  4. Apply the rules you read to the array from the file

An example of a rule would look like:

    {
        id: 1,
        type: greater-than,
        value: 10,
        name: Number was passed threshold,
    }

Each rule type was specified with a light description and an example was provided. A sample input (config and data) that, if applied correctly, would trigger each rule was also provided. In all there were 5 rules: greater, less, equal, not-equal, multiple (a rule that had a list of rules inside of itself that would all need to be applied true to trigger it. ex: greater-than 4 & not-equal: 8).

The instructions only specified one requirement. Something like:

    Provide a method for our engineers to run your program
    via command line:
        `./<your program name> <rules-file> <data-file>

> For JavaScript, it takes some time to setup a new project from scratch with linting and formatting configurations. Then documenting it in your README.

For your code to be formatted and for you to configure a linter are two separate goals. For JS (were standards vary greatly from org to org) I don't mind as long as there is a consistent formatting. Any IDE or editor has a format function that I would be happy to see used.

> If what you ask is of sufficient complexity

I hope that it is evident that this task was designed to not be complex for anyone but the most junior of developer.

> Of course, I don't think the fact that I did this is going to be noticed as it wasn't written out as a requirement

This sort of thing is exactly what I am interested in with this homework. You were given requirements and you reached for a set of tools that you thought were helpful to you as an engineer despite them not being required.

> If you notice what I'm nitpicking, it's all the setup and writing that becomes a time sink for most. In isolation, it wouldn't break the bank, but with all those requirements combined, you've lost much more than 3 hrs. When I'm given take homes for interviews, this is where I've burnt the most time - taking the time to document well.

I don't hold a lack of documentation or extra items against a candidate. That is made clear early on. There are tools that are very simple to setup or things you can do that would make it obvious to me that you know what you're doing.

You don't need a linter to format your code. You don't need a unit test framework to write unit tests:

    const apply = require('./submission').apply;
    const tests = [
        {rules: [], data: [], output: []}
    ];

    tests.forEach(function (test) {
        const output = apply(test.rules, test.data);
        if (output != test.output) {
           console.log("Failure");
        }
    });

Also, not having these isn't a negative. It just gives me a different context for the engineer I'm talking to. The only thing I think is bad is when an someone interviewing does one of the following:

    1. Code is obviously not working
    2. Code doesn't handle my sample data
    3. Code doesn't handle an obvious edge case (no data, for example)
    4. Code doesn't follow the one requirement (./<program> <config> <data>)

This assignment is a filter (if you can't program it is obvious) and provides background in the engineer (if they claimed on the phone to always do TDD and there isn't even extra sample data provided I am suspect of that claim).

Internal company standards make linters, testing, documentation, etc a "teachable" skill. I don't care if you do that on your own time because when you come to work for the company you will be doing what the company thinks is a good idea for it's source code.

(edit, forgot the "multiple" rule)