|
The project interviews are in-person. They take between 4 and 5 hours, so the time commitment is roughly the same as any other on-site interview process. No preparation is required. In fact, I could tell you the question ahead of time and I would still get useful signal out of the interview. Our process was to have someone "setup" the interview by introducing the project and helping the candidate get set up. Then the candidate would work on their own. Periodically, someone else on the loop would stop in, see how things were going, answer questions, etc. At the end, the last person on the loop would go over their code with them. This was a very important step. Hearing someone talk about their own code is just as important as the code itself. It's also interesting to see how candidates debug things, should any bugs arise. Finally, we would debrief. We'd bring up their code in a meeting room, test their project, ensure it meets the requirements, find bugs, etc. We'd discuss the code, see how clean it was, whether or not it was nonsensical, etc. Selecting a project for a project-interview is non-trivial. It's definitely possible to pick a bad project. For instance, sudoku solver sounds like a bad project. The evaluation criteria is too strict and it sounds like there is probably an "optimal" answer. Basically, it sounds like one long leet code question. A good project question has a number of qualities: 1. The project needs to be a language of the candidates choice. You want to see how they code, how they think about problems etc. If you give them a skeleton, you're constraining their thinking and will get a weaker signal. It's also very informative to watch a candidate setup a new project. Good engineers will easily setup a new project in the language of their choice. Weaker candidates will suffer. 2. Evaluation criteria needs to be clear. We had 5 requirements that the project needed to meet. It was objectively easy to say whether or not the requirements were met. Failure to meet any of the requirements was essentially an auto-fail. I've seen project interviews where they don't tell you exactly how your code is evaluated and then run a test suite against your code. Forgot to test a negative number? Auto fail. How is that reflective of work done on the job? Such opaque criteria are terrible. 3. The project needs to be well-defined, but still somewhat open ended. We had 5 criteria to meet, but it was amazing the variety of responses we got. New grads would struggle to get all 5 done in time. Experienced engineers would use the time to go above and beyond. Going "above and beyond" wasn't really necessary. We could still judge skill just from the basic requirements, but it was cool to see how far some people took the project in the short time given them. This is a reason something like "sudoku solver" is a bad question. 4. The problem needs to involve some sort of data model / state tracking. A lot of software engineering involves tracking state, so we found it's important to see how people actually do that. 5. The current team needs to do the interview themselves. For instance, going back to a sudoku solver, how many people could sit down and do that in a short period of time? How many people spend their time thinking about discrete math problems like that? Personally, I don't like doing that, so it seems weird that I would require that of a candidate. Everyone on the team should feel comfortable that the project is achievable and reflective of the general work done on the team. |