[mattkirman]

At the risk of playing devil's advocate, I would argue that giving candidates the ability to use virtually any language or framework and develop on localhost is not necessarily an advantage.

When I'm looking to hire a Rails dev then I want to know as soon as possible in the interview process that they actually know some of the basics of Rails and MVC - not just if they can build a web app. Furthermore, by hiding the candidate's choice of language/framework (and code quality) behind localhost the benefits of assigning an arbitrary coding challenge are lost. I fear that the only solution would be for the candidate to submit the full source code.

However, as a very quick "can this candidate actually write code" filter, I think you're well on track.

[ksat]

Yes that's a valid point. We do ask the candidate to submit the source code once the they have solved the challenge. So employers who need only say "Rails" developers can mention it in the instructions and check the source later.

"However, as a very quick "can this candidate actually write code" filter, I think you're well on track."

Yes, that's exactly what I wanted this to be. Quickly filter out the 80% and move to the next stage where you can do much better justice reviewing the candidate's past work, working with the candidate on a project etc.