[ksat]

Yes that's a valid point. We do ask the candidate to submit the source code once the they have solved the challenge. So employers who need only say "Rails" developers can mention it in the instructions and check the source later.

"However, as a very quick "can this candidate actually write code" filter, I think you're well on track."

Yes, that's exactly what I wanted this to be. Quickly filter out the 80% and move to the next stage where you can do much better justice reviewing the candidate's past work, working with the candidate on a project etc.