[joshuamorton]

>Another option would be to just throw them into a source code tree and say something like "Here's a tree of one of our projects. Talk to me about it."

"Its in perl, I don't know perl, and especially not your internally modified version of custom-magic perl that Steve wrote 7 years ago."

Not to mention that you are now either showing source code trees to random potential hires, or you have to audit/create/otherwise use some potential set of source code. Maybe you prescreen by asking them their favorite language, and you come in with an open source project, in their language of choice, but now you have to have one of your devs spend time familiarizing themselves with Redis or the Python interpreter or Hibernate Core or Angular or whatever, and what happens when they ask to do the interview in Haskell?

FWIW, I know some companies that do the interviews you're describing, but they're all relatively small (<100 employees), and they all do that kind of interview only after a technical phone screen with your conventional questions, because the time investment required by the company is so great.