[jghn]

I used to take "relatively rare" a step further and viewed it as a myth. I'd get into debates with coworkers that all one needs to do is have a conversation with a candidate to fully judge their abilities. No need for coding tests or the typical questions. Unfortunately after we brought in a ton of candidates there were a few people who fell into this bucket. I have no idea how it happened but they couldn't code themselves out of a wet paper bag when it came time to do so even they kicked ass in technical conversations. Sadly it became a bit of an "I told you so" moment where the outliers are made up to be more frequent than they actually are.

I like your idea though.

[MichaelGG]

Had a similar experience with a remote sysadmin. Interviewing, he seemed to understand everything. He explained how he'd setup networks, details on using SSH for bastion hosts, etc. Seemed to really know stuff.

After hiring him, he was unable to even SSH to a box. As in he didn't know how to give us his private key, or how to configure his SSH client.

Maybe one guy takes interviews under other people's names then lets them mess up the work.

[troels]

OTOH if he had given you his private key, surely he would have failed the sysadm test right there.

[DoofusOfDeath]

That happened to me once, during a phone interview with Google.

My brain simply froze during the live coding test. I looked like an idiot, even though the coding challenge was one I could have normally handled in my sleep.

My guess is that it was due to anxiety about doing well on the interview. But to this day I'm gun-shy regarding live coding tests.

[meggar]

I did that at a Google interview too, on a variation of a question I'd practiced a few days before no less.

[cookiecaper]

It shouldn't take much demonstration to see that they are basically competent, and there shouldn't really be a difference between asking them to loosely state the commands they would issue and the process they would take and watching them do it.

People normally perform better by verbalizing the process than they would by sitting at their computer and performing it, since a lot of the concern in an interview can be whether you're getting your code/commands syntax-perfect or not.

The only way I can really see this going badly would be if the interviewer was only asking generalized questions that someone like a non-coder HN reader would be able to pick up from the zeitgeist without actually having ever implemented. Instead of asking about trends or fads, it's probably better to ask detailed questions about implementation processes.

If they really have internalized all the applicable concepts but can't express them in computer language, well, it shouldn't be hard for them to get up to speed on the syntactical fineries. Make sure you're testing for grokkiness of the core concepts needed, not how well they're following the trends or the news.

Personally I use a very small code test and put the rest of the energy into a detailed conversation.