[sheepmullet]

"This question comes up a lot, and I'm not sure that it's really as hard of a problem as it's often made out to be."

The speed or novelty of a solution to an interviewers pet problem provides no insight at all into how the candidate will perform on the job.

At my last company we would re-interview a sample of our staff using the same interview questions we were going to ask externals in order to gauge the quality of the questions. Pet problems provided the least value out of all the interview techniques we tried.

"In other words, it might be difficult to tell the difference between someone moderately better than you and someone vastly better than you."

It is very difficult to determine if somebody is better, and how much better they are, than you when you are close in skill level.

On the other hand if somebody is 5x better than you it is easy to determine. You are right though in that it is impossible to tell the difference between somebody who is 5x better than you and 10x better.