[taneq]

Sure there is. You take each programmer’s self-ranked hardest N problems and get the other programmer to try and solve them.

[NateEag]

People are not actually fungible.

Each of us has different skills, insights, strengths, weaknesses, and life experience.

What caused one of us months of misery might be the next person's perfect problem, what they were born to solve flawlessly.

[taneq]

That’s why you average across several questions. If one is too hard just leave it. You’ll pretty soon get an idea who’s good at what.

[whatshisface]

And if one person has had a career of spending 1 year on each research problem while the other person has had a career of spending ten minutes on each UI micro-bug? You swap the problems, and person A spends 20 minutes, person B spends half a year, and you think person A is better even though they're actually 4x worse.

[taneq]

Just because you can contrive a pairing that’s less efficient to compare doesn’t mean it’s a bad test for colleagues.

[daniel-cussen]

Plot hole. Why did he spend 1 year on each research problem? After a year, what? A paper incorrectly proving the goal was impossible because that's what PhD's in industry say (I've seen this) so they're boss gets off their case?