[ammon]

So, we've gathered data on exactly these points over the last 2 years. And what we've found that the decision that gets the best signal is often not what feels most accurate to the interviewer.

For example, my guess before running these experiments would have been that simply looking at progress (how far a candidate gets through a problem) would be a bad measure of interview performance. I'd expect that things like style and how communication how careful a candidate was being would render pure progress a mad metric. However, when we compare pure progress to a subjective score the interviewer gives the candidate (ignoring specific reasons, does the interviewer think the candidate is a good engineer after the interview), we found that pure progress is more predictive of success at companies! (To be clear, we don't only look at progress at Tryplebyte. We've also found other things to be predictive.)

The same is true (among our candidates at least) for question difficulty. Easy, straightforward question (write a command line interface to store and retrieve key-value pairs) are more predictive of success at companies than question that try to target intelligence (calculate how much water would collect in a histogram)

[famil]

Have you tried IQ testing instead of algorithm questions to measure intelligence?

[roguecoder]

There is no such thing as a single "intelligence". "IQ" is measuring at least three different forms of cognitive ability, is an inherently flawed and culturally biased tool and may be illegal to administer unless you can prove it's relevant to the work.

Additionally, what matters is not natural talent but the set of skills and techniques an individual has built up using those talents, compensating for their weaknesses and taking advantage of their strengths. I've met plenty of very, very smart people who flounder the first time they encounter a code base they can't hold entirely in their head at once: someone who was less "smart", with a smaller working memory, but who has been developing skill with abstraction and system metaphors since CS 101 is often a better actual developer.

The myth that developers need to be "smart" is pernicious, and the cause of most of the really horrific code bases in this industry.

[dragonwriter]

> [IQ tests] may be illegal to administer unless you can prove it's relevant to the work.

Not any more than any other means of assessment on which outcomes differ on a protected axis of discrimination like race; yes, the rule was first articulated in a case involving a fairly blatant use of IQ tests to effect racial discrimination, but it is by no means restricted to IQ tests.