[Alex3917]

> And we've now reached the point where our automated assessment substantially outperforms human interviewers at evaluating technical skill.

What does this mean exactly? E.g. does the test successfully identify the people who have the best portfolios of things they’ve built previously?

[gwern]

It might mean something like the test predicts better than a randomly-selected interview whether a candidate will eventually be hired. The real question is whether the interview here is an unstructured interview or a structured interview. Industrial psychologists have been pointing out for decades that unstructured interviews are awful, even though everyone keeps doing them, so it wouldn't be surprising if they can be outperformed by even a simple checklist or test. (There are similar, somewhat infamous, results for things like judges and parole - where simple linear models can outpredict the experts.)

[ammon]

I'm one of the co-founders at Triplbyte. What this means is that our statistical model now significantly outperforms our own interviewers (and me) at performing a structured interview, and then making a judgment about whether a candidate will or will not go on to do well at interviews at other companies. It is slightly meta, because we're using an interview to predict other interviews. But I find it very exciting, because a) it means we're more accurate, and b) we know exacly what features we're feeding into our model, and can avoid a lot of the bias in the process.

[dabockster]

> does the test successfully identify the people who have the best portfolios of things they’ve built previously

Even with an algorithm doing the identification, any data produced is still going to be highly subjective (based on the "best portfolio" ideas of TripleByte and whoever worked on the algorithm).

[baxtr]

I wonder if the machine will tell me accurately if I’d like to hang out with the candidate after work for a beer or two. I’m not sure about that