[eckesicle]

I ran a largish statistical analysis where I was working on all interviews and employee ratings for the following year (16000 data points).

My conclusion was that:

* some interview problems carried zero signal

* some interviewers are really good at identifying strong candidates, and others correlate negatively! On average interviewers rating is a weak signal (<0.25. Correlation)

* geographic location is a medium signal (0.2)

* a relevant degree is a strong signal (0.4)

* choice of programming language is the strongest signal (>0.8 correlation (or anti-) for some languages, and some.

There is some selection bias though since candidates who were rejected did not get a chance at the job so we didn’t have a rating for them.

[hermannj314]

The choice of language being a signal is interesting. That goes against my intuition.

Can you elaborate?

[callahad]

Could rhyme with PG's "Python Paradox" essay from 20 years ago: http://www.paulgraham.com/pypar.html

> It's a lot of work to learn a new programming language. And people don't learn Python because it will get them a job; they learn it because they genuinely like to program and aren't satisfied with the languages they already know.

> Which makes them exactly the kind of programmers companies should want to hire. Hence what, for lack of a better name, I'll call the Python paradox: if a company chooses to write its software in a comparatively esoteric language, they'll be able to hire better programmers, because they'll attract only those who cared enough to learn it. And for programmers the paradox is even more pronounced: the language to learn, if you want to get a good job, is a language that people don't learn merely to get a job.

I wonder correlations eckesicle found, and what languages would fit that profile today.

[tough]

So like Erlang nowadays? Or Rust?

[callahad]

It may be my own bubble, but it feels like Rust left that niche around 2019?

I would've put Clojure on the list around 2010.

Elixir feels like a right answer today. Maybe Zig, too?

[yw3410]

My intuition would expect the two to be correlated.

All anecdotal evidence I've seen points to the language mattering a great deal since only those people who are enthusiastic about niche languages, tend to put more time in learning/practicing programming and all other things being equal, spending more time on a skill correlates to being better at said skill.

[eckesicle]

Yes. Interviewees were free to pick any language to attack their problem. We found that interviewees who picked certain languages performed consistently worse during their interviews, and also received worse feedback in their 360-review after working for a year (if they passed).

The bottom languages were: Java C++ Php

They also made up the bulk of interviews. I suspect that Java and c++ together accounted for about 75% of all interviews.

Languages like ruby, python and golang increased your chances of passing the interview as well as getting good peer feedback one year on.

More niche languages (rust, erlang etc) didn’t have enough data points to be analysed separately, but if you grouped them they were the choice of the strongest candidates.

[zephrx1111]

Didn’t know the language can be a factor. I was always told any language is fine.

[breckenedge]

Do you have a link to this research? This is amazing.

[eckesicle]

Sadly not. We ran the study on internal company data, so it was never published. The purpose of the study was to improve and inform on our hiring practices