[mattkrause]

Basic math too.

Suppose “ability” is normally distributed in the population and in your initial team too. Replace people when the new candidate improves the team’s median ability (supposedly what Amazon’s “Bar Raiser” checks).

I kicked together a simulation of this process. The first replacement is pretty easy (50/50), but the hundredth hire on a team of ten often takes tens of thousands of interviews, and sometimes over a million. No one is literally doing this.

[BigBubbleButt]

How many years does it take a team of 10 to get to their 100th hire? If they only replace 1 person a year, ~100 interviews a year sounds pretty normal to me (it actually sounds low compared to where I last worked where I sometimes interviewed multiple people a day).

[mattkrause]

I should have pointed out that it's literally an exponential growth, so it'll be fine for a while until, suddenly, it isn't.

It's also very vulnerable to "founder effects" if the teams are small: a few geniuses early on make it nearly impossible to hire someone new.

[vorticalbox]

Would love to see this code if you don't mind sharing.

Mostly interested on how you even create a simulation like this.

[mattkrause]

Mine is super-dumb, but let me walk you though it.

First, you need to a way to generate candidates of varying abilities. There are all sorts of tough questions related to measuring intelligence, but let's side-step those and make something like IQ. By construction, IQ is normally distributed with a mean of 100 and a standard deviation of 15.

    function generate_candidates(;n=1)
        return 100 .+ (15 .* randn(n, 1))
    end

With that, make the initial team and find the "bar" for a new hire:

    team = generate_candidates(;n=10)
    bar = median(team)

To replace a candidate, you just sit in a loop, "interviewing" candidates until one exceeds the bar.

    candidates_seen = 1
    while (new_candidate = generate_candidates(n=1)[1]) < bar
        candidates_seen += 1
    end

Once you find that person, you kick someone out of the team and replace them with the new candidate. I did it at random, but eliminating the lowest performer only exaggerates the effect.

    team[rand(1:length(team))] = c

Having done that, you need to recalculate the now-raised bar:

     bar = median(team)

We repeat this process a small number of times to simulate turn-over within a team. Since it's random, you want to repeat the entire process a number of times too. Complete code: https://gist.github.com/mrkrause/e33c589b901b4b8c96f940ea0a4...

I had a hunch this process would be exponential, and it certainly looks that way if you plot the results. This was meant as a quick-and-dirty way to check that: there are some tricks that might speed up the simulation and it might even be possible to do the whole thing analytically (but it's Friday afternoon).

FWIW, I highly highly recommend this sort of noodling around for building intuitions. At work, we recently spent a year and $$$ collecting some brain data, and a dumb model like this was the key to figuring out what was going on.

[Noumenon72]

The model might work better if you assume an increasing median among the population of candidates over time. This could be true if:

* Amazon's stock price and profits made it a more desirable place to work over time

* The increasing supply of computer science graduates improved the hiring pool

Another thing that could happen is the most skilled x% of each team leaves over time for better positions, so that the median slips back. Then raising the bar each time only keeps it in the same place.

Or candidates could be rated on multiple dimensions, so while you can't find someone uniformly better than your median employee, you can hire one person to raise the bar on build systems, one on microservices, and so on.

[CamperBob2]

    function generate_candidates(;n=1)
        return 100 .+ (15 .* randn(n, 1))
    end

How does this yield a Gaussian distribution?

[mattkrause]

`randn` returns draws from a standard normal distribution (i.e., a Gaussian with mean 0 and standard deviation 1); the n is for normal.

It's then stretched to have an SD of 15 via multiplication, and shifted to have a mean of 100.

This is Julia, but it's the same in Matlab and Numpy (np.random.randn); it'd be rnorm in R (which also lets you specify the mean and sd directly).

[CamperBob2]

Ah, thanks, I was reading it as "random number."