| I had similar questions. You're saying that if, for example, the quality among the pool of women applicants were higher than the quality among the pool of men applicants, then you would expect women to have greater representation among successful applicants than among applicants in general, and that if the female applicant pool were of higher quality and yet female applicants were accepted in the same percentage at which they applied then this would be evidence of an anti-female bias at Hacker School. In other words, you argue that the genders are accepted at the same rate at which they apply is evidence that either (A) the quality between the two pools is identical and Hacker School is not biased, or (B) the quality between the two pools is not identical and Hacker School is biased. And that from the evidence there is no way to distinguish from the evidence between the two scenarios. You might be able to distinguish between the two scenarios given the rate at which anonymized applications are accepted for further review. That would be your "metric that is blind to X". I guess that the fact that for (B) to be true, there would have to be some coincidental match between quality differential and reviewer bias argues against the case of bias. And also that each stage approves the same proportion of genders is an argument against some kinds of bias (e.g. reviewers who, at each stage, sheer off a different proportion of each gender's applicants). So as you point out, Hacker School's analysis requires some model assumptions. I think that the conclusion as written might have been too strong. I hope that any organization that might attempt to rigorously address the endemic issues of gender and race in our vocation will take exactly the steps you recommend regarding assessment of applicant quality, and if we find that males and females are entering into applications with different levels of quality then we will have another vector by which to address those issues. I took this post as a kind of informal report about steps that Hacker School has taken to lessen bias in their application process and the industry in general. And given what I've written above, I take their statements as evidence--not proof, but evidence--of good measures against bias. |