[kailpa1]

From `resume_evaluation_system_message.jinja`

> *SCORES MUST NEVER DEPEND ON THE FOLLOWING FACTORS:*

> - College, university, or educational institution name

> - CGPA, GPA, or academic grades

I don't understand why they would omit these factors from the evaluation.

[swiftcoder]

> I don't understand why they would omit these factors from the evaluation.

Only hiring MIT graduates sounds great to a lot of tech folks! Automatically rejecting applicants from HBCUs, however, sounds like a lawsuit

As to GPA thing, I think it's just to stop the LLM glomming onto an obvious numerical grade? LLMs like to rank things by obvious dimensions, and whether someone had a 4.0 or a 3.8 in grad school makes very little difference to their performance 10 years down the line.

[ceejayoz]

https://qz.com/1427621/companies-are-on-the-hook-if-their-hi...

> But it didn’t. After the company trained the algorithm on 10 years of its own hiring data, the algorithm reportedly became biased against female applicants. The word “women,” like in women’s sports, would cause the algorithm to specifically rank applicants lower. After Amazon engineers attempted to fix that problem, the algorithm still wasn’t up to snuff and the project was ended.

And in another org:

> After an audit of the algorithm, the resume screening company found that the algorithm found two factors to be most indicative of job performance: their name was Jared, and whether they played high school lacrosse. Girouard’s client did not use the tool.

https://www.npr.org/2024/04/11/1243713272/resume-bias-study-...

> Their working paper, published this month and titled "A Discrimination Report Card," found that the typical employer called back the presumably white applicants around 9% more than Black ones. That number rose to roughly 24% for the worst offenders.

It'll discriminate by proxy, basically.

[bulder]

I don't understand why they'd hand over those data points over to the model in the first place. If it's in the context window, it's impacting the output. To ensure that no weight is placed on those factors, they should be sanitizing them out before handing the data over to the model.

[sph]

Hopefully so that people like me, that dropped out of high school yet have had a successful career as a self-taught engineer, have a chance. [1]

Just kidding, my resumes are sent to /dev/null like everybody else’s.

——

1: In fact, I will be controversial and say that self-taught engineers tend to be the strongest in their own particular niche, because they are powered by sheer desire to learn and improve. I am routinely appalled by how many people go on forums to ask how to learn a new thing, completely unable to self-direct their learning. I blame the modern school system.

[kailpa1]

I'm a self-taught programmer as well, who dropped out of university, and these factors being omitted would benefit me as well, but I feel like good grades and a good university are still indicators of someone being or is capable of becoming a good programmer.

This system would drop a Harvard top graduate for someone having a year of experience in some outsourcing firm.

[Schiendelman]

Unfortunately, graduating from Harvard is a very good predictor of whether your parents were wealthy, and also that you are less likely to be black.

I worked for a very large job board for the last six years, it's the one you're thinking of. What we found is that the outcomes of paying attention to what school you went to are almost entirely discriminatory, and not predictors of success.

[kailpa1]

That's sad to hear. I guess I'm giving too much credit to schools as a measure of success.

[Schiendelman]

Absolutely. It's correlation, not causation. At least for the most part. The network you build at a very good school exposes you to all those other well connected people, that's the real value.

[goosejuice]

> I feel like good grades and a good university are still indicators of someone being or is capable of becoming a good programmer.

Really depends on the program. In my undergrad program there were some very smart CS students who got great grades that really struggled with the programming. Smart and capable people can be bad at programming and lack many qualities that make for a good hire.

[kailpa1]

Sure, but isn't this kind of person the exception? I feel like most of the time good grades mean good programming skills

[sph]

I started in an outsourcing firm (body rental actually) but I definitely get your point. Maybe they optimize for real world experience, or rather, how one is used to workplace politics and logistics. The top grad will have higher expectations, and all they want is a cog for the Machine.

[kailpa1]

Yep, I don't know either, but I guess they have their reasons for this.