Hacker News new | ask | show | jobs
by xenocyon 1713 days ago
"The more dimensions you analyze, the close to 'truth' you can get."

Data scientist here. The above is not the correct takeaway from Simpson's paradox. It is not generally correct that the trends seen in subdivided groups are closer to truth than overall groups; sometimes the opposite is the case. It depends entirely on what the divisions are and whether they make sense.

With regard to gender-based pay disparity, there are a multiplicity of factors, from the most obvious ("equal pay for equal work") to other factors such as the fact that professions largely staffed by women tend to get paid less than professions largely staffed by men. For instance childcare is miserably compensated, despite being a position of high responsibility and impact.

The consensus regarding women during the pandemic (not limited to tech workers) was that women have disproportionately sacrificed their careers to cover the needs of childcare and at-home schooling during the pandemic.

3 comments

Without defining truth and correct, the rejoinder makes little sense, your data science credentials notwithstanding.

What seems like a plausible interpretation to my un-credentialed, low-IQ self is: 1. Within dimension comparisons in this instance may suggest that “like” populations are similar across gender, suggesting that there is no bias in how tech compensates women and men. 2. The distribution of said dimensions differing between men and women may reflect exogenous effects that are not controllable by tech since they’re upstream of the tech hiring process.

Regarding Simpson's paradox, you can see a good treatment here [0] describing whether splitting a group is correct or not (based on causality); see Section 2.2 specifically.

[0] http://ftp.cs.ucla.edu/pub/stat_ser/r414.pdf

Well, that's definitely going on the reading backlog. Thanks!
> The above is not the correct takeaway from Simpson's paradox. It is not generally correct that the trends seen in subdivided groups are closer to truth than overall groups;

This is why the word truth was in scare quotes. And the word 'can' was also doing a lot of work in that sentence.

Should people who sacrifice their careers for noble goals be paid less?

Not that I am demanding an answer from you specifically. It's just a weird question in the context of a country where wage labor is the only real source of income for individuals.

What does “paid less” even mean in this context? If you sacrifice your career, you sacrifice the remuneration you were receiving from it. It’s part of the career…
That’s just simple economics - if they’re willing to settle for a noble job even if they get paid less then they’ll get paid less on average
In a market driven economy: yes. It’s the same reason why game developers are paid less than developers in other high skill jobs: game developers are willing to get paid less in return for working on games.
Your goal is the “noble” goal now, not your career.