[Spivak]

I think you're leaving out cultural inertia as a contributing factor to this. If you want training data that's purely descriptive of the current state of the world then fine, but such data will naturally encode our own biases into it. Reality unfortunately carries a lot of bias. There's nothing that says that plumbers have to be a male, that playing house is a game for girls, or online video games is dominated by boys. But huge amounts of social pressure, environments that aren't inclusive, and monkey-see-monkey-do keep the cycle going. I find it funny that you're making this argument on a board for programmers which is basically the canonical example of a profession that underwent a gender flip.

Talking about gender roles on HN is odd because programmers tend to think so binary. Gender roles in modern society are informed by biology inasmuch as La Croix's flavor is informed by fruit but all you get in the discussion is "see gender roles aren't entirely arbitrary so everything about the status quo must be justified."

A non men/women example is neighborhood crime rates. A totally "unbiased" record of crime rates will show that black neighborhoods have on-average crime rates. But that ignores the fact that less policing is done in areas with lower crime rates and if you don't look you won't find it.