So, you're making a very particular point, in such a way that it is easy to misunderstand. Normally, just saying "women" or "men" is fine, but in this instance, sports generally refer to biological sex, not social gender, because biology discriminates. Individuals with biological male sex have higher bone densities, more ability to gain muscle, tend to be taller, larger, etc. "Women", as in gender, not sex, should not be slower, this is true, but "women", as in sex, will be slower due to biology.
The physical disparity is not a social issue, it's biology; the social component can be argued, however. How we choose to categorize within sports is in our control and opinions will go either way - in my opinion, the separation of men's and women's sports was done to try to correct for that physical disparity caused by biology, if that's undone, I think the top records will be dominated by individuals with biological male sex, regardless of gender, in all categories.