If we really dig for an example - mountain climbing guides for Mount Everest, certain groups of people have evolutionary advantages that make them more suited for the job.
In an extreme environment where advantages of the tail end of population distributions are important then it's less likely the market will choose a diversified workforce.
If the goal is to have a diverse group then a group composed only of women isn't ideal either. Diversity is not about raising or lowering the bar.