I read on wikipedia [0] the other day a fairly disturbing statistic related to this, apparently human men have on average a ~10% bigger brain than women. It'd be interesting to know if that translates to a higher neuron count or the difference in volume is due to something else.
Probably just a consequence of overall physical size being larger. AFAIK there continues to be no evidence of a sex difference in overall intelligence, so slight difference in brain size is probably a red herring.