If most men had many wives, there must have been a lot less men than women, and/or a lot of men with no wife at all. Or perhaps wife sharing would account for it.
Or, like a lot of books, most of the stories were about rich men who had things others didn't. That part isn't glossed over at all. But that's just about the Bible stories. If you're talking about my comments on early humans, I think it's likely that sharing partners was more common.