It’s interesting that humans have decided to allow violence to be acceptable and commonplace, celebrated even, while sex is the thing that is taboo. Seems backwards, especially when you consider consent.
The thing you have to consider is genetics (parentage). Most don't men don't want to raise a child (and all the effort involved in that) that is not their own (without consent, outside adoption, etc). It's a very reasonable position and the basis for this arrangement appearing in most religions and legal systems.