We could get along just fine, professionally. I can get along just fine with many female colleagues at work. I am just not interested in keeping contact outside of the professional environment and neither in becoming friends, unless I am sexually attracted to a woman.
But now I have a wife and a baby, so I will not bother with other female relationships anyway. I like to keep things simple in life :)
I would recommend you do bother. Having a social life very enjoyable, even if we sometimes think it's not needed. Besides, you're not the only guy your wife is taking to, stop kidding yourself
I would say yours is invalid. I'm an asexual male and do seek friendship with both genders, more because of the need for seeking social interaction. And I do believe I do get along well with people even without second intentions.