I'm a man. I don't even like showing tremendous physical intimacy with the women I go out with on dates. I hate it when people at my workplace or church touch me. My brain says "Get away from me, all of you".
I did have plenty of physical intimacy from my parents and my father growing up. Lots of hugs, kisses from both parents etc. After reading some of the other comments and reflecting, I think it's that I feel like too many of the people in this world are trying to socially manipulate me through touch, when I wish to be away from them.