Interesting point. How can anybody know? If I have a penis, under today's standards I can claim to be female. If I have caucasian ancestry, can I claim to be black? Why not?
Because all of these factors will necessarily have to be taken into account when we come up with our speech rules, don't you see?
Sure it's complex, convoluted, and controversial.. but as long as we put the right people in charge to make the decisions for us, it will all work out fairly.
"We", meaning the readership of this site probably doesn't (I hope?), but it is a common trope that what you are allowed to say (meaning: be accepted without immediate backlash) is based entirely on your identity.
I'll not go into that concept further for fear of igniting another tired flame war.
More substantive comment: why do we care about a person's race or gender when they use particular words?