The main difference is that the person calling you female likely doesn't have an honest belief that you are female, they're just trying to be a dick.