There are lots of uses of "have" that are not property. "I have many friends," "I have a butcher in town who does great work," "I have 10 employees in my business."
Beyond a token effort to be reasonable in word choice it is not his responsibility to try and hedge against the unforeseeably long tail of potential misinterpretations. Your jump to conclusions response is your responsibility, not his. His language was fine.