[0815test]

That depends on what you mean by "inappropriate", of course. People say all sorts of things they don't mean to when they're made uncomfortable by something, and the line between "discomfort", "confusion" and "fatigue" is itself quite fuzzy. And yes, they apologized for it immediately afterwards - but apologies themselves can be strategically unwise, and leave one open to manipulative behavior in response to a perceived display of weakness. Their wife certainly didn't seem to think OP's behavior was inappropriate in the way you're implying, and she'd obviously care if it was!

All in all, I'm not sure why you seem to be willfully ignoring the whole part where OP explains what prompted their remark (whatever that might have been) in the first place; whether intentional or not, it seems that a somewhat "inappropriate" situation had already occurred by that time.