It's possible that median is a better measure of central tendency for those things. It also sounds like you're interpreting their use of 'average' as meaning median when you think that would make more sense, rather than asking them (and in fairness, I don't usually grill people on which measure they're thinking of when they say 'average' either). In my experience, people who actually understand the different measures of central tendency usually (on average, heheh) specify one explicitly. The rest of the population is probably only aware of arithmetic mean.
If someone says something about the "average person", they are not talking about the mean, they are talking about the median.