I think that's more of an ego thing that people don't like, such as, "I get a lot of important phone calls, so I might have to stop talking to you and start talking to the person in my ear at any moment." Maybe it's the same with Glass. I don't know.
I was thinking about that point as I was reading through this discussion. The whole "Borg" Bluetooth earpiece thing does seem to be passe. (Personally, my problem with them was always that they ended up being out of juice when I needed them so I just use a corded headset when I use one.) I'm not sure what this observation says for wearable computing, if anything.