[ahomescu1]

Society is made of many individuals, you can't disconnect a discussion about society from one about individuals, including "me". If you make high-level society-wide decisions that trample on individuals' rights, that also has society-wide implications (even if it's just a minority of individuals). What you're proposing is a tyranny of the majority, where the opinion of a single individual is discounted "for the greater good".

On the other hand, I believe society is just a big group of individuals, so "what about me?" does matter (in fact, it's maybe the most important question in all of society).