It leads me to wonder, if google is as smart as most people see them as, why don't they make an attempt at working this out, or at least defining a way to deal with it themselves?
They kind of do but this gives site owners a method to instruct Google on what they want stored about their sites. You can't figure that out with an algorithm.