For the subtle variants, I believe it's how they price discriminate. It's all the same part, but they use fuses to disable various peripherals. Way easier to do that than to fabricate a whole bunch of different chips.
Most likely when it's moderate/major features. Sometimes it can be as simple as a case of packaging (i.e. fewer pins so they can't bring everything out.) I've always assumed that some of the packaging options were driven by large customers who wanted something just a bit different for whatever reason. (PCB space, power consumption, just to be difficult... who knows)