[citrablue]

It's because Google's constraint is number of engineer-attention-hours. This kind of thing probably just isn't a priority, when they have a billion other things they could work on.

Keep in mind that Google is not a product company, they are a data company. They work on the biggest most impactful items, and don't have a lot of time for one-offs.

Finally, it might not be clear at all that these sites should be removed from the index.

[idlewords]

Google has vast resources and could easily get this stuff right, but chooses not to. Their protests about not having the capacity can get pretty comical. Note also that the claim about "working on the things that have the most impact" is tautological.