[solardev]

I don't know who Matt Butts is, but what do you mean? Those are just standard tools, along with robots.txt and a siremap, for telling well-behaved crawlers what to index and not. I've used them effectively across many sites. They won't help with rude crawlers (LLM training etc.) that ignore those signals, but should work with all major search engines. What am I missing?

But yeah, I think spam should be deleted altogether, not merely hidden from search engines.