[randomperson_24]

I think Google can probably not fix it. Users will have to be manually reporting as spam. These websites on seeing traffic from Google's crawler bots show a perfectly legit and highly SEO optimized website, but for anything else show other spam. If Google starts indexing from random IP ranges, most websites would probably block indexing from “unofficial IPs” or some companies (esp in EU) would file some lawsuit against Google. The reason being that some pay-walled news article websites won't be indexed properly, as the “unofficial IP-ed” Googlebot will not get the paywalled content.

If a website lies to Google itself, I believe the only way to solve it is by reporting the search result as spam or Google contracts people to somehow visit all billions of web pages (again the same problem – from different IP ranges) to verify it as a legit page.

I would like to know how Google currently handles it and probably how it could be improved

[panarky]

Google has all the text they've scraped.

They can see all the domains that have served a given snippet.

They also have history to identify where each snippet was first seen.

If SO has a lot of traffic and a good reputation, and if the same snippet is found first at SO and then later at bunch of newly created, low volume, low reputation domains, then show the SO result and not the others.

[Nextgrid]

The practice of "cloaking" has been around for ages and I'm sure Google has (or at least had) solutions against it.

I'm not sure on what grounds could someone sue for crawling from random, unaffiliated addresses as long as the crawling isn't causing a denial of service (they can always check robots.txt using the main IP then use that to throttle crawling from random IPs as to remain compliant).

> The reason being that some pay-walled news article websites won't be indexed properly, as the “unofficial IP-ed” Googlebot will not get the paywalled content.

Good riddance? That would be a welcome change.