Hacker News new | ask | show | jobs
by aaronwall 5985 days ago
>> care about not being branded <<

Absolutely right. They care about brand.

If you look at the content (and I don't mean the top 100 guides, but the hundreds of thousands of backfill pages) most of it is nothing without being part of some branded collection (leveraging Google's authority site loophole). And thus protecting their brand is crucial to protecting their livelihoods.

If a scraper site is branded as a scraper then it falls out of Google's good graces. And without that traffic the site loses probably over 50% of its revenue.

But if they removed the scraped content they lose a huge chunk of revenues as well (if I had to guess I would say at least 25% to 30%...and it is the all margin piece of their business).

I think the only longterm solution is some sort of editorial curation and removing nofollow from the links.

Sure it will increase costs in the short term, but in the longterm it increases the value of their operation significantly. And it sets up a huge competitive barrier between them and other would be semi-legit to illegitimate scraper sites.