Hacker News new | ask | show | jobs
by r_singh 228 days ago
My service only extracts public data major retailers, not indie sites, and deducts more credits for lower-traffic domains to offset load differences.

It would be great if there were reliable ways to distinguish good bots from bad ones — many actually improve discoverability and sales. I see this with affiliate shopping sites that depend on e-commerce data, though that impact is hard to trace directly.

The bad actors are the ones cloning sites or using data for manipulation and propaganda.