Hacker News new | ask | show | jobs
by EdwardDiego 1690 days ago
What were they scraping, if I can ask? Was it targeted or just wget -r style?
1 comments

It was a hybrid of low-effort vulnerability scanning and targeted inventory scraping. Many dealerships in the automotive space will pay gray-hat third parties to scrape and compile data on their competitors.

The irony for us as a provider is that it's one of our customers (party A) paying a third party to scrape data from another one of our customers (party B) which in turn affects the performance of party A's site. We've started blocking these third parties and directing them to paid APIs that we offer.

And how do you get your 'inventory data'? Aren't you scraping (or using scraped data) yourself? Oh the irony :)
No, we're a contracted provider for these customers. They ingest their data into our network through APIs or CSVs.
Makes little sense - customers upload data to you and they don't want any data back? Really?
It's not them who want it back, it's their competitors who want it.
I get it why someone else scrapes it. But why customers upload data in the first place? Aren't they interested in getting some OTHER data from you and that OTHER data may as well be scraped?