Hacker News new | ask | show | jobs
by kbenson 1614 days ago
Do you provide an API, paid or not, for the same data? An API which might even have limitations on use makes scraping a bit less defensible in my mind, but if you're offering something for free to the public and then getting upset when people take and use that free info, maybe free isn't the right business model, or maybe you should look into what those people are using that scraped data for and see if you can offer it better and cheaper.

The best way to stop someone trying to make a buck on your hard work is to go direct to their customers and do a better job. If you can't, what they're selling is something on top of your offering and you aren't serving that market, and you either should start serving it, or make a deal so the scrapers can continue to do it without impacting your service.

As someone that had to do scraping in the past, and went through having a free open API that served our needs perfectly replaced with an account based one that required we make 100x the queries, it was really frustrating that the company refused to even respond to queries for specific business accomodations to data.