Hacker News new | ask | show | jobs
by adinosaur123 1692 days ago
Are there any court cases that provide precedence regarding the legality of web scraping?

I'm currently looking for ways to get real estate listings in a particular area and apparently the only real solution is the scrape the few big online listing sites.

2 comments

https://en.m.wikipedia.org/wiki/HiQ_Labs_v._LinkedIn

That’s one of the bigger ones. Unfortunately recent events means scraping is still a gray area.

Legal gray areas are perfect for growth hacking. Just look at Uber and AirBnb.
And even bigger growth hack for a lot of companies would be having scraping protected by law.
Do you mean this case?

https://en.m.wikipedia.org/wiki/Van_Buren_v._United_States

I think it only applies to systems that aren't available to the general public, which in this case was the GCIC. Anything that is available to the public, even if it requires some sort of registration, would I think be legal to scrape. YMMV though.

I was involved in a scraping-related case, though in my situation we were scraping public domain data/facts/public domain media. Email me if you'd like additional info. :)

More related to the submission content -- at the time we used rotating proxies, both in-house & external (ProxyMesh - still exists & only good things to say about it); they allowed us to "pin" multiple requests to an IP or to fetch a new IP, etc...