Hacker News new | ask | show | jobs
by komlenic 5614 days ago
If you're just looking to prove out a concept and see if there could be any interest/traction for it, I would feel comfortable going ahead and scraping (but be smart about it). To me that means: know your target (big corporate sites vs JimBob's autos), throttle your requests (both the frequency with which you scrape and how rapidly you fetch pages), and how obvious it will be where you got your data.

But from your Q's it seems like you're looking at whether or not this is a longer-term viable idea. Hopefully some smart people will comment with experiences/facts on the legal subject.

All that legality may not matter so much if you can position yourself as a clearinghouse where these other parties just have to have a presence to survive, instead of a competitor?