Hacker News new | ask | show | jobs
by cheetos 3303 days ago
Did your service offer a paid API? Scraping happens because of a lack of better options. Surely you can understand why the scrapers didn't want to contact you beforehand.
3 comments

If you want my data on a paid API basis, then ask me about it. I need to know how big the demand is for third party users before I even prioritize building a paid API, having the god damn courtesy to ask for something would give me an idea.

If you're using my data to hijack my traffic, without asking, you could have all the right justifications in the world but you're still a prick. Who knows, maybe your orphanage building app will move me tears once I hear about it and I'll give you free access.

In what world is "engage a third-party scraping company" a better option than "drop a quick email to the site operator"?
Because with scraping you are in a legal grey area. But if you contact the site directly and they say "no", then there is no excuse to scrape.
Well, yeah, like if you ask someone to sell you their dog and they say "no". Doesn't justify stealing the dog.
Your analogy doesn't hold up. Your example is clearly theft, and is a criminal matter. Violation of a sites terms is a civil one, and again is a legal grey area. Scraping the site doesn't delete the content from a server...but there is only one copy of the dog.

EDIT - I can't reply to your comment below, but FWIW I agree that scraping sites in this manner is unethical. I am merely describing the logic that most scrapers go through for self justification and legal protection.

The analogy was "doing something anyway because you might be told 'no'", not "my server behaves like a canine quadruped". Copyright infringement is also potentially criminal in the UK, it's not as simple as you suggest.

But whatever. It just saddens me that the internet is a constant "don't be a dick" battle with companies like the scraper guys.

(edit - understood :) )

Yes, because they did not want to pay. This is not some complex issue.

Incidentally, I'll be shopping later. May I give you $5 for you to drive to London and take me there? I was going to steal your car, but then I figured I'd be a good citizen and demand you provide it to me at your own, prohibitive, loss.