Hacker News new | ask | show | jobs
by lazyjones 3599 days ago
> Scraping should fall under fair use policy

Even if the burden put on the scraped website is far from "fair"? At my last job we had half the server capacity used by scraping bots most of the time, despite blocking Tor (since it was used exclusively for scraping most of the time). Such use translates into real operating costs just for being scraped, depending on your way of monetizing with 0 (no ad views) or negative income from those "users" on top (sometimes it's the competition or agencies selling your data so people don't have to use your website).

2 comments

It is your right, of course, to use whatever technical means necessary to make scraping harder.

And it is your right to sue whoever resells your data without licensing it.

But a legal pursuit just for scraping per se looks mean to me.

Maybe your business model has an intrinsic flaw.

For instance on LinkedIn, how many of the 400M strivers do I really want to network with? Yet I (or my bot) can see them all.

Those are both stupid arguments.

> Maybe your business model has an intrinsic flaw.

Maybe business are free to decide on their business model and not forced to comply with anybody else hobby project.

> how many of the 400M strivers do I really want to network with?

Are you suggesting that linked should block your access to everything except what their ml algorithm decides will interest you?

> not forced to comply with anybody else hobby project

How is LI being forced to comply? It's the opposite, hobby project is legally forced to desist.

They are not, and they shouldn't be.

The argument I was against was that LinkedIn should bear the cost of scrapers, and if they can't they must change their monetisation strategy to something that can.