Hacker News new | ask | show | jobs
by throwaway6845 3301 days ago
Yes. Did that after this episode.
1 comments
tokenizerrr 3301 days ago
Were you seriously expecting bots to read your T&C? Or anyone, for that matter? Did you mention that it was okay for Google to scrape your site?
link
throwaway6845 3301 days ago
We're not talking generic "bots".

We're talking a custom scraper written for this site and this site only.

Yes, I am expecting the people who spend hours inspecting the source of my site, and then writing a custom scraper for it, to spend 30 seconds reading the T&Cs first.

link
tokenizerrr 3300 days ago
Not sure why you'd expect that. If my webbrowser can download your source code, my software will as well.

If you want people to read it put your content behind a sign up with a checkbox.

link
throwaway6845 3300 days ago
It is _already_ behind a sign-up with a checkbox. They scraped their way past that too.
link
samstave 3300 days ago
How? (Seriously, how does one do this?)
link
kossae 3300 days ago
Simply log in first, then perform the scrape programmatically.

Seen here: https://kazuar.github.io/scraping-tutorial/

link
tokenizerrr 3300 days ago
Ah, that changes things somewhat.
link