Hacker News new | ask | show | jobs
by wolrah 2028 days ago
Agreed 100%.

Basically anything that would stop a scraper from downloading or interpreting a single page will also get in the way of search engines and screen readers. The former is generally not desirable and the latter may actually be illegal for some organizations. Just give up if this is your goal.

Rate limiting and other approaches to restrict bulk downloading of your entire dataset are more practical, but are still generally easy to work around if one is sufficiently determined unless you require authentication for all access.