|
|
|
|
|
|
by echelon
146 days ago
|
|
|
> Many sites only allow Google scrapers in robots.txt and the public will fight you hard if you scrape them without permission. This just lets a monopoly replace the website instead of distributing power and fostering open source. The same monopoly that was already bleeding off the web's utility and taxing it. |
|
|