Hacker News new | ask | show | jobs
by fulco__ 636 days ago
I don't think I specifically got blocked. I haven't done anything to get around scraping restrictions like setting a fake User-Agent. If a website doesn't want to be scraped I'll just use inspect element.

Some other current limitations:

- Might miss items in a SPA that loads additional assets outside the initial html - Misses images/SVG's that are loaded with external css with background:url(...);