Hacker News new | ask | show | jobs
by estimator7292 185 days ago
What if your target isn't any WordPress website, but any website?

It's simply not possible to carefully craft a scraper for every website on the entire internet.

Whether or not one should scrape all possible websites is a separate question. But if that is one's goal, the one and only practical way is to just consume HTML straight.