[userbinator]

For example, moving from static HTML to a web framework would require your scraper to actually run the JavaScript to generate the DOM in the state that a reader might view it in

Or, as is often the case, the content is already there or fetched via an API in far more easily-consumed JSON format that you can use directly.

[throwaway894345]

That’s my point.

Granted, lots of APIs make it prohibitively difficult to authenticate such that it’s easier to simply scrape. Such is the case with just about every Microsoft product I’ve ever used, most recently the XBox Live API. I genuinely wonder what kind of nonsense goes on in Microsoft design review meetings.