Hacker News new | ask | show | jobs
by Ihateusernames2 3142 days ago
Chrome Devtools.

That sounds silly, and it is. But really, in many cases the data you want to scrape is already loaded to the page in a useful format. For example, take Politico's 2016 election results data: https://www.politico.com/2016-election/results/map/president...

If you go to devtools > network > XHR, you'll see the page for Colorado loads a file containing all of the results in a format you should be able to parse. This example is a bit tricky, as parsing it as a non-programmer might still be a bit challenging, but some sites will load data as a CSV.

I know this doesn't exactly answer your question, but I think it would help non-programmers to know this.