Hacker News new | ask | show | jobs
by timscott 4683 days ago
I've recently been learning all this the hard way.

1. Documented API. Failing that...

2. HTTP client fetching structured data (XHR calls). Failing that...

3. HTTP client fetching and scraping HTML documents. Failing that...

4. Headless browser

I recently found myself pushed to #4 to handle sites with over-complex JS or anti-automation techniques.