[unbearded]

One thing that I noticed is that if the website is behind Distil Networks, they will block on the first request and make cumbersome, if not impossible, to automate some task.

I get that is important to protect the information of their clients (which seems to be content aggregators), but there are legitimate use cases for allowing at least some scrapping to happen - one being when the information is about the person/company who wants to read it in an automated form so it can be used for further processing.