Is there a reason web scrapers aren't used for public sites with content? It seems like this would avoid the pitfalls of APIs and changing terms and prices?
They were, in a much less constrained way, until companies running platforms started suing people for scraping. And investing heavily in making it difficult technically. The difference is advertising. The vast majority of people do not want to be advertised to, given the choice, nor do they want to have their contract with a platform downgraded (by having to use the UI designed by the company). This situation is some of the users taking a stand about it.