Hacker News new | ask | show | jobs
by chumanak 2444 days ago
great article, the problem I found in those cases is that some pages like immobilienscout24 sometimes take a lot to load, and that increases the cost. Also in my case I found some websites (not immobilienscout24) blocking my requests, so I ended up implementing proxycrawl to unblock the requests, that allowed to bypass the blocks of google cloud but still I faced the loading times problem of some of the websites