[vjshah]

i had the same reaction, but i looked to see what google's tos was for things like this and i found https://serpapi.com/ , which apparently has been around for 5 years and also offers paid services for scraping google search. if their homepage is to be believed, there are large companies using their service. i'm very surprised that google hasn't taken steps to block them.

[benzible]

Note that part of SerpApi's product seems to be indemnification from liability. On their pricing page, the Developer plan specifies "No Legal US Shield" and the Production plan specifies "Legal US Shield".

From their FAQ:

> What are scraper legal protections?

> SerpApi, LLC assumes scraping and parsing liabilities for both domestic and foreign companies unless your usage is otherwise illegal.

[vjshah]

if i were evaluating serpapi, my concern wouldn't be my own legal liability. i'd be worried about serpapi getting blocked one day and my own application suffering as a result

[yellow_lead]

There's many services like this that want/need access to Google search results. Another is ahrefs which is geared towards SEO.

[michaelmior]

Interesting. I haven't heard of SerpApi but that is indeed surprising.

[sour-taste]

I was also serprised