[solardev]

Do you have to use Curl? It wouldn't render a lot of sites correctly anyway (anything that uses JS for rendering).

Can you run a puppeteer/playwright instance (which control real browsers) and add an ad blocker to that? e.g. https://github.com/ghostery/adblocker or https://github.com/microsoft/playwright-python/issues/782

[suramya_tomar]

I don't have to use curl, but in the past when I have setup something that opens browser instances it has usually been a bit unstable in the sense that it would crash intermittently.

I wanted something that I could kick-off as a daily cron job.

[solardev]

Hmm, can't you check for correctness somehow and retry on failure? Those headless browsers are often run in automated environments.