With bots that break through cloudflare captchas. As a human, I immediately close the browser tab when I see one. If you make your site inaccessible for humans, only the bots remain.
I assume they have their own index of the web. Or perhaps there are companies where you can buy such indices. And then mix and match those, perhaps with your own too.