Grab the Alexa Top 10,000 list and curl them. You’ll be amazed at the big brand names that block you.
This has big implications if you are trying to move some of your corporate edge to “the cloud”, or even your personal edge.
AWS needs to procure and curate a group of IPs to be used for this class of use case, ensure the services that the big brands subscribe to for their web firewalls are not blocking these IPs (very like curating IPs on anti-spam blocklists for SMTP/MX).
The idea of a "clean" IP range is great, but it won't help little guys like us. Amazon can either:
1. Make it available only to Fortune 500s or AWS accounts with a monthly spent >$X million; heavy penalties and even legal action for anyone caught spamming. This way the IP range stays clean, but is out of reach for little guys like us trying run an VPN edge.
2. Make it available to everyone. But then spammers will farm AWS accounts and abuse the clean IP range until it is banned just like any other AWS IP range.
I'm not sure what you're looking for in terms of examples, but I've had external tests for scraping code on different sites. We had to slowly disable tests that were failing because the sites were testing the request with a reCAPTCHA.
This has big implications if you are trying to move some of your corporate edge to “the cloud”, or even your personal edge.
AWS needs to procure and curate a group of IPs to be used for this class of use case, ensure the services that the big brands subscribe to for their web firewalls are not blocking these IPs (very like curating IPs on anti-spam blocklists for SMTP/MX).