Hacker News new | ask | show | jobs
by nunez 2203 days ago
Lambda is needed to get rotating IPs and scale while avoiding browser fingerprinting. SQS takes the results of those scrapes and puts them into a database, DynamoDB. It's a straightforward web scraping pipeline.
2 comments

Lambda isn't enough. You'll get blocked in a heartbeat. You still need a proxy service.
No, the access google over residental proxy servers provided by packetstream.io