Y
Hacker News
new
|
ask
|
show
|
jobs
by
nunez
2203 days ago
Lambda is needed to get rotating IPs and scale while avoiding browser fingerprinting. SQS takes the results of those scrapes and puts them into a database, DynamoDB. It's a straightforward web scraping pipeline.
2 comments
toddh
2201 days ago
Lambda isn't enough. You'll get blocked in a heartbeat. You still need a proxy service.
link
atesti
2203 days ago
No, the access google over residental proxy servers provided by packetstream.io
link