|
|
|
|
|
|
by henryw
3714 days ago
|
|
|
Looks pretty interesting to do async on python. I once did something similar in node (async by default) with a few lines of code. I think I scraped 12 or 20 million real URLs in 8 hours on a $5 cloud VM. It was limited by network bandwidth. |
|
|