|
|
|
|
|
|
by supermdguy
104 days ago
|
|
|
I’ve actually written a crawler like that before, and still ended up going with Firecrawl for a more recent project. There’s just so many headaches at scale: OOMs from heavy pages, proxies for sites that block cloud IPs, handling nested iframes, etc. |
|
|