Hacker News new | ask | show | jobs
by jessejjohnson 384 days ago
I tried, fairly impressed. How do you manage situations where sites may not block your request when trying to profile the provided url?
1 comments

Hi, thanks for sharing

We have multiple data sources, not just scraping. There are many solutions for this, but they can be expensive. For now, we’re using some infrastructure that we built ourselves, and it’s enough for our small scale.