Hacker News new | ask | show | jobs
by Exuma 3309 days ago
How do you handle scraping the products, do you just fetch all new products once per day? Or do you fetch them all in real time as the user searches
1 comments

Everything is done with real time calls to the eBay API. eBay's catalogue is too large to make it realistic to fetch it as a whole and, anyway, as far as I remember they don't offer a data dump (it's been a long time since I checked).

Luckily, eBay's API is very fast, but I also worked hard to keep the total response time of my code within a reasonable range.

Ah interesting, that makes sense. Thanks