Hacker News new | ask | show | jobs
by jabo 5133 days ago
I tried doing something like this a while ago. But HN kept blocking my IP after a few days of screen scraping. How did you get past that rate limit?
2 comments

I am using the bigrss feed as described in the latest post: http://ycombinator.com/newsnews.html
I wish it gave out the points the article had... does your data track that or just the position?
Just the position unfortunately, I would need to scrape the site itself in order to get comments and points info. It would be awesome if that data was added to the RSS feed
Were you hitting it more than once every 30 seconds?
No, I hit it once in 15 minutes!