Y
Hacker News
new
|
ask
|
show
|
jobs
by
jabo
5133 days ago
I tried doing something like this a while ago. But HN kept blocking my IP after a few days of screen scraping. How did you get past that rate limit?
2 comments
marcneuwirth
5133 days ago
I am using the bigrss feed as described in the latest post:
http://ycombinator.com/newsnews.html
link
cooldeal
5133 days ago
I wish it gave out the points the article had... does your data track that or just the position?
link
marcneuwirth
5133 days ago
Just the position unfortunately, I would need to scrape the site itself in order to get comments and points info. It would be awesome if that data was added to the RSS feed
link
peteretep
5132 days ago
Were you hitting it more than once every 30 seconds?
link
jabo
5124 days ago
No, I hit it once in 15 minutes!
link