Hacker News new | ask | show | jobs
Hn-api: a simple, ad-hoc Python API for Hacker News (github.com)
23 points by scottjackson 5975 days ago
2 comments

PG has asked a number of times that people avoid scraping HN, because it thrashes the server.

HN is also running really pretty slow today.

> HN is also running really pretty slow today.

Just my luck. As soon as HN took a little while to load for me, I knew that someone would mention it in this thread.

I made hn-api understanding that scraping HN is frowned upon, but also hoping that programmers wouldn't do dumb things with it. As I [said][1], I minimised the number of requests hn-api makes to HN.

In the readme, I say that hn-api is unofficial and unauthorised. I also tell people to be gentle with making requests to HN. Is there some other action I should take? Ideally I'd like to keep my work up, but if pg tells me to take it down, I will.

Great, now I feel all dirty like The Pirate Bay.

[1]: http://news.ycombinator.com/item?id=1112267

Best run that with some delays in there or you will definitely hit the velocity checks and you'll find your IP banned.
I hear you -- in fact, I think I'll put something in the readme about that.
Thanks for the link.

hn-api only makes one request when it gets stories or one request when you get a user's karma -- that doesn't seem like too much. I knew there was a velocity check on IPs (and that I was making an unofficial API), so I made it as light on HN as possible.

Can't you cache the users state ? It's not like you need karma updates every time you do a request to the API.
> It's not like you need karma updates every time you do a request to the API.

Obviously.

When a request is made to get stories from HN, no HackerNewsUser objects are created (and thus, no karma gets updated). hn-api will only update karma when you, the coder, make a HackerNewsUser object or call the HackerNewsUser.refreshKarma() method.

Like I said -- one request when it gets stories or one request when you get a user's karma.