Hacker News new | ask | show | jobs
by xpose2000 5109 days ago
I made sure to be nice to reddit. The scrapper is set to crawl reddit once every 12 hours for new "top monthly iamas".

Very good suggestion for nested threads. A good example of a reply to a question is on the westboro-baptist-church thread. I think it's possible to implement this suggestion. Will fool around with it on localhost and see what I come up with.
1 comments
Gnolfo 5109 days ago
Yeah, I had a limiter put in mine as well so that it only made a request every 6 or 8 seconds.

No worries on the suggestion. Those threaded comments can be tricky sometimes.

Hey, if you do hear back from them about their stance on this sort of thing, I'd really appreciate if you could let me know what they say. I sort of halted my project after a certain point because I had the fear I'd just have to take it down as soon as I completed it.

link
buttscicles 5109 days ago
You should be able to up that to 2 seconds with no problem, provided you're following the other usage rules of course.

https://github.com/reddit/reddit/wiki/API

link
xpose2000 5105 days ago
Threaded Q&A's are now supported and will show up for any new IAmA's collected.

Reddit has still not responded. However, I know a moderator of Reddit went to the site based on the link source. It seems like they don't care.

link