Hacker News new | ask | show | jobs
by meritt 4371 days ago
I thought Twitter's API only gave you access to a small percentage of real-time tweets. Did Twilert obtain full firehose access or is another method being used?
3 comments

The 'sample' streaming API endpoint indeed provides you with a small percentage of all tweets.

However, the 'filter' endpoint gives you access to all tweets matching to your search, up to a certain limit. Unless you're tracking very commonly used words or brands like 'selfie' or 'ipad', you're likely to get all tweets you are interested in without ever reaching the limit.

cf. https://dev.twitter.com/docs/faq#6861

Also, the API limits are per authenticated user, not per app.

Its probably the firehose via one of the two data providers, is why the service isn't free. Datasift is one of said providers, can't remember the name of the other.
Gnip is the other one. It was recently acquired by twitter though: https://blog.twitter.com/2014/twitter-welcomes-gnip-to-the-f...
If you want to get "general" data (without a search) you can only get a small sample - that's what we use for hashtagify.me - but it's also possible to download pretty much all tweets for a specific search. Without guarantees, of course!