Hacker News new | ask | show | jobs
by benlumen 2162 days ago
I'd love access to the Twitter API for research purposes for my writing, but I'm priced out for that purpose.

I can't prove it, but it feels like they only want big businesses researching their markets, and not people like me asking questions about Twitter itself - even though letting people do that would result in negligible additional traffic.

This might be controversial and I know people will say that they're a private company and can do what they want, but Twitter has positioned itself as society's de-facto official communication channel and I think the data on it should be a matter of public record at this point.

edit: Does anyone have experience with scraping tools and Twitter? Presumably it's hard to do.

4 comments

I've been collecting Twitter scrapers recently with the idea that I might eventually do something with them. This one [1] seems most promising but I haven't actually tried it.

[1] https://github.com/bpb27/twitter_scraping

This is just my own vague experience, but I don't think this specific to Twitter, nor does it really have to do with pricing people out.

I think that public APIs in general are a thing of the past, mostly because of bad actors, security problems and scale. Look at things like Clearview AI, just slurping up every photo they can get their hands on. All it takes is one or two abusers and then everyone has to be limited.

The simple instagram API that I used to just get my own most recent post 3-4 times per day, now requires me to submit an application, with multiple size app icons, justification for my use, privacy policy, ToS, etc. All to I can fetch my most recent post from my own profile.

It's just ridiculous how every large website is clamping down on their APIs, but with abuse left and right, they don't really have many options.

Have you tried contacting Twitter about this? They've given free access to researchers in the past. They'll likely ignore you or say no, and if they say yes it may be more restrictive than you like, but it's worth a shot.
Doesn't seem to support "from:{screen_id}" in search, a bummer.