Hacker News new | ask | show | jobs
by anubisresources 2996 days ago
How many petabytes does something like this take up?
2 comments
written 2996 days ago
I download profiles I'm interested in of course. Not everything. 70 profiles with 75000 tweets take 5GB for JSON + images + linked pages content + 60GB for videos.

I didn't optimize for space at all, twitter's json is quite wasteful.

link
llao 2996 days ago
The internet says there are about 500 million tweets per day. If we assume 500 bytes for storing each, this would be 250 Gigabytes per day.
link
written 2996 days ago
Average tweet JSON object size is 5300B.
link
llao 2995 days ago
Should compress nicely though.
link