Hacker News new | ask | show | jobs
by schmeckleberg 1466 days ago
i think you and the above poster are in vehement agreement. ingesting 2 MBs of logs per minute is impressive in its pluperfect unimpressiveness.

maybe the presentation was called "Timmy's first named pipe" or "Sally explores /etc/logrotate.d"

3 comments

That’s a LOT of text to describe me sitting on my couch. 2MB per minute is far more than the most detailed biography in existence.
220m users. Let’s imagine 50m are streaming concurrently. That’s 100TB an hour in logs lol. They could be storing an entire petabyte of logs a day. My friend did some data center stuff for the large hadron collider and wasn’t hitting these data ingestion states, and these are just to record me binging the office.
The comment said "log processing system". Sounds more like it's a stream and not stored logs.
2 MB/per minute/per stream at Netflix scale is crazy.