Hacker News new | ask | show | jobs
by creiht 1888 days ago
Late to the party, but one does not simply store 10PB of data :)

The short story is, ignore most of the advice, poach^H^H^H^H^Hhire someone who has done this, and leverage their expertise. There is no armchair quarterbacking infrastructure at this scale.

1 comments

Honestly, 10PB is small potatoes nowadays - and it's not "armchair quarterbacking" to think about storing it in a distributed environment

I work with one customer right now that's storing something like 27PB across ~100 systems for analysis on a rolling 90-to-365-day period (and they're a relatively small customer) with Splunk

If they were doing more storage than analysis, that storage cluster would be substantially smaller