[evmar]

Massive hard drives only useful for archival purposes, under this argument: if your hard drive is being used for live queries, you want to access all the data on it. Even if you have have all the pieces in place to stream 1gig per sec off of the drive, a 1pb drive would still take 1 million seconds = ~11.5 days to read.

So in practice in production it's more useful to have smaller hard drives in more places to work on the data in parallel. And in the truly archival cases there are other concerns (like redundancy) that mean there isn't as much demand for a single massive drive.

[AnthonyMouse]

> Massive hard drives only useful for archival purposes, under this argument: if your hard drive is being used for live queries, you want to access all the data on it. Even if you have have all the pieces in place to stream 1gig per sec off of the drive, a 1pb drive would still take 1 million seconds = ~11.5 days to read.

Your premise is that anything you don't use on a regular basis belongs in archival storage, presumably in some kind of central archive.

Suppose there is 1PB of static data of which you access a different 50GB every day. You can call it archival if you like but it's still going to save you 50GB/day of network traffic to have a local copy.

So for example, Netflix could make a box that came loaded with all their content and new content is added using IP multicast or P2P during off-peak hours. The peak hours bandwidth savings would be immense and you would be completely immune to crappy or unreliable network connections.

[dalke]

I've toyed with the idea that if you buy a 1PB hard disk it will come pre-loaded with, say, a copy of archive.org and search indices, or a large collection of movies. Also, if everyone had the same reference data set then (handwaving!) it could make for some fantastic compression methods.

I don't think the economics works out, since at 1G/s it probably takes too long to load the data, and as this essay points out most people will stream what they want on demand. I also doubt there will be a standard content set which is around long enough to assure that my imagined on-the-fly compression model-building-by-corpus-reference will take root.

[mchahn]

> (handwaving!) it could make for some fantastic compression methods.

You could have indexes of hashes and store any chunks of anything in a big flat address space. You wouldn't even need to know what you have. Just a massive amount of archived chunks of storage. (OK, that is more than hand-waving, maybe arm-waving?).

[TheOtherHobbes]

The problem is that new content is being generated at an increasing rate.

There was an estimate in 2011 that total storage of everything everywhere by everyone was more than 250 Exabytes, increasing by around 25% annually.

There's going to be a lot of duplication in that, and a lot of it won't be public. So as a ballpark guess a complete collection of public-only sources - including all available commercial content of all possible kinds ever recorded, academic papers, Wikis, news sites, forums, and such - is going to need 25-50 Exabytes, with maybe 25% compound of new content every year.

So you could get the entire Internet delivered by truck or two, but you probably wouldn't have anywhere to put it.

[digi_owl]

I seem to recall reading some worries that current sizes are already pushing the limits of RAID setups. This in that it takes so much time to rebuild a drive given current interfaces that you risk one of the others in the array to fail during the process, thus making recovery impossible.

[greglindahl]

... which is why many big storage systems keep 3 replicas of each chunk of data stored across different servers. A failed device's chunks can be rebuilt in parallel across the entire cluster.

Or, if you care capacity-constrained, you can use cross-server erasure coding. And even call it cross-server RAID, if you like.

[Viper007Bond]

My consumer storage RAIDs already take 1-2 days to rebuild. I can't imagine larger enterprise ones!

[BrainInAJar]

your consumer storage RAID is a lot dumber than enterprise ones, which can resilver more intelligently

[notacoward]

It's not just a worry. I've interacted with users at multiple sites who have actually lost data because the rebuild windows are so long that they had three effectively-simultaneous failures within a RAID-6 set.

[bit-player]

This is an important point. The Google white paper does address it, though only in the abstract: They want to maximize both capacity and IO bandwidth. One can imagine large disks with multiple independently actuated head assemblies. But quite possibly you're right that it's not worth the both.

[ChuckMcM]

The interesting thing about Google's storage infrastructure was that teams were optimizing IOPs per drive and talking to thousands of them over a gigabit link. I had an interesting conversation with Sean about that one day, asking him if he got 100 IOPs per drive, and had 10,000 drives, and a gigabit ethernet port, how much data on the drive could be part of any service being provided over that gigabit link? Plot that over a RPS (generic 'requests per second').

In my case I was trying to get him to sign off on powering down some of the drives that could not be reached to save power. But even with the data staring him in the face he could not go there. Network bandwidth gets better, and that exposes more data to the pipeline, but if you want < 500mS request response you have to balance the system.

[Shivetya]

on one of our production servers we have about 60tb spread across 240/250g drives.