[thereallurch]

While you make a good point, your end number is wildly inaccurate.

1) I highly doubt the NSA is buying $100 dollar low grade hard drives. I bet this data is backed up at least twice per site, at multiple sites.

2) The data is not unstructured, it will be structured in some sort of database. Storing your facebook photos alone will be potentially hundreds of megabytes.

3) I doubt NSA is spending it's entire budget every year on hard drives.

[VikingCoder]

Yes, that's why I quite carefully said, "Every year, the NSA COULD..."

Emphasis on the COULD. It's to emphasize how cheap storage is, and how relatively few U.S. Citizens there are. In our day-to-day lives, we think 311 Million is an enormous number. But in the scale of computing, it's actually quite small.

My real point is: storing the data is no longer a challenge. If you want someone (businesses, governments, etc.) to not store data about every citizen of the U.S., you need laws, regulations, and checks and balances.

[pessimizer]

1) If he's three or four orders of magnitude off, his point still stands.

2) Also, the data could be almost completely unstructured. You mine the data that you have to create structure. If you toss anything, or prematurely optimize things, you're missing out on what the algorithms that you come up with 10 years from now will give you.

3) If you're using enough redundancy to multiply $35,000 into $3,000,000,000, you're insane.