[film42]

I become more and more amazed at the price of storage but in the same breath I am still horrified at the price of data at scale.

Right now I have a VPS at Carat Networks to throw crap on and I pay $15/m. For that I get 50GB of space and 500GB of transfer. I understand the speed and reliability is greatly improved with S3, but as a simple file host, it still makes sense for me to throw it on a vps or low-end dedicated server at 1/10 - 1/3 the cost of ^this^ projection.

[bigiain]

" … I am still horrified at the price of data at scale."

This isn't "the price of data at scale", this is "the price of flexible, reliable, available data at scale"

I think what some people don't understand, is that Amazon _aren't_ trying to compete on price.

With Amazon, you're paying a premium for the ability to scale, both up and down, very quickly.

Rackspace, Linode, and some-guy-subletting-racks-in-some-local-datacenter can easily beat EC2 prices for "general purpose servers". What Amazon does differently is let you quickly and easily go from 1 "server" to 10 or 100 servers, then switch most of them back off again 4 hours later. I deal with a great local hosting guy, who can (and does) fast track provisioning for me at times, but if I called him and said "Ummm, the CEO is on Oprah tonight, I need 100 additional webservers, a load balancer or two, and a dozen database slaves; to keep my not-architected-for-scale-but-suddenly-in-need-of-it web app alive at 8:30pm tonight", there's no way he'd be able to do it. And even if he _could_ there's now way he'd agree to if I said "and I only want to pay for it all until midnight, then shut all the extra down and go back to charging me for my single instance".

"$1000 per terabyte per year" might seem crazy expensive if a sensible alternative for your data storage requirements is to go to BestBuy and grab a 2TB external drive for ~$100. But that's a _very_ different thing to what Amazon are selling...

[Negitivefrags]

I would rather pay 1/5th the price of Amazon for all the other days when I am not on Oprah though.

Our current CDN provides for 4k per month what amazon would charge 18k for.

Yes, that 4k is on a 12 month contract that we had to negotiate. We are paying for about 4 times the bandwidth per month that we are actually consuming at the moment, but it's just so much cheaper overall and the bandwidth we don't consume each month rolls over to the next. (We plan to consume it all one day!)

I firmly believe that the vast majority of AWS customers are paying for flexibility that they are not actually using 99% of the time.

[film42]

This is 100% accurate. My point was that S3 is good at being scalable infinitely, but you can use low-end hardware to scale up to point x for storage. This will be small and fail quickly when it comes to media companies, but for most web apps who need an image host or cdn, it'll go a long way at a fraction of what aws charges. My problem I guess is that I see younger companies looking at aws, linode and rackspace as the _only_ solution and I think that's unwise.

I realize I'm slowly going offtopic, sorry! AWS still rocks and I wish I could use my amazon gift cards there.

[jimwalsh]

Comparing a VPS to S3 is apples/oranges. The redundancy/backups/scale S3 provides over a single VPS is very valuable to people. This argument is silly on a post about S3. Yes if you want a cheap webserver to dump things you can get that. You can also get an EC2 instance with plenty of space pretty cheap to just dump things as well.

[hayksaakian]

s3 is not optimally priced for consumer file storage, i'd think it's better priced for webapps and such, where speed and reliability is super important.

if you web server is on ec2 you also get the lower latency of having everything in one place.

[film42]

Very true, and that's why S3 is excellent, but I still feel there's a lot of value in using low-end servers until you're running at a large enough scale where redundancy actually matters (not just we should use this cause it's what everyone else is doing).

[hayksaakian]

But the ultra low end tiers are free.

Your scale is limited by your wallet. You don't have to ref actor your infrastructure as your service grows.

[rb2k_]

not to forget that S3 has various backups of that data.

[cstejerean]

How much of that 500GB of transfer do you actually use every month?

[film42]

I use around 100GB of it, but at $15/m I don't consider that a wasted 400GB, instead it's just available for tunneling.

[nacs]

This is the reason why companies like your host can offer 500GB or terabytes or even "unlimited bandwidth" for such a low price -- its because they sell to a lot of people and pray that 90% of them won't even come close to using their full bandwidth allotment.

If everyone that was paying for the 500GB was using anywhere close to 500GB at that price that company would go bankrupt very quickly.

[ghshephard]

I thought I would run the numbers.

http://drpeering.net/white-papers/Internet-Transit-Pricing-H... is a good rough estimate of Transit Costs on the internet.

For 2012, it's around $2.34/megabit/second.

Testing the math "at break even" - $15 / $2.34 = 6.4 megabits/second. 6.4 megabits/second at 30 days in gigabytes = 2,073.6 gigabytes.

So, there's enough margin for everyone to be using the 500GB without that ISP going bankrupt. (Yes, I realize that they have costs for servers, cooling, real-estate, diesel, staff, security, etc..., but this shows we're in the right ballpark with a 4x margin)

[nacs]

Right, at 500GB it's definitely a reasonable cost -- I think VPS providers like the parents host are much more sensible with what they advertise.

Virtual hosts like Bluehost ("UNLIMITED Domain Hosting, UNLIMITED GB Hosting Space, UNLIMITED GB File Transfer") and Dreamhost ("Disk Storage Unlimited TB + 50GB Backups, Monthly Bandwidth, Unlimited TB") however are the ones who are especially bad with their advertised offers (all for around $5-7 a month). You start using even a couple hundred GB of bandwidth a few GB of storage and they're happy to kick you off for "abusing resources".