I'm not sure how this could be removed - the fundamentals behind it are basic building blocks of S3.
Maybe raising the cost of transient storage? e.g. If you have to pay for a minimum of a day's storage - but even if that was the case this would still be cost-effective, and at any rate it seems very unnatural for AWS to charge on such granularity.
+ I would guess that S3 is orders of magnitude more profitable for AWS than cross-AZ charges, so I'm not sure they'd consider it a loss-leader.
It would be fairly easy to change the pricing policy. GCP did something similar for cross-region https://cloud.google.com/storage/pricing-announce#network. This is pretty severe because it seems to affect all reads. However I can imagine an alternate implementation where the source AZ is tracked when data is written and egress fees are charged when the data is read (as if the data was always stored in the source AZ). This could even be done more complexly such as only charging the first time data is read in another AZ. Once you read once it is free as-if it is now cached in that new AZ forever. Another option would just be raising the minimum storage duration so that it basically costs all or most of what the data transfer would.
It would definitely piss a lot of people off as it is adding to their bill, but it could likely be done in a way that makes exploiting this for just data transfer not worth it without adding huge costs to most "real" use cases.
Yeah, I see what you mean - that'd indeed render this method ineffective. Like you said I'm sure this would bother a lot of customers, but it's not a completely unrealistic overhaul of S3 pricing.
That being said, that'd be sort of "mean" of AWS to do - the data is already replicated across AZs whether you pay for it or not because of how S3 works.
It's absolutely amazing that so many devs don't realise this. They seem to think that bandwidth should cost a few cents a month, when in reality it is virtually free. Perhaps the 7c/GB charge was reasonable when AWS came out 15 years ago, but networking has got orders of magnitude cheaper and faster in the intervening time period.
What's more odd now that 1gigabit+ home connections are available, it should be obvious to anyone doing the math that it can't cost that much, otherwise a 200GB CoD install would be costing the ISP $20.
I feel like an entire generation of devs have been weirdly brainwashed by cloud to believe that a ton of things need to be very complex and expensive.
Of course it’s also a zero interest rate phenomenon. We are exiting a >10 year era when the name of the game was simply to grow and anything in your way could be dealt with by just throwing money at it. Nobody cared about cost as long as growth numbers went up.
I hear you, and that is an egregious margin. Just wondering if part of their bandwidth pricing calculation is driven by a goal of constraining their infrastructure costs (or other considerations beyond profit). I'm actually wondering this exactly because it is so egregious.
There is of course a thing wherein if something is free people mindlessly use it. If all AWS customers did this with bandwidth, I wonder how it would impact total usage and AWS's subsequent infrastructure considerations.
I'm no fan of their pricing and I'm sure there's an unhealthy dose of greed in there. Your phrasing just prompted me to consider what other factors might also be involved. And, if part of the rationale is actually to influence customer behavior with disincentives, then by definition there would have to be some pain involved.
Yep. Big cloud bandwidth is a 200X markup from list price. Its ludicrous.
It serves two purposes for them. One is obviously a nice profit center. The other is that free ingress but expensive egress causes data to flow in but not out, creating a center of gravity and a form of lock in.
They're not paying for bandwidth, but their connections are not asymmetric, so they need to balance egress and ingress or they will incur fees or dropped traffic.
The pricing is there to maintain this balance. Since they're obviously egress heavy, it makes sense for them to charge for egress, and make ingress free.
People think AWS is using costs to "tax" you, what they're really doing is using to control the shape and size of their traffic.
If this is true then how do so many other companies not charge this way? VPS companies that charge radically less and bare metal / colocation hosts that charge flat rates are all profitable and their networks work fine.
Add to that the fact that people often explicitly choose these smaller providers because they have cheap bandwidth, meaning they're going to be a magnet for high bandwidth users like DIY CDNs, streaming, game servers, TURN servers, video conferencing relays, etc.
I find it hard to believe that AWS or GCP are getting core Internet bandwidth on worse terms than much smaller companies like Vultr, Hivelocity, Datapacket, or OVH.
The other companies have significantly different SLAs and drop packets far more readily. They also charge for bandwidth, in my experience, you get your 1TB/mo with your $5 VPS sure, but once you go over, you're facing per/GB charges that are very close to AWS default egress price.
They're not a magnet for these services for the reasons I just described as you reach your per VPS limit very quickly, and to get more "cheap bandwidth" you have to be prepared to run 100s of VMs per provider, and have to consider provisioning VMs you don't need just to get access to another $5 TB of transfer, or you're just going to end up paying the per GB fee anyways.
The terms aren't worse, but the service and their guarantees are different. Again, if you ask AWS for a bandwidth deal, they'll cut you one within a few minutes that will more than halve the price of your transfer if you pay up front. Which is AWSs way of saying, "if you make your usage predictable, we can make it way cheaper."
Why? Because they have fixed _capacity_ on their links. The costs manage that _capacity_.
Digital Ocean and Vultr are a fraction of AWS pricing. Vultr is $0.01/gb. Bare metal providers are often cheaper still, selling by size of pipe rather than bytes transferred.
In my experience GCP and AWS are pretty unwilling to budge on bandwidth pricing unless you are very large and making a long commitment. If you are not spending six figures a month forget about it.
You may be right about SLA but I run large volume services out of bare metal providers and do not experience meaningful packet loss or down time in practice. Bandwidth costs are easily hundreds of times less than AWS or GCP.
Ok “loss” is a relative word here… a loss compared to what they could have got from you.
Some how AWS has to rip you off so if there is a non rip off gateway to the ripoff then if you can use the non rip off to avoid another rip off, they will close the “loophole”.
Maybe raising the cost of transient storage? e.g. If you have to pay for a minimum of a day's storage - but even if that was the case this would still be cost-effective, and at any rate it seems very unnatural for AWS to charge on such granularity.
+ I would guess that S3 is orders of magnitude more profitable for AWS than cross-AZ charges, so I'm not sure they'd consider it a loss-leader.