Looks like it's a response to the recent cloud services market investigation by the CMA [1].
Which highlighted "Egress fees harm competition by creating barriers to switching and multi-cloud leading to cloud service providers entrenching their position" [2].
It's also interesting that they are calling out problems with software licensing, as that is another thing the CMA is investigating in their cloud market review.
I read through a couple of these responses to the CMA by MS, Google and AWS and their smaller competitors
as expected the hyperscalers refuse to acknowledge that the free ingress and expensive egress is a lock-in mechanism, and their smaller competitors complain bitterly about this
the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on)
if they want to play this game then the CMA should simply make them charge the same for ingress and egress
that way they can "fund their network costs" without issue, and if they want to make them both free then that's their decision
> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on)
This doesn’t pass the red face test IMO. The hyperscaler networks are indeed very expensive, but that’s because they need to provide non-blocking or near non-blocking performance within the availability zone, and the clouds don’t charge for this service.
The Internet egress part ought to be straightforward on top of this: plug as much bandwidth worth of connections into the aforementioned extremely fancy internal network. Configure routes accordingly.
It’s worth noting that the big clouds will sell you private links to your own facilities, and they charge for these links (which makes sense), but then they charge you for the traffic you send from inside the cloud to these links, which is absurd since they don’t charge for comparable traffic from inside the cloud to other systems in the same AZ.
I have, but not for a use case where this matters.
FWIW, Google has been working on these fancy nonblocking networks for a very long time. They’re very proud of them. Maybe they don’t actually use them for GCP, but Google definitely cares about network performance for their own purposes.
The whole concept of blocking is inapplicable to packet-switched networks. The whole time I was there I never heard anyone describe any of their several different types of networks as non-blocking. Indeed, the fact that they are centrally-controlled SDNs, where the control plane can tell any participant to stop talking to any other participant, seems to be logically the opposite of "non-blocking", if that circuit-switching terms were applicable.
Your message seems to imply that these datacenter networks experience very little loss, and this is observably far from reality. In GCP you will observe levels of frame drops that a corporate datacenter architect would consider catastrophic.
>but for some reason doesn't apply to ingress (which they're silent on)
The industry standard for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall.
My personal feeling is they're moving costs around so that egress has a big margin and other items have a smaller or potentially negative margin.
I've seen this at other providers. We did a competitive pricing exercise at my last company, and our overall cost went down, but the mechanism was per hosts costs went down significantly and egress costs went up significantly, and the per host cost decrease outweighed the egress cost increase.
It still doesn't make sense to charge for ingress, because everybody knows that should be free, unless you're a residential ISP.
Many companies in many industries do this. It’s often simply not practical or sensible to price every SKU “fairly “ on some value or cost basis. Your margin varies from item to item (including negative) and the whole bundle works out.
> for peering is paying the 95th percentile of egress or ingress depending on whichever is greater. Ingress is free for these clouds because egress > ingress overall
How about customers pay for actual usage, rather than some [fake] averaged-across-all-customers usage?
Why would the cloud provider charge for usage that doesn't actually cost them money? Unless usage patterns drastically change industry-wide, the ingress really doesn't matter to them. The egress does.
It seems entirely reasonable to look more skeptically at cloud providers' exact charges vs cost for egress, particularly when high egress fees might contribute to lock-in, and when the public price sheet vs the preferred customer pricing might differ radically. But asking them to totally restructure the charges, inventing a charge for ingress when their actual total ingress cost is zero and, short of major industry-wide usage pattern changes, will remain zero? Why would you do that?
Yeah, it's long been common for more traditional hosting providers to limit egress traffic and charge overage fees for going over that but have no similar limit on incoming traffic for basically the same reason: their network-wide traffic patterns mean that egress is what costs them money and ingress is effectively free on top of that.
> the hyperscalers say they have to charge egress fees to pay for the costs in building their networks, but for some reason doesn't apply to ingress (which they're silent on)
because ingress traffic volume is a fraction (in my experience, in website hosting of a well known household brand, barely 1%!) of egress traffic volume, and most peering connections are 1:1 in ingress/egress bandwidth so the egress bandwidth cost sets the price.
This also holds true on thebflip side for most consumer internet speeds (at least in the US).
Many people have a x00 Mbps or even x Gbps downstream, but most have no more than x0 Mbps upstream. Literally their ability to pull traffic from websites is 50X in some cases than to push information out. Going beyond that (greater uploads) often costs significantly more.
Whether or not these two are actually related isn't clear to me, but it is interesting.
That's due to the DOCSIS standard for cable modems. They specced out more channels for downstream than upstream because of the limited bandwidth of copper and consumer priorities. With fiber there's an order of magnitude more bandwidth available, so the uneven split is much less (if at all?) common with the big backhaul lines between datacenters. For consumer fiber you'll usually get symmetric but for the most part it doesn't make sense as the vast majority of consumers just don't make use of their upstream bandwidth.
For what the lock-in topic is concerned the expensive egress is just relevant when you want to leave your provider, not during daily operations with your consumers. Free ingress but prohibitively expensive egress is literally only there to make it as easy as possible for customers to come to the cloud provider and as hard as possible to leave. Coming to a provider or leaving them should cost the same or else it's just a trap.
This is like Comcast giving you the initial installation for free but charging you an arm, a leg, and a year of your life to allow you to leave because "the process is costly". Both practices are scummy and abusive towards the customer.
I don't see it that way at all. Those egregious egress charges still apply while you are actively using GCP, encouraging you to put everything in GCP, and eschew multicloud.
This seems to me more of a "try us out for free" play. Bring your big data here, if you end up not liking it, we won't penalize you for taking your data out. Given that GCP is running at a very distant 3rd, they need to make plays like this.
This. I see this as purely an attempt to influence AWS in lowering the barriers of migrating between clouds. But also makes sense for those who want the option to test the waters with no downside.
So on the one hand Google now accepts that egress fees are outrageous. Great! On the other hand, they're only reducing (removing) the fees when you leave them.
If this move were really about acting in customers' best interests, they would reduce the fees for everyone. Doing this only for departing customers feels performative.
Google is part of Cloudflare's Bandwidth Alliance [1] which is removing most egress fees. Google still charges for egress, but it's half of what AWS is charging. We moved everything off of S3 anyway and have been using Cloudflare's R2 storage along with Google Compute Engine instances. R2 can be a little flaky, but the cost savings for us more than makes up for it.
Unless they’re planning to kill GCP entirely. Then everyone is a departing customer /s
But seriously, it seems like a fair-ish compromise. The 60 day limit is tough though. As long as you’re continuing your usage and not using offsite backups, ingress/egress isn’t probably too problematic. It’s only problematic when you try to suddenly egress all data you’ve ever stored, which you probably wouldn’t do unless you’re migrating away.
It's problematic to me. I run a system partly on GCP and partly on another provider. I'd like to move some of the stuff that's on the other provider over to GCP VMs, but I am prevented from doing so solely by the GCP egress fees.
My general complaint is that high egress fees prevent users from developing hybrid cloud solutions which mix components from different cloud vendors. Instead, you are forced to choose a platform, and then you're locked into it. That seems textbook anti-competitive. We shouldn't be grateful for the opportunity to switch vendors, we should be angry that "choosing a single cloud vendor" is a thing at all.
The cynical might wonder if this is a precursor to Google closing down its cloud.
They can't really shut Google Cloud down and still be charging people to exit.
And if they had suddenly seen the light on egress fees then surely they would have cut egress fees everywhere..... the fact that it's only on account closure is kinda suspicious.
Shutting down a $30B+ business would be such a ridiculous, yet Google-esque decision. It would also be a huge shame for consumers - the public cloud is already an Oligopoly
All over the Internet people are slapping google on the back for..... pretty much nothing. In fact much of the praise seems to be worded as though Google had dropped all egress fees.
Google continues to charge the nothing-short-of-highway-robbery 12 cents per gigabyte unless you are in Australia in which case its 19 cents per gigabyte. This is astoundingly bad value.
What are people hailing Google's "free to exit" in such glowing terms. Even herein the comments people are cheering for Google.
Worth noting at this point that Cloudflare R2 charges 0 cents per gigabyte egress.
> Certain legacy providers leverage their on-premises software monopolies to create cloud monopolies, using restrictive licensing practices that lock in customers and warp competition.
I like to see them publicly call out Microsoft and Oracle.
I think the "Legacy" and "licensing" portions are specifically calling out MS and Oracle, but they very sneakily are calling AWS out too on the fact that egress makes it insanely expensive to leave their platform.
I've worked at some orgs where to either move their data out of S3 would cost $20k, or to even delete it would cost thousands in API calls.
> to even delete it would cost thousands in API calls
raises eyebrow
It's a well-known trick (proposed by AWS Support as well) to set S3 lifecycle rules to empty buckets with too many objects to cycle through via List calls. Doesn't cost anything.
That, and any of the smaller “clouds” with egress/delete fees that need to be considered when leaving. Seems massively disingenuous given that until this announcement they also had such fees (“look, those people try rip you just like we were doing until five minutes ago!”) but that is pretty standard for marketing materials.
This makes them a better option as the first cloud provider to try, other things being equal, because leaving (back to on-prem or to another provider) is easier. I assume they are trying to remove a little of the huge the distance between them and the two leading players by reducing concerns that might add on-boarding friction.
I don't. This is very much the pot calling the kettle black. Of the cloud providers, Google has the least pleasant business tactics. I would do business with AWS gladly, with Azure, but never with Google.
Google should have made the same announcement without the snarky mean-spirited bitterness.
Egress fees are way too high.
And it doesn't keep anyone in cloud, if you want to move out, you do it.
In fact, I think the fear of egress costs keeps more people OUT of cloud than it keeps people in.
This is a smart move that won't cost them anything and may increase their business.
I know plenty of companies who can't afford the Aws egress fees to get their data out of Aws, and it's far cheaper to just pay storage for one more month and kick the can down the road.
I clicked the link and you have to _apply_ and potentially be admitted just to get a free exit? So there's no guarantee??
Is requiring a Google committee review/application process a new trend with Google products? I recently was denied on another application through Google for API access to get one businesses GMB reviews, and it's frustrating because there's no recourse. Google is so opaque now.
Getting ahead of antitrust I see. Big tech has gotten so big that we will now see this peace meal stuff being done to appease regulators to stave off any major action.
Google Cloud has ~10% market share; I don't think this is an antitrust avoidance play. More likely, it's removing a concern companies might have with bringing workloads onto a relatively smaller player. Especially one that has a history of discontinuing products.
The issue isn't as much market share as it is how sticky picking a cloud platform is. At that point the market share doesn't really matter, especially if your name is Google.
don't think so. Google is the distant 3rd in the Cloud, so hardly a subject to antitrust. It is more likely that failing hard on their goals for the Cloud set to that division about 3 years ago (and it was rumored that either they achieve the goals or it is "or else" for the Cloud division) they are starting the blame game whining about licensing/unfair competition/etc. (whereis they should blame only themselves as they would never bend over to the enterprise customer like say AWS would do who for example developed MS SQL (Transact SQL) interface to their own db - that is how you deal with the competition and software lock-in instead of whining (reminds that phrase from Babel - "that is why Benja is the king, while we are sitting on the cemetery fence"). In this case for example I remember how AWS was hunting down laid off Sybase (where T-SQL comes from) engineers whereis being an experienced enterprise software developer is viewed by Google is more like a disadvantage and results in meager offer (several my acquaintances had similar experience), so GCP losing enterprise game doesn't look that surprising to me. And now that enterprise customers are starting to add huge AI related workloads the GCP is hardly ever mentioned)
3. You will automatically be refunded all network egress fees incurred in the final 60 days, capped at the number of gigabytes you had stored in our products in the preceding 60 days.
The process requires people to be aware of the offer and contact support. A few months down the road, nobody will remember this exists, so most migrations won't actually make use of it.
That's a cool thing for them to do, an interesting business choice, but cool. It certainly helps the companies feel a little better about vendor lock-in, which is terribly plentiful in the cloud.
The TLDR is that when you tell them you want to cancel, you have 60 days to do so and during that time you'll have no egress fees. Makes sense.
Biggest problem though is... if you have a substantial amount of data or need to do a complex and seamless transition - this probably won't work for you. I would have to be on the DevOps team that's told to move a complicated and data-heavy application, and they only had 60 days to do so. Also the bulk of the data movement is, in my experience, one of the first steps of migration - not the last.
My hope is that, if nothing else, this will spur similar behaviors in other large cloud providers ::cough::aws::cough::.
Sounds like they're trying to technically address EU regulators' concerns without providing any real value to customers.
In real life it's probably extremely unusual for any company to altogether cancel their Google Cloud contract. More likely is the scenario where you move the bulk of your cloud usage to a new provider, but still have various straggler infrastructure on the old one, which is not worth the effort to clean up. Or, you go to a multi-cloud strategy so you want to move half your data off Google but keep the other half around. Google's egress fees are still standing in the way of these cases.
Yep, for a sufficiently large account, this strikes me as an offer they know will never be taken. "Migrating" typically means "stopping $XXX,XXX spend per month" not "completely ceasing use of all GCE services."
They know this and this is mainly marketing, I think.
> I would have to be on the DevOps team that's told to move a complicated and data-heavy application, and they only had 60 days to do so. Also the bulk of the data movement is, in my experience, one of the first steps of migration - not the last.
Couldn't you wait to tell Google until you've more or less figured out the logistics, then tell Google that you're leaving, fees for egress gets disabled and you initiate the move. Then you have 60 days to complete the move.
But yeah, if the move takes 30 days because you have a ton of data, and you figure out after the move is complete, that you missed 10%, you only have 30 days to figure out how to get that out too.
Egress bandwidth costs have been discounted for years. Including GCP. Bandwidth Alliance... Like 10 years ago traffic cost was a real problem with the cloud. Today there are other areas that are expensive.
> Certain legacy providers leverage their on-premises software monopolies to create cloud monopolies, using restrictive licensing practices that lock in customers and warp competition.
This is fine issue of its own. But the way it is laced up here muddies the waters around this actual news.
Google was doing the wrong thing and is changing that while not really taking responsibility for doing the wrong thing.
But to make that less obvious, this other concern is brought into the story creating this high ground on a separate topic.
The strategic reframing of corporate communications is tiring.
The audacity of ranting about some cloud provider leveraging their licensing and then dropping a link to Azure in the next sentence, I love it, thanks Google.
I love that Google are willing to such a big swing at Microsoft in the text of this announcement - I just wish AWS wasn't so badly shackled busy it's marketing and PR people these days.
I think Azure is great if you are willing to do it Microsoft's way and are focused on B2B markets where factors like compliance matter in a way that can actually disrupt your sales cycle.
Put differently, if you can get over the principled nerd stuff like "everything must be open" and break out that wallet, you can stand to make a lot of money without as much headache as the guy who decided to roll his own IdP.
Getting "cost sensitive" on Azure is how you lose track of the rabbit. The whole point in my view is to trade money for reduced complexity and time. It lets you focus on solving really hard business that others simply can't seem to find the time to.
Hi. I fix AWS bills for a living and also shitpost a lot.
This is a smart play that costs them basically nothing. Remember that egressing data costs customers at worst 3x the monthly cost of storing it. Nobody is avoiding leaving because of the egress fee.
What this does do is assuage the “lock-in” fear common in cloud-reluctant customers, while presents them as being forward thinking.
I’ve never heard of a cloud migration where the data egress wasn’t at least an order of magnitude less than the engineering cost of the migration itself.
The parent poster https://twitter.com/QuinnyPig is one of my favorite Twitter accounts. Every day he gives me renewed hope that, no matter how much I wish I had more time to devote to developer experience for people using the internal tools and APIs I design on the startup side of things... at least it'll be better than the DX and customer service provided by the biggest players providing infrastructure to our entire industry :)
I share the view that nobody avoids an exit or migration because of egress fees. In fact, for online migrations the period of replicating data between providers might go on for months.
But all cloud providers leverage The Principle of data locality or data gravity, which states that compute benefits from being close to the stored data. If a customer moves the data elsewhere it follows that the compute will soon leave too.
At face value this is a good thing, but I gotta say it’s pretty rich for Google to try to diss other cloud vendors for leveraging their effective monopolies given its moves in search and now browsers. Sorry, but GCP doesn’t get a pass on Google’s other behavior just because currently a loser in the cloud infra market.
Which highlighted "Egress fees harm competition by creating barriers to switching and multi-cloud leading to cloud service providers entrenching their position" [2].
It's also interesting that they are calling out problems with software licensing, as that is another thing the CMA is investigating in their cloud market review.
[1] https://www.gov.uk/cma-cases/cloud-services-market-investiga...
[2] https://assets.publishing.service.gov.uk/media/652e958b69726...