Hacker News new | ask | show | jobs
by ashtonkem 1981 days ago
The big problem for me is trust. I don’t care what the feature set or performance is; I don’t trust Google enough to bet a business on it. And I’m not even worried about Google being malicious; I’m worried about them being mercurial and changing/removing things I need without warning.
6 comments

It’s an old and lazy/annoying complaint.

There’s a difference between consumer stuff and enterprise stuff with contracts. Grown up services like GSuite, AppEngine, etc have been alive and well for many years and aren’t going anywhere.

It makes sense to do risk assessment and avoidance where there is value. General emotional stuff isn’t productive.

Building your entire business on AWS Lambda, for example, is a risk you need to understand. In the past, I worked on a team that chose to put a critical business process on a IBM POWER/Aix platform... in our case we went through the options, identified risks/opportunities for standardizing a long lived process on a sole source platform, and made a decision. It was a decision that served us for about a decade before we moved on, so it was very successful.

>It’s an old and lazy/annoying complaint.

Granted, you offered a nuanced reply for someone building something on these clouds. However, when you're dealing with enterprise that are directly competing with the main cloud providers on other verticals, they are not being lazy or annoying, they are being careful, and not without a reason, and avoid using their competitor's cloud infrastructure.

Some of our clients have no problems using these cloud providers. Others wouldn't go through them because that would leak information.

If I'm not mistaken, Google used to buy datacenters in stealth mode, under different companies created for that purpose, to avoid getting on Microsoft's radar and keeping how successful search was from them.

We use GCP, for context.

> Google used to buy datacenters in stealth mode, under different companies created for that purpose, to avoid getting on Microsoft's radar and keeping how successful search was from them.

I believe, Google for a long time (and still does?) thought their infrastructure was the secret sauce, and that might have impeded them from competing with AWS in the early years (despite having all the pieces in place already)?

There are apprehensions like these all over the place to protect their user base.

This matters to us because we're building our machine learning platform[0] and we want flexibility as opposed to using their machine learning products, because each considers themselves "the only cloud". Therefore, we're compelled to do "multi-cloud"[sigh], because we want to be able to train models on X, deploy on Y, and have data on Z.

It's funny to watch, though, as if I recall correctly, there was an article on one of these companies where saying "multi-cloud" was blasphematory.

- [0]: https://iko.ai

Eh, even corporate GSuite changes from time-to-time. Google products and services are less stable than AWS. I’m not talking about uptime, necessarily, but for all of AWS half-baked ills you know that half-baked service is going to be around forever.

I don’t think it’s a tired argument: Google is much more likely to cut bait than Amazon.

If this was on occasion it wouldn’t bother me.

If my cloud provider said, “hey we released a half baked service and we’re deprecating it” at least that would give a solid reason to fix some obvious technical debt. Otherwise you may just be band-aiding technical debt for years.

Anecdotally, I’m thinking of ElasticSearch Service around 2017. We were pushing almost a terabyte an hour into ESS.

We ended up tacking on SearchGuard, ElastiAlert, some SSO proxy, and about 3-4 other products, when what we wanted was X-Pack.

It took a lot of toil before we convinced the org to go permit a migration off of ESS.

You might accept it, but I wouldn’t. Telling product owners that their timelines have been pushed because our cloud provider is removing something we depend on again is not a conversation I want to have.

Large enterprises are complicated beasts, and they value stability a lot. Even removing a single feature might cause dozens of teams to drop everything in order to go and fix the mess that someone else made. Why risk it? Especially if the alternative is someone who will wait to release a feature until it’s more than half baked and support it for a decade or more?

I said “on occasion,” not time after time.

I think purging some of the lower quality services from a catalog of over 175 (AWS) services would be a net positive, because orgs wouldnt come along and build on top of sore ice that may not be as extensible as you need it two quarters from now.

I disagree. Perhaps AWS should be more selective in what they choose to launch, but once a service is launched you should have a very compelling reason to deprecate it. This creates a positive feedback loop that AWS is safe —- in the sense that if you build on top of their services you know that they will continue to be around.

It makes the AWS dashboard a bit more cluttered, but if you use one of AWS’ half-baked services you know it will be around for as long as you want. Maybe you outgrow it; that’s fine. You can opt to move off, but AWS won’t force your hand.

There’s a ton of value in that stability. Your MySQL server running on EC2 still works today if you haven’t migrated to RDS. And if you migrate to RDS, you can be confident that it will be around until well after you’ve moved to your next job.

Ironically, this is related to Golang’s strong 1.X backwards compatibility guarantee. Knowing that what works today will work tomorrow has tremendous value. You don’t have to wake up and migrate everything from vendor to modules. You can build on ECS today and have confidence ECS will be around tomorrow.

It’s human trust. It’s a basic thing in all business dealings. Google has built this reputation themselves over the years by constantly sun setting new products. I mean there is a website showcasing it https://killedbygoogle.com. Google support is known for being notoriously bad and unhelpful. They are the first targeted tech giant going through new antitrust suits and who knows the outcome of them. Why would I trust a business I’d hope could be around for 10 to N years when AWS/Azure are around? Yes, the same could be said for them but the basic human element of trust is on their side.
Why do you trust AWS when Amazon kills failed non-AWS products left and right? I even remember Bezos bragging about this being a core part of the company culture. Why do you trust Azure when Microsoft's graveyard of non-Azure graveyards has long since overflowed?

For me, it's because Amazon killing their failed phone, their failed Yahoo Answers clone, their failed search engine, or their failed paypal clone says nothing about their commitment to their phenomenally successful $45 billion / year business with high margins and 29% growth.

Google Cloud is a $14 billion / year business growing at 45% / year. It is a smaller business than AWS, sure. But not by an order of magnitude. It's basically 3.5 years behind Amazon on the growth curve. Were you afraid in 2017 that AWS would be killed due to being too small a business?

>"Why do you trust AWS when Amazon kills failed non-AWS products left and right? I even remember Bezos bragging about this being a core part of the company culture."

I'd be really interested in this quote and its context. At AWS, things are always iterated upon and tested. But there's an incredible emphasis on two-way door decisions. Basically, don't make big decisions that can't be reversed if things go poorly. And once you've released something customer-facing (especially something as important as a new AWS service), you're locked in for the foreseeable future.

There are still CloudHSM v1 HSMs running out there in the wild. CloudHSM v2 was released in 2017. And CloudHSM v1 hasn't been available since at least 2019.

There are still EC2 instances running in EC2 classic--that is, EC2 before VPC was introduced.

It's a pretty common theme in what Bezos says, no? Here's some examples: https://www.investors.com/news/management/leaders-and-succes... has some of such quotes:

> "Failure comes part and parcel with invention," Bezos wrote in his 2013 letter to shareholders. "It's not optional. We ... believe in failing early and iterating until we get it right." Three years later, he added, "Amazon is the best place in the world to fail."

Or:

> “As a company grows, everything needs to scale, including the size of your failed experiments. If the size of your failures isn’t growing, you’re not going to be inventing at a size that can actually move the needle,”

Now, I'm sure that doesn't apply to AWS, and your culture is totally different. You're an enterprise product with real contracts, real commitments, and making real money. You're not the Groupon clone, online pharmacy or whatever that grocery service *Amazon killed today* was.

But if we're accepting that Amazon's general trigger-happiness when it comes to failed consumer products doesn't extend to AWS, why does Google Cloud not get the same treatment?

Google has absolutely abandoned things in their cloud. App Engine has been a moving target for years for just one example.
I beg to differ. We've run our business on AppEngine for 8+ years. We've never been forced to migrate, and when we've chosen to upgrade to newer runtimes like Java8, the transition was smooth.

Upgrading to Java11 will indeed be a big change, but Java8, with memcache etc, is still very much supported.

I led a team that wrote an app app engine using python 2. When Google upgraded to python 3 they completely rewrote entire libraries, like the one for datastore, so we were forced to either rewrite most of our app or to stay on a deprecated platform that stopped getting new updates.
Can you give a more clear example of what GCP has dropped support for?
Google provides a list of App Engine features they've removed. [1] Beyond that there is also a somewhat undocumented phase of working-but-forgotten. Classic App Engine features like the datastore, memcache, Users API, Python 2, Go 1.11 etc go under this category. These are things that still work, but get no updates. Instead you get constant e-mails and other notifications about how you should redesign your app to work with the 2nd generation App Engine system. Which means Firestore (in datastore mode) instead of datastore. Memorystore instead of memcache. Your own solution instead of the Users API etc.

--

[1] https://cloud.google.com/appengine/docs/deprecations

Yes, this. Thank you for explaining it better than I did. I loved this with a real production python 2 app, and my take-away, be very careful building against proprietary systems like classic app engine and all of its services.
*lived
Tell to the folks paying for the Maps API who built businesses around it.

What a scam. Has amazon EVER raised the price of an AWS service?

No, Amazon hasn't. If a service is poorly-priced and/or seldom used, they'll rebuild the service from scratch to offer better pricing and features. See Macie as an example.
This comment brought to us 9 days after an entire company was yeeted off the internet by a cloud provider.

Old, lazy, and annoying? Come now.

Most customers do not plan to repeatedly break their contract and makes no effort to fix it. Parler’s refusal to follow a legal agreement is only a concern to a small handful of other sites.
That’s quite an imaginative take on what happened.
You’re welcome to read the legal filings yourself so you can make less uninformed comments in the future. Here’s an example of the messages AWS contacted Parler about in November:

https://www.courtlistener.com/recap/gov.uscourts.wawd.294664...

The entire filing makes it clear that they were given considerable time and refused to moderate content which violated the terms of service which they had agreed to follow:

https://cdn.arstechnica.net/wp-content/uploads/2021/01/gov.u...

Maybe you should actually read your own links. They don't say what you think they say. Not to mention the source is Amazon. Of course Amazon thinks Amazon is in the right.

In a span of seven weeks, on a platform with 8 million users, Amazon found 100 examples of objectionable content.

How many examples of objectionable content do you think I can find on Reddit, Twitter, or Facebook right now?

It's not a question of whether Parler had offensive content on its site. It's a question of Amazon holding Parler to terms that they don't hold anyone else to, including Parler rivals. Parler had moderation tools, were moderating content, and had a clear Terms of Service that outlined the type of content that was not allowed.

Let us travel back in time to 2009. It's three years after Twitter was founded. How robust and effective do you think the Twitter moderation was back at that time?

Even today, in 2021, Twitter is absolutely FULL of illegal content and hate speech. When do you think Amazon will eject Twitter from AWS? Should be any minute now. Any minute.

Not to mention some of the ridiculous price increases, which could be enough to kill your business if you depended on them, e.g.

* Google Maps API increasing prices by 1400%+

* GKE price increasing from free to ~$73/month per cluster

I work for a huge international bank. We've had some of our non-production infra on GCP and it's been a very bad experience for us; mainly reliability-wise. We're moving away from it entirely.
I put a production AAA game on it (with horrible constraints for a cloud provider, like highly stateful connections and hard requirement to honour fsync in the underlying hardware)- and we had a truly great experience.

Can you go into more detail? Not saying you’re wrong of course, but your experience is massively contra to my own, and like I said, we have production on there with little to no issues.

Or some ML crob job algo going haywire and closing your account - with no human to contact to get it resolved.
Google's services are so perfect they don't even need to invest in customer service!

If you have a problem with anything, it's clearly your fault and you're doing it wrong. /s

Sarcasm aside, at $COMPANY we have AWS reps integrated into our Slack, and they work hand in hand with our engineers quite often. Even for things as low down as “why is this query so slow on AuroraDB?” They even hop into war rooms for big events in case we need immediate assistance during high visibility outages.

It’s hard to overstate how important this level of close support is for an enterprise that is literally betting the farm on a cloud provider. The idea of counting on Google’s historical level of support is an absolute non-starter for us.

The role you describe is a technical account manager and Google has hired many. I’m a Google cloud partner and every project I’ve worked on has had multiple TAMS doing exactly what you describe, working hand in hand with customers, connecting the customer to product engineers, to support, navigating Black Friday and Christmas, etc...
To me, Google has always felt like the Stack Overflow of enterprise companies. Our questions to Google have been met with "You're trying to do it this way, but what you really want to do is this". Or worse, they'll send us a link to a document which we've already read ourselves and then just go silent.

They've never tried to deeply understand our use cases or business at all. Hopefully they turn this around but IMO, Google is absolutely horrible at B2B, they're just not currently set up to do it correctly.

I had this experience too around 2015-2017, but I've noticed a big difference between 2018 and now. They've hired a large number of people whose job is primarily focused on understanding the customer's business and use case. Most of these people have empathy in my experience.

In the large enterprise migrations I've worked on the past 2 years, I haven't heard a Google employee or partner say, "you're doing it wrong." Sometimes there is a response along the lines of, "we don't recommend doing it that way for the following specific reasons. For your consideration, here are alternatives we recommend for your use case. We've filed your use case as a feature request with product."

Additionally, I've seen issues which block a migration to GCP get escalated and fixed quickly. Issues which in ~2015 might have garnered a, "you're doing it wrong" response.

I've noticed a real shift in attitude and execution. There is genuine empathy and an attempt to understand how enterprise customers run their workloads in the cloud, multi-cloud and hybrid-cloud.

A while back before AWS was so popular their support at lowest tier developer accounts was totally ridiculous. I'd messed something up, out of their scope to fix (they could have just said nothing wrong with X). Instead some AWS engineer with my permission jumped in and got application side to fix things up and walk me through the configs etc.

What!! So I paid $50 for one month for support, and got a full on support expert who'd bill (at least in my world) $250+/hr. Honestly, the support cost was maybe even LOWER, and I'd just signed up for support to submit my ticket.

Google does not care. You GSuite calendar is not accessible on google home devices etc etc despite TONS of requests from paying customers. That same calendar integrates easily with ALEXA from amazon. Huh? Someone is paying attention, and it's not google.

I do like project based permission / approach in GCP. I like a lot of other things there too. But I've had stuff running for years on AWS without issue - so the trust with AWS keeping things going mostly is there.

this exactly. Microsoft has known it for years: to be successful with enterprise accounts, people matter. Account managers, sales/support engineers, direct access to product teams (sometimes). I've never worked with Amazon/AWS but from what I've heard anecdotally, they are of the same mindset. (It would be consistent with Bezos' customer service mantra so perhaps unsurprising).

I know of only 2 colleagues who've looked at GCP. Both are medium-large financial institutions. Both said they had very little human-to-human contact with Google representatives. In one case they chose AWS instead; the other is still evaluating GCP.

It's interesting that other posts in the thread are complimentary about the quality of the GCP products. I can believe that; Google has built its fair share of impressive technology. But I see no evidence of an enterprise supplier I could trust: one that would be there to help out when the world went belly up. Given that, it doesn't matter how good their products are.

I'd actually like to see GCP being successful. As well as bringing some competition and diversity to the market, it would give Google a more honest and above-board revenue stream than ads. I don't see that happening though, at least not without a change in leadership.

Agree, we have an integrated AWS contact too and they're really invaluable for getting insight on what the actual limits are on their services.
Long-time GCP user here, from a big conservative enterprise. GCP is a wonderful technical platform, but if I ran an oil or coal or tobacco or weapons company, or any other business susceptible to future boycotts and political activism, I would only adopt GCP in a cloud-neutral way. Google employees have disproportionate power over what gets done, versus for example AWS where customers seem to have a lot of power.
I know it's easy for me to say, but shouldn't nobody be building their business to be critically dependent on some other business, regardless of whether cloud provider or other class of service? Especially when the balance of power clearly favours one of the parties, this seams like a bad idea if it can be avoided. I don't know how hard it is to develop architectures which run on several cloud services. If it is possible, shouldn't this be standard practice?
> I don't know how hard it is to develop architectures which run on several cloud services. If it is possible, shouldn't this be standard practice?

I think the answer to this question depends on a multitude of factors. For example, nowadays you can use container technologies to create images of your software, which will run on any bit of infrastructure that supports them, be it AWS, GCP, Azure, or even your on-prem servers with something like Rancher or even Docker Swarm running on it. However, creating software like that needs some special thought put into it, this site covering some of those aspects: https://12factor.net/

And while this will make your code more reproducible and migrating between different clouds more easy (or even running on multiple clouds simultaneously), it'll do so at the expense of making development slower and a bit harder for you. For some, this will be worth it, while for others it'll be less so. There are definitely valid criticisms of container technologies and some of the aspects they don't handle all that well yet (Kubernetes being overcomplicated for some projects, whereas Docker Swarm isn't "trendy" or in active development, then there's managing storage and needing to work with either network file systems, or distributed file systems, or even using bind mounts even though they're considered bad practices, then there's routing and the complexity of service meshes etc.).

Of course, there are also people who really don't want to think about infrastructure that much and just want their apps to run in a semi-managed manner, like Heroku does, or perhaps just want to use one of the cloud vendors' managed database or messaging system offerings. Not everyone has a large amount of resources to invest in engineering and running infrastructure.

A multi-cloud strategy is generally not a good idea.

Some links that will hopefully explain it better than I can:

* https://www.lastweekinaws.com/blog/multi-cloud-is-the-worst-...

* https://www.infoworld.com/article/3428682/your-multicloud-st...