[byteofbits]

We’re currently migrating to Spanner for a variety of reasons - but the mandatory downtime on their Postgres CloudSQL offering will be the part I miss the least.

It’s insane that even with all of their HA and failover turned on they take the whole cluster down for as long as they like every few months!

[ethbr0]

One thing that customer obsession at an entire-organization-depth level does is encourage broad customer use awareness.

To an engineer, things are things, because of how they architect and build them.

To an engineer who understands a customer, things are things and all the things people actually use them for. Big difference.

It also makes "Well, that customer is using it wrong" less of an exceptable engineering dodge.

[throwaway894345]

Surely product makes these decisions, not engineers, right? I agree that customer empathy is important, but I don't think we can conclude that the engineering team (rather than the product team) is the source of the deficiency?

[majormajor]

> Surely product makes these decisions, not engineers, right? I agree that customer empathy is important, but I don't think we can conclude that the engineering team (rather than the product team) is the source of the deficiency?

I haven't worked inside AWS or GCP, but I've never seen product get everything they want, especially around maintenance/downtime. If "less downtime" is on the roadmap but engineering is constantly pushing back "that'll be really really hard and take a long time and they're just using it wrong anyway," I can't imagine it getting done as quickly as at a place where the engineering team was also focused on customer satisfaction.

[throwaway894345]

> that'll be really really hard and take a long time

It probably is hard and intensive. Engineering shouldn't lie and promise that it will be easier. Product has to take that engineering estimate and determine whether to work uptime or some sexy feature (and sexy features usually win because of perverse incentives).

Moreover, I have a hard time believing this for a couple reasons: first of all, I've scarcely met engineers who were opposed to improving product reliability, maintainability, etc. The portrait of Google engineers arguing that database services fundamentally shouldn't be HA (and customers are "using it wrong" for wanting HA DBs) is particularly incredulous. Secondly, I've never heard of an organization where engineering held political power over product decisions, but I have worked in several places where product dictated engineering solutions. Businesses trust product more readily than engineering because the things that engineering is always petitioning for are abstract and "costly" (deferring some immediate profit for reduced costs in the long run) while the things product wants are usually tangible and profitable.

[rutthenut]

Sounds familiar. Product and sales areas of the business look at immediate revenues - and hopefully profit - but are always more keen to do that at the cost of increasing technical debt within the engineering side of the business. Unless sales see a real impact to technical debt, they will always choose the short-term approach.

[throwaway894345]

Agreed, and I don't think that's even necessarily the worst thing. It just means that engineering and product/sales have to have a conversation, mutual trust, and a shared vision that extends beyond the next quarter. These are hard things to cultivate, however.

[lmeyerov]

It's a full-company culture problem where ~everyone has personal responsibility.

Behavioral properties are end-to-end, like speed, security, customer success, uptime, etc. Everyone and everything on the hotpath for that has to be on board, as just one violator means nobody is achieving it. Operationally, part of achieving that means company norms, such as when collaborating, planning, executing, etc.

It's 100% fine not to provide any of these, and is even reasonable given it's hard to change one's DNA end-to-end... but better to be an explicit decision. Likewise, if starting a company... a lot easier to pick & ingrain these habits ahead of time.

[jiggawatts]

I tried explaining this to some Azure product teams, and they gave me a blank stare in return.

Sure, you can have zone-redundant Azure SQL Databases, but not Azure App Service!

You can have zonal Azure App Service, but with a different network model than the default, so it's a breaking change for some apps.

It's as if nobody has actually sat down at Microsoft to build something similar to what their customers are building. It's all just tech demos, "get your blog hosted in 5 easy steps", and crap like that. Nobody at Microsoft has ever built anything of substance with the majority of their own platform.

It was the same story with Windows Presentation Foundation (WPF). It was hot garbage when it was first released, but Microsoft kept telling all their partners to prefer it over the legacy GDI+. People tried and failed to write GUIs in it. When after many years Microsoft finally tried to convert Visual Studio to WPF -- the first time they had used their own framework for one of their own apps -- magically the core issues were fixed and WPF become viable for real apps.

[mathattack]

My experience with MSFT is that the account reps are clueless by design. Even the sales assistants with fancy technical titles are short of clues. Easier for them to oversell when they don’t know what’s really going on underneath. We have to go 3 or 4 deep into their product org to find someone who will fess up to product issues that we are already aware of.

[kaliszad]

Also they now require you to become Microsoft Partner, if you want to use the oauth2 login not only for personal Microsoft accounts but also other Microsoft accounts such as those provided by a business running Office 365. They changed it ~ 3 months ago and are now not able to verify people in time. Even if they just want to check your name, email and maybe a profile picture and nothing else. The process is much less obvious than for the same thing with Google or Facebook. Even the ZIP-Code has to include spaces as if it was written on a letter, otherwise you will not be able to send the form, aaaand there is no hint about this requirement.

In general, Microsoft, Google and others are behaving really enterprise-y in that they are slow, you cannot reach anybody of importance to solve massive issues with their products and everything is actually quite expensive for what it does.

[adamparsons]

I had no idea this was a thing, is there any justification at all for this behavior? Sounds like a massive dealbreaker to me.

[renewiltord]

I use Aurora just for this reason. RDS is just a great product. Doesn't even need the cloud sql proxy equivalent since you can just install the authenticator into the database.

[paulryanrogers]

Aren't RDS and Aurora two different things?

[dragonwriter]

> Aren't RDS and Aurora two different things?

NO, RDS includes non-Aurora versions of many DBs, plus MySQL and Postgres-compatible Aurora servers, plus MySQL and Postgres compatible Aurora Serverless.

[renewiltord]

Aurora is one of the DB things you can launch in RDS.

[paulryanrogers]

Thanks for clarifying. I'd associated RDS with the non-Aurora offerings of Mysql and Postgres. Judging by upstream version compatibility it appears Aurora is more heavily forked than their non-Aurora siblings.

[dragonwriter]

I don't think Aurora MySQL/Postgres are just forks, I think they are a completely custom datastore behind a MySQL or Postgres-compatible interface (which probably uses a lot of non-engine code from the open-source base database.)

[paulryanrogers]

Regardless it looks like anyone choosing Aurora should not hold their breath for Mysql 8 or Postgres 10+ compatibility. Seems like only one major version bump has happened since they launched the first one (Mysql 5.6-to-5.7).

Which is fine. It can just be a little confusing as they drift and the caveats grow.

[ransom1538]

WAIT. Be careful. That is a super expensive product with a high likelyhood of lockin. It doesn't support all SQL features. Also! I run hundreds of GCP databases and never ran into: "but the mandatory downtime on their Postgres CloudSQL", maybe it is only Postgres?