[dh2022]

In Azure - which I think is at Google scale - everything is dynamically linked. Actually a lot of Azure is built on C# which does not even support static linking...

Statically linking being necessary for scaling does not pass the smell test for me.

[mbreese]

I never worked for Google, but have seen some strange things like bit flips at more modest scales. From the parent description, it looks like defaulting to static binaries is helping to speed up troubleshooting to remove the “this should never happen, but statistically will happen every so often” class of bugs.

As I see it, the issue isn’t requiring static compiling to scale. It’s requiring it to make troubleshooting or measuring performance at scale easier. Not required, per se, but very helpful.

[btilly]

Exactly. SRE is about monitoring and troubleshooting at scale.

Google runs on a microservices architecture. It's done that since before that was cool. You have to do a lot to make a microservices architecture work. Google did not advertise a lot of that. Today we have things like Data Dog that give you some of the basics. But for a long time, people who left Google faced a world of pain because of how far behind the rest of the world was.

[btilly]

Azure's devops record is not nearly as good as Google's was.

The biggest datasets that ChatGPT is aware of being processed in complex analytics jobs on Azure are roughly a thousand times smaller than an estimate of Google's regularly processed snapshot of the web. There is a reason why most of the fundamental advancements in how to parallelize data and computations - such as map-reduce and BigTable - all came from Google. Nobody else worked at their scale before they did. (Then Google published it, and people began to implement it. Then failed to understand what was operationally important to making it actually work at scale...)

So, despite how big it is, I don't think that Azure operates at Google scale.

For the record, back when I worked at Google, the public internet was only the third largest network that I knew of. Larger still was the network that Google uses for internal API calls. (Do you have any idea how many API calls it takes to serve a Google search page?) And larger still was the network that kept data synchronized between data centers. (So, for example, you don't lose your mail if a data center goes down.)

[arccy]

perhaps that's why azure has such a bad reputation in the devops crowd.

[dh2022]

Does AWS have a good reputation in devops? Because large chunks of AWS are built on Java - which also does not offer static linking (bundling a bunch of *.jar files into one exe does not count as static linking). Still does not pass the smell test.

[arccy]

In AWS, only the very core Infra-as-a-Service that they dogfood can be considered "good", Everything else that's more Platform-as-a-Service can be considered a half baked leaky abstraction. Anything they release as "GA" especially around ReInvent should be avoided for a minimum of 6 months-1 year since it's more like a public Beta with some guaranteed bugs.

[dh2022]

In AWS, only the very core Infra-as-a-Service that they dogfood can be considered "good" - large chunks of which are, by the way, written in Java. I think you are proving my point...

[arccy]

which just means Java isn't affected? or your definition of not not counting bundled and not shared jars as static linking is wrong, since they achieve the same effect.