[ca508]

ah that's my bad - I wrote this in Dec, we only published in Jan. Obv. missed updating that.

Timeline wise; - we decided to go for it and spend the $$$ in Oct '23 - Convos/planning started ~ Jan '24 - Picked the vendors we wanted by ~ Feb/Mar '24 - Lead-times, etc... meant everything was ready for us to go fit the first gear by mostly ourselves at the start of May (that's the 5mo) - We did the "proper" re-install around June, followed closely by the second site in ~ Sep, around when we started letting our users on it as a open beta - Sep-Dec we just doubled down on refining software/automation and process while building out successive installs

Lead times can be mind numbing. We have certain switches from Arista that have a 3-6 mo leadtime. Servers are build to order, so again 2+ months depending on stock. And obv. holidays mean a lot of stuff shuts down around December.

Sometimes you can swap stuff around to get better lead-times, but then the operational complexity explodes because you have this slightly different component at this one site.

I used to be a EEE, and I thought supply chain there was bad. But with DCs I think it's sometimes worse because you don't directly control some parts of your BoM/supply chain (especially with build-to-order servers).

[Twirrim]

From working at a cloud, and speaking with capacity folks regularly when I was in certain roles, the supply chain strikes me as one of the biggest nightmares. Even at scale when vendors really, really want (or want to keep) your business. At times it almost seems like someone sneezes somewhere and whoops, there goes your hardware delivery timelines.

The advantage at cloud scale is a lot of constant signal around capacity delivery, demand etc. so you can build mathematical models to best work out when to start placing orders, and for what.

[justjake]

FWIW this is the advantage of being able to run in the cloud and on perm

If we have to we can “burst” into the cloud