[small_scombrus]

Its not going to be amazingly professional and 100% best practice, but you can set up docker-composes and/or design containers to pull everything they need on first run.

That plus a decent backup system would work for a small array of servers with fail-safes. Though I would die inside if a production user-focused app under any level of proper load was set up like that.

[imp0cat]

You'll need more than a backup system. At least some sort of a load balancer to switch between different groups of running docker containers (so that upgrades, backups, etc... can happen without service being interrupted).

[code_runner]

not every single system requires 100% uptime... some folks still use release windows for specific applications/hardware. I'd argue that most systems out there can support a release window.... and the overlap between "docker-compose works for us" and "we have release windows" is probably quite high.

We needed to do inference on remote machines stationed at random points across North America.... the machines had work to do depending on external factors that were rigidly scheduled. really easy to make upgrades, and the machines were _very_ beefy so docker-compose gave us everything we needed.... and since we shipped the data off to a central point regularly (and the data had a short shelf life of usability) we could wipe the machines and do it all again with almost no consequences.

I needed to coordinate a few small services and it did the trick beautifully

[vundercind]

My experience has been that the vast majority of systems could tolerate a few minutes offline per month for upgrades. Many could tolerate a couple hours per month. No or negligible actual business harm done, enormous cost savings and higher development velocity from not adding the complexity needed for ultra-high uptime and zero-downtime upgrades.

What's vital is being able to roll back a recent update, recover from backups, and deploy from scratch, all quickly. Those are usually (not always) far easier to achieve than ultra-high-uptime architecture (which also needs those things, but makes them all more complicated) and can be simple enough that they can be operated purely over ssh with a handful of ordinary shell commands documented in runbooks, but skipping them is how you cheap out in a bad way and end up with a system down for multiple days, or one that you're afraid to modify or update.