[edofic]

"Since there's no way to roll back server upgrades"

It is if you run a modern filesystem like ZFS or btrfs. You just do a cheap snapshot before upgrading(can be automated) and roll back if there are problems. Even works with lvm.

[KaiserPro]

Sadly as zfs isn't as widely available as I'd like, you can use LVM to provide snapshots.

Its not as friendly as ZFS snapshots, but it is at least available in centos 5 https://www.centos.org/docs/5/html/Cluster_Logical_Volume_Ma...

(sadly somethings need long term support)

[lsc]

rolling back a LVM snapshot involves dding off the snapshot, and on to whatever you want your production disk to be (or just running off the snapshot forever, which has.... performance consequences with LVM.)

Yes, LVM snapshots exist, but they are of limited utility compared to ZFS and the like.

I've been experimenting with CentOS6 and ZFS on Linux; so far it looks pretty good. it handles failing consumer grade hard drives vastly better than lvm on md, and snapshots are inexpensive.

[b101010]

You can also roll back lvm snapshots using the merge option

https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...

[lsc]

nice. I had not seen that before... that makes it a lot more useful.

[KaiserPro]

Oh yes, LVM is abhorrent, along with mdadm.

Thats why I'm not so happy about the direction the tools are going with BTRFS. If ZFS isn't around, LVM is your only real option.

However, RHEL usually sets up LVM, so you might as well use it.

[thaumaturgy]

Ooh, thanks. That's a good idea. I'd never considered running ZFS on my small servers.

[keithpeter]

Or perhaps simply gzip of the / partition before upgrading. If insuperable problems, zap new / and restore old known working one?

Assuming data/sites are on different partitions.

[kbenson]

While initially conceptually easier to grasp, that is far inferior to using a snapshot.

Here's a short list of ways in which that may cause you problems:

1) gzip of a path is not point in time, synced files may no longer be in sync since they were backed up at slightly different times (e.g. I hope you didn't expect database consistency to actually mean anything).

2) gzip of a path will take a while, because it has to actually function on every file (a snapshot is generally copy on write, meaning it's "free" (not quite) for every file until it's changed. Throw away the snapshot before a change and there's no need to copy the file.

3) gzip will take more room (see #2)

[keithpeter]

Excellent reply. The implications of a significant number of transactions per minute had escaped me.

[davexunit]

Or, you could try out NixOS/GuixSD, which support transactional upgrades and rollbacks for the full system. No need to take a disk image (outside of your normal backup routine, of course).

[vacri]

btrfs still isn't production ready. I went to a talk from a btrfs dev two weeks ago, and it was still "use btrfs with caveats", not "just use btrfs".

[vel0city]

Or, if you're using VMs, just take a snapshot and roll back if something doesn't work right. For anything that isn't virtualized, there's ZFS.