| I think the complexity of your infrastructure is not helpful - so I would start in reducing it. I personally use one single server (Fujitsu D3417-B, Xeon 1225v5, 64GB ECC and WD SN850x 2TB NVMe) with Proxmox and an OpenWRT Router (Banana Pi BPI-R3). The Proxmox draws ~12W and the OpenWRT around 4.5W in Idle and with NodeJS I can run MeshCommander on OpenWRT for remote administration. Since I use ZFS (with native encryption), I can do a full backup on an external drive via: # create backup pool on external drive
zpool create -f rpoolbak /dev/sdb
# create snapshot on proxmox NVMe
zfs snapshot -r "rpool@backup-2024-01-19"
# recursively send the snapshot to the external drive (initial backup)
# pv only is there to monitor the transfer speed
zfs send -R --raw rpool@backup-2024-01-19 | pv | zfs recv -Fdu rpoolbak
An incremental backup can be done by providing the -I option and two snapshots instead of one to mark the starting point and the stop of the incremental backup: # create new snapshot
zfs snapshot -r "rpool@backup-2024-01-20"
# only send everything between 2024-01-19 and 2024-01-20
zfs send -RI --raw rpool@backup-2024-01-19 rpool@backup-2024-01-20 | pv | zfs recv -Fdu rpoolbak
That is easy, fast and pretty reliable. If you use this `zfs-auto-snapshot` you can rewind the filesystem in 15 minute steps.This is also pretty helpful on rewinding virtual machines[1]. Recently my NVMe (a Samsung 980 Pro) died and restoring the backup via zfs (the backup shell command in reverse order from rpoolbak to rpool) took about 2 Hours for 700GB and I was back online with my server. I was pretty happy with the results, although I know that ZFS is kind of "experimental" in some cases, especially encryption. 1: https://pilabor.com/series/proxmox/restore-virtual-machine-v... |