[a20eac1d]

The one thing that gives me hypertension when self hosting a code repository is backups.

The hard drive of your server can fail at any time and when self hosting you are responsible for your backups.

This is giving me night terrors, especially when its on a cloud server and I don't have access to the hardware.

Currently, I'm running a cron task once per day executing a simple backup script that does the following:

Stop the Gitea container, copy the entire Gitea directory (including the docker-compose.yml and the data directory) to a backup folder, restart the container, sync that folder to a Backblaze bucket, delete the backup folder.

Restoring the backup is (should) be as easy as downloading the bucket from Backblaze and simply docker-composing it up.

I'm looking for other ideas advice that will help me sleep at night. Thanks!

[LinuxBender]

For what its worth, another thing you might consider looking into is using RSnapshot [0]. RSnapshot helps me sleep at night. This will create multiple directories that are hard links to files that did not change, thus saving disk space and giving you multiple days or weeks of snapshots, in the even something was corrupted and you want to roll back at the filesystem level. Most questions one could come up about rsnapshot with are answered on Serverfault [1] There are many how-to sites [2] with usage examples. You can create snapshots locally or remote. On mac you can brew install rsnapshot.

[0] - https://rsnapshot.org/

[1] - https://serverfault.com

[2] - https://linuxconfig.org/guide-to-rsnapshot-and-incremental-b...

[hnlmorg]

Snapshots aren't a backup. However they do obviously have their merits too. Personally I'd recommend ZFS over RSnapshot and then make use of raidz so you have redundancy at the hardware level as well as at the snapshots.

[LinuxBender]

Rsnapshot creates a backup and then creates filesystem diffs from that backup. The backup can be local and/or remote.

[hnlmorg]

You can do that with ZFS as well but that still doesn't make snapshots a backup. The backup is when you send those snapshots to a remote volume. If a backup isn't stored remotely it isn't a backup (it also shouldn't be mounted locally).

[LinuxBender]

Yup, aware what ZFS could do. I was not going to suggest it since they are talking about a personal git server. ZFS likes memory, lots of it. Rsnapshot is a simple perl script that uses rsync. The backups and snapshots are wherever you point it to. Local, remote, both, 20 other locations if you are so inclined.

[hnlmorg]

The stories about ZFS memory consumption are largely exaggerated. But I do take your point :)

[GuB-42]

One of the advantages of systems like git is that you just need to clone and you have a complete copy of the original repository, you can synchronize with push/pull. No need to mess with backblaze and containers unless you want to back up your server configuration too. If you lose your server, just push your working copy on a new server. With enough people, losing data becomes almost impossible.

In fact you don't even need a server, just push/pull between your machines, a server just makes things more convenient.

[hellcow]

Do you not have all your code locally as well? It seems like if your drive failed for a personal git host, you could just re-push everything back up to it.

[3np]

Have you considered a distributed filesystem?

For example:

Glusterfs Replicated 3 (1 arbiter/parity), put on top of zfs filesystems with checksumming.

Users/consuming services can Fuse mount over the network

Incremental backups of bricks (1 should be enough) to a mirror or spinning rust

Then all you need to worry about is offsite

If you really only need to solve for gitea, this is prob overkill but if you have more services putting stuff on disk it could be worth it. Works great for me.