[fncivivue7]

Sounds like you want Borg

https://borgbackup.readthedocs.io/en/stable/

My two 80% full 1tb laptops and 1tb desktop backup to around 300-400G after dedupe and compression. Currently have around 12tb of backups stored in that 300G.

Incremental backups run in about 5 mins even against the spinning disk's they're stored on.

[0cf8612b2e1e]

Python programmer here, but I actually prefer Restic [0]. While more or less the same experience, the huge selling point to me is that the backup program is a single executable that can be easily stored alongside the backups. I do not want any dependency/environment issues to assert themselves when restoration is required (which is most likely on a virgin, unconfigured system).

[0] https://restic.net/

[SomeoneOnTheWeb]

You can also take a look at Kopia (https://kopia.io/).

I've been using Borg, Restic and Kopia for a long time and Kopia is my personal favorite - very fast, very efficient, runs in the background automatically without having to schedule a CRON or anything like that.

Only downside is that the backups are made of a HUGE number of files, so when synchronizing it can sometimes take a bit of time to check the ~5k files.

[wanderingmind]

Highly recommend Kopia that has a nice UI and can work with rclone (so any cloud back end)

[klodolph]

I’ve been using Kopia, I recommend it.

[wereallterrrist]

No, I distinctly don't want borg. It doesn't help or solve anything that Syncthing doesn't do. The obsession with borg and bup are pretty baffling to me. We deserve better in this space. (see: Asuran and another who's name I forget...)

Critically, I'm specifically referring to code sync that needs to operate at a git-level to get the huge efficiencies I'm thinking of.

Syncthing, or borg, scanning 8 copies of the Linux kernel is pretty horrific compared to something doing a "git commit && git push" and "git pull --rebase" in the background (over-simplifying the shadow-branch process here for brevity.)

re: 'we deserve better' -- case in point, see Asuran - there's no real reason that sync and backup have to be distinctly different tools. Given chunking and dedupe and append-logs, we really, really deserve better in this tooling space.

[formerly_proven]

borg et al and "git commit" work in essentially the same way. Both scan the entire tree for changes using modification timestamps.

[dragonwriter]

> borg et al and "git commit" work in essentially the same way. Both scan the entire tree for changes using modification timestamps.

But git commit doesn’t do that. If you want to do that in git, you typically do it before commit with “git add -A”.

[codethief]

I don't think GP was talking about backups (which is what Borg is good for) but about synchronization between machines which is another issue entirely.

[_dain_]

They work together. I use syncthing to keep things synchronized across devices, including to an always-on "master" device that has more storage. Then borg runs on the master device to create backups.