Hacker News new | ask | show | jobs
by ananonymoususer 1338 days ago
I use this all the time. It's a big time saver on multi-core machines (which is pretty much every desktop made in the past 20 years). It's available in all the repos, but not included by default (at least in Ubuntu/Mint). It is most useful for compressing disk images on-the-fly while backing them up to network storage. It's usually a good idea to zero unused space first:

(unprivileged commands follow)

dd if=/dev/zero of=~/zeros bs=1M; sync; rm ~/zeros

Compressing on the fly can be slower than your network bandwidth depending on your network speed, your processor(s) speed, and the compression level, so you typically tune the compression level (because the other two variables are not so easy to change). Example backup:

(privileged commands follow)

pv < /dev/sda | pigz -9 | ssh user@remote.system dd of=compressed.sda.gz bs=1M

(Note that on slower systems the ssh encryption can also slow things down.)

Some sharp people may notice that it's not necessarily a good idea to back up a live system this way because the filesystem is changing while the system runs. It's usually just fine on an unloaded system that uses a journaling filesystem.

1 comments

Alternative way of zeroing unused space without consuming all disk space: https://manpages.ubuntu.com/manpages/trusty/man8/zerofree.8....
Thanks. If run as an unprivileged user, the dd command will not consume ALL of the disk space (so privileged processes will not be disrupted). It will consume up to the free space limit (default 5%) as described here: http://blog.serverbuddies.com/using-tune2fs-to-free-up-disk-...

The zerofree command looks useful, but I don't know how portable it is. The dd method works across many platforms (such as AIX).