Hacker News new | ask | show | jobs
by jpdoctor 4794 days ago
Would love to hear ideas on the best replacement from other HNers.
15 comments

Open-source utilities running from livecd.

   dd if=/dev/sda bs=16777216 | gzip -c9 > /path/to/sda.img.gz
If you have the pv utility installed, you can get progress bars:

   dd if=/dev/sda bs=16777216 | pv -c -W | gzip -c9 | pv -c -W > /path/to/sda.img.gz
To restore (WARNING: THE FOLLOWING COMMAND IS EXTREMELY DANGEROUS, USE IT ONLY IF YOU KNOW EXACTLY WHAT YOU'RE DOING):

   pv -c -W /path/to/sda.img.gz | gzip -cd | pv -c -W | dd bs=16777216 of=/dev/sdx
Of course, you can use your favorite compression program instead of gzip; on Debian-like systems, bzip2 or xz should be drop-in replacements available by default.

If you want a compressed image which you can mount read-only without uncompressing (e.g. if you want to be able to reach into the backup and pull out a single file or directory), you can pipe the dump to a FIFO and then use the pseudo-file feature of mksquashfs [1]. E.g. something like this (commands not tested, there may be typos):

    mkfifo sda.fifo
    echo 'sda.img f 444 root root cat sda.fifo' > sda.pf
    dd if=/dev/sda bs=16777216 > sda.fifo &
    mkdir empty
    mksquashfs empty sda.squashfs -pf sda.pf
    mount -o ro,loop sda.squashfs /mnt
You then have an uncompressed image visible. If it's a whole-disk image (/dev/sda instead of /dev/sda1), you need to use a program called kpartx to make device nodes for each partition.

I recommend reading the man pages and playing around with these utilities in a Virtualbox VM with a small disk.

Of course, this approach doesn't pay any attention to filesystems. Which means it works with Windows partitions (unlike LVM or btrfs snapshots). But there are limitations; it reads and stores "empty" space not occupied by files (I recommend making a large file full of zeroes beforehand to make the empty space compress better), restoring to a smaller disk is difficult if not impossible, restoring to a larger disk requires you to resize manually afterwards if you want to use the extra space.

[1] It's a little easier conceptually to make a squashfs directly from an image file, but that approach requires you to store the image file, which requires temp space equal to the size of the disk you're backing up. The pseudo-file approach has a benefit of not needing temporary space equal to the size of the disk being backed up, you just need enough space for the compressed result.

That misses much of what Ghost does. Ghost is file-system-aware, so it's not a byte-for-byte cloner. It can copy one volume to a large volume, and vice versa.

You can accomplish the same thing in Linux, of course, but it requires a lot more than just "dd".

For making frequent backups without shutting down Windows, nothing I've tried compares to ShadowProtect Desktop [1]. It makes a complete incremental backup of my C: drive every 15 minutes. Those backups often take only a few seconds, and they don't interrupt my work at all.

I used Acronis before that and it worked fine, but the incremental backups took many minutes with a lot of disk activity - even if I'd only touched a few files. Running a backup every 15 minutes wouldn't be in the cards. So I'd run an incremental manually when I felt worried enough to do so - if I remembered to do it!

It's a real stress-reducer to know that everything is backed up every 15 minutes and I can read files or restore the system from any of those 15-minute checkpoints.

ShadowProtect does this by making sector-by-sector backups with a limited amount of filesystem knowledge: it skips free space and pagefile.sys/hiberfil.sys, and that's about it. It has a driver that keeps track of disk writes while Windows is running, so it doesn't have to scan the disk or filesystem for changes.

If that tracking data becomes invalid (e.g. because of a bluescreen or forced power-off) it falls back to a full disk scan for the next incremental backup. It still makes the same small incremental that it would have made otherwise, it just takes longer - about as long as a full backup.

Because the backup is sector-by-sector, if you change just part of a large file, only the changed sectors need to be backed up. If you move or rename a large file, it takes hardly any time at all to back it up, because it only has to back up the filesystem metadata and not the file data itself.

You can mount a backup volume and open the files in any Windows app, or you can use their restore CD to do a bare metal restore, including a hardware independent restore. Of course other backup programs have those features too, they just don't have the crazy fast incrementals.

I've used the full restore many times and only had a problem once, when I migrated a Windows 8 installation from a ThinkPad X220 Tablet to a ThinkPad W520. It wouldn't boot at first, but I ran the automatic repair from the Windows 8 DVD and that fixed it right up.

One limitation with ShadowProtect is that you can't do a restore onto a hard drive smaller than the original one, only the same or larger. This is a consequence of the backup image being a full disk image and not a set of files. (You can of course mount a backup image on a running system and copy files from it regardless of disk or partition size.)

Also, it doesn't provide disk cloning, only image backup and restore. Here there are more good options; my favorite right now is Drive Copy 12 Professional from Paragon Software [2]. (After you make their CD and boot it, it launches a simplified wizard environment; check the bottom of the screen for the "advanced launcher" where all the good options are.)

[1] http://www.shadowprotect.com/

[2] http://www.paragon-software.com/home/dc-professional/

Macrium Reflect: http://macrium.com/ I haven't had occasion to use its recovery features yet, but works great as a disk backup solution.
Not entirely sure what Ghost does, but... dd?
Ghost is like dd, but with filesystem knowledge (i.e., so it won't dump block belonging to deleted files, and can restore do a different size disk). Somewhat like what dump/restore is on Unix file systems, but of course with Dos/Windows support.
>Ghost is like dd, but with Microsoft Windows filesystem knowledge,

And limited or no support for other filesystems. It was very convenient if you were a MS-only, and could be troublesome if you tried to do anything fancy with another OS. In versions after 2003 Norton began adding some support for other filesystems.

imagex from Windows AIK (or windows ADK if you want win 8 version)
Windows Image Backup and also disk2vhd ( http://technet.microsoft.com/en-us/sysinternals/ee656415 )

I love the VHD format!

Also wim is a nice choice (created by the imagex.exe in the Windows Preinstall Environment based AIK toolkit, also used by Vista and Win7 setup): Not block level based, but contains all ACLs etc. needed to deploy an image

I've used Drive Snapshot for years with great success. XP, Win 7, 2003, 2010, etc. It is my go-to tool when migrating disk images. It handles live snapshots and differential images very well when working with Exchange, etc...

http://www.drivesnapshot.de/en/

True Image is a good commercial alternative (now it's a full fledged suite of products)
Seconded. It does quite fast differential backups and gives you direct access to individual files in the backup.
DriveImage XML (for Windows cloning) - http://www.runtime.org/driveimage-xml.htm
I've had good results with Casper (the "Friendly Ghost"?) from Future Systems. Does what it says on the proverbial box.
http://dar.linux.free.fr/ can help, with some scripting.
check this out, if you like g4l:

http://www.feyrer.de/g4u/g4l.html

If using NTFS, you can't beat imagex.exe.
Sure you can. Per Microsoft,

  ImageX currently does not support the following NTFS features:

    - Extended attributes.
    - Object IDs.
    - Reparse points that are neither symbolic links nor junctions.
      ImageX will fail to apply them.
    - Sparse files. (They can be captured and applied, but they are
      no longer sparse after they are applied.)
A decent disk cloning utility should support all these things.
Fascinating, didn't know all that. But I guess for basic imaging these aren't a big deal. It's what the Windows installer has used since Vista.
dump(8)/restore(8)