Hacker News new | ask | show | jobs
by discreditable 2969 days ago
This is doing it the hard way. Make the image a loop device:

   losetup /dev/loop0 /path/to/your/image.img
Now you can use /dev/loop0 like it's a normal disk. If you're not seeing your partitions (/dev/loop0p1, etc.):

    partprobe /dev/loop0
When you're done, detach the loop device:

    losetup -d /dev/loop0
I use this trick to transparently compress ddrescue images with brtfs: https://brashear.me/blog/2017/11/14/how-to-create-a-compress...
7 comments

If you have a full dd image of the disk, you can just do losetup -Pf /path/to/your/image.img and that will pick a loopback device and perform the partprobe on it automatically.
Thanks, not OP but that looks useful to me. For context, I've used the strategy in OP to debug a small linux system running on an SD card, if I have one version that has a bug and one that doesn't, and I don't know the difference, I can image them both, mount them and diff the whole system (i.e. with meld.) And it can be done with the same card over time. I'd like to figure out how to do this with VM disks as well but I've never had a pressing need.
> I'd like to figure out how to do this with VM disks as well but I've never had a pressing need.

Everything that you will (probably) need is here: http://libguestfs.org

If you happen to need something that isn't already included, the APIs make it pretty easy to build your own tools as well.

Conveniently, those tools are also already packaged up for your distro (unless you use some esoteric distribution, perhaps).

I commonly use loopdevs to create and partition bootable disk images for clonezilla. I like that I can create them sparse so that they take up less space.
I came here to say using the loop device just makes things much easier. No math to get your offsets and fat fingering the numbers this way.
Interesting I have used kpartx which uses device mapper instead of Partprobe.

Wonder why kpartx exists; will have to look into that. Maybe not everything can be partprobed.

I'm not sure. All I can think is that partprobe is provided by parted and I have noticed people tend to gravitate towards using fdisk or parted and ignoring the other. Notice the author of the article is an fdisk guy.

I might be mis-remebering but I think parted used to be considered lame before fdisk lagged to support GPT.

Personally I’d call myself an fdisk/gdisk person - I like the safety net those programs provide by staging disk changes prior to committing to the MBR/GPT, if I’m doing things by hand this is my preferred tool.

I don’t like how parted will automatically commit your changes as you go, though this is useful for scripting your changes and is my preferred method for disk geometry modifications when preparing maintenances etc

Right tool for the job and all that.

From what I remember partprobe has had a history of occasional unreliable behaviour under RHEL - but I understand it works properly under RHEL 7.3+

kpartx is an older tool that was used with 2.6.xx-era Linux systems. partprobe/losetup are the newer mechanisms for doing this.

A lot of StackOverflow questions have answers that use kpartx, and those answers are out of date.

There's just one thing I don't get with this approach. How do you make use of the space savings? You've explicitly created an outside disk image which is larger than the ddrescue image, and the ddrescue image is inside that larger file. So while the inside image is taking less space inside the mounted loopback volume, the outside disk image will always take the original amount of space. Do you trim the outside volume after you're done or do you somehow extract the lzo-encoded disk image after?
It's called sparse files, it's files with holes in them where no data is written to disk. You can have terabytes of files on small disks as long as they contain a high % of bocks of only zeros.
Can't believe I've gone so long without knowing about sparse files.
Millions of zeroes in a row tend to compress pretty well.
There’a actually an even easier way for many use cases. Just use libguestfs. One big benefit it has over the losetup process is that it works in unprivileged containers because nothing needs to be mounted.
You can make that even easier:

    # losetup -Pf /path/to/disk.img
    # mount /dev/loop0pX /mnt
Additionally, this method is not susceptible to race conditions when other loop devices are being configured.