The standard technique is to reserve a big file on the old filesystem for the new filesystem metadata, and then walk all files on the old filesystem and use fiemap() to create new extents that point to the existing data - only writing to the space you reserved.
You only overwrite the superblock at the very end, and you can verify that the old and new filesystems have the same contents before you do.
I believe that is also the method [btrfs-convert](https://btrfs.readthedocs.io/en/latest/Convert.html) uses. A cool trick that tool uses is to keep the ext4 structures on disk (as a subvolume), which allows reverting to ext4 if the conversion didn't go as planned (as long as you don't do anything to mess with the ext4 extents, such as defragmenting or balancing the filesystem, and you can't revert after deleting the subvolume of course).
My conversion went fine, but there were so many misaligned sectors and constant strange checksum errors (on files written after the conversion). With the cherry on top being that if there’s more than X% of checksum errors, btrfs refuses to mount and you have to do multiple arcane incantations to get it to clear all its errors. Real fun if you need your laptop for a high priority problem to solve.
Lesson learned: despite whatever “hard” promises a conversion tool (and its creators) make, just backup, check the backup, then format and create your new filesystem.
I've never had the conversion corrupt a filesystem for me (plenty of segfaults halfway through, though). It's a neat trick for when you want to convert a filesystem that doesn't have much on it, but I wouldn't use it for anything critical. Better to format the drive and copy files back from a backup, and you probably want that anyway if you're planning on using filesystem features like snapshots.
Windows used to feature a similar tool to transition from FAT32 to NTFS. I'd have the same reservations about that tool, though. Apple also did something like this with an even weirder conversion step (source and target filesystem didn't have the same handling for case sensitivity!) and I've only read one or two articles about people losing data because of it. It can definitely be done safely, if given enough attention, but I don't think anyone cares enough to write a conversion tool with production grade quality.
I believe you are right. You can only convert back to the metadata from before. So any new or changed (different extents) files will be lost or corrupted.
So best to only mount ro when considering to rollback. Otherwise it's pretty risky.
No, it also covers the data. As long as you don't delete the rollback subvolume, all the original data should still be there, uncorrupted.
Even if you disable copy-on-write, as long as the rollback subvolume is there to lay claim to the old data, it's considered immutable and any modification will still have to copy it.
I understood it as "it doesn't touch/ignore the data". But I guess we mean the same thing.
You are right. All of the old files will be in areas btrfs should consider used. So it should correctly restore the state from before the migration.
Thanks!
This is a weird level of pedantry induced by holding many beers tonight, but I've always thought of "Hold my beer" as in "Holy shit the sonofabitch actually pulled it off, brilliant". I think it's perfectly fitting. Jumping a riding lawnmower over a car with a beer in hand but they actually did the math first. I love it.
It’s referring to a comment a drunk person would make before doing something extremely risky. They need someone to hold the beer so it isn’t spilled during what’s coming next.
{\off I think they used "hold my beer" correctly. It can be used for any weird idea, that a drunk person would actually try (usually with a stretch), regardless if they succeed or not. I don't think that "the SOAB actually pulled it off" is part of the usage.}
Apple did something like this with a billion live OS X/iOS deployments (HFS+ -> APFS). It can be done methodically at scale as other commenters point out, but obviously needs care).
You don’t need to look that far. Many of us here lived through the introduction of NTFS and did live migrations from FAT32 to NTFS in the days of Windows 2000 and Windows XP.
Yea thanks for recalling this. I totally forgot about that because I never trusted Windows upgrade, let alone filesystem conversion. Always backup and clean install. It's Microsoft software after all.
Craig Federighi on some podcast once said they conducted dry-runs of the process in previous iOS updates (presumably building the new APFS filesystem metadata in a file without promoting it to the superblock) and checking its integrity and submitting telemetry data to ensure success.
Apple doesn’t just deploy to the whole world in an instant though.
First it goes to the private beta users, then the public beta users, and then it slowly rolls out globally. Presumably they could slow down the roll out even more for a risky change to monitor it.
Sure, but still whoever wrote the patch had his ass on the line even shipping to a batch of beta users. Remember this is Apple not Google where the dude likely got promoted and left the team right after pressing click :)
"WinBtrfs is a Windows driver for the next-generation Linux filesystem Btrfs. A reimplementation from scratch, it contains no code from the Linux kernel, and should work on any version from Windows XP onwards. It is also included as part of the free operating system ReactOS."
Not sure what point you're making here. WinBtrfs is a driver for the same btrfs filesystem that Linux uses. It's most common use case is reading the Linux partitions in Windows on machines that dual-boot both operating systems
As someone who has witnessed Windows explode twice from in-place upgrades I would just buy a new disk or computer and start over. I get that this is different but the time that went into that data is worth way more than a new disk. It's just not worth the risk IMO. Maybe if you don't care about the data or have good backups and wish to help shake bugs out - go for it I guess.
The userland tools included with Windows are very lacking, but that's more of a distro problem than a filesystem problem. VSS works fine -- boringly, even -- and people take advantage of it all the time even if they don't know it.
The standard technique is to reserve a big file on the old filesystem for the new filesystem metadata, and then walk all files on the old filesystem and use fiemap() to create new extents that point to the existing data - only writing to the space you reserved.
You only overwrite the superblock at the very end, and you can verify that the old and new filesystems have the same contents before you do.