Hacker News new | ask | show | jobs
by dmitrygr 1162 days ago
So much confusion in this discussion. Let me try to clarify (all of this is very very simplified).

1. NAND in SD cards has page size (minimum writeable piece) something like 4kB or 8KB, or maybe even 64KB. But as per spec SD cards must allow writes at 512 byte granularity. Aligning your file system clusters with these flash pages will allow SIGNIFICANTLY better performance and lower wear. How? Well in the CSD register (accessible using SD commands), the NAND geometry is given, so all it takes is reading that and creating a file system that is properly aligned.

2. Why not just mkfs.vfat? You can, but unless you do some of the above work yourself and some math, you’re unlikely to get the alignment right and will suffer because of it, especially on small or random writes. You CAN do it yourself. Just takes a little work.

3. What about my gopro, my phone, or $DEVICE-X that can format SD cards? Well, most device firmwares DO indeed do this correctly as the SD spec advises to do.

4. Why doesn’t the card just do it in firmware all by itself? Curiously sony memory stick does this. Why not sd? There isn’t a command in the spec for that and adding it now is too late - nobody will use it. Thus we have this tool and ones like it.

5. What is this about the secure area? The tool just says it doesn’t touch it. Almost nobody does. Costs a lot of money to license the SD security stuff. Nobody uses it. It was meant for securing MP3s on cards. Secure area exists on all cards (yes you are paying for flash you cannot use). Luckily it isn’t big. Special commands are used to access it. This tool and basically any device you’ll ever encounter in your everyday life do not use them. Lacking those commands, it is as if the secure area doesn’t exist.

6. Why dont windows/Linux/macOS do it right themselves? My guess? Separation of concerns. The formatting code there only formats a partition. For proper alignment you may need to start the partition on the right block as well. Simple example. Say in your fictional card, we have 4KB pages. That’s 8 512-byte sectors. So we want our first FAT cluster to start at a multiple of 8 (and be a multiple of sectors in size). Say we work out that our various FS Structures need 17 sectors. That means that we need the partition to start at sector 7. Thus we make the MBR do so. Wait! You might say that FAT already has “resevered sectors” value we can use for this. No MBR touching needed. Well, you’d be surprised how many devices/firmwares break with nonstandard values there. Also, what makes you so sure that mkfs.vfat can even find out how many sectors into /dev/sdb, /dev/sdb1 starts, to do the math right? For that matter, what makes you think it would even know to check for SD-card-ness of /dev/sdb?

1 comments

> Well in the CSD register (accessible using SD commands), the NAND geometry is given

Is this difficult to obtain (e.g., via ioctl(2)) and parse?

[edit] looks like it's readable from sysfs: https://www.kernel.org/doc/html/latest/driver-api/mmc/mmc-de...

Parsing is another matter. It would be really useful if e.g., parted could read/parse that and then automatically align created partitions!

Partition alignment also depends on the file system options you plan to use to format it. Eg: depending on cluster size you choose, the size of the FAT differs, which in turn shifts the start sector of the first cluster. So, really, partitioning and formatting NEED to be done cooperatively and not independently. That’s the purpose of this tool, really.
Yeah good point. Back in the distant past, parted manged the creation of filesystems as well as the partitions on which they are created. It's a shame that was all removed because this is exactly why it's needed!