[dataflow]

WIM is the closest thing Windows has to a full file-based capture, but I've noticed that even that doesn't capture everything, unfortunately. I forget exactly, but think it was extended attributes that DISM wouldn't capture, despite the /EA flag. Not sure if that was a file format limitation or just a DISM bug.

[qwerty456127]

Very sad. Cross-platform extended attributes are the very thing I would love. I even imagine a new archive format which would be just a key-key-value (I mean it - two keys, a set of key-value pairs for every top level key - this is EA / NTFS streams) store with values compressed using a common dictionary (also possibly encrypted/signed with a common key). Needless to say such a format would enable almost any use case, especially if the layout of the file itself is architectured right. MacOS wouldn't have to add their special folder (they add to every ZIP) anymore, tagging files and saving any metadata about them would be possible, saving multiple versions of the same file, alternative names (e.g. what you received it with and what you renamed it to) for the same file.

I even dream about the days when a file main stream would be pure data and all the metadata would go to EAs. Imagine an MP3 file where the main stream only records the sound but no ID3, all the metadata like the artist and the song names are handled as EAs and can be used in file operation commands.

This also can be made tape-friendly and eliminate need in TAR. Just make sure files (streamms/EAs are written contiguous, closely-related streams go right near, compression is optional and the ToC+dictionary is replicated in a number of places like the beginning, the middle and the end).

As you might have guessed I use all the major OSes and single-OS solutions are of little use to me. Apparently I'd just use SquashFS but it's use is limited on Windows because you can hardly make or mount one there - only unpack with 7zip.

[mappu]

It's easy to forget about supporting EAs on Windows - they are extremely uncommon because you practically need to be in kernelspace to write them. Ntoskrnl.exe has one or two EAs, almost nothing else does.

(ADS are super commonplace and the closer analogue to posix xattrs.)

[qwerty456127]

I didn't know this, thanks. I thought xattrs and ADS are synonymous. Do SquashFS, ext4, HFS+ and APFS have ADS then?

I am looking forward to write my own cross-platform app which would rely on attaching additional data and metatata to my files.

"need to be in kernelspace" does not sound very scary because a user app probably doesn't need to do this very job itself - isn't there usually an OS-native command-line tool which can be invoked to do it?