[panic]

People like the simplicity of the file/folder paradigm. That said, it may make sense to build your OS's file/folder abstraction on top of a more powerful database system. The ad-hoc, user-oriented nature of filesystems causes lots of problems when you try to build reliable software on top of them. There's no reason an app's internal network cache (for example) needs to be stored in the same way as a folder of user-facing documents.

[derefr]

I feel like the modern OSes have all developed a two-layer storage paradigm, where, from the kernel and base-userland's perspective, you just have an ordinary filesystem; and then, from the Desktop Environment's perspective, this filesystem gets combined with a content indexer to derive a more database-like system that tracks "documents."

And documents themselves can be databases (e.g. SQLite3 databases) in a way that's understood by this same database-like system, such that there can be a vertical integration between changes to "documents" that happen through Desktop-Environment-level APIs, and events elsewhere in the system.

(For example: the interaction in macOS/iOS between Core Data "documents" and iCloud. If you have a KV-store-typed Core Data "document", then iCloud backs it with a KV-store on its backend, and syncs changes between your KV-store and the cloud KV-store keywise, rather than just re-uploading the entire document whenever it changes.)

From a userland perspective, it's a bit hard to understand why these abstractions haven't been "pushed down the stack" into the base system, such that even POSIX utilities can see and deal with "documents" as a whole.

But from a systems-development perspective, I think I understand why it hasn't happened—modern OSes are developed from the inside out, in a way where development on the kernel or the base-system usually involves stripping away all the complexities of the higher level. A good example of this layering is Android: there's the Linux base-system, and then the Android runtime on top.

People who need to develop new kernel extensions for Android devices, need to work on them "at a Linux level", rather than trying to figure out what's going on by peering into the Linux part of the stack from the Android-runtime part of the stack. When they're doing that, none of the Android niceties are available. If the whole filesystem was vertically integrated such that you didn't have one before the Android runtime successfully booted, that'd be really annoying in this case. A filesystem is very useful for doing this kind of development. (Picture if Linux didn't have the initramfs abstraction—if, before the real root filesystem was mounted, there was just no filesystem at all, and so no POSIX userland utilities to rely on. That'd make developing a boot process much harder.)

[panic]

Yeah, this "two-layer storage paradigm" is exactly what I'm getting at. There are a lot of other ways you could slice it, though. System services want simple APIs with high performance -- they'd prefer not to pay the cost of the higher-level user document representation.

Personally, I think user documents shouldn't be a kernel concept -- it should be a protocol implemented by user-space system services. This would let any app expose its storage as "documents" without having to change its internal representation.

[snuxoll]

The concept is nice, but every time somebody has tried to implement it there has been some miserable failure either from a business or technical sense. See the Be File System for the former, and WinFS for the latter.

In fact, the only successful implementation of the concept (even if only partially) I've seen is the object-oriented storage on IBM i (formerly i5/OS and OS/400).