[metadat]

Parallel Filesystems aren't a new or novel concept, and there have been lots of implementations.

The first one I encountered was DrFTPD circa 2004. But these days, any object storage system qualifies because they all support varying replication schemes and reading from any valid in-sync replica.

[mrlongroots]

Now we are getting into the definition of what a parallel filesystem is.

In my book, a parallel filesystem is not just pooling together a bunch of nodes, but something that can actually support the synchronized accesses needed by a parallel workload. So not just decoupling between data and metadata, but scaling out of the metadata layer as well.

That and a hierarchical namespace (I could be sold on compromising some POSIX compliance for performance reasons, but it has to fundamentally be a hierarchical namespace with similar semantics). So object stores would not qualify.

[funcDropShadow]

Object stores are not filesystems. They have a paths, but they are not hierarchical.

[em-bee]

unless it's a hierarchical object store, like the one i am using.

[em-bee]

thinking more about this, the difference between a hierarchical and non-hierarchical storage i the existence of additional indexes representing the hierarchy.

in most filesystems files are addressed by an inode. and directories are just lists of inodes. if you remove those then you end up with a KV store: inode->file.

consequently i see no difficulty to convert a KV store into a hierarchy by adding the necessary tables/directories representing the hierarchy.

[funcDropShadow]

In a hierarchically system renaming a directory is an O(1) operation, in a non-hierarchical system with paths it is an O(n) operation.

[em-bee]

the KV store i am talking about doesn't use paths, it uses IDs. the paths are in a hierarchical directory that is added on top of the KV store, or alongside of it. the result is a filesystem.