[EdSchouten]

With it being stateless, I was referring to just the part where I wanted to point out the difference: file handles are stateless, while FUSE's equivalent (nodeids) are stateful.

Regardless of whether it's an in-memory or persistent solution, the problem remains: fuse-t has little choice but to leak resources of the underlying FUSE file system, as there is no valid point in time in which you can issue FORGET operations. This means that any file system with some form of churn rate will leak memory.

Also note that delegation is effectively optional. If a server simply always replies with OPEN_DELEGATE_NONE, the client has to interact with the file as if it's stored remotely.

There is no need to implement SEQUENCE, by the way. macOS implements NFSv4.0, while SEQUENCE is part of NFSv4.1 and later.

[boulos]

> macOS implements NFSv4.0, while SEQUENCE is part of NFSv4.1 and later.

Whoa. That's a real mistake. v4.0 is widely considered a "oops, we shouldn't have released this". TIL!

[concerned_ctzn]

I don't understand the comment about leaking memory, nfs file handles don't have to be persistent. Besides, how many FUSE filesystems implement FORGET? Anyway this is the reason I went with NFSv4. I started the project with NFS v3 but then discovered many limitations: being stateless, no named attributes, questionable locking support, etc. So Eventually I dropped it and re-implemented everything on NVSv4.

[EdSchouten]

> I don't understand the comment about leaking memory, nfs file handles don't have to be persistent.

As long as the file the NFSv4 file handle refers to is still usable (i.e., linked into the file system), the NFSv4 file handle must remain usable. Note that this is not a universal requirement, but at least one that the macOS NFSv4 client enforces. It only implements FH4_PERSISTENT. This doesn't seem to be documented explicitly, but is somewhat revealed by this printf():

https://github.com/apple/darwin-xnu/blob/main/bsd/nfs/nfs_vf...

> Besides, how many FUSE filesystems implement FORGET?

Any file system that wants to remove files in the background (meaning: not by calling unlink() through the FUSE mount) must likely do proper refcounting on such files, and provide an implementation of FORGET.

For example, a file system that can give information on live football matches may want to remove files/directories belonging to matches that have already ended. In that case you want the node IDs to remain valid, but refer to files that have already been unlinked from the file system hierarchy. The FORGET operation allows you to determine when those files can be removed from the FUSE server's bookkeeping entirely.

Here is an implementation of FORGET that I wrote for Buildbarn:

https://github.com/buildbarn/bb-remote-execution/blob/master...

[concerned_ctzn]

I'm not sure I understand you point, if an inode gets removed, then GETATTR or LOOKUP would fail as it should unless you're talking about open files. In the latter case the inode will get removed when the last open handle to the file is closed

[EdSchouten]

Sure, LOOKUP would obviously fail, as the object is no longer present in the file system under any name. GETATTR is a different story. Consider this sequence of operations:

    $ mkdir foo
    $ cd foo
    $ rmdir ../foo
    $ stat .                                                                                    
    16777221 401089477 drwxr-xr-x 2 ed staff 0 64 "Sep  6 06:41:05 2022" "Sep  6 06:41:05 2022" "Sep  6 06:41:05 2022" "Sep  6 06:41:05 2022" 4096 0 0 .

Notice how we removed a directory, and were still able to obtain its attributes afterwards using GETATTR. In fact, I can even go ahead and modify some of its attributes using SETATTR:

    $ touch .
    $ stat .
    16777221 401089477 drwxr-xr-x 2 ed staff 0 64 "Sep  6 06:45:34 2022" "Sep  6 06:45:34 2022" "Sep  6 06:45:34 2022" "Sep  6 06:41:05 2022" 4096 0 0 .

So that's what FORGET is for. It allows the kernel to hold on to inodes, even if they have been unlinked from the underlying file system, regardless of whether they are opened or not.

I think it's important to realise that a FUSE nodeid _does not_ represent an identifier of an inode in the file system. Instead, they are identifiers of the inode in the kernel's inode cache. Every time the object is returned by LOOKUP, MKDIR, MKNOD, LINK, SYMLINK or CREATE, should their reference count be increased. FORGET is called to decrease it again.