[duped]

Really what you'd like to see is a way to write the mount command for each file type (do one thing well) and another command to detect the file type and dispatch accordingly (probably similar to the `file` command), all in user space.

The only thing standing in the way of this today is that MacOS doesn't expose a user space file system API. You can do this on Linux, Windows, and BSDs today.

(No, file provider extensions don't cut it, Apple devs who read this, please give us a FUSE equivalent, we know it exists).

[stuaxo]

I want stuff to work like ZipMagic did in the early 90s/ early 2000s.

You could cd into zip files, they would act as directories and files at the same time.

I seem to remember Linus saying a file could act like a directory in Linux a long time ago too.

Though I don't think Linux has filters for the filesystem like Windows does so implementation might be more tricky.

[Groxx]

Does https://osxfuse.github.io/ cover this? Or is there some fundamental issue? (beyond "it's not built in")

[skissane]

Recent macOS versions do have a general purpose built-in API for user mode filesystems. That API is incompatible with FUSE. The big problem is it is undocumented and you need an entitlement from Apple to use it, and Apple won’t give you that.

Apple do have a publicly available API for cloud file systems (Dropbox-style products), but it makes a lot of assumptions which makes it effectively unusable for other use cases.

Then there are third party solutions like osxfuse. These have the problem that they rely on kernel extensions and Apple keeps on making those harder and harder, and is aiming to get rid of them; plus, they are all now proprietary licensed, albeit often with a free license for open source use.

One approach that does work without any kernel extensions or private APIs is to make your user filesystem an NFS server and then mount that. One competitor to osxfuse does that, but it also is proprietary

[ranger_danger]

According to the MacFUSE author, their specific approach is not actually undocumented:

> Apple has put it in an umbrella called "unsupported" (in the kernel interfaces section) ... either Apple will not take this interface away, and if they do, it will be to provide a better interface

http://preserve.mactech.com/articles/mactech/Vol.23/23.03/Ma...

[skissane]

Yes, that’s not what I’m talking about though

MacFUSE uses the kernel mode VFS API

I’m talking about the undocumented user mode filesystem LiveFS/UserFS/com.apple.filesystems.lifs API which was added in Monterey (macOS 12), and since Ventura (macOS 13) is used to implement the OOTB FAT and exFAT filesystem support. Using that requires private entitlements (e.g. com.apple.private.LiveFS.connection) which Apple (thus far) won’t give to anyone else

[ranger_danger]

What about the File Provider API?

[skissane]

It is designed for the cloud storage use case (Dropbox, Google Drive, etc) - it creates local copies of files and synchs them with remote ones. Not what you want to do in the general case.

[w10-1]

> it makes a lot of assumptions which makes it effectively unusable for other use cases

What use-cases are foreclosed and why?

[duped]

Well that requires a kext so it's a nonstarter, and fuse-t uses NFS which is extremely janky and unreliable on MacOS.

The fundamental issue is that macOS doesn't provide an API for this natively.

[skissane]

> fuse-t uses NFS which is extremely janky and unreliable on MacOS

I was wondering what issues you were talking about, and then I found this - https://github.com/macos-fuse-t/fuse-t/issues/45 - data corruption

> The fundamental issue is that macOS doesn't provide an API for this natively.

The API is there, Apple just doesn’t want to give anyone outside of Apple the entitlement that lets them use it. I don’t understand why Apple won’t.

Well, I understand that would require them to document it, ship public headers, and support it for external developers - but why not?

[duped]

> The API is there, Apple just doesn’t want to give anyone outside of Apple the entitlement that lets them use it.

If no one can call it it's not an API, it's an implementation detail. And I don't even think its exposed by headers, just alluded to by people who claim APFS is implemented in user space.

> I was wondering what issues you were talking about, and then I found this

Worse than this, it's possible to DoS a Mac with an NFS server just by refusing to reply to a request. That's unacceptable for a user space file system (although FUSE is only kinda better, in that it can force processes that read from the FS into uninterruptable sleep that prevents them from being killed).

> Well, I understand that would require them to document it, ship public headers, and support it for external developers - but why not?

Because Apple doesn't give a fuck about developers. Every developer will eventually learn this, but for those that haven't - Apple doesn't want you writing software for their platform, unless you're an Apple employee and on an Apple team paid to do it. It's why their docs suck, it's why to learn anything you need to watch ADC videos instead of read manpages, and it's why all the cool stuff is behind protected entitlements that you can't get or will be limited in using.

[mike_hearn]

No, it's almost certainly not because they don't give a fuck about developers. They definitely do.

It's much more likely that they want to:

a. Dogfood the API using internal use cases first when they can still make changes to the API without breaking anything. Note that the latest MacOS releases moved some filesystems into userspace using this new API. They probably learned some stuff by doing that.

b. Work out how to protect system stability from crappy userland filesystems. As you point out, bugs in FUSE providers can hang apps.

c. Work out how such an API interacts with their sandboxing system and how to avoid FUSE-style filesystems being used to subvert the sandbox. This is a common source of exploits in FUSE-style systems and is one of the key learnings from GNU/Hurd: UNIX software is written on the assumption that filing systems aren't malicious and invalidating that assumption creates new bug classes.

d. Work out what the most important use cases are and try to ensure those use cases will have a good or at least uniform UX first.

Providing a FUSE-like API is presumably also just not a high priority. By far the most common use case in terms of number of users is the Dropbox use case. FUSE is mostly used for toys and experiments beyond that (like filefs). Those matter and I'm sure there are friendly geeks on the Darwin team who'd like to enable those, but Linux also works for exploration. Certainly Apple management would not be happy about an engineer who decided to enable nerd experimentation but undermined the security system whilst doing so.

And it's worth remembering that you can have root on macOS. It means disabling SIP and adding a kernel boot arg, but that only takes a few minutes and then you can grant apps any entitlements you like:

https://github.com/osy/AMFIExemption

That's no good for people who aren't developers, but most FUSE filesystems are designed for developers anyway.

[skissane]

> Worse than this, it's possible to DoS a Mac with an NFS server just by refusing to reply to a request.

I wonder if their SMB/CIFS client implementation has these kinds of issues? It probably gets used more heavily

> And I don't even think its exposed by headers

Apple (accidentally?) released some of the private headers for this feature in one of their open source releases: https://github.com/apple-oss-distributions/msdosfs/blob/rel/...

[duped]

Maybe? It's kind of hard to tell. It's not exactly easy to write any of these servers from scratch to find out. But I wouldn't be surprised - they want app developers to be using the file provider extension API, which is unsuitable for everyone who isn't making a Dropbox clone.

That link is very interesting. It doesn't smell like any other Apple API as they're exposing a vtable with good documentation comments. It would be interesting to hack with this with SIP disabled to see how it works. I'm especially curious about how mount/unmount work and how the plugin registers itself with the OS, or what application is the client/host.

[nine_k]

> why don't they

It would make macOS more of a general-purpose OS, would increase the amount of functionality from which third parties would benefit, but Apple themselves would likely not. That would increase the number and variety of tech support requests, ever so slightly but still, and would introduce a few new attack surfaces.

Instead, Apple's strategy is to tighten the macOS more and more, and turn it into a specialist OS completely controlled by Apple, with a few companies like Adobe and Ableton licensing access to its internals.

[samatman]

I've been using OSX since 2003, and developing on it for more than ten years. At no point have I seen anything that it's reasonable to call "tightening macOS", let alone the absurd claim of complete control except for an inner circle of elite companies.

The closest thing would be adding the attestation system, so that unsigned binaries have to be explicitly given permission to run... once. That's a security feature which trades a bit of convenience for a lot of protection, especially for the average user. I have no problem with that sort of thing.

I see this sort of sentiment very frequently from non-users of the operating system, but never from those of us who actually use it. Go figure.

[skissane]

Apple used to be a lot more developer-friendly company. It is part of what got them where they are now - the fact that so many developers use Macs, which in turn encourages business software vendors to support Macs

Stuff like this is of little interest to ordinary users (at least not directly), but appeals to developers

By de-emphasising the developer is experience, they are undermining one of the factors that got them to where they are today