The benefit also isn’t as big as you might expect. Most of Linux’s recently found security vulnerabilities were due to ToC/ToU bugs. Fil-C would not magically fix these problems.
It would sometimes be a good trade off, for some users. But I don’t think many regular users would choose to pay this cost.
If you’re using SeL4, userland processes are already strictly sandboxed. There’s still some benefit to Fil-C, since the added memory safety would make it much more difficult to take over a process. But the blast radius of a compromised program in SeL4 is much smaller because of the capability model.