[evilotto]

Not even just the semantics, the performance is awful. Even when the fork is virtual (as any modern fork is) and there's no memory copying because it's COW, all the kernel page tables still need to be copied and for a multi-GB process that's nontrivial. That's why any sane large service that needs to fork anything will early on start up a slave subprocess whose only job is to fork quickly when the master process needs it.

[dexen]

>all the kernel page tables still need to be copied and for a multi-GB process that's nontrivial

Only in the pathological case where the large process is backed solely by the 4kb pages. The hardware has long now supported large pages - on x86 since Pentium Pro, if memory serves - and huge pages. The popular OSes (Linux 2.6+ and Windows 2003+) also do support large and huge pages. A 2GB process can easily be three pages: r/x code, r/w stack, r/w data (2gb). Granted, it gets a bit more complex if mmapped I/O or JIT are used, but since both are mature technology now, it's fine to point fingers at any inefficiency and demand better. Another caveat would probably be shared libraries loading at separate address ranges, which, IMO, is another reason to ditch shared libraries for good.

Contrary to popular wisdom, OS research is still relevant.

[temac]

You want to ditch shared libraries and mmap to map your big processes using GB pages to make fork fast again (despite it not being the main and only drawback)???

OS research might be relevant, and it's good that some people have wild idea, but honestly I doubt this one will go anywhere :P

[dexen]

Ah sorry, only want to ditch the shared libraries; added mmap is a later edition didn't realize it's unclear. Of course mmap is necessary.

[temac]

About shared libraries, I know that there is this line of thought considering them "evil" (well at least sufficiently to want to get rid of them); but I'm quite unsure about what a modern system would look like without them (although this is less a problem at the application level on e.g. Android, the system level is still extremely important)

With Spectre, proper process bounds (well, address spaces) are more important than ever -- and oh well even without that I'd still have cited them as incredibly important, in the sense that I'd rather have more than fewer. Given that, code reuse involves shared libraries, for several good reasons; the obvious one being not wasting RAM, but then there is the update problem (how to patch programs when security holes are discovered, especially if multiple parties are involved), and on top of that there is the cache pollution problem, which is related to the code duplication problem, and which is quite insidious because it is probably simultaneously hard to benchmark and very real (ambient loss of perf, just not in very hot paths, but this will still have an impact on the general perf of a system, quite like Spectre mitigations are having a big impact)

Now we could like address space boundaries so much that we would want to just use even MORE processes in place of shared libraries, but this obviously does not work for all services (and Spectre is biting us again because context switches are not cheap), plus if you take it to the extreme this makes systems extremely hard to design, and even bigger. This is part of the reasons we are using Linux instead of Hurd... (well Linux is too much in the opposite direction, but there are hopes that it will in the long term evolve toward a middle ground)

And anyway that does not fit the narrative at all of using more huge pages.

Now there are the usual radical ideas about how everything should be running on some kind of VM (sometimes even including the kernel), drastically reducing the amount of "native" code; but given the reality of our current systems that "everything" both relies on multiple VMs and I doubt it will tend to only one, nor should it (because of the monoculture this would induce). Plus the ambient perfs are still lower than native code, and TBH I don't expect that to change ever.

So, why and how would you like to get rid of shared libraries?

[pjmlp]

We are using Linux instead of Hurd due to manpower.

Most high integrity real time OSes are microkernels.

Interesting that you mention Android, one of the key points of Project Treble is using separate processes for drivers with Android IPC to talk to the kernel (including hardware buffer handles).

[temac]

Well in the end we are using any X rather than Y tech because of manpower, regardless of pretty much any other characteristics.

So let's put manpower kind of aside for the bulk of the dev (where thousands of man-years are needed for any big project) and look at what could actually be achieved with the very small amount of manpower bootstrapping those projects. At this point you understand that the manpower thing is only a convenient narrative, while the reality is that even at the early time, Linux based systems worked really better than Hurd based systems.

Because general purpose micro-kernels based systems are hard, and especially those with a design as ambitious as the Hurd. (When you start to want to strongly isolate FS from VM code, it even stops being just hard and starts to be really HARD.)

And this was even worse at the time for perf reasons (but perf reasons are still applicable even today, given the impact on mobile and datacenter workloads)

However, yes, I'm in favor of more isolation today, because for a shitload ton of drivers it literally won't make any difference whether or not you take 1us vs 50us if you need to execute once every few seconds. So it is retarded to the highest level to run in kernel space if you don't actually need it. Sadly, Linux is way behind on that subject today.

That being said, and back to the original subject, a microkernel or at least a less monolithic one won't really get us in the less shared-libraries direction if it just re-implements the same perimeter of features of a monolithic ones, nor in the huge pages everywhere direction...

[apta]

> Contrary to popular wisdom, OS research is still relevant.

Is it really popular wisdom though, or is it the opinion of one person and it got hyped up, much like the same hype happened on a subpar programming language that same person worked on?

[lloeki]

> That's why any sane large service that needs to fork anything will early on start up a slave subprocess whose only job is to fork quickly when the master process needs it.

I don't think that's (entirely) true. This is more because a large service with some potent master process will have said process Do Stuff(tm) that will involve opening files, threads, signal handling, or whatever things that need to be taken care of one way or the other when forking to a worker (or whatever other child) process. It's therefore much simpler to fork a master subprocess into a child spawner earlier on, when it has yet to do anything. You significantly reduce your chances of screwing up if you have nothing to clean up for.

[evilotto]

That's true, it's not the only reason. Dealing with threads and buffers and pthread_atfork and the associated heartbreak is a biggie also. But the performance is nothing to laugh at.

I just did a quick test, a 100mb process generally takes >2ms to fork, while a 1mb or less process takes 70us. It seems like its pretty much linear with process size.

[megous]

Does using hugepages help?

[evilotto]

It probably would, but I don't have a system I can test it on.

[zbentley]

The performance is awful, but in return you get the COW memory you mentioned. That's a pretty huge benefit for a lot of programs with huge, seldom-changing memory state at startup. If those programs want to parallelize themselves without duplicating that memory or paying startup time/CPU overhead, fork() is a pretty handy way to achieve that.

[nitwit005]

These days, you can usually start a process without forking through posix_spawn/vfork. Although, I gather some servers still do it so they can set the current working directory more easily.