[kbolino]

That's fair. If so, then you still can have things like drivers and HAL and so on too. However, there's no hard security barriers.

How do multiple processes actually work, though? Is every executable position-independent? Does the kernel provide the base address(es) in register(s) as part of vfork? Do process heaps have to be constrained so they don't get interleaved?

[jlokier]

There are many options. Executables can be position-independent, or relocated at run-time, or the device can have an MPU or equivalent registers (for example 8086/80286 segment registers), which is related to an MMU but much simpler.

Executables in a no-MMU environment can also share the same code/read-only segments between many processees, the same way shared libraries can, to save memory and, if run-time relocation is used, to reduce that.

The original design of UNIX ran on machines without an MMU, and they had fork(). Andrew Tanenbaum's classic book which comes with Minix for teaching OS design explains how to fork() without an MMU, as Minix runs on machines without one.

For spawning processes, vfork()+execve() and posix_spawn() are much faster than fork()+execve() from a large process in no-MMU environments though, and almost everything runs fine with vfork() instead of fork(), or threads. So no-MMU Linux provides only vfork(), clone() and pthread_create(), not fork().

[kbolino]

Thanks! I was able to find some additional info on no-MMU Linux [1], [2], [3]. It seems position-independent executables are the norm on regular (MMU) Linux now anyway (and probably have been for a long time). I took a look under the covers of uClibc and it seems like malloc just delegates most of its work to mmap, at least for the malloc-simple implementation [4]. That implies to me that different processes' heaps can be interleaved (without overlapping), but the kernel manages the allocations.

[1]: https://maskray.me/blog/2024-02-20-mmu-less-systems-and-fdpi...

[2]: https://popovicu.com/posts/789-kb-linux-without-mmu-riscv/

[3]: https://www.kernel.org/doc/Documentation/nommu-mmap.txt

[4]: https://github.com/kraj/uClibc/blob/ca1c74d67dd115d059a87515...

[zgs]

Under uClinux, executables can be position independent or not. They can run from flash or RAM. They can be compressed (if they run in RAM). Shared libraries are supported on some platforms. All in all it's a really good environment and the vfork() limitation generally isn't too bad.

I spent close to ten years working closely with uClinux (a long time ago). I implemented the shared library support for the m68k. Last I looked, gcc still included my additions for this. This allowed execute in place for both executables and shared libraries -- a real space saver. Another guy on the team managed to squeeze the Linux kernel, a reasonable user space and a full IP/SEC implementation into a unit with 1Mb of flash and 4Mb of RAM which was pretty amazing at the time (we didn't think it was even possible). Better still, from power on to login prompt was well under two seconds.

[inkyoto]

> The original design of UNIX ran on machines without an MMU, and they had fork().

The original UNIX also did not have the virtual memory as we know it today – page cache, dynamic I/O buffering, memory mapped files (mmap(2)), shared memory etc.

They all require a functioning MMU, without which the functionality would be severely restricted (but not entirely impossible).

[jlokier]

Those features don't require an MMU.

The no-MMU version of Linux has all of those features except that memory-mapped files (mmap) are limited. These features are the same as in MMU Linux: page cache, dynamic I/O buffering, shared memory. No-MMU Linux also supports other modern memory-related features, like tmpfs, futexes. I think it even supoprts io_uring.

mmap is supported in no MMU Linux with limitations documented here: https://docs.kernel.org/admin-guide/mm/nommu-mmap.html For example, files in ROM can be mapped read-only.

[inkyoto]

That is not how a VMM subsystem works, irrespective of the operating system, be it Linux, or Windows, or a BSD, or z/OS. The list goes on.

Access to a page that is not resident in memory results in a trap (an interrupt), which is handled by the MMU – the CPU has no ability to do it by itself. Which is the whole purpose of the MMU and was a major innovation of BSD 4 (a complete VMM overhaul).

[jlokier]

You're right about the VMM.

But three out of those four features: page cache, dynamic I/O buffering and shared memory between processes, do not require that kind of VMM subsystem, and memory-mapped files don't require it for some kinds of files.

I've worked on the Linux kernel and at one time understood it's mm intimately (I'm mentioned in kernel/futex/core.c).

I've also worked on uClinux (no-MMU) systems, where the Linux mm behaves differently to produce similar behaviours.

I found most userspace C code and well-known CLI software on Linux and nearly all drivers, networking features, storage, high-performance I/O, graphics, futex, etc run just as well on uClinux without source changes, as long as there's enough memory, with some more required because uClinux suffers from a lot more memory fragmentation due to needing physically-contiguous allocations).

This makes no-MMU Linux a lot more useful and versatile than alternative OSes like Zephyr for similar devices, but the limitations and unpredictable memory fragmentation issues make it a lot less useful than Linux with an MMU, even if you have exactly the same RAM and no security or bug concerns.

I'd always recommend an MMU now, even if it's technically possible for most code to run without one.

[saati]

The original UNIX literally swapped processes, as in write all their memory to disk and read another program's state from disk to memory, it could only run as many processes as many times the swap was bigger than core, this is a wholly unacceptable design nowadays.

[inkyoto]

It also supported overlays on the PDP-11, although not in the beginning. I do not think that anybody makes use of the overlays anymore.

[int_19h]

In an embedded scenario where the complete set of processes that are going to be running at the same time is known in advance, I would imagine that you could even just build the binaries with the correct base address in advance.

[actionfromafar]

A common trick to decrease code size in RAM, is to link everything to a single program, then have the program check its argv[0] to know which program to call.

With the right filesystem (certain kinds of read-only), the code (text segment) can even be mapped directly, and no loading into RAM need occur at all.

These approaches saves memory even on regular MMU platforms.