[pritambaral]

> How do you have two processes writing to the same place in memory without memory mapping a file?

In increasing order of modernity: System V SHM, POSIX SHM, and `mmap(... MAP_SHARED | MAP_ANONYMOUS ...)` (e.g., on Linux).

[CyberDildonics]

These are still memory mapping files using file paths and returning file descriptors as far as I know, which makes sense because you have to have something coordinated between the two processes.

[pritambaral]

> These are still memory mapping files using file paths ...

No, they're not. The entire purpose of MAP_ANONYMOUS is to avoid using files.

Sources:

1. The Linux Kernel source code [1], where it comes with the code comment: "don't use a file".

2. The glibc source code [2], where it comes with the same code comment: "Don't use a file".

3. The Linux man-pages project documentation of mmap [3], where it is documented thus: "The mapping is not backed by any file; its contents are initialized to zero. The fd argument is ignored"

Similarly for SHM, but if you still don't get the point about MAP_ANONYMOUS, I doubt you'll get it for SHM either.

> ... and returning file descriptors

A socket is a file descriptor. An epoll handle is a file descriptor. On modern Linux kernels, a pid handle is a file descriptor. None of them are backed by "files".

> ... because you have to have something coordinated between the two processes.

FDs are not the only things processes can share, even if you go back to the venerable, original Unices, so I don't see what you mean.

----

[1] https://elixir.bootlin.com/linux/0.99.14r/source/include/lin... (the first version it was released in) to https://elixir.bootlin.com/linux/v5.16.1/source/include/uapi... (the latest version)

[2] https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/lin...

[3] https://man7.org/linux/man-pages/man2/mmap.2.html

[CyberDildonics]

This just goes back to the same question - what do two processes use to map the same memory into their memory space if it isn't a path to a file?

I'm not saying there isn't anything, I'm just seeing an extreme avoidance to an actual answer. The other guy went down a rabbit hole of syncing that memory to storage, which has nothing to do with anything.

[jstimpfle]

I'm starting to think you're even more confused than I had assumed. You were literally given a reasonable possible answer to your question multiple times (MAP_ANONYMOUS). And if there wasn't a big confusion you wouldn't be asking these questions in the first place because you could just make up your own answer.

I'm also left uncertain if you're assuming Linux and not talking about it. At least your objections to general statements are weirdly specific, while you never clarify the context (e.g. what OS you're talking about), and you seem to assume that there couldn't be other ways of achieving stuff. There seems to be a weird lack of understanding of the basics in your comments.

At the core, everything you need to share memory is that the participating processes agree about the (physical) address range of that memory (e.g. a 64-bit starting address and a 64-bit size). You could literally hardcode a physical address range, map this range to arbitrary (and possibly different) virtual address ranges in each of the processes, and start communicating through that shared memory. Note that the mappings are stored in the RAM and CPU, it has nothing at all to do with any files or filepaths.

And this whole discussion is completely pointless anyway because it started of YOU misunderstanding what I meant by "file-backed memory", which is not my fault at all. The term is completely unambiguous, it means (as opposed to POSIX SHM / MAP_ANONYMOUS / whatever) page cache memory that gets synced to an underlying file on a filesystem.

Please stop questioning and start experimenting and understanding what we're saying. We know what we're talking about. You don't.

[CyberDildonics]

"MAP_ANONYMOUS|MAP_SHARED mapped memory can only be accessed by the process which does that mmap() call or its child processes. There is no way for another process to map the same memory because that memory can not be referred to from elsewhere since it is anonymous."

https://stackoverflow.com/questions/4991533/sharing-memory-b...

misunderstanding what I meant by "file-backed memory"

No, it started by talking about using atomics for lock free interprocess communication, something MAP_ANONYMOUS can't do.

You hallucinated writing to storage as being part of this, didn't explain yourself and are getting upset about it. Atomic instructions that manipulate memory is orthogonal to what the OS does is the background. No one would think an operation on the order of nanoseconds has anything to do with writing permanent storage.

clarify the context (e.g. what OS you're talking about)

This thread is about mmap - it says it in the title.

it has nothing at all to do with any files or filepaths.

Two processes need some way to map the same memory and they do it through file paths.

[jstimpfle]

> something MAP_ANONYMOUS can't do.

> No one would think an operation on the order of nanoseconds has anything to do with writing permanent storage.

I literally just explained to you why with a file-backed mapping it is not nanoseconds but potentially an infinite time: https://news.ycombinator.com/item?id=29977672

> it has nothing at all to do with any files or filepaths.

I literally just explained to you that is doesn't have to be filepaths (and even with shm_open() POSIX standard, pedantically IT IS NOT A FILEPATH).

[jstimpfle]

> This thread is about mmap - it says it in the title.

I was asking what YOU are talking about. And also, this thread is actually about the approach of memory-mapped file I/O, not about POSIX mmap() specifically.

That's why I was (clearly) making statements that are not tied to any particular OS or platform, from the beginning.

[pritambaral]

> ... they do it through file paths.

>> ... or its child processes

[pritambaral]

Please just RTFM on `mmap(... MAP_SHARED | MAP_ANONYMOUS, -1, 0);`. There is no file path involved.

That's all I can say to you now, apart from "this is not StackOverflow".

[CyberDildonics]

But then that isn't interproccess communication.

If your boss said "we need these two programs to have lock free IPC through memory" and you said "use MAP_ANONYMOUS" they would say "that is local to the process tree and won't work".

You can try to ignore the context of this thread, but if someone wants IPC, this doesn't work.

[pritambaral]

> But then that isn't interproccess communication.

It is. It may not be _generic_ IPC, but it is IPC all the same. E.g., this is how postgres does IPC across its processes.

> that is local to the process tree and won't work

Isn't that what SHM is for? But, oh I see, you're willfully ignoring the fact that SHM keys _are not file paths_. So, yeah, I guess in _your_ world, non-file-backed IPC can't work.

> If your boss said ...

Sucks to be your boss, since _you_ don't get the fact that SHM keys and the filesystem are entirely separate namespaces.