[AnotherGoodName]

A well written bespoke function can beat a generalized function at a specific task.

If you have the resources to write and maintain the bespoke method great. The large database developers probably have this. For others please don't take this link and go around claiming mmap is bad though. That gets tiresome and is misguided. Mmap is a shortcut to access large files in a non linear fashion. It's good at that too. Just not as good as a bespoke function.

[dist1ll]

This paper isn't aimed at random developers, and it's not a criticism of mmap in general.

This is an appeal to core database engineers to stop using the wrong tool for the job.

[formerly_proven]

mmap can be handy but usually is not a good idea when you care about ACID properties. So it tends to be most useful outside databases.

[josephg]

Can you give some examples where mmap is useful?

[duped]

I once improved a parser's performance a huge amount (iirc, something like 500x) when parsing large (>1GB) text files by mmap'ing the files instead of reading them into a byte array. It's not a magic bullet but it was alright for that application.

Another technique that can only be done with mmap is to map two contiguous regions of virtual memory to the same underlying buffer. This allows you to use a ring buffer but only read from/write to what looks like a contiguous region of memory.

[dataflow]

If your data is likely to already be in the system cache, memory mapping can achieve zero copying of the data, whereas reading will perform at least one memcpy. So there can be a performance advantage depending on the usage pattern.

Also, I've never tested this, but I believe mapped files will get flushed as long as the system stays running. So if you only need resilience against abnormal termination rather than system crashes, it seems like a good option?

[amluto]

> Also, I've never tested this, but I believe mapped files will get flushed as long as the system stays running. So if you only need resilience against abnormal termination rather than system crashes, it seems like a good option?

Linux will not lose data written to a MAP_SHARED mapping when the process crashes.

But! Linux will synchronously update mtime when starting to write to a currently write protected mapping (e.g. one which was just written out). This means (a) POSIX is violated (IMO) and (b) what should be a minor fault to enable writes turns into an actual metadata write, which can cause actual synchronous IO.

I have an ancient patch set to fix this, but I never got it all the way into upstream Linux.

What you can do is mmap a file on a tmpfs as long as you trust yourself to have some other reliable process handle the data even if your application terminates abnormally. This is awkward with a container solution if you need to survive termination of the entire container.

[whartung]

It’s useful in a world of several processes sharing things. This is much less common today in a world of “single process” containers and VMs as well as monolithic processes using threads or async techniques.

However, Java can build a special library file of the core JRE classes that it can mmap into memory with the intent to speed up startup times, mostly for small Java programs.

Guile scheme will mmap files that have been compiled to byte code. You can visualize a contrived (especially today) scenario where Guile is used for CGI handlers, having the bulk of their code mapped, the overall memory impact of simultaneous handlers is much lower, as well as start up times.

The process model is less common today so the value of this goes down, but it can still have its place.

[fatboy]

One place I've seen it used was a lib by a guy called DHoerl for reading images that are too big to fit in memory (this was years ago on iOS).

A very over-simplified and probably a bit incorrect description of what it did was to create a smaller version of the image - one that could fit in memory - by sub sampling every nth pixel, which was addressed via mmap.

It actually dealt with jpegs so I have no idea how that bit worked as they are not bitmaps.

[otterley]

glibc itself uses mmap under the covers when doing malloc in certain situations. Granted, it's anonymous and not file-backed, but it's still proven to be performant. See, e.g, mallopt(3).