Yeah, mmap, I think this is the relevant line [1].
Fun fact, very early on, we used to create one mmapped file per serialized object, but that very quickly broke down.
Then we switched to mmapping one large file at the start and storing all of the serialized objects in that file. But then as objects get allocated and deallocated, you need to manage the memory inside of that mmapped file, and we just repurposed a malloc implementation to handle that.
Fun fact, very early on, we used to create one mmapped file per serialized object, but that very quickly broke down.
Then we switched to mmapping one large file at the start and storing all of the serialized objects in that file. But then as objects get allocated and deallocated, you need to manage the memory inside of that mmapped file, and we just repurposed a malloc implementation to handle that.
[1] https://github.com/ray-project/ray/blob/21202f6ddc3ceaf74fbc...