Hacker News new | ask | show | jobs
by Jenya_ 1910 days ago
The problem solved by arena allocators here is to avoid many calls to malloc. System allocator is slow, especially in multithreaded environment. Instead of calling malloc 1000 times to allocate 1000 small pieces (like 16 bytes sized each), malloc is called once and then all the small pieces are carved from the big memory block.

The arena allocators are also popular in game development. Memory is allocated from the arena allocator during frame processing (16 ms), and all this allocated memory is deallocated at once at the end of the frame.

1 comments

Again, I don't understand why you wouldn't just allocate large arrays, especially knowing that excessive memory allocation is slow and going to lock on top of that (you could use jemalloc, but that is orthogonal here).

> Instead of calling malloc 1000 times to allocate 1000 small pieces (like 16 bytes sized each), malloc is called once and then all the small pieces are carved from the big memory block.

No one who has any idea about performance is saying lots of memory allocations are ok. It sounds like what you are doing though is allocating arrays but then using them for 'arena allocators' when you could just use them directly and use indices for ordering if you need that.

> The arena allocators are also popular in game development. Memory is allocated from the arena allocator during frame processing (16 ms), and all this allocated memory is deallocated at once at the end of the frame.

I would ask why the memory needs to be deallocated at the end of each frame. A C++ vector can be cleared without releasing its capacity. What you really want is to have all the memory that you need allocated once so it can be reused over and over easily. There is no reason to involve an allocator on a frame to frame basis of a video game unless you go over what you already thought you would need.

If you have 'millions' of 'memory managers', why not just use a few big arrays? It really seems like an over complication of a problem that doesn't need to be difficult. I'm surprised no one would take the time to clean that out if there are terabytes of memory used and multiple days for single program runs.

Lol, so many questions :). I myself only use linked list when I do not know in advance how many items I will get. Otherwise I do allocate an array.
I think these are the types of questions that should be asked long before run times end up being measured in days and memory usage is measured in terabytes.

Linked lists are good for when you need inserts and deletes in the middle of a list and traditional linked lists made from memory allocations and pointers are obsolete.

Even if you don't know how many items you will have in advance, allocating large arrays still works. You can always chain large arrays together. The strange thing is that you are saying you use linked lists, but you are allocating them out of arrays that are already allocated.