Hacker News new | ask | show | jobs
by Out_of_Characte 2311 days ago
What an interesting concept. Good programmers always consider certain behaviours to be wrong. Memory 'leaks' being one of them. But this real application of purposefully not managing memory is also an interesting thought exercise. However counter intuitive, a memory leak in this case might be the most optimal solution in this problem space. I just never thought I would have to think of an object's lifetime in such a literal sense.

Edit; ofcouse HN reacts pedantic when I claim good programmers always consider memory leaks wrong. Do I really need to specify the obvious every time?

3 comments

Cleaning up memory is an antipattern for many tools, especially of the EVA/IPO model (input-process-output). For example, cp(1) in preserve hard links mode has to keep track of things in a table; cleaning it up at the end of the operation is a waste of time. Someone "fixed" the leak to make valgrind happy and by doing so introduced a performance regression. Another example might be a compiler; it's pointless to deallocate all your structures manually before calling exit(). The kernel throwing away your address space is infinitely faster than you chasing every pointer you ever created down and then having the kernel throw away your address space. The situation is quite different of course if you are libcompiler.
> The kernel throwing away your address space is infinitely faster than you chasing every pointer you ever created down and then having the kernel throw away your address space.

In this case you normally want to allocate an arena yourself.

> Another example might be a compiler; it's pointless to deallocate all your structures manually before calling exit().

And now the compiler can no longer be embedded into another application, e.g. an IDE.

It's a reasonably pragmatic way of thinking, but beware the consequences. One benefit of working with custom allocators is that you can have the best of both worlds. Unfortunately, custom allocators are clumsy to work with.

Solve the problem you have now, not the problem you may not have later. You can worry about that when the time comes, if it ever comes.

In the case of compiler, one solution would be to replace all calls to `malloc` with something like `ccalloc` that simply returns pieces of a `realloc`'d buffer which is freed after the in-IDE compiler has finished compiling.

"Throwing away" a bunch of address space also happens when freeing up an arena allocation, and that happens in user space. This means that you might sometimes be OK with not managing individual sub-allocations within the arena, for essentially the same reason: it might be pointless work given your constraints.
Is there is a way to tell Valgrind that a certain memory allocation is intentionally being "leaked", and should not produce a warning?
No, they did what good engineers do: They analyzed the problem and found a feasible and robust solution. Following rules without thinking is not what good programmers do. I’d argue that most problems of modern software development stem from this mindset, even when it’s a rule that should be applied 99% of the time.
All good until years later, a new team builds, unaware of the leak, same system into a longer range missile...
Right, but that doesn’t invalidate the previous decisions made at that time. And it’s not only a problem in software. Accidents can happen because engineers trade material strength to reduce weight only to find out that over a number of generations they slowly crept above the safety margin that was decide upon 10 years ago, resulting in catastrophic failure. I’m sorry I don’t have the actual story at hand right now, but it’s not unimaginable either way. The problem you described has more to do with proper passing of knowledge and understanding of existing systems rather than strictly adhering to a fixed set of best practices.
It is interesting what you can come up with if you rely on constraints in the physical realm to inform your virtual realm choices. I've been looking at various highly-available application architectures and came across a similar idea to the missile equation in the article. If you are on a single box your hands are tied. But, if you have an N+1 (or more) architecture, things can get fun.

In theory, you could have a cluster of identical nodes each handling client requests (i.e. behind a load balancer). Each node would monitor its own application memory utilization and automatically cycle itself after some threshold is hit (after draining its buffers). From the perspective of the programmer, you now get to operate in a magical domain where you can allocate whatever you want and never think about how it has to be cleaned up. Obviously, you wouldn't want to maliciously use malloc, but as long as the cycle time of each run is longer than a few minutes I feel the overhead is accounted for.

Also, the above concept could apply to a single node with multiple independent processes performing the same feat, but there may be some increased concerns with memory fragmentation at the OS-level. Worst case with the distributed cluster of nodes, you can simply power cycle the entire node to wipe memory at the physical level and then bring it back up as a clean slate.