Hacker News new | ask | show | jobs
by tychver 3132 days ago
500MB per worker is totally standard. What happens is a job causes a huge array or hash to be allocated, and after it‘s finished the memory can’t be returned to the OS due to heap fragmentation. Java does some crazy stuff with compaction. C programs typically try and internally allocate into arenas to avoid it.