I haven't hosted modded servers in many years but a while ago the advice was to avoid (4, 8] GB (might have been 6.5 instead of 8, I don't remember, but the point is there's a dead zone) because of something weird with the garbage collection. I think it had to switch to 64-bit pointers or something. Anyway the worse garbage collection outweighed the memory advantage. I personally never hosted with more than 4gb.
IIRC more memory also increases GC time so after a certain point the GC causes such annoying lagspikes it's often better to play with less.
I’ve done testing and at some point more memory just changes how long/large the GC cycles are. More memory can cause larger hiccups when mods are leaking memory.
I played with a few of them. A server I used to run had a leaky mod that would, when a base was large, leak so much that the GC would hit twice a minute with a 5 second stall. It didn’t matter what GC was used. I never found a solution.
IIRC more memory also increases GC time so after a certain point the GC causes such annoying lagspikes it's often better to play with less.