It's like caching, in kind but not in type. Once you add it, people will stop trying to be parsimonious with resources and just reach for the cache every time. They'll just lean into it. In a hot minute you will discover you can't turn it off because people have lost their brains and the data flow of the app is through the cache and not through the call tree.
If you tune for allocation patterns that are in the code, then you are cementing those as continuing in perpetuity. Better to cut the fat first, so that you can tune for the necessary complexity instead of the accidental. That will be self-correcting because any new misuses will be taxed with higher performance regressions.
I worked on Java code at AWS for a few years and nobody tried to optimize allocations. Then I changed jobs and started working on a Java MPP database and my first code review was brutal. You were expected to avoid allocations as much as possible (mostly by using the SoA pattern everywhere). At that scale no GC could save you from excessive allocations.
I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision.
People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough".
Well, for string, yes, but not for arbitrary objects.
The most surprising allocation pressure I constantly run into is primitive boxing.
The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.
The especially nasty thing about that boxing pressure is that when I still paid attention to the JVM, they had some sort of cache for boxing of numbers under about 128, which meant that if you did tests with toy inputs, identity checks would pass in code that needed an equality check instead, and benchmarks built the same way would see no GC pressure in those call trees but see quite a lot of pressure in production. I remember being really mad, almost to the point of a sense of betrayal, when I learned that. It's kind of a DieselGate but with more plausible deniability. You can't say it was deliberate, but if you already had trust concerns with that person it would move the dial farther into the red.
One of my least proud (most proud?) hacks when working with very large data sets is something like this
Map<Integer, Integer> intCache = new HashMap<>();
while (loading) {
Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i);
}
This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int values are frequently the same.
This saves a bunch of memory and ultimately GC pressure as a result.
I worked on projects where we indeed needed the manual stringbuffer refactor on hot paths in the code. It's not difficult work but boy is it tedious. Spent a lot of time with headphones on during those days. Only to work on another project a few years later where I paid for my sins by reversing the same change on someone else's codebase.
There's a version of Java, I can't recall which but I want to say 4? Maybe 3? Where someone rewrote parts of the Swing backend as native code to speed it up. Then Hotspot got good enough in the next version that the generated code was faster than the native code + FFI overhead. FFI was pretty high at the time. So they reverted the native code migration and went back to the old code.
In this post I look at a simple event to response latency benchmark, MarketDataSnapshot to NewOrderSingle at 50K/s for 30 minutes using JLBH to test Chronicle-FIX. The goal is to compare a system which is doing redundant work (in this case logging each message using SLF4J), compared with not logging (Chronicle-FIX records every message internally using Chronicle Queue) and how this changes the choice of Garbage Collector
Reading this gives me considerable pause - I can’t think of many classes within the codebase I work on that don’t have @Slf4j at the top…
Since there wasn’t a link to the source code in that post, can you help me understand this - for the SLF4J baseline is your logger impl a console appender, a file appender, or a network service like an OTel collector? Does any of that matter for GC context?
All common logging backends create a LogEvent or similar object for each logging call, and logging calls also typically construct new strings, which usually means a new StringBuilder object, its internal array (multiple ones if it grows), the final array it is copied to, and the String object that wraps that array.
These are typically short-lived objects and therefore cheap. Nevertheless, continually creating many such objects increases GC pressure, in particular if the logging happens in code that doesn't otherwise create many objects.
Cheap, not free, and even pretty simple to accidentally fool the GC on the lifetime of these objects.
Consider, for example, if you have a log message like this
logger.info("Hello {}", myOldObject);
if "myOldObject" is large enough or contains references to large things or has just been around for a while, it may be a part of OldGen at this point. And if that's the case, the LogEvent objects will end up automatically promoted to OldGen. Meaning the only time those can be be claimed is in an expensive major collection. The end result is that these things will ultimately fill up old gen and trigger more of the expensive old gen collections.
That's why it can be faster in some circumstances to write the more wordy
if (logger.isInfoEnabled()) {
logger.info("Hello {}", myOldObject.toString());
}
Nothing saves you, however, if your string being logged is too long. It can be autopromoted to old gen if you are trying to log a 10mb string.
> And if that's the case, the LogEvent objects will end up automatically promoted to OldGen.
Why do you think this would happen? There's no mechanism that makes young gen objects that reference old gen objects (or are referenced by old gen objects) get promoted faster. You have to survive a certain number of collections.
Ah shoot, you're right this wouldn't be what I'm thinking.
I was thinking of "GC Nepotism" [1]. That's the case where an object in old gen pointing to an object in new gen will automatically promote that new gen object into old gen. This can be particularly problematic with graph structures.
> All common logging backends create a LogEvent or similar object for each logging call, and logging calls also typically construct new strings, which usually means a new StringBuilder object, its internal array (multiple ones if it grows), the final array it is copied to, and the String object that wraps that array.
Which then gets discarded because that was a Log.verbose and your minimum log level in production is WARN.
Which is why many libraries have moved towards making your log message returned by a lambda. One constant lambda allocation (so, not a lot, an invokedynamic is absolutely fuck all.) that allows you to straight up skip allocating a full string that most likely is interpolating things and attempting to reach for context present on other threads is strictly better in 99.9% of the cases. The GC pressure is kept minimal and most importantly, constant.
> Which then gets discarded because that was a Log.verbose and your minimum log level in production is WARN.
This isn't true for the LogEvent or equivalent object, which only gets created after the log level is tested to be applicable by the logger implementation.
For call-site object allocation, you can wrap the logging call into an if statement that checks for the corresponding log level. The lambda allocation isn't constant if it captures anything from the surrounding scope, which will generally be the case for logging calls. (Unless by "constant" you mean that it's a single allocation per execution.)
> I don't like garbage. I don't like littering. My ideal is to eliminate the need for a garbage collector by not producing any garbage. That is now possible.
If you tune for allocation patterns that are in the code, then you are cementing those as continuing in perpetuity. Better to cut the fat first, so that you can tune for the necessary complexity instead of the accidental. That will be self-correcting because any new misuses will be taxed with higher performance regressions.