[cogman10]

The most surprising allocation pressure I constantly run into is primitive boxing.

The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.

[hinkley]

The especially nasty thing about that boxing pressure is that when I still paid attention to the JVM, they had some sort of cache for boxing of numbers under about 128, which meant that if you did tests with toy inputs, identity checks would pass in code that needed an equality check instead, and benchmarks built the same way would see no GC pressure in those call trees but see quite a lot of pressure in production. I remember being really mad, almost to the point of a sense of betrayal, when I learned that. It's kind of a DieselGate but with more plausible deniability. You can't say it was deliberate, but if you already had trust concerns with that person it would move the dial farther into the red.

[throawayonthe]

isn't that kinda how python preallocates integers -5 to 256? https://docs.python.org/3/c-api/long.html#c.PyLong_FromLong

[hinkley]

Sounds like it. I wasn't familiar with how python does this but it looks like the identity operator is 'is' so while someone could definitely screw that up, it would stick out more in a code review than == versus ===

[drob518]

Yep, and sometimes just a small code change can flip on boxing.

[cogman10]

One of my least proud (most proud?) hacks when working with very large data sets is something like this

    Map<Integer, Integer> intCache = new HashMap<>();
    
    while (loading) {
      Integer feild1 = intCache.computeIfAbsent(getField1(), (i)->i);
    }

This is a terrible thing that shouldn't be as useful as it is to us... but it is really useful. We have a bunch of objects that can optionally have Integer values (hence a null is valid) but those int values are frequently the same.

This saves a bunch of memory and ultimately GC pressure as a result.

Valhalla can't come soon enough for us.

[drob518]

I do something similar for Java Time local dates. Financial data in particular has lots of redundant date info and benefits from being memoized. Converting to epoch millis also works.

[hinkley]

There was a generation of date processing in Java when the class wasn't reentrant. The API was functionally flavored though, so people would instantiate one instance configured how they liked and reuse it across all calls. You would only see this problem under heavy load, such as in production, and it basically took encountering someone for whom it had already happened to spot what was going on. I think I found out from the Java issue tracker, thanks to some other dev filing an accurate bug report, but then I was that person for others at a full handful of other jobs afterward. Everyone was surprised, as one would be. They eventually fixed it but that was busted for a long long time.

[actionfromafar]

If I squint, is this a special kind of heap compression?

[cogman10]

Not really heap compression or special, it's just reusing a reference to an object already allocated on the heap.

Right now, if I do this

    LocalDate a = LocalDate.of(2020, 1, 1);
    LocalDate b = LocalDate.of(2020, 1, 1);

A and B reference 2 different object allocations on the heap even though they are the same date. a != b.

In Java, that can be pretty expensive even for an object as light as a LocalDate. By running the cache and doing

    var cache = new HashMap<LocalDate, LocalDate>();
    LocalDate a = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i);
    LocalDate b = cache.computeIfAbsent(LocalDate.of(2020, 1, 1), (i)->i);

Now you have the situation where `a == b` and you immediately end up dropping the object allocation for b on the next GC.

The technique works best when you have a lot of repeated objects which are immutable. It is also only really needed because Valhalla isn't here. Once "value types" become a thing, then the representation for `LocalDate` inside the JVM can become just the fields and not a reference. The JVM is also free to do the sort of de-duplication optimization all on it's own for larger objects.

[actionfromafar]

Now I understand what Valhalla is (new JVM vesion!), I thought it was some kind of dark joke at first. Thanks for the explanation!

[drob518]

Right. And this is safe now since Java Time classes are immutable. That’s key. Before, you had to worry about someone modifying the object.

[to11mtm]

I mean FWIW this looks to me much like the sort of hacks I had to get into last time I worked with Java so I can't really blame you....

> Valhalla can't come soon enough for us.

This will be interesting to see for sure, I think it will raise the bar for competitors as well; .NET GC has lingered in some ways for a while in progressing [0][1], there is a long standing github discussion around a lower latency GC where there is a potential alternative [2] but nothing really signaling that it could be integrated in future as an option. Valhalla might finally put enough pressure on microsoft to do something about the lag in this space.

[0] - The inferred stackalloc stuff on the JIT level is awesome but I don't count that as improvement to actual GC

[1] - Pinned allocs took a long time and we still can't get aligned pinned allocs (have to manually pad instead)

[2] - https://github.com/dotnet/runtime/discussions/115627

[spullara]

can't wait for Project Valhalla, going into preview shortly