[mknyszek]

Looking at only the CPU numbers from this benchmark is misleading. This site requires the use of default configurations for each language runtime, and JVMs tend to have a much larger default heap than the Go runtime. Tracing GCs tend to have a CPU/memory tradeoff built into them [1]. Compare the memory footprint of the best Go and best Java programs in terms of wall time [2] (the site doesn't make it easy, you have to go back and forth between the two links) and the difference is enormous (these Go programs are running with much smaller total heap sizes, so much less runway for the GC).

If you use GOGC and GOMEMLIMIT to even the playing field (and note, use a Go program that isn't using sync.Pool) the difference in wall time is far less stark (though it's still there, maybe 5-15%; don't quote me, it's been a long time since I measured this and I don't remember exactly). (The difference in total CPU time is bigger.)

And finally, keep in mind this benchmark is hammering as hard as it can on the GC. How it impacts real applications depends on how much the application relies on the heap.

[1] https://go.dev/doc/gc-guide#Understanding_costs

[2] https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

[igouy]

> [2] (the site doesn't make it easy, you have to go back and forth between the two links)

? Go binary-trees programs shown together with Java programs here:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

(But stuff on those pages will be moving around for a few days.)

[mknyszek]

I just meant that you cannot easily see wall time and memory use together on the same page. (I would love to be wrong about that.)

[igouy]

Are you reading on your phone? Portrait orientation may lose columns.

On the page you referenced:

— secs is elapsed seconds aka wall time aka wall clock

— mem is memory use

— gz is source code size

— cpu secs is cpu seconds

secs, mem, cpu secs as reported by BenchExec

[mknyszek]

Yep! I was on my phone, sorry about that.

Did it ever used to only show wall time? Or am I just completely misremembering?

[igouy]

It's been showing both for at-least 15 years (and before that claimed to be showing CPU secs, not Elapsed secs).

[gf000]

I can see a ~3x difference in memory usage in case of JIT-compiled Java vs Go, but AOT-compiled Java comes in at less memory usage while still being faster than Go. I believe Java AOT uses the Serial GC, while the "normal" version defaults to G1, so there is that (the GC code is actually reused between Graal and OpenJDK, so we could remove this "variable")

Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program.

And I deliberately linked this benchmark, as the topic at hand is the GC itself.

[mknyszek]

Fascinating. I could see the Serial GC, if it's generational, just totally crush this particular benchmark. I wonder what the heap size heuristic is for the Serial GC.

> Don't forget that the JVM has to allocate memory for all its subsystems as well, like the JIT compiler, so that 3x memory is not entirely heap usable by the program.

That's fair. I recall doing my due diligence here and confirming it is actually using mostly heap memory, but again it's been a while and I could be wrong. (Also if the actual heap size is only ~100s of MiB and the rest of the subsystems need north of a GiB, that's much more than I would have anticipated.)

> And I deliberately linked this benchmark, as the topic at hand is the GC itself.

Sure. Not trying to suggest the benchmark doesn't have any utility, just that even for just GC performance, it doesn't paint a complete picture, especially if you're only looking at wall time.

[gf000]

> That's fair. I recall doing my due diligence here and confirming it is actually using mostly heap memory, but again it's been a while and I could be wrong.

No, I think you are right - I'm not trying to claim that most of that 1.7 GB is used by the core JVM, just that that's another factor (besides simply the different base heuristics on how much space to claim).

The only fair thing would probably be to re-run the benchmark multiple times with different available RAM (via cgroup) and see the graph. Though I'm fairly confident that Java would beat Go in this particular benchmark at any heap size.