[throwaway894345]

> The point of the btree example is to test how good programming languages are at allocating tree-like structures that can't be preplanned.

Forcing allocations for every node isn't justified by a desire to demonstrate dynamically sized binary trees. A naive dynamically-sized tree would just keep a list of node buffers and allocate a new node buffer every time the previous one fills up (perhaps with subsequent buffers doubling in size). The benchmark is, by all appearances, contrived to be slower.

[igouy]

> … a desire to demonstrate dynamically sized binary trees…

Cart before horse — the binary trees are justified by a desire to demonstrate memory allocation.

http://hboehm.info/gc/gc_bench/

[throwaway894345]

1. That’s plainly not the case here since other languages are allowed to use custom allocators

2. Why use a binary tree benchmark in the first place if you’re going to limit the implementation to certain naive implementations (and again, only for one language)? Why not just measure allocations outright or at least call the benchmark “allocator performance”?

3. Showing allocation performance doesn’t help anyone understand the actual performance of the language, which is transparently what everyone uses these benchmarks for. If they wanted a general idea for language performance they would allow trivial, idiomatic optimizations. A benchmark that shows allocation performance is worthless, and a suite of benchmarks that includes a benchmark for allocation performance but not GC latency is worse than worthless: it’s misleading because latency is the more important concern and it’s what these bump allocators trade in order to get their fast allocation performance.

[lalaithion]

Looking at the fastest Java, Haskell, Racket, OCaml, JavasScript, C#... they're all doing per-node allocation using the standard allocator, and all beating Go. The limit is not just for Go. I don't know why you think that Go is the only one being disadvantaged here.

[throwaway894345]

I believe this is all addressed in my original post: https://news.ycombinator.com/item?id=28293381. If you have specific questions/concerns, I'm happy to address them, but I don't see the point in repeating myself.

[igouy]

1. Please be specific.

2. Again, not limited to only one language.

3. You are allowed an opinion.

[throwaway894345]

1. Rust, C++, C

2. It is in practice, but regardless of the size of the cohort there’s no compelling reason for these limitations.

3. And you’re entitled to ignore reason. It cuts both ways.

[igouy]

1. Which program?

2. Again, not limited to only one language. You are allowed an opinion about what is or is not compelling.

3. As before.

[throwaway894345]

1. btree; see the Rust version which uses a bump allocator for example

2. Doesn't matter whether it's exactly one language.

> You are allowed an opinion about what is or is not compelling.

It's not a matter of opinion. The definitional purpose of benchmarks is to indicate something about reality; if you contrive rules that cause the benchmarks to deviate from reality, they lose their utility as benchmarks. I've demonstrated that the rules are contrived (i.e., they prohibit real-world, idiomatic optimizations), so I think we can say as a matter of fact that these benchmarks aren't useful.

Of course, no one can force anyone else to see reason (but I don't have any interest in talking with unreasonable people).