[jonbarker]

It's not uncommon for up to 1/3 of usage and therefore the bill on VMs in the cloud to be consumed by garbage collection. So if you can rewrite it without a garbage collector, you can save money. A great book on this topic is "The Beast Is Back" by jetbrains. Advocating C++ in that case (written in 2015). If GC makes you more productive, that's good, but at some point rewriting things without GC makes sense.

[stcredzero]

So if you can rewrite it without a garbage collector, you can save money.

Save money over what? It's usually a lot easier to optimize memory use than to rewrite code. Going by the 80/20 rule, 80% of the memory pressure on the GC will be created by 20% of the code. So amortize the price of the rewrite over the expected lifespan of the app. Then compare this to, let's say, the cost of optimizing 10% of the app to eliminate 40% of the memory pressure. Then, also factor in the likely rate of bugs introduced in a rewrite compared to the optimizing GC and the cost of incurring, finding, and fixing those bugs.

Going by this analysis, I would suspect that in some environments where rapid iteration is key, progress is fast, and apps have short lifespans, it might be better to optimize GC instead of rewrite to eliminate it. I'd also expect that in other cases, it is better to rewrite to eliminate GC.

[ncmncm]

This analysis assumes that not using GC costs something. However, in modern C++ code, as in Rust code, you can root around as long as you like and not find any code outside low-level, standard libraries that does any memory management. Avoiding GC costs exactly nothing in progress or in iteration time.

So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.

[grumdan]

One point is that the rewrite itself costs developer time that could be spent on something else.

Aside from that, rewriting something in C++ based on code in a higher-level language with better abstractions might cost additional developer time, maintainability, and quality.

[staticassertion]

> One point is that the rewrite itself costs developer time that could be spent on something else.

But they're already advocating for a rewrite, just within the same language.

[stcredzero]

I thought someone was also advocating for rewriting something in a managed language with GC to a language like Rust.

[stcredzero]

Avoiding GC costs exactly nothing in progress or in iteration time.

This might be true in new development. The specific context in this discussion was rewrites.

So, the analysis of GC overhead is always going to pit cost X against cost zero, no matter how low you manage to get X.

Again, you're talking about new development. That's not going to fit everyone's situation.

[ncmncm]

OK, I get you.

The problem is that whatever level of GC overhead you start with, or achieve, it will be non-zero, and its actual magnitude, including typically big cache-footprint knock-on effects that show up attributed, in perf results, to mainline processing, will be practically impossible to estimate reliably without comparing against a rewrite.

So, instead, you generally have to say: we compared some similar(-ish) program Y that was rewritten and cut the number of server instances required to meet demand by 30%, 60%, or what-have-you. But, exactly for the reasons you cite, comparisons published are against performance under GC after that optimization has already been done, as much as was practical.

[stcredzero]

including typically big cache-footprint knock-on effects that show up attributed, in perf results, to mainline processing, will be practically impossible to estimate reliably without comparing against a rewrite

It sounds like you're most familiar with, and you're arguing from a situation where the cost/benefit tradeoff overwhelmingly makes efficiency the king. So sure, in that situation, don't use GC. Write code with those cost/benefit tradeoffs in mind. That's valid, probably awesome and wonderful in certain situations. However, that doesn't mean that those particular cost/benefit tradeoffs rule over everything, everywhere, and everyone else should follow suit.

that optimization has already been done, as much as was practical

Which could mean a lot of things. I think, for the sake of your argument, you're arguing that we can assume it means, "very close to maximum optimization." Sorry, but you can't make this assumption everywhere and always. It depends. As Matt Easton says: Context!

[ncmncm]

> that doesn't mean that those particular cost/benefit tradeoffs rule over everything, everywhere, and everyone else should follow suit.

Indeed, a very great deal of code running on servers is in Python or Ruby. That's the low-hanging fruit before you start talking about GC overhead in a compiled language. But of course most of that is somebody else's code. People tend to be interested in what it costs to run their own code, not the next office over's.

If you believe your own argument, it would be foolish to start a rewrite while easy GC optimizations still await. Not everyone chooses wisely every time, but it's the charitable assumption.

[icebraining]

How much is the usage consumed by malloc/free or equivalent?

[ncmncm]

malloc/free are not used in modern C++.

The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.

[jchw]

> malloc/free are not used in modern C++.

I don’t think that was their point.

> The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.

This is at least slightly misleading, though.

Obviously, memory comes from somewhere. You can amortize costs by not needing to allocate new pages often.

Minimizing the amount of memory consumed by an application dynamically is the only way to absolutely reduce the cost. There’s lots of ways to do this, and plenty of C++ and Go software aim for “zero allocations.” However, there is still actually allocations in many softwares with “zero allocations” because they still use the stack.

For deeply concurrent applications, the stack ends up being a lot of memory. If you reduce the amount of stack memory per fiber, it reduces memory usage initially, but then fibers are more likely to hit the guard page and allocate more stack.

There’s strategies to reduce dynamic allocations pretty much all over (even in GC’d languages like Go.) The fact is, though, avoiding it is much akin to avoiding the GC. In Go, its actually identical to avoiding the GC.

(As a note in post, I acknowledge that not every concurrent application uses fiber style concurrency, but I believe with minor adaptations this point still stands for many classes of applications. Fully avoiding OS allocations is possible, but it definitely isn’t the “default” for C++ apps.)

——

This isn’t to say your point is not correct at least for some viewpoint, but it’s not actually that simple, which is absolutely worth noting.

[steveklabnik]

They're not directly invoked, but that's still what's called under the hood.

[ncmncm]

In Rust as in C++. But the fact that the actual calls don't appear in your source code means you don't incur any programmer cost relying on them. Yet, where runtime cost would be a problem (typically, affecting latency on a hot path) it can be avoided entirely.

[lossolo]

> malloc/free are not used in modern C++.

I think using term "are abstracted away" is better choice here. You still allocate memory, dosen't matter if it's malloc/free, mmap/unmap or compiler is doing it for you. It cost time and space. Sometimes the cost is negligible but not always, depends on application.

[vardump]

> The fraction of runtime involving allocation and deallocation, at the level where they happen, is typically negligible. In servers, after program startup, it is often (and deliberately) exactly zero.

In that case you don't use std::string, std::vector... or any containers at all? Or anything else that allocates, directly or indirectly.

[weberc2]

C++ still needs to allocate and free memory too and for all we know naive C++ might spend more time in memory management code than a GC language.

[ncmncm]

There is a fair bit of C++ code in use. We have no need to guess.

And, the answer is that real programs in obligate-GC languages spend overwhelmingly more time in GC than C++ or Rust. Much of this time is spent waiting on cache misses, which are hard to track to the responsible bit of code.

[weberc2]

Do you have evidence for this? Specifically C++ and Rust tend to be written for applications where tight control over memory is necessary and so any sample like you're describing is going to be biased by these carefully tuned C++/Rust applications. Even the standard libraries for C++ and Rust differ considerably in allocation behavior from Java or Python--these languages are conventionally designed to allocate differently, but that doesn't mean that the GC is the problem. Further, different programming paradigms allocate memory differently and the distribution of these paradigms across GC languages and non-GC languages (or whatever terms you like) are almost certainly varied. There are lots of confounding variables to control for, and until you control for them you're pretty much just guessing.

[nicoburns]

I think it's pretty obvious if you use these languages. A lot of things that require you to heap allocate in Java or Python (like classes!) have stack-allocated versions in rust/c++. Which means that code which avoids slow heap allocation is much nicer than equiavlent code in Java that must avoid high level abstractions.

You're right that the GC isn't necessarily the issue. It's more the forced heap allocation which most GC languages come with.

[weberc2]

Agreed, although to pick a nit (because it's an interesting nit IMO) "Classes" is orthogonal to allocation. In C++ you choose where you allocate your class and in Java the escape analyzer chooses for you. C# and Go have a GC and (more or less) semantics for heap-vs-stack allocation.

[xfer]

That heap allocation is most likely either optimized away and allocated in stack/ allocated in minor heap which is same as bumping a pointer just like in stack!

[weberc2]

Only for Java, not Python. And in any case, the cache performance will still be worse, but that’s a more minor cost.

[vardump]

> ...for all we know naive C++ might spend more time in memory management code than a GC language.

That's actually true, as long as all things are equal. GC can amortize memory management cost. Of course at cost of more jitter.

Unfortunately many GC language users tend to do way more allocations as well, diminishing the advantage and even turning it into negative.

[codesushi42]

>we know naive C++ might spend more time in memory management code than a GC language.

That's not really true anymore after the introduction of smart pointers. Basically C++ implements reference counting to manage memory as a result.

[ncmncm]

Smart pointers make no difference to the time spent allocating and freeing memory.

But we know that GC always imposes huge costs that point benchmarks uniformly fail to reveal. Often the costs are tolerable, or even negligible. At issue is the set of recourses available when they turn out not to be.

[codesushi42]

In what regard? GC just amortizes the cost. But memory allocation doesn't fundamentally work any differently.

In fact, it can be worse in Java because you have no control over whether an object lives on the stack instead of the heap.

[ncmncm]

No. GC imposes expense in addition to actually managing the memory in use, or newly not. If that were not true, there would be no reduction in hardware footprint after a rewrite.

Make no mistake, rewriting is a huge expense, rarely embarked upon without readily demonstrated benefit. (Exceptions tend to be rewrites in Java for organizational / political reasons. But I digress.) Not spending the time, instead, adding features, and delaying new features until there is a place to put them, can dwarf the base cost of the rewrite. That rewrites are done frequently enough to be discussed tells you there are huge operational benefits available.

[codesushi42]

The OP mentioned nothing about organizational costs, I was talking purely about performance. The OP was incorrectly claiming you'd spend more time in C++ allocating memory, not less.

[weberc2]

Well said, although that's not the only thing at issue; the prominent tradeoff for that generally smaller (or less powerful) set of recourses is that you have easy, automatic memory management for the default case. I.e., you don't need to make decisions about how memory is managed unless you need to make those decisions. In C++ and Rust, you have to constantly make those decisions (which kind of pointer to use, where will it allocate, what happens to ownership, how will this affect callers, etc).

[wbl]

"We know". Where is the evidence?