Hacker News new | ask | show | jobs
by felixge 263 days ago
I'd love to hear more! What kind of profiling issues are you running into? I'm assuming the inuse memory profiles are sometimes not good enough to track down leaks since they only show the allocation stack traces? Have you tried goref [1]?. What kind of memory pressure issues are you dealing with?

[1] https://github.com/cloudwego/goref

Disclaimer: I work on continuous profiling for Datadog and contribute to the profiling features in the runtime.
2 comments
tsimionescu 263 days ago
I for one am still mystified how it's possible that a GC language can't expose the GC roots in a memory profile. I've lost so many hours of my life manually trying to figure out what might be keeping some objects live, information the GC figures out every single time it runs...
link
felixge 263 days ago
Do you think the GC roots alone (goroutine stacks with goroutine id, package globals) would be enough?

I think in many cases you'd want the reference chains.

The GC could certainly keep track of those, but at the expense of making things slower. My colleagues Nick and Daniel prototyped this at some point [1].

Alternatively the tracing of reference chains can be done on heap dumps, but it requires maintaining a partial replica of the GC in user space, see goref [2] for that approach.

So it's not entirely trivial, but rest assured that it's definitely being considered by the Go project. You can see some discussions related to it here [3].

Disclaimer: I contribute to the Go runtime as part of my job at Datadog. I can't speak on behalf of the Go team.

[1] https://go-review.googlesource.com/c/go/+/552736

[2] https://github.com/cloudwego/goref/blob/main/docs/principle....

[3] https://github.com/golang/go/issues/57175

link
defraudbah 263 days ago
no, haven't heard of goref yet but will give it a shot!

usually I go with pprof, like basic stuff and it helps. I would NOT say memory leak is the biggest or most common issue I see, however as time goes and services become more complicated what I often see in the metrics is how RAM gets eaten and does not get freed as time goes, so the app eats more and more memory as time goes and only restart helps.

It's hard to call it memory leak in "original meaning of memory leak" but the memory does not get cleaned up because the choices I made and I want to understand how to make it better.

Thanks for the tool!

link
prerok 263 days ago
Sorry if this is a basic question but are you setting the GOMEMLIMIT?

Also, are you running the code in a container? In K8s?

link