You don't have to know only the tools to use, but to know how to use them. Looking only at histograms only tells you what fills the memory, not how it does it. Exploring the object tree in the heap dump and thread dumps are way more useful.
Last month we used VisualVM to get rid of the OutOfMemory errors and to decrease the run time of an application from >110 hours to 40 hours. We found that a big 3rd party library was keeping some kind of undocumented "undo history" of every action, that could not be flushed.
Couldn't agree more. In a garbage collected language, object retention graphs and call stacks are everything when it comes to object lifetime. The garbage collector looks at them to figure out when it's time to clean up an object, so you need to be looking at them if you want to figure out why the GC isn't cleaning up that object.
If those features aren't being used then there's a good chance that the problem is in the chair, not in the profiler.
What? I'm sorry, but what? The point of profilers is to find memory leaks and performance bottlenecks while disregarding the performance of the app with the profiler running.
Who cares if the profiler is making the app run slower. You aren't going to run it all the time in production!
If you can't find the memory leak or performance bottleneck, either you are doing something wrong or the profiler is not giving enough information. Getting a stack trace of where those byte[]'s are being created would be a good start.
This was my thought as well. While memory profilers tend to be fairly useless in my experience, disregarding them simply because they degrade the performance of your application (and really, 3-4x in the case of YourKit is astoundingly good) is silly. Heavy use of a debugger will also degrade the performance of your application -- should we not use debuggers, even when they're the right tool for the job?
In my own experience I've had quite a bit of success with the valgrind family of tools when I was debugging memory leaks in a GSM message codec (written in C, not a managed language) I wrote a few years ago. I guess it depends on the nature of the code and memory leaks.
I've also had some success tracing memory use with VisualVM for the purpose of finding out where memory is allocated so it could be used more efficiently (both for performance and do prevent leaks).
You don't even need a stack trace per se, the profiler should show you where everything is rooted in memory, or you should at least be able to track it back item-by-item to see where they are being held for enough instances to get a general picture of where memory consumption is occurring.
You cannot run it in production at all. No product owner will tolerate such overhead. And if you cannot run it in production long enough, then how and when will you find your memory leak?
> In all described cases only instructions in "Getting started" or demo video were followed, as every newcomer would do. It is quite possible, that there are some more advanced techniques, which lead to more satisfying results.
From experience in other languages, profilers are perfectly capable of pointing the source of leaks. It's what they're for. They can do so by displaying a weighted object graph (drilling from byte[], in this example, using back references), and by taking stack snapshots of some percentage of allocations.
Profilers will not show you any back references. Memory dump analyzers will do. The point of that blog post was to show that profiler alone will not help you. You need that memory dump and back references and dominators etc.
I tried it for WebLogic monitoring. It looks good and has nice features (like transactional support), but we could not trust it for the PROD environments. It requires a java agent to be installed on the JVM, and we were not prepared to retest the environment at that level.
We ended up using JMX monitoring for most of the needed metrics.
Last month we used VisualVM to get rid of the OutOfMemory errors and to decrease the run time of an application from >110 hours to 40 hours. We found that a big 3rd party library was keeping some kind of undocumented "undo history" of every action, that could not be flushed.