Using NetBeans 6.0 to Find that Elusive Memory Leak

Last week I spent some time on the phone with Gregg Sporar, a NetBeans evangelist, looking at the new NetBeans 6.0 Profiler. Some time ago I’d given Gregg a chunk of code and asked him to find the memory leak using NetBeans. The only ground rule was; he was to avoid looking at the code. Some time later Gregg came back and showed me how he’d used NetBeans to locate the leak.

Though Gregg had found the leak, the process was less than satisfactory from my point of view. Gregg had used a lot of his vast experience with solving these problems in order to solve this problem. And, he had to dive into the code far earlier in the process than what I would have liked. Even though the code is trivial, it still took a few hours before he nailed it.

What I really wanted was a process that someone with far less experience than Gregg could follow. I also wanted to avoid the code can follow yet still be successful. Some of the answer to this problem is in better tooling. Fortunately that improvement in tooling can be found in Netbeans 6.0 M7. The two most promising features IMHO are generations and path to closest GC Walker. I’ll start with generations and show how it enabled Gregg to quickly find the leaking object.

Where as our age is computed by tallying the number of winters we survive, an objects age is calculated by the number of garbage collection cycles that is survives. So it stands to reason that if we look at the age of each object we should find that the ones that leak will be much older than ones that don’t. There are a few obvious exceptions to this. For example, data that is loaded at startup and is kept in memory for the duration. That said, a quick examination of a list of really old objects with some semantic knowledge should reveal objects that are leaking. As it so happens, NetBeans profiler provides you with that information.

The feature, called generations, provides the list of objects and ages that can be used to help determine just which object is leaking. Using this feature, Gregg found what was leaking in minutes. However, it didn’t tell him where the leak was. Without the heapwalker tool, Gregg was forced to dive into the code.

Just last week Gregg and I reloaded the other half of the problem, finding where the object leaking. For that we used Heap Walker found in NetBeans 6.0 M7. The first set in using heapwalker is to get a binary heap dump. You can do that using -Xrunhprof or simply triggering a dump using jconsole or jmap. Once imported into NetBeans, the profiler offers a view of all objects in the heap. From the classes view you can select an instance and from there see it’s reference information. From the reference information you can display the shortest path to a GC root.

What Gregg and I found when we did this was not only the source of the leak but we made a more interesting observation. The source of the leak was the only domain object between the GC root and the object that was leaking. After a quick back and forth I concluded that this wasn’t an accident and in fact it could be the basis for a process to help in the diagnosis of memory leaks. To test this out we quickly cobbled together yet another test case. In this case it was a leak on a the http session object. The problem was; the application was too simple to confirm the initial observation. IOWs we just didn’t have a domain object to show up between the leakee and the GC root. However the session state is an implied collection and it was very easily identified as the source of the leak.

Where does all of this stand? It seems that by using a combination of generations and heapwalker one should be able to quickly diagnose memory leaks. It also seems that there maybe a way to automate the diagnostic process just as deadlock detection has been baked in today. However, we only used the technique on a very small number of applications so it could all be a red-herring also. As soon as I get some time, I plan on trying out this more difficult case described by Attila Szegedi. I’m also interested in hearing about any experiences other have in using the new profiling features.

tags: netbeans  memory  leaks  java