The cost of an object

It would seem that Brian Goetz’s comments on object allocation in Java has stirred up some controversy. The latest entry being from Markus Kohler of SAP. Normally I’d commend directly on the blog entry but sorry Markus, I’ve really no interest in subscribing to your blog just to make a quick comment.

That said Markus does rightfully claim that object allocation in Java is very cheap and it is the cost of garbage collection that creates the problem. As is the case in matters such as this, both Brian and Markus are right.... and wrong at the same time. The reason being that garbage collection *can be* expensive. This doesn’t mean that it has to be expensive. However the expense is not so easy to calculate as there are many extenuating variables.

First point, garbage collection in young generation is very inexpensive. The trick to tuning most garbage collection problems is to try an catch as much of it in young as you possibly can. This is because the cost of a collection in old *can* be very expensive. While it certainly will be more expensive than a collection in young, a significant part of the total cost of a collection in old is having to deal with survivors. Think of it this way, garbage collection is a process that run to failure. In this case we define failure as the inability to collect an object. So recognizing a dead object is cheaper than recognizing that an object is alive. However that isn’t the end of the story. There are also compaction costs that you don’t see in Eden. The Survivor spaces resemble hemispheric GC and as such compaction happens naturally. Cleaning Eden and the dormant survivor is as easy as wiping the slate clean. You can’t do this in old space.

There are numerous other tricks that are used to reduce the strength of the GC process but most of them work on the premise that the collector is collecting. If that isn’t the case then things go off very quickly. All of this suggests that pooling objects will hurt GC performance. And wouldn’t you have it, I have a couple of benchmarks that demonstrate that point very nicely (they will be published in a new series of articles that I’m writing).

So now the point that Brian where Brian is wrong. I have another benchmark that I use in my performance tuning course to discuss architecture and design. The question asks, what are the limits to scalability (vertical or horizontal) and how would one go about fixing them. Well with the help of Cliff Click I managed to have the bench run on an Azul box containing 700+ CPUs. As written, the application scaled out to 200 CPUs before GC pauses halted any further performance improvements. Cliff looked at the code with the help of an Azul profiler and determined that one section was responsible for most of the garbage. A quick tweak later and now we were filling out 700 CPUs all doing *useful* work. This is a bottleneck that I would never have found on conventional hardware. That said, Azul is not using conventional GC technology but that is a different story. Point is, excessive object creation *was* the final limiting factor on scalability where even the almost pauseless Azul GC just couldn’t keep up.

If there is a point to this tale I believe it is; performance is about balance between trade-offs. In other words I am quite certain that Brian is all for the use of caching and object reuse. I’m quite certain that Markus wouldn’t suggest excessive use of pooling and object reuse. However both of these techniques are not free in that you will incur a cost for using them. The question becomes at what point does benefit out-weigh the cost? Another question is; are there techniques that I can use to reduce the costs?

For object creation the answer to the later question is to limit the scope of every object to the narrowest point possible. Strangely enough this performance best practice is also a design best practice.

So why do we typically use caching, pooling, and object reuse? There are several reasons among them are; pooling objects that are expensive to obtain so they can be reused. The most common class of these objects are those that access and utilize system resources. Caching query results is something that I would consider however only after obtaining some evidence that I can truncate several calls to the database by doing so. To do this requires that you measure the activity between your application and the database. Thread pooling is an effective way to calm and control traffic through your system. Clustering important data for the purposes of resiliency is yet another use case. What is missing from the equation is the effect it has on GC which I believe is more to Brian’s point.

tags: java  performance  azul  brian  goetz  cliff  click  markus  kohler