Java Performance Services
Training, Seminars, Benchmarking, Tuning

Java Performance Tuning Course


Chania Crete, May 17-20, 2010


Sun Extreme Learning EXL-2025

Houston, December 1-4,2009
New York, December 8-11, 2009
Washington DC, January 5-8, 2010



San Francisco, January 11-14

Anti-if

I have joined Anti-IF Campaign

Calendar

««Nov 2009»»
SMTWTFS
1234
5
67
891011121314
15161718192021
22232425262728
2930

Performance Anti-Patterns

My Top Tags

                                       

Mailing List

My RSS Feeds








JDJ publishes yet another language comparison.

posted Wednesday, 16 June 2004
I see that the JDJ was published yet another article designed only to produce controversy in a vain attempt to drive traffic to the dreaded Sys-con/java website. Judging by my inbox, they've been successful. I fail to see the interest in 4 GL language comparison benchmarks. This is especially when one attempts to compare languages that have significant commercial success. The biggest problem is defining the benchmarks themselves. Though benchmarks are about the "real world", they themselves cannot be real world. They must ask a question and then answer it. They cannot take into account all questions, only the question that has been asked. One of the questions that you must ask when you are validating a benchmark is what is the question being asked and does this particular benchmark answer that question. The usefulness of a benchmark is a much more subjective measure. Just what is it that we asking when we ask is language X faster than language Y? If we assume that we are limiting ourselves to run time execution speed of the resulting binaries, then what we are really asking is which compiler will produce the most efficient set of machine code instructions. To answer this question, we are assuming that we will be starting with equivalent pieces of source code but can we really be sure of this. Languages are expressions or a notation convenience to express your thoughts about how to solve a particular problem. Some languages are more expressive in a particular problem domain. We may need to take entirely different approaches to solve the same problem using different languages. In these scenarios, how do we define "equivalent" source code? Given that some languages are better suited for some problem domains, we also need to ask, which problem domains do we need to consider. We can’t consider every problem domain and nor can we easily determine if source code written in different languages is equivalent for many problems. This doesn’t make the process of benchmarking less important or irrelevant, it just means that we need to understand the limitations of this tool. We need to understand that we cannot answer broad sweeping questions such as what was asked. You can ask is this best language for my domain based on benchmarking work that you may have done. You may even look at existing benchmarks to see if they are relevant. But don’t expect a definitive answer because they will only come from snake oil salesmen. One last point, doesn't faster hardware make this point moot? The problem is that we still cannot develop applications at the rate desired by those sponsoring projects. Certainly the largest gains to be had come from being able to express ourselves efficiently with our most important tool, language.



1. Jason Bell left...
Wednesday, 16 June 2004 9:33 am

Kirk if you are reading a publication for the sole point of picking holes in it, then you have completely missed the point. You have an option. Don't read. You are starting to sound like someone who has been burnt... There's no point walking into the fire.

I don't see Sys-con commenting on any of what has happened over the last month. They are showing business as usual. And if you are posting things like this then they full well know that you are reading their publication.

I don't read online or print materials anymore. I've got my own stuff to be getting on with. I keep an eye on blogs and they can be a bind as well.

I know the article was benchmarking type thing, but are we going to see ex JDJ staff just hammer JDJ articles for the sheer hell of it.

If you are going to create a new publication then put your money where your mouth is and get on with it. I know you can do it, it's just a case of doing it.

jaseb

Visit me @ http://jasonbell.blog-city.com


2. Kirk Pepperdine left...
Wednesday, 16 June 2004 10:35 am

Sorry Jason, but I think you've totally missed the point of the blog. The fact that JDJ published it is irrelevent. To say that one language is faster than another based on a few benchmarks is just plain silly. As for JDJ, you very well know that they have been digging for controversial subjects for the purpose of driving traffic. It would be much better if they could drive traffic with some useful bits of information but, then can't and we both know why.

As for being burnt by JDJ... I've not been burnt by JDJ, They've not done anything (yet) to me. If I wanted to sound like someone who was burnt, I'd be complaining about how the Sys-con not didn't appreciate my efforts and how the continue to my email. In fact, I'd be talking about how Fuat and others in Sys-con totally disgraded Joe's email. Oh my gosh, now I sound like someone who's been burnt!


3. Jason Bell left...
Wednesday, 16 June 2004 3:28 pm

Kirk, the second paragraph says it all. :)


4. a reader left...
Wednesday, 16 June 2004 4:09 pm

The most glaring difference between C++ and the Java was the MethCall test. Looking through the code I noticed what I think is the reason why.

Unlike Java, the C++ explicity frees up the memory used by the test every single iteration. The Java version like most other Java programs uses the garbage collection mechanism. And just because the memory is not being used, it does not automatically get deallocated on the spot.

I think if you see the maximum memory usage for the test you'd notice that Java is orders of magnitude larger than the C++ version which cleans up after itself.

Archimedes Trajano [trajano@yahoo.com]


5. a reader left...
Tuesday, 6 July 2004 9:33 pm

I think the point of the C++ vs. Java test was not intentionally for the memory usage. Those of us who have used Java and C++ for programming know that there's a give and take with the way both languages work.

C++ and Java naturally do memory differently. Java is a language that is based on the Smalltalk model, which implemented a garbage collector for memory management. The whole point is not to force Java to take care of the memory, but to let Java take care of that itself. Calling "System.gc()" to 'validate' a test is completely useless. That's like telling C++ to perform a few extra calls that are not necessary.

Java takes care of the memory itself, which makes the argument that "Java should free memory using System.gc() at odd places" a moot argument. C++, however, requires that you call new and delete on objects. Naturally, because it's not a garbage collected language.

If you want to argue about which language is better, and which is worse, then let's get to the argument about cross-platform.

You cannot write a program with one set of instructions in C++ and expect it to not only compile on one box, but also transport to other boes and work just like it did on your box. Java is the opposite. For the most part, I can take a Java program and compile it on one platform, take those JAR files and run them on any other platform that has a supporting JRE.

The argument that memory is the most important part of the test is totally irrelevant, and all of you are missing the point if that's all you're focusing on. The point is, the tests run on the languages are legitimate, and they show the performance of the libraries in question - not necessarily the language.

IMHO, a language is only as good as the libraries available to it. The performance is gathered directly from those libraries. If the libraries suck, chances are, the language will also.

Kenji Hollis [kenji.hollis@gmail.com]


6. a reader left...
Thursday, 15 July 2004 3:33 pm

IMHO, I agree with Kenji: with this kind of benchmarks your answer a question about how smart and skilly the libraries programmers were.

It's highly likely that you could get diferents result compiling the same C++ code with difentes libraries.

Barto


7. a reader left...
Wednesday, 28 July 2004 2:26 am

Java is better than C++ for all the good reasons:

1. There are more libraries in Java than in C++.
2. Libraries in Java interoperate easily, while this almost never happens in C++.
3. Compilation in Java is super fast when compared to C++. This means I can code 10 times faster, because refactorings in C++ are expensive.
4. Number 3 implies that writing huge applications in C++ is a waste of time and money.
5. The gap between Java and C++ is only widening.
6. Garbage collection in Java can now run in a concurrent thread, while other processors are still running some threads. C++ would need at least 100 years to get that feature.
7. Java used to call JNI to get more speed, but JNI was always slower than the same functionality in 100% pure Java. JNI now is only used to get access to the OS, but in the future most OSes will be written in Java.

JavaCoder


8. CppPro left...
Saturday, 5 August 2006 6:00 pm

I write software for investment banking trading apps. In C++ my module manages to precify 8000 vanilla option operations per second on a good desktop. In Java a similar module (actually simplified) hardly gets past 1000.

I realize that is not a scientific test but it is enough for me (and for the bank).


9. Antonio left...
Thursday, 12 October 2006 6:30 pm

Interesting grain of salt, CppPro. Java and C++ represent different toolsets, and mindframes, is a known fact. Even if the former is prevailing more and more (is it?), it is still not the best fit for every task. And most of all, complex tasks are still hard to tackle, however less than with C++.


10. Hero2Zero left...
Friday, 13 October 2006 6:38 pm

Lets not lose sight of the supposed "real issue" here. The argument has always been that C/C++ execution of code is just plain faster than the same code (or as close as possible given language differences) written in Java and ran inside the JVM. In the beginning this was undeniably true! Now, it's not so much. Java is faster at some things for various reason, C/C++ at some things for various reasons. On the whole, having used both languages, I tend to think it's currently a wash and therefore a moot argument. Now, can we please get back to bashing Micro$oft for the real faults in computerland!? Who's with me? LOL

-H2Z