Edit: ninja'd but i'll leave what i said anyway:
Ah, yes but because C++ is compiled once, and cannot[or atleast would be... insanely difficult] change its source code at runtime [which java not only can, but does, constantly], it can make optimizations on a per-machine/per-situation basis.
Since java 6 or so, more and more comparisons are showing them as basically equivalent in speed, or with java being faster. As of version 6 - update 10, java took another leap forward with at runtime deciding what needs to be on the heap.
It's just its been very slow coming, and the amount of runtime optimizations the Java virtual machine could do are finally starting to make up for the fact that it has to go through the JVM layer [using just-in time compiled code]
Ironically, C# is using the same sort of optimizations to beat C++ in performance!
Basically, Java and C# can more effectively use the hardware because they can evaluate the hardware in runtime rather then being force to make educated guesses at compile time or more likely the time the programmers making the code.
Now, thats not to say C++ is slow, and if you program for a specific case on a specific piece of hardware you can certainly beat java and C# easily using the same algorithm.... that is if you assume you don't get more time to optimize in java and C# since in general you have to spend less time debugging them....
However, if you are really programming for insane speed on specific hardware, you shouldn't be writing it in C++, or even C, instead you should be writing it in machine code.
//But if your writing some general code for hardware that is unknown to you [ie, most games], you may want to seriously concider writing it in java or C# instead... [unless your working with people who are using C++ of course..] More and more C++ programmers are switching to C#.[microsoft is pushing C# pretty hard]
I wouldn't be surprised if C++ fades away a few decades down the road.