New Benchmark Ranks Supercomputers More Thoughtfully

This is an interesting benchmark, and it is valuable as another benchmark data point. That's really all it is though. The problem with benchmarks is that they don't necessarily predict how the computer will run your problems. For instance, the Blue Gene architecture is great for a very specific class of problems that are very scalable. Cray's are great at graphing problems like this.

If you are working on the more blue collar types of jobs like structures, most CFD, etc. then you are better off with a more standard configuration. The reason is simple. The speedup of something like a Cray isn't worth the cost. A linux cluster will run you $100,000. Move to an SGI and you are up to $3M. Cray's start at $10M. Blue Gene is in the "Don't ask don't tell" category. Performance goes up as you spend more money, but it is generally more like 20% on each jump. Unless your problem is specifically suited to that machine.

The two benefits of this new benchmark are:

1) If you can predict the performance of your jobs based on this benchmark, then you can easily make comparisons of different machines. That's a big if.

2) Cray's have gotten a bad rap because they don't show good price/performance in linpack and other standard benchmarks. They can often get overlooked when you need to solve a Cray type problem.