Oak Ridge 'Jaguar' supercomputer is World's fastest

"Supercomputer modeling and simulation is changing the face of science and sharpening America's competitive edge," said Secretary of Energy Steven Chu. "Oak Ridge and other DOE national laboratories are helping address major energy and climate challenges and lead America toward a clean energy future."

To net the number-one spot on the TOP500 list of the world's fastest supercomputers, Jaguar's Cray XT5 component was upgraded this fall from four-core to six-core processors and ran a benchmark program called High-Performance Linpack (HPL) at a speed of 1.759 petaflop/s (quadrillion floating point operations, or calculations, per second). The rankings were announced today in Portland at SC09, an international supercomputing conference.

In 2004, DOE's Office of Science set out to create a user facility that would provide scientists with world-leading computational research tools. One result was the Oak Ridge Leadership Computing Facility, which supports national science priorities through the deployment and operation of the most advanced supercomputers available to the scientific community.

"Our computational center works closely with the science teams to effectively use a computer system of this size and capability," said James Hack, director of the National Center for Computational Sciences that houses Jaguar in the Oak Ridge Leadership Computing Facility.

Jaguar began service in 2005 with a peak speed of 26-teraflop/s (trillion calculations per second) and through a series of upgrades in the ensuing years gained 100 times the computational performance. The upgrade of Jaguar XT5 to 37,376 six-core AMD Istanbul processors in 2009 increased performance 70 percent over that of its quad-core predecessor.

Researchers anticipate that this unprecedented growth in computing capacity may help facilitate improved climate predictions, fuel-efficient engine designs, better understandings of the origin of the universe and the underpinnings of health and disease, and creation of advanced materials for energy production, transmission, and storage.

The Oak Ridge computing complex is home to two petascale machines. In addition to DOE's Jaguar system, the National Institute for Computational Sciences, a partnership between the University of Tennessee and ORNL, operates another petascale Cray XT5 system known as Kraken, which was ranked 3rd on the November Top500 list at a speed of 831.7 teraflops.

"The purpose of these machines is to enable the scientific community to tackle problems of such complexity that they demand a well tuned combination of the best hardware, optimized software, and a community of researchers dedicated to revealing new phenomena through modeling and simulations" said ORNL Director Thom Mason. "Oak Ridge is proud to help the Department of Energy address some of the world's most daunting scientific challenges."

Simulations on Jaguar have primarily focused on energy technologies and climate change resulting from global energy use. Scientists have explored the causes and impacts of climate change, the enzymatic breakdown of cellulose to improve biofuels production, coal gasification processes to help industry design near-zero-emission plants, fuel combustion to aid development of engines that are clean and efficient, and radio waves that heat and control fuel in a fusion reactor.

"The early petascale results indicate that Jaguar will continue to accelerate the Department of Energy's mission of breakthrough science," said Jeff Nichols, ORNL's associate laboratory director for computing and computational sciences. "With increased computational capability, the scientific research community is able to obtain results faster, understand better the complexities involved, and provide critical information to policy-makers."

Hack, a leading climate modeler, concurs. "The speed and power of petascale computing enables researchers to explore increased complexity in dynamic systems," he said. As an example he cited the world's first continuous simulation showing abrupt climate change, led by scientists at the University of Wisconsin and the National Center for Atmospheric Research. Run on Jaguar earlier this year, the computer's speed made it possible to publish the results by July in Science.

Through the Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program, DOE's leadership computing facilities at Oak Ridge and Argonne national laboratories will employ a competitive peer review process to allocate researchers 1.6 billion processor hours in 2010. In 2009 the Oak Ridge Leadership Computing Facility allocated 470 million processor hours on Jaguar through the INCITE program.

Scientists in industry, academia, and government have requested more than 2 billion processor hours on Jaguar for 2010. The six-core upgrade on Jaguar will enable Oak Ridge to allocate 1 billion processor hours. Equipped with unprecedented computer power, materials scientists can simulate superconducting materials and magnetic nanoparticles with greater realism. Climate scientists can improve accuracy, resolution, and complexity of Earth system models, and physicists can simulate quarks and explore masses, decays, and other properties of the fundamental constituents of matter.

Enlarge

Molecular dynamics simulation of confinement and dispersion of small molecules within carbon nanostructures, mimicking the dynamics of electrolytes in porous carbon materials. The visualization was generated by the SciDAC code VisIt. Simulation: Dr. Vincent Meunier, ORNL. Visualization: Jeremy Meredith and Sean Ahern, ORNL.

• Cray XT computer system
  
• Top500 rank: 1st for XT5 component and 16th for XT4 component
  
• 2.595 petaflop/s peak theoretical performance for the combined system (2.332 petaflop/s from XT5 and 0.263 petaflops from XT4)
  
• Superlative speed: 1.759 petaflop/s actual performance for the XT5 and 0.205 petaflop/s for the XT4 on HPL benchmark program
  
• 255,584 processing cores
  
• XT5: 37,376 AMD six-core Istanbul OpteronTM 2.6 gigahertz processors (224,256 compute cores)
  
• XT4: 7,832 AMD four-core Budapest OpteronTM 2.1 gigahertz processors (31,328 compute cores)
  
• InfiniBand network connects XT5 and XT4 components
  
• Cray SeaStar network interface and router
  
• System memory: 362 terabytes (almost three times that of the second largest system)
  
• Unmatched input/output bandwidth to read and write files: 284 gigabytes per second
  
• Sizable storage: Spider, a 10-petabyte Lustre-based shared file system
  
• Speedy Internet connections enable users to access Jaguar from around the world.
  
• High-end visualization helps users make sense of the data flood Jaguar generates.