IBM and ETH Zurich unveil plan to build new kind of water-cooled supercomputer

Making computing systems and data centers energy-efficient is a staggering undertaking. In fact, up to 50% percent of an average air-cooled data center’s carbon footprint or energy consumption today is not caused by computing but by powering the necessary cooling systems to keep the processors from overheating—a situation that is far from optimal when looking at energy efficiency from a holistic perspective.

“Energy is arguably the number one challenge humanity will be facing in the 21st century. We cannot afford anymore to design computer systems based on the criterion of computational speed and performance alone,” explains Prof. Dr. Poulikakos of ETH Zurich, head of the Laboratory of Thermodynamics in Emerging Technologies and lead investigator of this interdisciplinary project. “The new target must be high performance and low net power consumption supercomputers and data centers. This means liquid cooling.”

With an innovative water-cooling system and direct heat reuse, Aquasar—the new supercomputer, which will be located at the ETH Zurich and is planned to start operation in 2010, will reduce overall energy consumption by 40%. The system is based on long-term joint research collaboration of ETH and IBM scientists in the field of chip-level water-cooling, as well as on a concept for “water-cooled data centers with direct energy re-use” advanced by scientists at IBM’s Zurich Lab.

The water-cooled supercomputer will consist of two IBM BladeCenter servers in one rack and will have a peak performance of about 10 Teraflops.

Each of the blades will be equipped with a microscale high-performance liquid cooler per processor, as well as input and output pipeline networks and connections, which allow each blade to be connected and disconnected easily to the entire system (see image).

Water as a coolant has the ability to capture heat about 4,000 times more efficiently than air, and its heat-transporting properties are also far superior. Chip-level cooling with a water temperature of approximately 60°C is sufficient to keep the chip at operating temperatures well below the maximally allowed 85°C. The high input temperature of the coolant results in an even higher-grade heat as an output, which in this case will be about 65°C.

The pipelines from the individual blades link to the larger network of the server rack, which in turn are connected to the main water transportation network. The water-cooled supercomputer will require about 10 liters of water for cooling, and a pump ensures a flow rate of roughly 30 liters per minute. The entire cooling system is a closed circuit: the cooling water is heated constantly by the chips and consequently cooled to the required temperature as it passes through a passive heat exchanger, thus delivering the removed heat directly to the heating system of the university in this experimental phase. This eliminates the need for today’s energy-hungry chillers.

“Heat is a valuable commodity that we rely on and pay dearly for in our everyday lives. If we capture and transport the waste heat from the active components in a computer system as efficiently as possible, we can reuse it as a resource, thus saving energy and lowering carbon emissions. This project is a significant step towards energy-aware, emission-free computing and data centers,” explains Dr. Bruno Michel, Manager Advanced Thermal Packaging at IBM’s Zurich Research Laboratory.

From the industrial side, the project is part of IBM's First-Of-A-Kind program (FOAK), which engages IBM's scientists with clients to explore and pilot emerging technologies that address real world business problems. It was made possible by the support of IBM Switzerland and the IBM Research and Development Laboratory in Boeblingen, Germany.

This liquid cooled supercomputer research is planned as a three-year collaborative research program called Direct Re-Use of Waste Heat from Liquid-Cooled Supercomputers: Towards Low Power, High Performance, Zero-Emission Computing and Datacenters, which is funded jointly mainly by IBM, ETH Zurich and the Swiss Competence Center for Energy and Mobility (CCEM). Part of the system will be devoted to further research into cooling technologies and efficiencies by scientists of ETH Zurich, ETH Lausanne, the Swiss Competence Center for Energy and Mobility, and the IBM Zurich Research Lab.

The computational performance of Aquasar is a very important part of the research. Aquasar will be employed by the Computational Science and Engineering Lab of the Computer Science Department at ETH Zurich, for multiscale flow simulations pertaining to problems encountered at the interface of nanotechnology and fluid dynamics. Researchers from this laboratory will also optimize the efficiency with which the respective algorithms perform within the system, in collaboration with the IBM Zurich Lab. These activities will be supplemented with algorithms of other research labs participating in the project. With this supercomputer system, scientists intend to demonstrate that the ability to solve important scientific problems efficiently, does not need to have an adverse effect on the energy and environmental challenges facing humanity.

Source: IBM