1. The energy consumption in data centers becomes an increasingly crucial matter in terms of energy balance and costs. Energy efficiency moves in the center of attention. Which role play air-conditioning and cooling in using a data center more energy efficient?
 Dr. Michael K Patterson |
Dr. Michael K Patterson: After the IT equipment itself, the cooling system is typically the next largest energy user. It also is the easiest part of the data center to operate poorly. For example, the power delivery architecture may have some inefficiencies in it, but once it is on-line it pretty much just feeds power. The cooling systems have so many more variables in them that can be done poorly. But if we design it right and operate it right it is also the one place where significant energy savings can be had. |
 Dr. Rainer Weidmann |
Dr. Rainer Weidmann: I can’t emphasize this enough. At present, air conditioning accounts for 35-50% of the total power requirement of an efficient electronic data processing center. With less efficient systems, the PUE (Power Usage Effectiveness) can even reach the value 3. A very good electronic data processing center that uses no fresh air, only recirculated air, would have a value of around 1.7. In the DataCenter 2020, we have set ourselves the target of 1.3. When looking to save energy, the focus is primarily on air conditioning. This is in heavy contrast to uninterrupted power supply (UPS) units, where the level of efficiency is currently 92 percent. There is therefore only a small margin for optimization in this area: approximately 8 percent. |
2. Which approaches and methods of air-conditioning and cooling do exist that differ fundamentally?
Dr. Michael K Patterson: The first option is air-cooling vs liquid cooling; although liquid cooling is often misunderstood. It primarily means liquid to the rack-level, and not liquid to the component-level. There are so many different sources of heat in the platform; the CPU, memory, chipset, drives, power supplies, and voltage regulators – the CPU for example is generally only 30-35% – that to cool them all with liquid becomes prohibitively expensive and complicated. The one place where this might make sense is in High Performance Computing, where the interconnect requires very high density.
For air-cooling there are also many options, cooling with a central chilled water system vs a direct-expansion (or DX) cooling system. Then in the data center itself, the airflow management strategy makes a difference, this ranges from the simple hot aisle/cold aisle arrangementall the way to hot-aisle containment strategies. Also important is whether the data center has any economiser modes to allow for “free cooling” schemes.
Dr. Rainer Weidmann: There are a number of different approaches. The first of these to mention is operation with purely recirculated air (with or without indirect free cooling), which doesn’t require a supply of fresh air from the outside. Or direct free cooling, a type of cooling with outside air. Natural resources can be used too, for example surface water or well water. And we must not forget direct CPU cooling with liquids.
3. Which factors influence the energy efficiency of a refrigeration system in the data center?
Dr. Michael K Patterson: The overall efficiency of the cooling system has a multitude of parameters that can affect it, but most of these are part of the cooling system design and well removed from the data center. But in the data center itself there are some very important factors. First is airflow management. This matters in two ways: good airflow management will limit excess fan energy and will also provide warmer return air to the cooling systems. That warmer air will allow the cooling system to operate more efficiently. Second is running the data center at the right temperature. ASHRAE guidelines state that the IT equipment inlet can be as warm as 27C. Unfortunately, the cooling system return air is often measured rather than the supply at the IT equipment leading to overcooled data centers, so that needs to change. That wouldmake a huge difference in getting to a more efficient system. As for measuring the temperature at the server inlet, most Intel Architecture with today’s Intelligent Servers can tell you that temperature.
Dr. Rainer Weidmann: Absolutely right. The outside temperature is one of the main factors, as is humidity. The design also plays a part: is direct or indirect free cooling implemented? Important too is the Delta-T value, which indicates the temperature difference between return air and supply air. It is also crucial to maintain the temperature of the servers at the right level: this is a matter of selecting the correct inlet temperature. Airflow and current should be taken into account as well. Wherever possible, cold air and warm air should be prevented from mixing.
4. Please describe briefly the benefits of an air-cooled and liquid-cooled system
Dr. Rainer Weidmann: Cooling with outside air can prove to be extremely energy efficient, particularly during the colder months. However, controlling humidity is costly, and there is a risk of environmental influences such as air pollution being carried into the electronic data processing center. Just as challenging is the installation of outside air sensors to guard against such problems. With purely forced air cooling, and possibly with indirect free cooling, we are talking about two separate circuits. Humidity can be regulated very well here and there is also no exposure to outside influences. This system does, however, carry higher investment costs. A very efficient solution is water cooling of the CPU directly, which enables extremely high water flow temperatures. The installation costs for this are, however, quite considerable. Liquid Cooled Racks (LCR) are in principle no different to recirculated air systems, just smaller in volume and with tremendous power in the rack.
Dr. Michael K Patterson: Both systems can be very efficient. The challenge is applying the right one in the right place. Water cooled systems are generally more space efficient and can add a lot of capacity to a space-constrained data center. Water cooling also helps when you are looking at the very high-end of the HPC segment. Air-cooling can also be very efficient and if the final heat sink is the outdoor air, an air-side economizer in the right location may be the most efficient of all systems. Air-cooling is also a little bit simpler from a hardware and operational perspective.
5. What about reliability and availability of the different cooling systems? What is to consider when running a data center?
Dr. Rainer Weidmann: Environmental influences play a crucial role and require in-depth analysis in advance of planning and implementation. A critical point to note is the matter of data security and availability requirements. When defining protection objectives, it is essential to establish which hazards can potentially occur, which data is to be protected, and how data security will be implemented. External influences also include natural disasters and accidents such as fires. The ideal availability of the electronic data processing center must also be clarified in advance. Global warming, too, has repercussions for infrastructure planning. A rise in temperature necessitates additional cooling, and the units must be sized accordingly.
Dr. Michael K Patterson: Reliability and availability can be good or bad in any type system. It’s all just a matter of proper engineering design. Applying the right amount of redundancy, and working thru the tools available (such as Failure Modes and Effects Analysis) will provide the level of reliability and availability that the system requires.
6. In which scenario and/or environment would you recommend which cooling system? Is there also a benefit in combining different cooling systems?
Dr. Michael K Patterson: As mentioned before, the local conditions and data center design and operational concepts will tell which way to go. Imagine a data center with a 30cm raised floor that is full of piping and cabling and a new rack of 14 kW is required. There is very little if any airflow available. In a case like this a liquid cooled rack would be ideal. On the other hand in a new data center with a decent raised floor and/or good airflow segregation, then a liquid cooled rack may simply cost extra. In general mixing the two systems will not be a benefit and will likely wind up costing more as the data center owner must essentially install twice the infrastructure. But at the same time there are very few absolute rules, and in this case our first example answer to this question was a mixed system, but in general, it’s not a good way to go.
Dr. Rainer Weidmann: I can’t emphasize this enough. A combination is generally only beneficial if it creates real redundancies. But you always have to check the on-site circumstances.
7. Does the external location/environment play a key role for running a data center energy efficient? (located in a northern region, close to rivers)
Dr. Rainer Weidmann: The local surroundings play an absolutely vital role. The outside temperature is essential for the duration of free cooling. The use of surface water for cooling can also be extremely efficient. With this approach, however, existing water laws must be observed and all relevant permissions must be sought from the district or municipal authorities.
Dr. Michael K Patterson: The external environment should play a key role in both the design and operations of the energy efficient data center, in both what you can do, and what you should not. For example trying to run a data center with an airside economizer in an area with significant pollution could be a real problem for reliability. Also an engineering analysis of the climate and the options will generally tell you what type of cooling system makes the most economic sense. Dry climates encourage the use of evaporative cooling, cool climates also should take advantage of compressor-less design options. On the other hand there may not be an acceptable ROI for installing these types of systems in a place with a very warm, humid environment.
8. What do you think of the approach to use outside air for cooling a data center (when the outside temperatures are lower than room temperature) – (Free Cooling) ?
Dr. Michael K Patterson: It’s a great approach if you do it right. First you have to be aware of the possibility of pollution causing corrosion on the IT equipment. Second the air flow needs adequate filtration. Third the data center truly needs to use a strict airflow segregation scheme. And finally, and most importantly, the design has to start with this cooling plan in mind. The inlet air vents will be very large, and they must be comprehended by the architects in the beginning of the design or else the concept is in danger of failing.
Dr. Rainer Weidmann: As I mentioned, this is an option that can be extremely energy-efficient, especially during the colder months or in more northerly regions. The point to bear in mind is the humidity, which can be costly to regulate due to the large volume flow. There is also a risk of problematic influences from outside, such as air pollution from opening the door to the electronic data center. It is therefore crucial to install sensitive sensors and implement measures for protecting against contaminants to the outside air.
9. Which are the key characteristics of the data center of the future? What is trend-setting? What should be in the center of attention when running/building/modernizing a data center?
Dr. Rainer Weidmann: The data center of the future will develop clearly in the direction of modularity and expandability. It will boast an outstanding energy balance sheet and optimally incorporate climatic and geographical conditions as early as the design phase. Energy efficiency and ecology will be essential criteria for the design and operation of an data processing center and will automatically contribute to cost savings and reduction of the carbon footprint.
Dr. Michael K Patterson: Simple – the data center of the future will be modular, expandable, flexible, with a higher level of integration between the IT equipment and the infrastructure. The two systems have to be better integrated to allow for optimum efficiency. I see the servers themselves telling the cooling system how much cooling they need, and all of this in a well controlled space with excellent airflow segregation.
10. What is your recommendation for companies that have to decide whether building a new data center or modernizing their existing one? Is there a rough guide?
Dr. Michael K Patterson: Great question. Often you find that a simple refresh of the existing servers coupled with an upgrade of the infrastructure works just fine. We are seeing a 9x IT system performance improvement over just the last 3 years. The quickest and easiest way to save money and get efficient is by using the latest hardware coupled with the right power and cooling upgrades to your data center. Marginal capacity increases can best be handled with server refresh in existing spaces. Larger capacity increases may require a new data center, and if needed using the technologies discussed above can yield a world class data center with outstanding efficiency.
