Energy consumption at data centers can be reduced using methods that are easy to effect. The researchers at T-Systems and Intel were able to show reductions in energy consumption of the data center. The measures were straighforward at a reasonable cost.

Find out more about the deployed methods of T-Systems and Intel:

Download (PDF)

Knürr system solutions in the world of information and network technology are part of an adaptive architecture from Emerson Network Power, which flexibly adapts to changes concerning security, high-density and all associated capacities. Companies are provided sustainable and long-term benefits from the high IT availability, operative flexibility and impressive reduction in investment and operating costs.

The Knürr Technical Furniture business unit develops, produces and distributes technical workstation systems for control consoles, operation control centers and electronics labs, as well as mobile equipment carriers for the automotive and medical technology industries.

Knürr AG is recognized around the world as one of the leading developers, manufacturers and distributors of rack and enclosure platforms in the indoor and outdoor area, including all relevant active/passive components in 19” construction and the technologies associated with it. Knürr is part of Emerson Network Power.

Knürr is certified in accordance with EN ISO 9001 and EN ISO 14001, and its quality management continuously guarantees the highest standards in all areas of the company.

Dr. Markus Leberecht is going to present results of Datacenter2020 at the “International Conference on Energy-Aware High Performance Computing” www.ena-hpc.org in Hamburg, Germany on Sept 16, 2010.

“Macro and Micro-level Energy-Savings Through COTS Technologies”, Markus Leberecht, Intel GmbH
Data center infrastructure optimization and energy-aware programming are two important practices in the toolbox of energy-efficient IT operation. In this talk, we are going to demonstrate how Intel engages in these two fields to drive highly pragmatic approaches towards lowering total and relative energy use in enterprise and high-performance computing.

The biggest section of the company, data infrastructure solutions, provides components for networks and computer centres. These include products like the REDi-Way system, comprising pre-assembled copper or fibreglass trunk cabling, patch cables, RJ-45 modules, distribution panels and junction boxes. The cable assembly section of WGD Datentechnik produces customized cables according to the drawings and specifications of its customers.

MPK-Slim trunk cables from WGD Datentechnik are in use at the DataCenter 2020. “WGD Datentechnik is a long-standing partner for our computer centres and our first port of call when it comes to copper cabling solutions,” explains Manuel Mair, project leader for DataCenter 2020 at T-Systems. “The trunk cables that have been installed are used to exchange information between IP-compliant devices and the central building control system. Altogether, we have dispatched over 1000 metres of these cables in the DataCenter 2020.”

High packing density saves energy

The 6-core WGD MPK-Slim trunk cable that has been used is specified for up to 10 Gigabit Ethernet and complies with various certifications (incl. the GHMT PVP) to ensure good connection quality during data transfer. Other important features of the trunk cable from WGD are its high packing density and small diameter. Because of these, the cable consumes about 52% less space than conventional cabling. The result is that cable channels can be smaller, reducing the potential fire hazard, because they contain less flammable material.

The fact that the cable takes up less space is also a benefit for energy efficiency. “Because the cable is so densely packed, there is more room in the server rack for the cooling system. Also, since there is less interruption to the air flow and hardly any air turbulence, the air conditioning in the rack is far more efficient, saving valuable energy,” says Manuel Mair. “We achieve the same effect in the false floor.”

Wolfgang Gerlach, Director of WGD Datentechnik, is delighted that his company is able to help the energy efficiency requirements of the DataCenter 2020, thanks to the MPK-Slim trunk cable: “As a longstanding partner of T-Systems for their infrastructure cabling, it was only natural that we should provide our slimline cables for the research computer centre. There, we can test our cabling solution, which measures only 13.5 millimetres, using six complete RJ45 boxes, in intensive and changing conditions, thus gaining valuable experience for our own research.”

One more advantage: the Slim-Kabel is pre-assembled and therefore ready for use as soon as it is delivered. “The MPK-Slim trunk cable from WGD can be reused and is very simple to lay, almost Plug & Play,” adds Manual Mair of T-Systems, delighted. WGD supplies the cable in any required length (up to 60 m for AWG26 /up to 90 m for AWG23) with protected plugs, an optional cable pull and the relevant measurement logs.

Whitepaper Data Center 2020 (english Version)

Diane M. Bryant, Chief Information Officer at Intel Corporation, and Olaf Heyden, member of the board of management and Head of ICT Operations at T-Systems.

The companies were able to reduce their power usage effectiveness (PUE) from 1.9 to less than 1.4 at their test data center in Munich. This value shows how much of the energy used is actually converted into computing performance. Since not only the processors but also the server cooling system, for example, use energy, this value has been at 1.9 on average.

Green is the way to go

Since September 2009, both companies have been doing research at DataCenter 2020, which was especially designed for this purpose. Intel and T-Systems experts have experimented with various scenarios to reduce the PUE value. For example, the companies found the point which creates a perfect balance between cooling energy and processor performance. The result showed that data center cooling did not need to be as high as was often assumed. The experts also improved the data center infrastructure by closing open gaps between servers. This way they were able to optimize air flow in server rooms.

This showed that energy consumption at data centers could be reduced partially with simple measures. And this is not only better for corporate expenses but also for the environment. If you are interested, you can access the complete results via the link to DataCenter 2020 in the right-hand column on this page.

For the DataCenter 2020, Emerson Network Power is providing two Liebert Hiross HPM L12UC forced air cooling devices and one for the humidification and dehumidification of the electronic data processing center. The company is also supplying the Liebert Hiross HPC-S water chiller, complete with free cooling, on the roof of the building. Christian Richter, Marketing Manager for UPS and air conditioning at Emerson Network Power in Bavaria, considers DataCenter 2020 to be an outstanding test area in which to research the interplay of different control parameters in the data center: “We can put the results of the extensive tests in the lab to good use in our product development. T-Systems is also a long-standing customer with enormous expertise in the field of energy efficiency.”

Forced air cooling devices

In the DataCenter 2020, two Liebert Hiross HPM L12UC forced air cooling devices are being used: one with an EC motor, and one with an AC motor and frequency converter. A third device from Emerson Network Power is also being used exclusively for the humidification and dehumidification of the air in the test lab.

“Deutsche Telekom operates approximately 35 electronic data processing centers around Germany, some of which still use out-dated technology such as AC motors,” explains Manuel Mair, Project Manager of DataCenter 2020 at T-Systems. “With the two types of forced air cooling device, we are finding out whether it makes sense to upgrade the older AC motors with a frequency converter, or whether it is better to just buy a new device with an EC motor.” According to Mr. Mair, the forced air cooling devices must always be considered in the context of the overall cooling system in the electronic data processing center, e.g. the cold aisle containment or the water chiller.

EC motors consume around 30 percent less power than AC motors. As they use no energy from frequency converters, they are free from electromagnetic interference. Electronic control allows the EC ventilators to be very well regulated and optimally set to the prevailing circumstances. The Liebert Hiross HPM series devices are fitted with iCOM control, which links several units via Ethernet and enables the synchronization of numerous air-conditioning units. The iCOM control also allows the airflow of the ventilators to be continuously increased if necessary.

Water chiller

On the roof of DataCenter 2020 is a compact design Liebert Hiross HPC-S water chiller with built-in free cooling facility. The device uses the outside temperature (especially in winter) to cool the water, therefore achieving energy savings of up to 30 percent. The Liebert Hiross HPC-S with its hermetic scroll compressors is at the cutting edge of technology and is redundantly configured with two separate cooling circuits to increase the reliability. The screw compressor is continuously adjustable and specially developed and optimized for air-cooled liquid chillers in air-conditioning systems.

“We have already built several control cabinets for the SICLIMAT X building automation system from Siemens. Siemens then asked us to be involved in the DataCenter 2020 project,” adds Thorsten Jokiel, manager of JOKIEL’s Munich office. “It is very interesting for us to see how T-Systems and its partners gain new expertise in energy efficiency in the test center using state-of-the-art technologies. We want our control cabinets to contribute to exploring possibilities for the data center of the future.”

The metal control cabinet in the DataCenter 2020 is one-of-a-kind, measuring in at 1.60 meters wide and 2 meters high. The team from JOKIEL, which called the project the “T-System EIP Muc test lab”, needed roughly 250 man-hours to build the cabinet including planning, production, and installation. The control cabinet physically connects the sensors to the building automation system by a cable; this system analyzes the measurements that are taken, using them to regulate room temperature. “We didn’t just want the control cabinet to fulfill its essential function as the link between the technical infrastructure and the building automation system, we also wanted it to have a strong visual appeal. And we succeeded,” emphasizes Manuel Mair. “After all, we have a lot of visitors coming to the DataCenter 2020.”

Dr. Michael K Patterson

Dr. Michael K Patterson: After the IT equipment itself, the cooling system is typically the next largest energy user. It also is the easiest part of the data center to operate poorly. For example, the power delivery architecture may have some inefficiencies in it, but once it is on-line it pretty much just feeds power. The cooling systems have so many more variables in them that can be done poorly. But if we design it right and operate it right it is also the one place where significant energy savings can be had.

Dr. Rainer Weidmann

Dr. Rainer Weidmann: I can’t emphasize this enough. At present, air conditioning accounts for 35-50% of the total power requirement of an efficient electronic data processing center. With less efficient systems, the PUE (Power Usage Effectiveness) can even reach the value 3. A very good electronic data processing center that uses no fresh air, only recirculated air, would have a value of around 1.7. In the DataCenter 2020, we have set ourselves the target of 1.3. When looking to save energy, the focus is primarily on air conditioning. This is in heavy contrast to uninterrupted power supply (UPS) units, where the level of efficiency is currently 92 percent. There is therefore only a small margin for optimization in this area: approximately 8 percent.

2. Which approaches and methods of air-conditioning and cooling do exist that differ fundamentally?

Dr. Michael K Patterson: The first option is air-cooling vs liquid cooling; although liquid cooling is often misunderstood.  It primarily means liquid to the rack-level, and not liquid to the component-level.  There are so many different sources of heat in the platform; the CPU, memory, chipset, drives, power supplies, and voltage regulators – the CPU for example is generally only 30-35% – that to cool them all with liquid becomes prohibitively expensive and complicated.  The one place where this might make sense is in High Performance Computing, where the interconnect requires very high density.

For air-cooling there are also many options, cooling with a central chilled water system vs a direct-expansion (or DX) cooling system.  Then in the data center itself, the airflow management strategy makes a difference, this ranges from the simple hot aisle/cold aisle arrangementall the way to hot-aisle containment strategies.  Also important is whether the data center has any economiser modes to allow for “free cooling” schemes.

Dr. Rainer Weidmann: There are a number of different approaches. The first of these to mention is operation with purely recirculated air (with or without indirect free cooling), which doesn’t require a supply of fresh air from the outside. Or direct free cooling, a type of cooling with outside air. Natural resources can be used too, for example surface water or well water. And we must not forget direct CPU cooling with liquids.

3. Which factors influence the energy efficiency of a refrigeration system in the data center?

Dr. Michael K Patterson: The overall efficiency of the cooling system has a multitude of parameters that can affect it, but most of these are part of the cooling system design and well removed from the data center.  But in the data center itself there are some very important factors. First is airflow management. This matters in two ways: good airflow management will limit excess fan energy and will also provide warmer return air to the cooling systems.  That warmer air will allow the cooling system to operate more efficiently.  Second is running the data center at the right temperature.  ASHRAE guidelines state that the IT equipment inlet can be as warm as 27C.  Unfortunately, the cooling system return air is often measured rather than the supply at the IT equipment leading to overcooled data centers, so that needs to change. That wouldmake a huge difference in getting to a more efficient system.  As for measuring the temperature at the server inlet, most Intel Architecture with today’s Intelligent Servers can tell you that temperature.

Dr. Rainer Weidmann: Absolutely right. The outside temperature is one of the main factors, as is humidity. The design also plays a part: is direct or indirect free cooling implemented? Important too is the Delta-T value, which indicates the temperature difference between return air and supply air. It is also crucial to maintain the temperature of the servers at the right level: this is a matter of selecting the correct inlet temperature. Airflow and current should be taken into account as well. Wherever possible, cold air and warm air should be prevented from mixing.

4. Please describe briefly the benefits of an air-cooled and liquid-cooled system

Dr. Rainer Weidmann: Cooling with outside air can prove to be extremely energy efficient, particularly during the colder months. However, controlling humidity is costly, and there is a risk of environmental influences such as air pollution being carried into the electronic data processing center. Just as challenging is the installation of outside air sensors to guard against such problems. With purely forced air cooling, and possibly with indirect free cooling, we are talking about two separate circuits. Humidity can be regulated very well here and there is also no exposure to outside influences. This system does, however, carry higher investment costs. A very efficient solution is water cooling of the CPU directly, which enables extremely high water flow temperatures. The installation costs for this are, however, quite considerable. Liquid Cooled Racks (LCR) are in principle no different to recirculated air systems, just smaller in volume and with tremendous power in the rack.

Dr. Michael K Patterson: Both systems can be very efficient.  The challenge is applying the right one in the right place. Water cooled systems are generally more space efficient and can add a lot of capacity to a space-constrained data center. Water cooling also helps when you are looking at the very high-end of the HPC segment.  Air-cooling can also be very efficient and if the final heat sink is the outdoor air, an air-side economizer in the right location may be the most efficient of all systems. Air-cooling is also a little bit simpler from a hardware and operational perspective.

5. What about reliability and availability of the different cooling systems? What is to consider when running a data center?

Dr. Rainer Weidmann: Environmental influences play a crucial role and require in-depth analysis in advance of planning and implementation. A critical point to note is the matter of data security and availability requirements. When defining protection objectives, it is essential to establish which hazards can potentially occur, which data is to be protected, and how data security will be implemented. External influences also include natural disasters and accidents such as fires. The ideal availability of the electronic data processing center must also be clarified in advance. Global warming, too, has repercussions for infrastructure planning. A rise in temperature necessitates additional cooling, and the units must be sized accordingly.

Dr. Michael K Patterson: Reliability and availability can be good or bad in any type system.  It’s all just a matter of proper engineering design.  Applying the right amount of redundancy, and working thru the tools available (such as Failure Modes and Effects Analysis) will provide the level of reliability and availability that the system requires.

6. In which scenario and/or environment would you recommend which cooling system? Is there also a benefit in combining different cooling systems?

Dr. Michael K Patterson: As mentioned before, the local conditions and data center design and operational concepts will tell which way to go. Imagine a data center with a 30cm raised floor that is full of piping and cabling and a new rack of 14 kW is required. There is very little if any airflow available.  In a case like this a liquid cooled rack would be ideal.  On the other hand in a new data center with a decent raised floor and/or good airflow segregation, then a liquid cooled rack may simply cost extra.  In general mixing the two systems will not be a benefit and will likely wind up costing more as the data center owner must essentially install twice the infrastructure.  But at the same time there are very few absolute rules, and in this case our first example answer to this question was a mixed system, but in general, it’s not a good way to go.

Dr. Rainer Weidmann: I can’t emphasize this enough. A combination is generally only beneficial if it creates real redundancies. But you always have to check the on-site circumstances.

7. Does the external location/environment play a key role for running a data center energy efficient? (located in a northern region, close to rivers)

Dr. Rainer Weidmann: The local surroundings play an absolutely vital role. The outside temperature is essential for the duration of free cooling. The use of surface water for cooling can also be extremely efficient. With this approach, however, existing water laws must be observed and all relevant permissions must be sought from the district or municipal authorities.

Dr. Michael K Patterson: The external environment should play a key role in both the design and operations of the energy efficient data center, in both what you can do, and what you should not.  For example trying to run a data center with an airside economizer in an area with significant pollution could be a real problem for reliability.  Also an engineering analysis of the climate and the options will generally tell you what type of cooling system makes the most economic sense.  Dry climates encourage the use of evaporative cooling, cool climates also should take advantage of compressor-less design options.  On the other hand there may not be an acceptable ROI for installing these types of systems in a place with a very warm, humid environment.

8. What do you think of the approach to use outside air for cooling a data center (when the outside temperatures are lower than room temperature) – (Free Cooling) ?

Dr. Michael K Patterson: It’s a great approach if you do it right.  First you have to be aware of the possibility of pollution causing corrosion on the IT equipment.  Second the air flow needs adequate filtration.  Third the data center truly needs to use a strict airflow segregation scheme.  And finally, and most importantly, the design has to start with this cooling plan in mind.  The inlet air vents will be very large, and they must be comprehended by the architects in the beginning of the design or else the concept is in danger of failing.

Dr. Rainer Weidmann: As I mentioned, this is an option that can be extremely energy-efficient, especially during the colder months or in more northerly regions. The point to bear in mind is the humidity, which can be costly to regulate due to the large volume flow. There is also a risk of problematic influences from outside, such as air pollution from opening the door to the electronic data center. It is therefore crucial to install sensitive sensors and implement measures for protecting against contaminants to the outside air.

9. Which are the key characteristics of the data center of the future? What is trend-setting? What should be in the center of attention when running/building/modernizing a data center?

Dr. Rainer Weidmann: The data center of the future will develop clearly in the direction of modularity and expandability. It will boast an outstanding energy balance sheet and optimally incorporate climatic and geographical conditions as early as the design phase. Energy efficiency and ecology will be essential criteria for the design and operation of an data processing center and will automatically contribute to cost savings and reduction of the carbon footprint.

Dr. Michael K Patterson: Simple – the data center of the future will be modular, expandable, flexible, with a higher level of integration between the IT equipment and the infrastructure.  The two systems have to be better integrated to allow for optimum efficiency.  I see the servers themselves telling the cooling system how much cooling they need, and all of this in a well controlled space with excellent airflow segregation.

10. What is your recommendation for companies that have to decide whether building a new data center or modernizing their existing one? Is there a rough guide?

Dr. Michael K Patterson: Great question.  Often you find that a simple refresh of the existing servers coupled with an upgrade of the infrastructure works just fine.  We are seeing a 9x IT system performance improvement over just the last 3 years.  The quickest and easiest way to save money and get efficient is by using the latest hardware coupled with the right power and cooling upgrades to your data center.  Marginal capacity increases can best be handled with server refresh in existing spaces.  Larger capacity increases may require a new data center, and if needed using the technologies discussed above can yield a world class data center with outstanding efficiency.

For the power distribution system, Hager outfitted the DataCenter 2020 with two individual switchboards from its “Univers N” series, various power distribution components (EIB standard European installation bus) and the tebis KNX Touch Panel PC for integrated control of lighting, blinds, leakage panels or the smoke generator which makes leaked air visible in the DataCenter 2020.

“We have already been working intensively with T-Systems in the Euro Industrial Park in Munich since 2004 and have achieved excellent results in terms of power distribution,” explains Torsten Schulz, sales director of Hager’s sales organization in Germany. “The results of the research conducted in the DataCenter 2020 are also very interesting for us. We are using these findings to adapt our products to the requirements of state-of-the-art data centers.”

Power distribution system

Hager’s Univers N series is a flexible interior installation system for power distribution. Deployed in the DataCenter 2020 is a main distribution board connected to a sub-distribution board for reliable distribution of power from the central input to the individual servers. Various components such as jumpers, terminal blocks and adapters can be found in the boards themselves.

“Every board supplied by our long-standing partner Hager is one-of-a-kind and thus designed exactly to our specifications with power feed switch, extra monitoring, an exact number of outputs and much more,” explains Manuel Mair, project manager at T-Systems for the DataCenter 2020. “The quality behind Hager’s products is extremely important to us because the central distribution hub cannot be allowed to malfunction. Every single component has to be made from high-quality material to prevent problems from occurring.” A malfunction in the power distribution system can lead to a failure of the IT hardware. The top-of-the-line components in Hager’s Univers N series thus help achieve the high level of availability in the data center.

Integrated control with touch panel

T-Systems has also deployed, in addition to Univers N, a range of power distribution components in the DataCenter 2020 that are controlled using the Hager tebis KNX Touch Panel PC. “It is very user-friendly and convenient because the central touch panel gives us the capability to control many functions such as light switches, blinds, smoke generator or the leakage panels with a single tool,” says Manuel Mair.

In other words: the power distribution components are based on the European installation bus (EIB), a standard in accordance with EN 50090, in the current version as KNX standard also in accordance with ISO/IEC 14543-3. It describes how sensors (buttons, dimmers, blind switches, brightness and temperature sensors, etc.) and actuators (relays, light control units, blind motors, etc.) are connected to each another in a building by a twisted pair bus wire. Lighting, heating, air-conditioning, ventilation or alarms can all be connected to one another, for example, by means of EIB.

The tebis KNX Touch Panel PC mentioned above is used for the central control and operation in the DataCenter 2020 and comes with an 8 or 15-inch monitor. In addition to the KNX connection, there are Ethernet and USB ports and a DVI and a PS2 interface provided. This means that the user not only has touch control of all KNX functions ranging from lighting to heating and blind control with the panel, he can also use all of the multimedia technology.