Dynamic Control Optimizes

Facility Airflow Delivery

A look at several easy-to-implement solutions that
optimize cooling by minimizing airflow delivery, saving
energy and reducing cooling capital costs

David L. Moss

Dell Data Center Infrastructure

Dynamic Control Optimizes Facility Airflow Delivery

This document is for informational purposes only and may contain typographical errors and
technical inaccuracies. The content is provided as is, without express or implied warranties of any
kind.

2012 Dell Inc. All rights reserved. Dell and its affiliates cannot be responsible for errors or omissions
in typography or photography. Dell, the Dell logo, and PowerEdge are trademarks of Dell Inc. Other
trademarks and trade names may be used in this document to refer to either the entities claiming the
marks and names or their products. Dell disclaims proprietary interest in the marks and names of
others.

March 2012| Rev 1.0

ii
Dynamic Control Optimizes Facility Airflow Delivery

Executive Summary
Facility cooling optimization can represent a significant operational and capital expense reduction
opportunity. The implementation of an intelligent solution that automatically and dynamically matches
facility air delivery to IT consumption yields a reduction in facility fan power and associated cooling
infrastructure costs as well as the opportunity to stretch the cooling system to handle more IT
equipment.

Introduction
The ideal data center cooling solution yields low operating costs, stretches HVAC capital expenditures
to cover more equipment, and simplifies infrastructure planning and deployment. Considering these
three tenets, very few solutions do well on all of them simultaneously. You might have one solution
that does really well on controlling operating expenses, for example, but it might be more expensive
than competing solutions. Dell has a very simple solution that delivers on all three. It is dynamic, in
that it can automatically adjust cooling delivery to support varying compute loads. Based on
raised-floor cooling and down-flow CRAC units (chilled water CRAH or DX-based CRAC), it has very low
upfront costs. Cooling is ideally matched to IT equipment consumption, making it easy to plan for
additional cooling equipment in a just-in-time manner.

Three technical aspects are required to achieve an ideal cooling solution. With each piece of IT
equipment operating under its own thermal algorithms, each instantaneously monitor s and adjusts
airflow consumption based on inlet temperature and multiple internal temperatures that serve as
proxies for the variance in component configuration and utilization. The first technical aspect is to
have a facility volumetric control strategy that can understand and adjust to the aggregate, dynamic
airflow consumption of all the IT equipment. Secondly, there must be good isolation of the supply and
return paths of air to each and every rack of equipment. Third, while continuously adjusting to the
aggregate consumption of airflow, the facility must deliver it spatially relative to where it is being
consumed.

The Dell solution uses under-floor pressure control (UPC) as the method to control and match the
overall volumetric delivery of the facility to the aggregate consumption of the IT equipment. This
satisfies the first technical aspect. The preferred method to isolate the air paths and to get the air
proportioned to the needs of the equipment is to use the Dell PowerEdge Energy Smart containment
rack enclosure. It is the preferred containment method because of its ease of use and relative cost.
The solution is most effective when applied facility-wide. We recognize, however, that it will be hard
for many to justify moving operational equipment into a new rack. In this paper we propose several
aisle containment options for retrofitting existing racks that can be configured to work like and be
deployed alongside new equipment deployed in Energy Smart racks.

3
Dynamic Control Optimizes Facility Airflow Delivery

The Dell containment solution

Responds dynamically
Integrates tight rack-level containment
Each racks volumetric requirements are met by a negatively pressurized, passive, front,
containment plenum
CRAC maintains facility-level delivery through UPC

The Energy Smart containment rack provides an integrated front plenum with a flexible seal at the
floor offering tight containment around floor venting. This passive cold containment enclosure utilizes
the intelligence of the IT systems to extract only the required amount of air from the raised floor. With
negative pressure generated in the front plenum of each rack, this also has an effect on the pressure
beneath the floor. Any change in airflow through the rack causes an under-floor pressure change to
which the CRAC units can respond. CRAC units are controlled in unison, striving to maintain a specific
under-floor pressure as measured by a single or multiple points under the floor. If using multiple
points, some method of averaging would be appropriate. For more detailed information on the Energy
Smart rack, refer to the referenced white papers 1,2.

Additional rack or equipment deployments are countered by an increase in CRAC flow rate when UPC
senses a pressure drop due to the additional air consumption. Whether the flow through the racks is

1
For more information regarding the benefits of the Energy Smart Rack, see Managing Data Center Costs with Dell
PowerEdge Energy Smart Containment Rack Enclosures by David Moss, Dell Inc., May 2011,
http://www.dell.com/us/enterprise/p/d/business~large-business~en/Documents~
energy-smart-containment-rack.pdf.aspx.
2
For more information regarding deployment of the Energy Smart Rack, see Dell PowerEdge Energy Smart
Containment Rack Deployment Guide by David Moss, Dell Inc., Sept 2011,
http://content.dell.com/us/en/enterprise/d/shared-content~solutions~en/Documents~energy-smart-
containment-rack-deployment-guide-dell.pdf.aspx

4
Dynamic Control Optimizes Facility Airflow Delivery

changing due to varying compute needs, intentional power management, virtual server moves, or any
other reason for the IT systems to adjust flow rates up or down, pressure control enables the air
handlers to adjust accordingly and minimize flow to just what is needed.

This sets up a very appealing scenario in terms of CRAC deployment planning. Unless vents are
intentionally added elsewhere in the facility, vents only exist under the deployed Energy Smart racks.
The only path out of the floor for the cold air is through the equipment installed in an Energy Smart
rack. The CRAC units only pump enough air into the floor to meet the air flow needs of that
equipment. With all the CRAC units operating at the same speed, it is easy to establish how much
reserve volumetric flow rate is available. With a very clear picture of the amount of reserve, it is much
easier to plan when an additional CRAC unit is needed. In conjunction with controlling CRAC unit air
movers by pressure, the temperature is best controlled by sensing supply air temperature.

The Dell containment solution results in minimized CRAC unit flow rate, which eliminates
overprovisioning. It also helps to establish a measurable airflow reserve capability upon which you can
more precisely plan future CRAC unit additions. These factors can provide reductions to both operating
and capital expenditures.

Cold aisle containment solution options

Discussion thus far has centered on the Energy Smart rack as the preferred containment enclosure, but
what about optimizing cooling for existing racks? It can be difficult to justify the powering down of
equipment for a move into a new rack; one solution for this situation is to implement containment
around the existing racks. Cold aisle containment is compatible alongside the Energy Smart rack. For it
to work like the Energy Smart rack with similar control over air distribution, several options might be
considered.

Except in situations of low-density racks with very little flow rate, pressure inside the Energy Smart
rack front plenum is negative, meaning the equipment in the rack is responsible for determining how
much air passes through the floor venting into the rack. The UPC set point can be adjusted to a low
value, and there is often a positive tradeoff of CRAC fan energy relative to IT fan energy with the IT
fan energy increased slightly for extracting air from the floor. Most of the cold aisle options will
operate with a positive aisle pressure and require a higher UPC set point, which is one point of
differentiation relative to the Energy Smart rack.

There are two main differences between each aisle containment option presented in this paper. The
first is in the method used to control how much air enters the aisle and whether the air is driven by the
IT systems, the UPC (the CRACs), or a third control system. The second is in the criticality of sealing
the containment solution, and whether or not aisle pressure will be the primary driver of the
volumetric flow rate.

Options associated with a higher UPC set point

Options with higher under-floor set points are options where the CRACs need to drive air into and
positively pressurize the cold aisle. These options will use more CRAC fan energy but may use less
energy elsewhere to move air. CRAC failures can cause a significant drop in aisle flow rate. Sometimes
the drop is concentrated in one area, and it may not be compensated for well by the remaining CRAC
units, regardless of the sensor placement. In some of the other options where the CRAC units are not
the driver of flow into the aisle, this is not a problem.

5
Dynamic Control Optimizes Facility Airflow Delivery

Option 1
Responds dynamically

Tight containment not as critical

CRAC drives proper flow rate by maintaining slightly pressurized aisles

Pressure measured in each aisle; control based on worst-case aisle; potential for
over-provisioning in other aisles

In option 1, each cold aisle containment pod would have its own pressure measurement. Aisle
containment sealing is not very critical; it needs only to be tight enough to ensure a positive aisle
pressure relative to the room (P1 to P2). CRAC units are controlled together and seek to maintain a
positive pressure in all cold aisle pods. This option may need more pressure sensors than others, and it
will adjust flow to the worst-case aisle. Other aisles could end up over-provisioned, which translates
into waste. If significantly different loads are deployed in each aisle, tuning with different resistance
floor tiles can minimize over-provisioning. All tuning should be done at the IT equipment peak load to
ensure adequate flow.

6
Dynamic Control Optimizes Facility Airflow Delivery

Option 2
Responds dynamically

Tight containment not as critical

Actuated vent damper(s) regulates flow into aisle by maintaining slightly positive aisle
pressures (separate control system)
CRAC maintains facility-level delivery through UPC

T2

CRAC

P2
T1

P1

With option 2, the overall facility flow rate is controlled by the UPC setting. The variability of airflow
into each aisle is controlled by multiple, actuated dampers with their own control system. They adjust
open or closed to maintain a small positive pressure in the cold aisle. There are several methods for
doing this; temperature difference is pictured in the figure above. By establishing a minor, intentional
leak in the aisle containment and sensing whether the leak is an egress (cold, T2 similar to T1) or an
ingress (hot, T2 dissimilar to T1), the vents can be adjusted to maintain a slight positive pressure in the
aisle. Two other measurement methods have been studied and compared3 involving velocity and
pressure differential rather than a temperature difference. The UPC setting is still high because it is
the CRAC units that are driving the aisle pressure. Dynamic consumption within the racks is countered
by damper movement that adjusts the resistance of the floor and ultimately affects the under-floor
pressure addressed by UPC. The number of damper-equipped vents depends upon the amount of
variability in IT consumption and/or the amount of additional equipment planned for installation into
each aisle.

3
An Investigation into Cooling System Control Strategies for Data Center Airflow Containment Architectures, by
Michael K Patterson, et al, IPACK2011-52090, 2009

7
Dynamic Control Optimizes Facility Airflow Delivery

Option 3
No dynamic response; fixed volumetric aisle delivery

Tight containment not as critical

Volumetric delivery in each aisle established by number and resistivity of vents

CRAC maintains facility-level delivery through UPC

Option 3 is simple cold-aisle containment. The only difference between this option and option 2 is that
it contains no actuated dampers; it therefore has no way to respond to dynamic IT changes or to
significant equipment additions. A constant flow can be established and tuned to approximate IT
consumption within the aisle by experimenting with the number and resistivity of the vents and by the
UPC set point. Optimization should be done during peak consumption of the IT equipment so as to not
under-provision the aisle. Aisle delivery should remain adequate as new equipment is added to other
portions of the data center since floor pressure is being maintained by the UPC set point.

Options with a lower UPC set point

Options using a lower UPC set point have something other than pressure created by the CRAC units
driving volumetric delivery into the rack or aisle. These options use less CRAC fan energy, but may use
more energy elsewhere to move air. As mentioned before, however, it is often the case that there is a
net positive energy tradeoff when using a lower UPC set point to decrease CRAC fan energy and using
energy elsewhere to control air delivery locally. Similar to the Energy Smart rack, these options may
offer better protection during CRAC failures. Sometimes a CRAC failure can result in a very severe local
drop in flow rate. A natural under-floor vortex can enlarge during a CRAC failure; if it occurs in the
vicinity of a cold aisle, it can have a large effect on the natural flow into the aisle. This is compensated
for in options where air is drawn out of the floor using a method like the Energy Smart rack or options 4
and 5.

8
Dynamic Control Optimizes Facility Airflow Delivery

Option 4
Responds dynamically

Tightest aisle containment requirement

IT equipment drives negatively pressured aisle to control volumetric delivery in each aisle

CRAC maintains facility-level delivery through UPC

Option 4 would require the tightest level of aisle containment with good seals between, above, under,
and around racks and at all seams. With this option, the aisle is tight enough that negative pressure
presented by the racks (IT equipment) translates into negative pressure in the cold aisle which, in turn,
allows the aisle pressure to help determine how much air comes out of the floor. Differential pressure
is measured between P1 and P2, where P1 is the bulk under-floor pressure measured at a single point
or some average of multiple points, and P2 is the room pressure. This option represents the closest
comparison to the Energy Smart rack.

9
Dynamic Control Optimizes Facility Airflow Delivery

Option 5
Responds dynamically

Tight containment not as critical

Fan-assisted vent regulates flow into aisle by maintaining slightly positive aisle pressures
(separate control system)
CRAC maintains facility-level delivery through UPC

T2

CRAC

P2
T1

P1

Option 5 is very similar to option 2. Rather than using actuated dampers below the vent(s), aisle flow
control comes from fan-assisted vents. Like option 2, the number of fan-assisted vents would be based
on the expected dynamic fluctuation of the IT equipment and any additional IT equipment planned in
the aisle. Controlled similarly to option 2, the fans will be controlled by establishing a slight, positive
aisle pressure.

Like option 2, the cold aisle would not have to be extremely tight. Some fan products do offer greater
flow rates than can be attained by an actuated vent that would offer aisle density advantages over
option 2. This option also enables a lower UPC set point and allows for lower CRAC fan energy at the
expense of the aisle fan energy. From that standpoint, it is similar to the operation of the Energy Smart
racks, which leverage the IT equipment fans .

Fan-assisted vents and vents with actuated dampers are available from multiple vendors , such as Tate
Access Floors and Degree Controls, Inc. would be two such examples. These products typically use
temperature-based control systems. Consult the vendors for flow performance data and control
options.

10
Dynamic Control Optimizes Facility Airflow Delivery

Summary
Containment should be considered a best practice and become plan-of-record in all data centers.
Paired with UPC, the Energy Smart rack offers a simple and effective way to optimally deploy and
manage rack-level containment and reduce data center operating costs. With the IT systems only
consuming according to their need and the facility fans adjusting to that need, net fan energy is
minimized, and the reserve air can be used elsewhere. In one lab test with Energy Smart racks , we
experienced up to a 3:1 savings in fan energy, with 3 watts saved at the CRAC for every watt increase
in the IT equipment.4 This innovative containment solution also helps in stretching the capital
expenditures for CRAC units and in the ideal timing of their purchase.

Aisle containment is a good option if you are unable to move equipment into Energy Smart racks. It will
work similarly to and alongside Energy Smart racks if you use one of the dynamic options suggested in
this paper. It will, however, require more attention to detail in terms of aisle sealing and choosing the
proper number and types of vent tiles, whether fixed or variable. Solutions where the CRAC units are
not the primary director of airflow (Energy Smart rack, options 4 and 5) allow lower UPC set points and
are likely to result in lower net energy use. These solutions are also helpful to smooth out airflow
anomalies, if they exist, during CRAC failures.

UPC is an effective means to establish a link between the HVAC and IT airflow control systems. With
the pressure sensors under the floor, very few sensors are needed. You may, in fact, get by with a
single sensor. Unlike other attempts to monitor airflow, a sea of sensors approach is simply not
needed. By combining UPC with containment, you can optimize your facility cooling in order to reach
the goal of matching air delivery to IT consumption.

4
Global Ad # G12000548

11