Facebook, Inc.

May 2014

MIT Sloan School of Management

Final Report for S-Lab Project with Facebook, Inc.

Introduction & Objective

Facebook, Inc. (“Facebook” or “the company”) owns and operates data centers in Oregon, North
Carolina, and Sweden with a new site currently under construction in Iowa. As the company
scales, it is critically important for the company to grow its server infrastructure as well. Energy
efficiency and sustainable design are at the heart of Facebook’s design philosophy. Its data
centers are designed with the perspective that the servers, the building, and the operational
systems are part of one, interconnected system. More information on Facebook’s sustainability
efforts can be found at www.facebook.com/green.

Facebook engaged students from the MIT Sloan School of Management (“MIT Sloan team”) to
provide recommendations on how to improve the design and sustainability of its current and future
portfolio of data centers. The MIT Sloan team evaluated existing and emerging initiatives for
Facebook’s consideration. This includes (but is not limited to): site location, employee
engagement, water and material usage, holistic design thinking, and industrial partnerships. The
company expressed particularly interested in non-energy related initiatives to complement its
existing efforts.

The information contained within this report highlights additional background on Facebook and the
industry. It then provides a brief overview of the project approach followed by data center
compilation of projects and initiatives for Facebook’s consideration. These recommendations
include the development of a site selection tool, the exploration of co-location opportunities, and an
evaluation of cooling and ancillary systems. Details are provided on feasibility, financial metrics
(where available), energy implications (if applicable), and other qualitative sustainability
measurements.

Background

Company. Facebook has strong buy-in to continue increasing the efficiency and sustainability of
its data centers over time. Facebook’s internal team of architects, designers, and electrical
engineers is continuously working to improve its data centers’ designs and construction methods.

Facebook also launched the Open Compute Project Foundation in 2011. Open Compute is an
online community and knowledge share platform to improve the overall efficiency of the data center
industry. It offers suggestions on the following categories designed to address serve and hardware
development and construction: server, storage, data center design, networking, hardware
management, certification, open rack, and solution providers. The reported result is “a data center
full of vanity free servers which is 38% more efficient and 24% less expensive to build and run than
other state-of-the-art data centers.” 1

1
http://www.opencompute.org/

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 1

Facebook, Inc. May 2014

Industry. Despite a commonality in functions, organizations utilize different sustainability strategies

for designing and operating their data centers.

Google touts one of the most energy-efficient portfolios of data centers and servers in the world -
averaging about 50 percent less energy use than comparable operations. There are two primary
sets of initiatives that have allowed Google to achieve this. First, Google reduced non-server
energy consumption to 12 percent of server usage, a Power Use Effectiveness (PUE) of 1.12. 2
PUE is an industry metric that compares total data center energy use to energy used by the
server/IT equipment. A data center’s ability to reduce energy consumption from heating, cooling
and other functions thus improves the efficiency of the whole structure. Google has also
implemented a “reuse and recycle” policy for its older servers - first attempting to reuse outdated
equipment (over 300,000 successfully) and then recycling unusable components. In addition to
Google’s work to improve its data centers efficiency, the company has also funded government
research targeting the impacts of cloud computing. Google has been a strong supporter of the
Cloud Energy and Emissions Research (CLEER) Model developed by the Lawrence Berkeley
National Laboratory. This open-access model assesses the energy implications of varying market
adoption rates for cloud-based services. 3

Amazon, on the other hand, is much less transparent with its effort, and lack of transparency
suggests that efficiency may not be a high priority at this time. Nevertheless, Amazon argues that
by providing service to hundreds of clients in every data center, it is effectively eliminating
hundreds of individual data centers by combining them into a single location. Such consolidation
can reduce real estate, labor, and backup generator requirements. Parallel to Google’s fund
efforts, Amazon has contributed to the Xen Project. The Xen Project is a Linux Foundation Group
project dedicated to software designed for lower operating costs. Xen hypervisor is a software
capable of running on low-power server chips that can run many applications in a single server,
resulting in servers that consume less power, thus reducing the power needed by data centers. 4

Challenges. As the world’s population grows - coupled with more people emerging from poverty
and having more devices connected to the internet, data usage will continue to grow and
computing requirements will become more important. Data centers are thus a growing source of
energy usage and environmental impact, as well as an area for continuous improvement. A 2006
EPA report estimated that data centers and servers consumed 1.5 percent of U.S. electricity (61
billion kWh, costing $4.5 billion). 5 A more recent report from Stanford University found that global
and U.S. data center energy usage increased by 56 percent and 36 percent between 2005 and
2010, respectively. 6

2
https://www.google.com/about/datacenters/efficiency/internal/
3
http://cleermodel.lbl.gov/
4
http://www.xenproject.org/
5
U.S. EPA. “Report to Congress on Server and Data Center Energy Efficiency.”
6
Analytics Press. “Growth in data center electricity use 2005 to 2010.”

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 2

Facebook, Inc. May 2014

Due to advancements in processing speed, heat densities are also getting higher and causing
temperatures to rise, which requires more energy to cool. Cooling typically involves evaporative
cooling, which requires extensive amounts of water. Many areas of the world face drought
conditions and water scarcity, making data center water use an area of growing concern. Data
center expansion has other environmental and social implications as well. The need for back-up
generators (typically supplied by diesel fuel and tested regularly) leads to additional energy
consumption via fuel burn, as well as noise pollution. Furthermore, data centers present zoning
and siting complexities for local and regional governments - such as brownfield vs. greenfield site
selection and weighing land prices with security considerations.

Project Overview

Approach. In April 2014, the MIT Sloan team evaluated and analyzed potential technologies and
initiatives for data center improvements. The MIT Sloan team will evaluate ideas against current
Facebook and industry practices, their impact on resource consumption and regeneration, their
ease of implementation based on technical and organizational complexity, and their financial
impact. Potential ideas were prioritized if the fell into one of Facebook’s priority areas: site location,
employee engagement, water and material usage, holistic design thinking, and industrial
partnerships.

Selected Initiatives. From the ideas generated, the MIT Sloan team selected the initiatives
captured in Table 1 below. These initiatives represent high-level categorizations of specific
technologies or actions that can be implement at new and existing data centers.

Table 1. Recommended data center sustainability initiatives

Initiative Description Sustainability Benefit
Site selection Evaluation of sites (regionally, then locally) Clean energy, water usage
based on financial, environmental and
operational criteria using multi-decision
criteria analysis
Co-Location Evaluation of locating with or in close water usage, waste heat usage,
proximity to organizations, input providers greening of work spaces,
and governments based on feasibility, employee engagement
financial benefit, payback and employee
engagement.
Cooling & Evaluation of liquid cooling systems based waste heat usage, water usage,
ancillary systems on heat transfer properties and impact on pollution reduction, social benefit
the building design. Research into
developing technologies with potential
energy and environmental benefits in the
near future.

Assumptions. Facility location and site selection is a complex decision involving specific local and
regional information (e.g. potential tax incentives offered by governments) as well as proprietary,

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 3

Facebook, Inc. May 2014

internal/operational considerations (e.g. cost of capital, technical issues, etc). Site selection
recommendations are therefore provided irrespective of geographic location. Additionally, the MIT
Sloan team prioritized cutting-edge technologies and initiatives over low-hanging fruit based on
preliminary conversations with Facebook employees. Consequently, detailed financial assessment
was limited and difficult to perform. Where available, analysis is listed.

Resources. This report was prepared by Ricky Ashenfelter (MBA 2015), Kari Hodges (MBA 2014),
and Alberto “Beto” Luna (LGO 2015) under the mentorship of Professor John Sterman. Two
additional MIT Sloan students supported with third-party research and knowledge sharing,
including Al Rabassa (Data Center Manager for the U.S. Government) and Bruno Pimentel (MIT
Sloan Visiting Faculty Fellow from Vale Institute of Technology). Bill Weihl (Manager of Energy
Efficiency & Infrastructure Sustainability) served as Facebook’s main point of contact to the MIT
Sloan team. He was supported by Karen Cooper (Sustainability Program Manager) and Lyrica
McTiernan (Infrastructure Manager).

Site Selection

Site selection is an important consideration for data center development. While Facebook has
sales offices around the world, data centers are often located in remote areas; consequently,
choosing a location where Facebook already has a presence is not a priority. The location in which
a data center is built carries with it many environmental implications, including water availability,
temperature, cleanliness of energy generation, and susceptibility to natural disasters. Without
greater visibility into Facebook’s future expansion plans, however, specific locations were not
selected or recommended.

Before any decisions are made, it is important that Facebook continue considering carbon
accounting and additionality standards when selecting site locations. If construction of a data
center in an area with low-carbon energy (e.g. hydro) leads to the displacement of someone else’s
clean energy, then the cumulative environmental benefits are more difficult to discern. Location
decisions that cause new, incremental no/low carbon generating capacity that would not previously
have been constructed (or used, in the event of under-utilization of existing resources) are ideal.
That said, data center expansion to regions at capacity can also offer financial and societal
benefits if it directly or indirectly attracts future companies, governments, and/or states to “raise the
bar” in these regions - such as through new renewables capacity generation, investment in more
sustainable infrastructure, or broader economic improvement.

Recommendation. We recommend Facebook build a multi-criteria decision analysis as a tool for

future evaluation of site selection. One example is the Analytic Hierarchy Process (AHP), which
was developed by Thomas Saaty in 1977 as a process for Multi-Criteria Decision Analysis. 7 While
AHP is used here as an example, other frameworks and methodologies exist and should be

7
T. Saaty. "Decision making with the analytic hierarchy process."

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 4

Facebook, Inc. May 2014

considered as well. These include Evidential Reasoning (ER) and Intelligent Design System (IDS)
modeling. 8

The first step is to determine a set of quantitative or qualitative criteria to be used in the decision
problem and establish objective comparisons between them. In Facebook’s case, the following
“environmental” criteria should be incorporated into site selection:

1. Electricity cost: price to purchase each unit of energy (quantitative, measured in USD per
MWh)
2. Cleanliness of energy: takes into consideration energy generation fuel mix; more
favorable for cleaner sources of energy such as hydro, wind, solar, geothermal, and other
renewable energy sources; displacement vs. additionality considerations (quantitative,
measured in tons CO2 per MWh)
3. Natural disaster vulnerability: an area’s susceptibility to flooding, seismic activity,
tornados, hurricanes, snow, and wildfires (qualitative or quantitative, using categorized risk
maps) 9
4. Water availability: short term susceptibility to drought; longer term vulnerability to scarcity,
water rights and policy (qualitative or quantitative, using risk maps); see Appendix 1 for
examples from the USDA’s Drought Monitor and WRI’s Aqueduct tool
5. Climate: finding the right balance between temperature of ambient air and moisture
content; (quantitative, measured in degrees and air moisture content); ideal conditions is
approximately 70 degrees and 40 percent humidity; see Appendix 2 for examples of free
cooling mapping in Europe and the United States.

Facebook should also consider fiber network connectivity; security; the availability of tax, land and
economic incentives; labor and construction costs; and education levels. Facebook will need to
weigh environmental, financial, and operational considerations in choosing how to perform these
pairwise comparisons and minimize total cost of ownership (TCO). Proper due diligence, location
selection, design, construction, and operations/maintenance can ultimately both improve
sustainability and lower TCO.

Another opportunity for site selection is at brownfield sites for industrial parks or with canal
systems. Canals provide cheap real estate, areas of environmental reclamation, and can leverage
flowing water for both cooling and turbine-electricity generation purposes.

A comparable system has been implemented at the Massachusetts Green High Performance
Computing Center (MGHPCC) located in Holyoke, MA. MGHPCC is a data center supporting five
Boston Area research universities (including MIT) and where hydropower and solar installations
make up more than 70 percent of the facilities energy use. 10 Additional locations around the
country and world may also yield ideal conditions.

8
https://php.portals.mbs.ac.uk/Portals/49/docs/jyang/XuYang_MSM_WorkingPaperFinal.pdf
9
FORTRUST. “Geographic and risk mitigation factors for data center site selection.”
10
http://www.mghpcc.org/about/what-are-the-green-design-aspects-of-the-mghpcc/

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 5

Facebook, Inc. May 2014

Implementation. Facebook can craft multi-criteria decision analysis around those factors that are
most relevant. See Attachment 1 for an example buildout of a decision model - capturing eight
decision criteria and evaluated based on both quantitative and qualitative criteria. The resulting
analysis shows that fiber connectivity and energy fuel mix are the most important factors for
deciding site selection, though this model is meant to be customized based on Facebook’s
requirements, strategy, and future plans. For example, it excludes other factors of production and a
community’s willingness to tolerate construction or backup (diesel) generators. It also dependent
on Facebook’s ability to extract favorable tax breaks or electricity price discounts.

As built, the existing equations suggests Denmark is the most optimal choice, followed by Brazil.
Screenshots from the model are illustrated in Appendix 3. Of course, this analysis omits many of
the factors listed above; it will largely be up to the Facebook team to decide which factors are most
important and aligned to broader business strategy. Given this model is meant for a handful of
locations only, it is recommended that Facebook first use a tool like this after it has first identified a
region that aligns with its core business growth or strategy (e.g. continent such as Asia, Africa,
Europe). The tool could then be used to identify the country (e.g. South Korea, China, or Singapore
within Asia) or the region (e.g. Guangzhou, Shanghai, Beijing, or Shantou within China) that meets
these expansion criteria.

The larger the number of choices, the larger the number of pairwise comparisons with the
qualitative criteria are needed. If the number of choices becomes too large, other approaches,
such as mathematical programming may be used; this approach carries with it added complexity.

Co-Location

After business and site location considerations, co-location can greatly impact data center
sustainability. By sharing location, infrastructure, and waste products, co-location maximizes the
use of inputs and byproducts. For example, co-locators utilize data center waste heat as a source
of hot water for local communities. Additionally, co-location creates opportunities for community
and employee engagement and positive brand impact, such as named or sponsored community
gardens.

Decision Making Framework. During weekly phone calls, Facebook set co-location priorities.
Specifically, Facebook indicated the following: co-location should be cost efficient with clear
payback. Co-location should align with Facebook’s culture and uniqueness. Facebook also
indicated that security restrictions might limit co-locator access to the data center campus and
buildings. The MIT team worked within these parameters and with the assumption that Facebook
can resolve its security needs.

Because future data center locations are confidential, the MIT team did not recommend specific
co-locators or countries/states/cities. Instead, we recommended a framework for assessing co-
location opportunities and provided successful examples that met with the included criteria. While
CHP was initially considered a co-location priority, the low temperature waste heat Facebook data

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 6

Facebook, Inc. May 2014

centers produce is more aligned with yet to be commercialized CHP technologies. Appendix five
provides EPA resources on data center CHP as well as information on the Department of Energy’s
CHP Technical Assistance Partnership, which works with private industry on CHP technology
innovation. 11

In Facebook’s case, the following criteria should be incorporated into co-location decision-making:

1. Water treatment plant proximity: municipal water treatment plants often provide
greywater at no cost. Evaporation is a desirable alternative to releasing greywater into
local rivers and stream. While many of these systems are gravity dependent, a pump can
also be used to transport treated water. With Facebook’s current energy efficiency, we
believe water efficiency is a reasonable next step with evident financial benefits (see
appendix 4).
2. Non-potable water source access: provides a desirable source of water without taxing
the local drinking supply. This minimizes water costs, increases efficiency, and provides
future potential for energy/ cooling water technologies.
3. Proximity to co-locators with waste heat as an input resource: residential
neighborhoods, offices and eco-industrial parks consume wasted heat, as do gardens.
Close proximity helps maximize heat recovery in low temperature waste heat transfers and
minimize waste heat transfer losses.

Co-locating with Facebook created entities. Given the complexity of co-locating with external
organizations, we recommend Facebook’s co-location strategy start with internally derived
initiatives that meet the following criteria:

4. Alignment with Facebook values: assess the Impact, Fastness, Boldness, Openness and
Social Value of each co-location opportunity.
5. Community and employee engagement: for initiatives without measurable financial
payback, employee and community engagement levels provide another means of
assessment through surveys, social media mentions, and participation rates.
6. Potential for positive brand/ PR impacts: initiatives should have positive impact on the
Facebook brand and the public’s understanding of Facebook’s commitment to
sustainability.

The following sustainability recommendations are sourced from working prototypes at data centers
worldwide and meet both the co-location and engagement model criteria outlined above.

Onsite Co-location Incubators. Incubators provide a unique opportunity to utilize data center
space and further Facebook’s community engagement priorities. For example, IBM’s innovation
center in Cambridge, MA provides coworking space as well as software and systems support for

11
Charlie Goff, Consultant, and Stephanie DeGabriele. Eastern Research Group, Inc. EPA CHP Partnership.
EPA CHP Partnership Helpline: 703-373-8108 or chp@epa.gov, Charlie.Goff@erg.com.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 7

Facebook, Inc. May 2014

start-ups 12 while Google’s backed incubator supports a social mission of creating more female run
tech startups. 13 With an incubator, Facebook can maximize data center use while maintaining
maximum control over the physical site, the type of companies to support, and the financial
investment. An onsite incubator also allows Facebook to vary its investment depending on access
to funding and long-term interest or facility needs. This model has high potential for any future
urban data center locations.

Greenhouses. In Iowa, recovered ethanol plant waste heat is pumped to a co-located greenhouse
to support local vegetation production. Given the low temperature of the ethanol plant’s waste heat,
a heat pump is used to increase the temperature from 90°F to the 120 to 140°F range. The acre
greenhouse’s waste heat needs vary by month. The majority of the heat is used during Iowa
winters to ensure growing temperatures are kept consistent and that food can be produced year
round. A one-acre greenhouse uses approximately 7210.35 (MBtu/h) of heat annually. 14

The cost of building a greenhouse varies largely by size, material, and growing method (soil vs.
water). For example, the Hometown Harvest Greenhouse http://www.hometownharvestseiowa.org/
in Iowa took seven months to build and cost approximately $70,000, with much of the resources
and labor donated. 15 The greenhouse does not track its waste heat usage. Instead, its curator Jan
Swinton attempts to maintain a 70-degree differential between the greenhouse and the
surrounding environment for Iowa’s six months of cold weather while incorporating 244 Btu/ hour of
lost heat into her configurations (see Appendix 7).

On weekends without waste heat production, the greenhouse relies on a propane heater to warm
the facility. During the coldest season, propane costs reach about $100/ week. Cultivating the
produce requires labor (an additional cost). 16 Ms. Swinton, uses ten volunteers working three-
hour/week shifts as well as three master growers, to harvest the 90 lbs. of produce weekly. This
produce is sold to local schools for approximately two dollars per pound. Additional greenhouse
costs include water and drainage infrastructure as well as 20 amps of electricity used for water
pumps.

Node Greenhouse. Another example of greenhouse co-location is Notre Dame’s node data
center. The Notre Dame Center for Research Computing placed “high-performance computing
nodes” at the South Bend Greenhouse and Botanical Garden, in order to heat the facility. A grid
scheduling software controls the temperature range of the greenhouse. “Compute jobs” are sent to

12
http://www.thewire.com/technology/2014/03/google-backed-incubator-looks-increase-women-tech-25-
percent/359061/
13
]https://www304.ibm.com/partnerworld/wps/servlet/ContentHandler/isv_inv_tsp_iic_cambridge_overview
14 Leopold Center. Final Report: Potential to Operate Greenhouses and Aquaculture in Conjunction with Iowa’s Ethanol Plants.

February 29, 2008. http://www.agmrc.org/media/cms/GreenhouseAquaculture_2008_11806A2892322.pdf

15
http://www.hometownharvestseiowa.org/the-greenhouse
15
Jan Swinton has agreed to be a Greenhouse resource. She can be reached at: jan@pathfindersrcd.org
16
Leopold Center. Final Report: Potential to Operate Greenhouses and Aquaculture in Conjunction with
Iowa’s Ethanol Plants. February 29, 2008.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 8

Facebook, Inc. May 2014

the greenhouse node to increase the temperature. Early results indicate that the less than pristine
environment, including higher moisture levels, do not negatively affect the node’s functionality. 17
This model is also attractive because not only does it decrease the gas heating costs for the
greenhouse but it may also provide an opportunity for the greenhouse to pay, in part, for the node’s
electricity usage. Specific information on waste heat usage is currently unavailable from Notre
Dame’s greenhouse.

While the payback for a greenhouse is hard to assess, strategically co-locating with a greenhouse
has other benefits. For example, greenhouse produce could offset Facebook’s food expenditures.
Facebook’s cafeterias provide an extensive selection of food for employees. Co-locating with a
greenhouse could help Facebook decrease food and transportation costs and ensure year-round
access to high quality low-cost organic food at its offices while decreasing costs and transport
emissions.

This co-location also provides positive PR, and community engagement opportunities. In the case
of the Southeast Iowa Greenhouse project, the resulting press for Schaus Vorhies, which supplies
the greenhouse with waste heat, has been extremely positive related to fostering healthy eating in
children, supporting local communities and helping to increase organic food production.

Arboretum. The Condoret data center in Telecity, France built arboretum space into its data
center design to create an in-house use for waste heat. This on-site garden enhances the work
environment and provides a makeshift research center for local university climate scientists. The
Société Forestière and the French National Institute for Agricultural Research use the arboretum to
determine plant adaptability to changing climate conditions. As universities and laboratories often
have security and disclosure needs of their own, this partnership may be extremely attractive if
security remains a concern. Although pricing information is not available for this option, as the
arboretum is built into the data center structure, this cost is absorbed into the building process. 18

Swimming Pool. In Switzerland, waste heat will heat a community swimming pool. In this process,
waste heat flows through heat exchangers to warm the water that is pumped into the pool located
on the data center’s campus. 19 For Facebook to maximize the use of its low temperature waste
heat, the community pool and the data center can share a wall. Although exact waste heat usage
information is not available for the swimming pool model, it takes 8.34 Btu to increase one gallon of
water one degree Fahrenheit. 20 The pool also works for Facebook employees. An employee pool
encourages employee health and fitness, which has been linked to fewer employee sick days and
better stress management. If incorporated into a larger gym or fitness center, a pool heated by
waste heat can help decrease Facebook’s overall healthcare costs and expenditure on gym

17
http://www.datacenterknowledge.com/archives/2008/05/16/data-center-heats-a-greenhouse/

18
http://www.telecitygroup.com/our-company/news/2010/telecitygroup-opens-new-state-of-the-art-
data-centre-in-paris.htm
19
http://www.datacenterknowledge.com/archives/2008/04/02/data-center-used-to-heat-swimming-
pool/
20
http://smartenergy.illinois.edu/pdf/newsletter6_6.pdf

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 9

Facebook, Inc. May 2014

subsidies at future sites. Additionally a pool has high community and employee engagement
potential, which speak to two of Facebook’s core values: building social value and focusing on
impact. 21

Integration into Urban Settings. Several data centers have integrated into urban settings and
used waste heat to provide heat and hot water to local communities. Since many of these
initiatives are still in the experimental phase, hard data on financial benefits and waste heat usage
for this type of co-location is not available. Additionally data centers we’ve contacted are hesitant to
disclose their waste heat distribution infrastructure. In general, 0.24 Btu heats one cubic foot of air
one degree Fahrenheit. 22

There are several successful examples of data center waste heat heating urban areas in close
proximity. For example, waste heat from Telehouse West data center will produce nine megawatts
of power and heat the local community immediately surrounding the urban data center. In Finland,
waste heat from one underground data center warms pipes that heat 500 nearby homes. In
Canada, pneumatic baffles regulate waste heat supplied to an office directly above the data center
via a connected duct system. In all these instances, waste heat was used successfully in the
district heating of offices, homes, and water supplies in urban areas. 23 These initiatives have high
impact on lowering community heating expenditures and energy use, and have resulted in positive
media coverage and community engagement.

Cooling and Ancillary System

After site selection, the design of the data center is the next important consideration for its function
and performance. The building design must integrate all systems needed for the operation of the
data center, such as power and cooling, plus it needs to allow incorporation of co-location
possibilities like the ones presented in this report. Currently, Facebook’s data centers are stand-
alone structures which primarily house the servers and the employees who work in the building.
The cooling systems have been designed into the building to provide temperature and humidity
controlled air to the servers and extract hot air via a network of fans, ducts, and aisles. Although
some of the waste heat is used to control the climate for the building, the design can be taken

21 http://www.datacenterknowledge.com/archives/2008/04/02/data-center-used-to-heat-swimming-pool/

22 Samuel C. Sugarman, Sam Monger.2000. Testing and Balancing HVAC Air and Water Systems. Fairmont Press.

23http://www.datacenterknowledge.com/archives/2013/02/14/telehouse-to-expand-in-london/
http://searchdatacenter.techtarget.com/news/1314324/Companies-reuse-data-center-waste-heat-to-improve-energy-

efficiency

http://www.treehugger.com/clean-technology/data-center-housed-under-cathedral-heats-homes-in-
finland.html

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 10

Facebook, Inc. May 2014

further, getting cooling systems closer to the heat-generating components and integrating waste
heat use beyond just climate control.

Liquid-based cooling. Air is the preferred method of removing heat away from servers. However,
liquid-based cooling has a higher capacity to remove and transfer heat than air-based cooling.
Some benefits from liquid-based cooling include the ability to run servers at more efficient
operating temperatures due to the increased heat removal, as well as the reduction in
infrastructure needed for air cooling. Removing heat-using liquids also facilitates alternative uses
for waste heat. Liquids store heat better than air, allowing for heat transport over longer distances
and more efficient heat exchange. The ability to transfer heat over longer distances increases the
possibilities for the design of co-located facilities.

There are two methods of liquid-based cooling that could benefit Facebook: cooling of heat-
generating components with liquid-filled heat sinks, and submerging the servers in liquid. The
former brings the fluid directly to the CPU, GPU, and other components on the server, extracting
heat through a network of tubes and heat exchangers at the rack 24. The latter submerges the
entire rack in a non conductive, dielectric liquid which is circulated to remove heat from the servers.
Both methods utilize liquid in a closed loop system to control the quality of the liquid, and then use
heat exchangers to remove the heat away from the racks using facilities’ water (or possibly
wastewater).

Both of these technologies are developed and available. Component liquid cooling systems may
be retrofitted into existing server and rack designs, while submerged cooling would require a higher
capital investment as well as a significant amount of effort to switch from standard to submerged
server racks. Liquid cooling will also require additional considerations. Personnel must be trained
on the new systems and on the new safety procedures that must be developed to avoid injuries,
spills, and damage to the electronics.

The benefits from liquid cooling systems could impact Facebook in multiple ways. For a data
center site like Prineville, Oregon, where large amounts of air are circulated per minute, these
systems reduce the amount of circulated air, the level of humidity and cooling control, and the need
for cold and hot aisles. These reductions translate into smaller facilities with smaller air ducts,
lower ceilings, and fewer ancillary equipment like fans and humidifying water lines. Liquid cooling
systems also have the potential of improving the Power Usage Effectiveness (PUE) closer to
1.05 25. In addition, waste heat removal can be coupled with the site’s fresh water system to provide
year-round hot water. As an alternative, the use of wastewater for heat removal could reduce the
use of freshwater, freeing up potable water for the surrounding community. Uses for the hot water
can range from climate control to floor and sidewalk temperature control, as well as transferred to
nearby facilities, like pools and greenhouses.

24
http://asetek.com/data-center/technology/how-liquid-cooling-works.aspx
25
http://www.grcooling.com/liquid-cooling-retrofit-savings/

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 11

Facebook, Inc. May 2014

Ancillary Systems. There are ancillary technologies in different phases of development which
could prove useful for Facebook at reducing emissions and pollution. The three explored
technologies were energy storage, wastewater treatment, and air pollution reduction. Although
these systems may not be considered primary systems, they can reduce dependence on backup
generators, reduce freshwater usage, and reduce the carbon footprint for the company.

Energy Storage. Existing energy storage systems are expensive, and their use may have
environmental implications. Two developing technologies stand out due to their small
environmental impact: liquid metal batteries and compressed air energy storage (CAES). Liquid
metal batteries made from abundant metals and a molten salt electrolyte 26 will provide a low cost,
modular, silent, and emissions-free alternative for energy storage. Compressed air energy storage
uses mechanical energy to store compressed air for use at peak hours. One technology in
particular uses the heat generated from compression to improve energy extraction during peak
hours 27. Although unconfirmed, removed heat from the servers could potentially contribute to the
performance of the CAES system. These two systems are in different stages of development.
Liquid metal batteries have developed through to prototype manufacturing facilities, while the
CAES system has gone through several successful funding cycles. The environmental implications
for Facebook would arise from a reduced dependency on diesel backup generators, thus lowering
GHG emissions and noise pollution. These two technologies could benefit Facebook in the near
future.

Wastewater Treatment. Facebook’s selection of evaporative cooling for its data centers has
minimized its energy needs for the cooling system but requires the use of freshwater. The
company’s data center design allows it to recirculate the water and maintain an annualized Water
Usage Effectiveness (WUE) of 0.42 liters per kWh of energy. Facebook could reduce its freshwater
use by connecting to wastewater supplies and incorporating water treatment equipment. New
commercially available purification systems have become small enough to supply 60,000 liters of
potable water per day without the use of chemicals 28. Developing purification technology uses
bacteria to purify wastewater while producing methane 29, and, subsequently, electricity. Used in
combination with liquid-based cooling, these technologies would allow Facebook to replace
freshwater consumption with wastewater usage, reducing costs and benefiting the surrounding
communities.

Air Pollution Reduction. Although Facebook data centers only generate on-site GHG emissions
when the backup generators are in operation, energy consumption and other activities contribute to
the company’s carbon footprint. Facebook could compensate for its carbon emissions by using
pollution reduction technology. The use of a titanium dioxide coating on architectural structures has
been used in places like Mexico City to reduce nitrous oxides (NOx) to water, carbon dioxide, and

26
http://www.ambri.com/technology/
27
http://www.lightsail.com/
28
http://www.swiss-cleanwater-group.com/en/our-water-cleaning-products.html
29
http://cambrianinnovation.com/solutions/ecovolt/

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 12

Facebook, Inc. May 2014

calcium nitrate 30 (a fertilizer). Titanium dioxide is a photocatalytic activated by sunlight. This

characteristic has made it ideal for direct application to building structures. The modules currently
available are priced based on size, but a preliminary estimate is approximately $42 per square
foot. The coating has an expected life of 10 to 15 years, at which time recoating would cost around
$5 per square foot. Its use at Facebook facilities in pollution intensive locations would provide the
company the opportunity to compensate for its carbon impact and improve the air quality of the
surrounding communities.

Conclusion and Next Steps

Energy efficiency and sustainable design are at the heart of Facebook’s design philosophy. With
this in mind our MIT Sloan team evaluated existing and emerging initiatives with respect to site
location, employee engagement, water and material usage, holistic design thinking, and industrial
partnerships. These priority areas were distilled into three initiatives: site selection, co-location, and
cooling and ancillary systems. These initiatives provide Facebook with sustainable benefits such
as clean energy, water usage, and social impact.

The initiatives allow Facebook to explore new opportunities for improvement. As the company
scales up operations, site selection will be an increasingly important consideration. The proposed
multi-criteria decision analysis tool may help identify critical elements for data center location, such
as fiber connectivity and energy fuel mix. Furthermore, co-location considerations may influence
site selection. Co-location has significant implications for resource consumption. Opportunities like
greenhouses and arboretums can boost both employee and community involvement. In addition,
proximity to wastewater sources can facilitate improvements in other aspects of the data center,
such as liquid based cooling. Incorporating technologies like liquid based cooling, energy storage
systems, and pollution reducing coatings may further enhance Facebook’s power and water usage
effectiveness and reduce the company’s environmental impact.

This report was prepared with generalized recommendations for the different initiatives. The next
step for Facebook is to consider evaluating each initiative further for potential benefit and
applicability. The multi-criteria decision analysis may prove to be a useful tool for site location once
the relevant criteria for selection are identified. The co-location options and criteria may be
incorporated into the site selection tool and the design of new data centers to improve employee
health, employee engagement, and positive community impact. The liquid cooling technologies can
be implemented as commercially available or integrated into the open compute design.

We believe that these recommendations will benefit Facebook in multiple areas, and we look
forward to seeing them implemented in the next generation of their data centers.

30
https://www.youtube.com/watch?v=g2tCnub5Zlk#t=183

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 13

Facebook, Inc. May 2014

Appendix 1. Water scarcity mapping 31

31
USDA Drought Monitor. World Resources Institute Aqueduct.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 14

Facebook, Inc. May 2014

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 15

Facebook, Inc. May 2014

Appendix 2. Sample free cooling maps 32

32
The Green Grid

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 16

Facebook, Inc. May 2014

Appendix 3. Sample multi-criteria decision analysis

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 17

Facebook, Inc. May 2014

Appendix 4. U.S. wastewater treatment plants 33

33
EPA. “U.S. wastewater treatment plants”.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 18

Facebook, Inc. May 2014

Appendix 5. U.S. EPA and DOE CHP Resources.

ICF and Department of Energy Report. “Opportunities for Combined Heat and Power in Data
Centers.” March 2009:
https://www1.eere.energy.gov/manufacturing/datacenters/pdfs/chp_data_centers.pdf

U. S. Environmental Protection Agency Combined Heat and Power Partnership Report.

“The Role of Distributed Generation and Combined Heat and Power (CHP) Systems in Data
Centers.” August 2007:
http://www.epa.gov/chp/documents/datactr_whitepaper.pdf

Data Center Energy Resources:

http://energy.gov/eere/femp/data-center-energy-efficiency

Waste Heat Recovery: Technology and Opportunities in U.S. Industry Report:

http://www1.eere.energy.gov/manufacturing/intensiveprocesses/pdfs/waste_heat_recovery.pdf

Center of Expertise for Energy Efficiency in Data Centers:

(http://datacenters.lbl.gov/)

DOE CHP Technical Assistance Partnerships

(http://www1.eere.energy.gov/manufacturing/distributedenergy/chptaps.html#).

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 19

 Facebook, Inc. May 2014

Appendix 6. CHP Incentives and Grants

www.epa.gov/chp/policies/database.html :
Provides an interactive database of CHP related incentives and grants. The database can be narrowed to
include states with high data center build potential.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 20

Facebook, Inc. May 2014

Appendix 7. Iowa Hometown Harvest Greenhouse

http://www.hometownharvestseiowa.org/the-greenhouse via Jan Swinton.

MIT Sloan School of Management | 15.915 S-Lab | Draft Final Report | 21