Cloud Computing:

The Next Revolution in Information Technology

1

Green Cloud Computing

Energy-Efficient Cloud Computing:

Opportunities and Challenges

Dr. Rajkumar Buyya

Cloud Computing and Distributed Systems (CLOUDS) Lab
Dept. of Computer Science and Software Engineering
The University of Melbourne, Australia

www.cloudbus.org
www.buyya.com
www.manjrasoft.com
Major Sponsors/Supporters

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Clouds offer Subscription-Oriented IT

Services: {compute, apps, data,..} as a
Service (..aaS)
Public Cloud

Cloud
Manager

Clients

Private
Cloud

Other
Cloud Services

Govt.
Cloud Services

Cloud Computing
3 Main Types or Personalities

Software-as-a-Service (SaaS): A wide

range of application services delivered via
various business models normally available as
public offering

Platform-as-a-Service (PaaS): Application

development platforms provides authoring
and runtime environment

Infrastructure-as-a-Service (IaaS): Also

known as elastic compute clouds, enable
virtual hardware for various uses

Animoto,
Animoto, Sales
Sales Force,
Force, Google
Google
Document
Document
User Applications

Saa
S

Google
Google AppEngine,
AppEngine,
MapReduce,
MapReduce, Aneka,
Aneka, Microsoft
Microsoft
Azure
Cloud Programming Environment and Azure
Tools:
User-level and
infrastructure level
Platform

Web 2.0, Mashups, Concurrent and Distributed

Programming, Workflow
Cloud Hosting Platforms: QoS Negotiation
Admission Control, Pricing, SLA Management,
Monitoring

Amazon
Amazon EC2,
EC2, GoGrid,
GoGrid,
RightScale,
RightScale, Jovent
Jovent

Infrastructure

Cloud Physical Resources: Storage, virtualized

clusters, servers, network.

Cloud
Economy

Iaa
S Paa
S

Scientific Computing, Enterprise ISV, Social

Networking, Gaming

Public Cloud
(IaaS)

User

User
Middleware

Master Node

Private Cloud
(Heterogeneous
Resources)

Hybrid Cloud

Slave Nodes
8

Slave Nodes
(Cluster)

Several Benefits
Service
Oriented

Elastic

Virtualized

Cloud
Computing

Dynamic
(& Distributed)

Autonomic
Market
Oriented
(Pay As You Go)
9

Shared
(Economy of
Scale)

Dark side..

10

Gartner Report 2007: IT industry contributes

2% of world's total CO2 emissions

U.S. EPA Report 2007: 1.5% of total U.S. power

consumption used by data centers which has
more than doubled since 2000 and costs $4.5
billion

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

11

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Powering Cloud Infrastructure

Modern data centers, operating under the Cloud

computing model, are hosting a variety of
applications ranging from those that run for a
few seconds (e.g. serving requests of web
applications such as e-commerce and social
networks portals) to those that run for longer
periods of time (e.g. simulations or large dataset
processing).
However, Cloud Data Centers consume excessive
amount of energy:

According to McKinsey report on Revolutionizing Data Center

Energy Efficiency :

12

A typical data center consumes as much energy as 25,000 households.

The total energy bill for data centers in 2010 was over $11
billion and energy costs in a typical data center doubles every
five years.

Where Does the Power Go?

Power Consumption in the
Datacenter

Server/Storage
Computer Rm. AC

50%
34%

Conversion

7%

Network

7%

Lighting

2%

Compute
resources and
particularly servers
are at the heart of a
complex, evolving
system!
Source: APC
13

Clouds Impact on the

Environment

Data centers are not only expensive to

maintain, but also unfriendly to the
environment.

14

Carbon emission due to Data Centers worldwide is

now more than both Argentina and the Netherlands
emission.
High energy costs and huge carbon footprints are
incurred due to the massive amount of electricity
needed to power and cool the numerous servers
hosted in these data centers.

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

15

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Background

16

Traditionally, HPC (commodity clusters) & Data

center community has focused on performance
(speed).
At the same time, microprocessor vendors have
not only doubled the number of transistors (and
speed) every 18-24 months, but
they have also doubled
the power densities.
Moores Law for
Power Consumption:

Research Motivations of Power

Aware/Energy Efficient Computing

Rapid uptake of Cloud Data Centers for hosting industrial

applications
Reducing the operational costs of powering and cooling Data
Centers:

The tremendous increase in computer performance has come with an even

grater increase in power usage.
According to Eric Schmit, CEO of Google, what matter most to Google is
not speed but power, because data centers can consume as much
electricity as a city.

Improving reliability

As a rule of thumb, for every 10C increase in temperature, the failure rate of
a system doubles.
Computing environment affected the correctness of the results.

17

The 18-node Linux cluster produced an answer outside the residual (i.e., a silent error)
when running in dusty 85F warehouse but produced the correct answer when running in
a 65F machine-cooled room.

Reliability/Implications

18

Reliability of
Leading Edge
Supercomputer
(D. Reed, 2004)

Estimated Cost
of An hour of
system
downtime (W.
Feng, (ACM
Queue, 2003):

Power Aware Computing

Power Aware (PA) computing/communication:

System level power management

19

The objective of PA computing/communications is to improve power

management and consumption using the awareness of power consumption
of devices.
Power consumption is one of the most important considerations in mobile
devices due to the limitation of the battery life.

Recent devices (CPU, disk, communication links, etc.) support multiple

power modes.
Resource Management and Scheduling Systems can use these
multiple power modes to reduce the power consumption.

DVS (Dynamic Voltage Scaling)

DVS (Dynamic Voltage Scaling) technique

Reducing the dynamic energy consumption by lowering the supply voltage at

the cost of performance degradation
Recent processors support such ability to adjust the supply voltage
dynamically.
The dynamic energy consumption = * Vdd2 * Ncycle

Vdd : the supply voltage

Ncycle : the number of clock cycle

An example

deadline

Power

Power

deadline

5.02
2.02
10 msec
20

(a) Supply voltage = 5.0 V

25 msec

10 msec
(b) Supply voltage = 2.0 V

25 msec

DVS-based Power Aware Scheduling

Motivation:

21

Develop Resource Management and Scheduling Algorithms

that aim at minimizing the energy consumption at the same
meet the job deadline.
Exploit industrial move towards Utility Model / SLA-based
Resource Allocation for Cloud Computing

Taxonomy of Power Management

Techniques

22

Data Center Level

23

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

24

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Cloud Providers Measures

Cloud service providers need to adopt measures to ensure

that their profit margin is not dramatically reduced due to
high energy costs.

25

Amazon.coms estimate the energy-related costs of its data centers amount

to 42% of the total budget that include both direct power consumption and
the cooling infrastructure amortized over a 15-year period.
Google, Microsoft, and Yahoo are building large data centers in barren
desert land surrounding the Columbia River, USA to exploit cheap
hydroelectric power.

There is also increasing pressure from Governments

worldwide to reduce carbon footprints, which have a
significant impact on climate change.
Carbon Tax (July 2012 in Australia) on industries

Green Cloud: performance

energy efficiency

26

As energy costs are increasing while

availability dwindles, there is a need to shift
focus from optimising data center resource
management for pure performance alone to
optimising for energy efficiency while
maintaining high service level performance.
We propose Green Cloud computing model
that achieves not only efficient processing and
utilisation of computing infrastructure, but also
minimise energy consumption.

Green Cloud Computing

Revenue

Power
Consumptio
n

27

Cloud Usage Model

Cloud
Datacenter
A

LAN and
Gateway router
(Network
Devices)

End
User

Cloud
Datacenter
B
Interne
t
Service
Provide
r

Router
s

Interne
t

28

VM and
Storage
(Server)
Air Conditioning,
and Chiller
(Cooling Devices)

UPS, PDU,
lighting
(Electrical
Devices)
Cloud
Datacenter
C

Datacenter

Cloud
Computin
g

Green Cloud Computing

Architecture
Consumers
(or their
Brokers)

Green
Service
Allocator

Broker

Green
Negotiator

Service
Analyzer

Consumer
Profiler

Pricing

Energy
Monitor

Service
Scheduler

VM
Manager

Accounting

Virtual
Machines
(VMs)

Consumer
Interface
Cloud
Interface

QoS and
energy-based
provisioning

Cloud Service
Provider

Cloud Service
Provider

Physical
Machines
Power Off

29

Power On

Power On

Cloud Infrastructure

Power Off

Cloud Service
Provider

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

30

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Case Study 2: Dynamic VM

Consolidation

31

Three Sub-Problems

When to migrate VMs?

Which VMs to migrate?

VM selection algorithms

Where to migrate VMs?

32

Host overload detection algorithms

Host underload detection algorithms

VM placement algorithms

Proposed Power-Aware Algorithms

Host overload detection

Adaptive utilization threshold based algorithms

Regression based algorithms

Minimum Migration Time policy (MMT)

Random Selection policy (RS)
Maximum Correlation policy (MC)

VM placement algorithms

33

Migrating the VMs from the least utilized host

VM selection algorithms

Local Regression algorithm (LR)

Robust Local Regression algorithm (LRR)

Host underload detection algorithms

Median Absolute Deviation algorithm (MAD)

Interquartile Range algorithm (IQR)

Heuristic for the bin-packing problem Power-Aware Best Fit

Decreasing algorithm (PABFD)

Performance Metrics

SLA violation metrics

A combined metric that captures both energy

consumption and the level of SLA violations, Energy
and SLA Violation (ESV):

34

Overloading Time Fraction (OTF) - the time fraction, during

which active hosts experienced the 100% CPU utilization
Performance Degradation due to VM Migrations (PDM)
A combined SLA Violation metric (SLAV):
SLAV
= OTF * PDM

ESV = Energy * SLAV:

Simulation Setup

CloudSim with a power package

A Data Center consisting:

More than 1000 Heterogeneous VMs

corresponding to Amazon EC2 instance types
Workload traces from more than 1000 VMs from
servers located in more than 500 places around
the world.

35

800 heterogeneous physical servers containing HP

ProLiant ML110 G4 and HP ProLiant ML110 G5
servers.

The data were obtained from the CoMon project, a

monitoring infrastructure for PlanetLab

Best Algorithm Combinations and

Benchmark Algorithms

Dynamic VM consolidation significantly reduces energy consumption compared to non-power aware

allocation and static allocation policies, like DVFS, NPA (non-power aware)
36

Case Study 1: Key Observations

37

Dynamic VM consolidation algorithms significantly outperforms

static allocation policies.
Heuristic-based dynamic VM consolidation algorithms substantially
outperform the optimal online deterministic algorithm (THR-1.0) due
to a vastly reduced level of SLA violations.
The MMT policy produces better results compared to the MC and
RS policies, meaning that the minimization of the VM migration time
is more important than the minimization of the correlation between
VMs allocated to a host.
Dynamic VM consolidation algorithms based on local regression
outperform the threshold-based and adaptive-threshold based
algorithms due to better predictions of host overload, and therefore
decreased SLA violations and the number of VM migrations.

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

38

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Green Cloud or Brown Cloud?

Ideally, for every server virtualized, save

Plus

39

~$700 and ~7,000 kWh / year

4 tons of CO2 emissions / year
Power down underutilized physical servers, saving 40%
Desktop management, saving 35% / year

But currently

Cloud
datacenters

Location

Estimated power
usage
Effectiveness
1.21

Google

Lenoir

Apple

Apple, NC

Microsoft

Chicago, IL

1.22

Yahoo

La Vista, NE

1.16

%
of
Dirty
Energy
Generation
50.5% Coal,
38.7% Nuclear
50.5% Coal,
38.7% Nuclear
72.8% Coal,
22.3% Nuclear
73.1% Coal,
14.6% Nuclear

% of Renewable
Electricity
3.8%
3.8%
1.1%
7%

Some Observations
Datacenters has heterogeneous properties

40

Source:

Geographically distributed datacenters (different

environmental factors and electricity prices)
Each resource site has different CPU configurations
Each site has different energy efficiency
Different Carbon-footprint

Best Practices for Data Centers: Lessons Learned from Benchmarking 22 Data Centers by Lawrence Berkeley
National Laboratorys report

40

Green Cloud Architecture

41

Third Party: Green Offer and Carbon

Emission Directory

Carbon Emission Directory

Contains data on Power Usage Effectiveness (PUE), cooling

efficiency, carbon footprint, network cost
Helps user to select cloud services with minimum carbon
footprint
Incentive for providers

Require more carbon transparency from providers

Government role by enforcing policies such as Carbon Tax

Green Offer Directory

Incentive for users

42

Advertising tool to increase the market share, e.g. Google

Choosing more carbon efficient hours

Lists services with their discounted prices and green hours

User: Green Broker

A typical Cloud broker

Green Broker

43

Lease Cloud services

Schedule applications

1st layer: Analyze user

requirements
2nd layer: Calculates cost
and carbon footprint of
services
3rd layer: Carbon aware
scheduling

Provider: Green Middleware

44

Case Study: IaaS Cloud

45

Carbon Emission Directory: Stores all carbon

emission rates for each IaaS provider

Green Offer Directory: Receives number of VMs

that can be initiated at a particular time for
maximum energy efficiency

Green Broker: Computes schedule with the

lowest carbon emission based on application
requirements

Carbon Efficient Green Policy

(CEGP)

Collect resource requests from user and

resource site information such as VMs, carbon
emission rate, DCiE, CPU power efficiency
Sort jobs based on deadline
Sort resource sites based on carbon footprint:
Carbon
Emission

46

Datacenter
Efficiency

Energy
Efficiency of VM

Schedule greedily the most urgent deadline jobs

on the most power efficient resource site.

Simulation Setup
Parallel Workload: first week of LLNL Thunder
trace from Parallel Workload Archive (PWA)

Configuration of Cloud resource sites2:

D. Irwin, L. Grit, and J. Chase, Balancing risk and reward in a market-based task service, in Proc. of the 13th IEEE International Symposium on High
Performance Distributed Computing, Honolulu, USA, 2004.
L. Wang and Y. Lu, Efficient Power Management of Heterogeneous Soft Real-Time Clusters, in Proc. of the 2008 Real-Time Systems Symposium, Barcelona,
1

47

Deadline generated based methodology proposed by

Irwin et al. (2004)1

EDF: Carbon-Efficient (CEGP) VS EST

(Early Start-time) Algorithm (EST)

48

Case Study 2: Summary

49

Presented a Carbon Aware Green Cloud Framework to

improve the carbon footprint of Cloud computing.
Proposed framework provides incentives to both users
and providers to utilize and deliver the most Green"
services.
Proposed a Carbon Efficient Green Policy (CEGP) for
IaaS providers.
Green Policy CEGP can save up to 23% energy while
reducing the carbon footprint by about 25%.

Outline

Cloud Computing at a Glance

Powering Cloud Infrastructure

50

Trends, Foundations, Issues, Taxonomy

Green Cloud Computing: Framework

Energy-Efficient Resource Management

Energy Consumption, Costs, Implications

Power-Aware Computing

Cloud Benefits and Challenges

Within a Cloud Data Center

Across Multiple Data Centers (InterCloud)

Summary and Thoughts for Future

Conclusions

Clouds are essentially Data Centers hosting application

services offered on a subscription basis. However, they
consume high energy to maintain their operations.

Proposed heuristics for energy-efficient dynamic VM

consolidation that significantly reduce energy
consumption, while providing a low level of SLA violations.
Presented a Carbon Aware Green Cloud Framework to
improve the carbon footprint of Cloud computing
Open Issues:

51

high operational cost + environmental impact

EE Data Structures + Algorithms

EE Resource Management for other workloads (e.g., workflows)

References

Keynote Paper

Taxonomy + EE InterClouds:

52

R. Buyya, A. Beloglazov, J. Abawajy,

Energy-Efficient Management of Data Center Res
ources for Cloud Computing: A Vision, Architec
tural Elements, and Open Challenges
, Proceedings of the 2010 International
Conference on Parallel and Distributed
Processing Techniques and Applications
(PDPTA2010), Las Vegas, USA, July 12-15,
2010.
A. Beloglazov, R. Buyya, Y. Lee, A. Zomaya,
A Taxonomy and Survey of Energy-Efficient Dat
a Centers and Cloud Computing Systems
, Advances in Computers, Volume 82, 47-111pp,
M. Zelkowitz (editor), Elsevier, Amsterdam, The
Netherlands, March 2011.
S. Garg, C. Yeo, A Anandasivam, R. Buyya,
Environment-Conscious Scheduling of HPC Applic
ations on Distributed Cloud-oriented Data Cent
ers
, Journal of Parallel and Distributed Computing,
71(6):732-749, Elsevier Press, Amsterdam, The
Netherlands, June 2011.

Wiley Press, New York, USA,

Feb 2011

Thanks for your attention!

Are there any

Questions?
Comments/Suggestions

We welcome you to:

Study/Research with Us | Do Business with us!
http:/www.cloudbus.org | www.Manjrasoft.com
rbuyya@unimelb.edu.au | raj@manjrasoft.com
53

54

Green Cloud Computing

Simulation Results: ESV

7

ESV, x0.001

6
5
4
3
2
1
0

55

Simulation Results: Energy

130

Energy, kWh

120
110
100
90
80
70
60

56

Simulation Results: SLAV

9

SLAV, x0.00001

8
7
6
5
4
3
2
1
0

57

Simulation Results: the Number of

VM Migrations
VM Migrations, x1000

22.5
20.0
17.5
15.0
12.5
10.0
7.5
5.0

58