A Market Design For IaaS Cloud Resources

A Market Design for
IaaS Cloud Resources
Proefschrift voorgelegd op 30 mei 2011 tot het behalen van

de graad van Master in de Wetenschappen,
bij de faculteit Wetenschappen, aan de Universiteit
Antwerpen.
Promotoren:
prof. Dr. Jan Broeckhove
Dr. Kurt Vanmechelen
Joris Roovers
Research Group Computational

Modelling and Programming
Contents
List of Figures ii
List of Tables iv
Preface vi
Abstract 1
1 Introduction 2
1.1 Cloud Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 IaaS in Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.3 IaaS Cloud Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.5 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.6 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2 Market Design Principles 32

2.1 Introduction to Markets . . . . . . . . . . . . . . . . . . . . . . . . . . 32
2.2 Financial Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
2.3 Auction Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4 Desirable Market Properties . . . . . . . . . . . . . . . . . . . . . . . . 44
2.5 Designing a Commodity Market . . . . . . . . . . . . . . . . . . . . . . 47
3 IaaS Cloud Computing Market: The Continuous Reverse Auction 60

3.1 Continuous Double Auction for IaaS Resources . . . . . . . . . . . . . 61
3.2 Issues with the CDA model for IaaS resources . . . . . . . . . . . . . . 78
3.3 The Continuous Reverse Auction for IaaS Resources . . . . . . . . . . . 85
4 Market Implementation 101

4.1 Implementation of the Continuous Reverse Auction . . . . . . . . . . . 101
i
4.2 Implementing Tags and Constraints . . . . . . . . . . . . . . . . . . . . 106
4.3 Implementation of the prototype . . . . . . . . . . . . . . . . . . . . . 113
5 Conclusion 117
5.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
5.2 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
Appendices 121
Appendix A Cloud Computing in Practice: SaaS and PaaS 122

A.1 Software-as-a-Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
A.2 Platform-as-a-Service . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Appendix B Compound Requests 126
ii
List of Figures
1.1 Difficulty of accurately provisioning IT infrastructure . . . . . . . . . 4

1.2 Onion representation of the cloud computing hierarchy . . . . . . . . 7
1.3 Private, Public and Hybrid clouds . . . . . . . . . . . . . . . . . . . 9
1.4 Total costs of using a reserved small instance vs. using an on-demand
instance on Amazon’s EC2 platform . . . . . . . . . . . . . . . . . . 14
1.5 Evolution of the spot price of a small instance on Amazon’s EC2
platform in the EU-region . . . . . . . . . . . . . . . . . . . . . . . . 15
1.6 Retail competition model of the electricity market . . . . . . . . . . 23
1.7 Retail IaaS market based on the electricity retail market. . . . . . . 26
2.1 Evolution of the electricity spot price for 10th of February 2011 on
the European Energy Exchange. . . . . . . . . . . . . . . . . . . . . 38
2.2 Market actors in an commodity market . . . . . . . . . . . . . . . . 49
2.3 Market Settlement: On-demand Resources . . . . . . . . . . . . . . . 57
2.4 Market Settlement: Futures Resources . . . . . . . . . . . . . . . . . 57
2.5 Market Settlement: Reserved Resources . . . . . . . . . . . . . . . . 57
2.6 Market Settlement: Spot Resources . . . . . . . . . . . . . . . . . . . 58
3.1 Overview of the Continuous Double Auction for IaaS Resources . . . 73

3.2 Importance of taking into account the usage of additional costs com-
ponents (1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.3 Importance of taking into account the usage of additional costs com-
ponents (2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3.4 Schematic overview of the workings of the Continuous Reverse Auction 86
3.5 Price discovery in a CRA-based market . . . . . . . . . . . . . . . . 92
3.6 Hierarchical market setup to reduce CRA communication bottleneck 99
4.1 Implementation of a CRA-based market using webservices . . . . . . 102

4.2 Passing encrypted tickets between the PriceRequestService and the
ExecutionService via the consumer . . . . . . . . . . . . . . . . . . 106
iii
4.3 UML diagram of the webservices of a CRA-based IaaS market . . . 107
4.4 UML diagrams of a Tag and TagSet . . . . . . . . . . . . . . . . . . 107
4.5 UML diagrams of a ConstraintDescription and ContraintParameters
that specify a standard constraint . . . . . . . . . . . . . . . . . . . . 108
4.6 Using a constraint lookup service to determine constraint semantics 109
4.7 Implementing a standard constraint library using reflection . . . . . 110
4.8 Standardized tags in the standardized constraint library . . . . . . . 112
4.9 Implementation of custom constraints using webservices . . . . . . . 112
4.10 Prototype deployment . . . . . . . . . . . . . . . . . . . . . . . . . . 114
4.11 Screenshot of the resource specification user interface as offered by a
broker to a consumer . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.12 Screenshot of the user interface offered to administrators to monitor
and manage the market . . . . . . . . . . . . . . . . . . . . . . . . . 116
iv
List of Tables
1.1 Examples of Amazon EC2 instance types . . . . . . . . . . . . . . . 13

1.2 GoGrid Cloud Server sizes . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3 Comparison of the electrical power grid with current IaaS clouds . . 22
2.1 Well-known stock exchanges . . . . . . . . . . . . . . . . . . . . . . . 34

2.2 Well-known stock market indices . . . . . . . . . . . . . . . . . . . . 35
2.3 Well-known commodity exchanges . . . . . . . . . . . . . . . . . . . 37
3.1 Examples of additional constraints that are regularly used when de-
scribing IaaS resources. . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.2 Bandwidth prices of different IaaS providers . . . . . . . . . . . . . . 78
3.3 Resources and services that are commonly charged separately by IaaS
providers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4 Provider specified table that defines the relations between usage and
price tags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.1 Technologies used to implement the prototype . . . . . . . . . . . . . 115
v
Preface
The writing of a thesis is an essential part of the curriculum of the Master of Com-
puter Science programme at the University of Antwerp, as it fulfills the role of
demonstrating that the author is able to autonomously study and reason about a
complex scientific subject. The particular subject of this subject, “A Market De-
sign for IaaS Cloud Resources”, was chosen by me as I have always been intrigued
with the potential capabilities of cloud computing, as well as by the fact that I have
always had a strong interest in financial and commodity markets. To add a goal-
driven aspect to the thesis, I decided early on to focus on the design of market that
has concrete applicability in today’s IaaS ecosystem. In order to achieve this, an
incremental approach has been used; starting from abstract and simple models and
iteratively taking more complexity into account. The final result of this approach is
this thesis document and an accompanying prototype implementation.
When it comes to acknowledgements, I first and foremost wish to thank my

promoter Prof. Dr. Jan Broeckhove, co-promoter Dr. Kurt Vanmechelen and
mentor Ruben Van den Bossche. In particular, I wish to express sincere gratitude
to Dr. Kurt Vanmechelen who helped me tremendously in shaping and ordering
my thoughts; I am certain his help made a huge difference on this thesis’ outcome.
I would also like to thank my mother and father, and my two sisters Marjolein
and Annelies for their continuous moral and emotional support through all of the
past years. My gratitude goes also towards many close friends, in particular Ilse
and Geert, for understanding my daily challenges and doubts as well as pulling me
through on more difficult moments.
Finally, I wish to express very sincere gratitude and appreciation to my fellow

students, without whom I would not have been able to get through the past years.
I wish to mention Kurt, Robbe and Nico in particular, as I’ve had the luck to work
very closely with them on numerous occasions, in which time they became close
friends of mine.
vi
Abstract
In today’s world computers and the Internet have become an integral part of every-
day life and have introduced new ways of mass communication and collaboration
among people from across the globe. The latest iteration in the Internet ecosystem
has been the conception of Cloud Computing, a new service-oriented computing
paradigm that tries to realize the long-lived dream of a global utility-like computing
model by introducing the notion of on-demand, pay-what-you-use, elastic resources.
The so-called Infrastructure-as-a-Service (IaaS) providers materialize the cloud

computing model by offering consumers access to (virtualized) hardware resources,
and charging them based on how they use these resources, rather than charging a
one time fee for the allocation of the hardware. Besides the ease of deployment and
management of applications on third party remote resources, the IaaS paradigm also
increases the possibilities for trading IaaS resources on a scale that moves beyond
the individual provider level. Such inter-provider trading is used in other commodi-
ties markets such as electricity markets, to efficiently balance supply and demand on
relatively short timescales. In addition, an open market for IaaS resources can effi-
ciently open up the cloud IaaS provider space. At present however, the possibilities
for creating such open markets for IaaS resources have been largely unexplored.
This thesis is an attempt to shed some light on this topic by investigating whether
it is feasible to design an open market for IaaS resources, as well as elaborate on
the structure of such an IaaS market by pinpointing its core components and mech-
anisms. This thesis spends particular attention to the Continuous Double Auction
(CDA) mechanism that is used in many commodity markets, and finds it unsuit-
able as a market mechanism in the current IaaS setting due to the complexity and
diversity of price models used by providers. To overcome this, a new market mecha-
nism called the Continous Reverse Auction (CRA) is introduced, which solves some
of these issues. In order to validate its practical applicability, some implementation
considerations about the CRA mechanism are made, and a prototype is implemented
that is able to cope with the heterogeneity of different real-world providers.
CHAPTER 1
Introduction
In today’s world computers and the Internet have become an integral part of every-
day life as they are used by millions of people at any given time. The vast online
network of computers that is the Internet, has introduced new ways of mass com-
munication and collaboration among people from across the globe. Although it is
only a few years ago that the Internet was just a collection of simple connected
webpages, the Internet of today is increasingly being defined by complex webap-
plications, social networks and online communities. From a user’s point of view,
the advantages of using such online tools are apparent: they are immediate, easily
accessible, location/time independent and social. Because of this, more and more
users are leaving desktop applications for what they are, and have started using
their online counterparts.
As the online community continues to grow, it is only logical from a company’s

point of view to increasingly use the Internet as a platform on which to build ap-
plications and software. Doing so, is however not without challenges as the Internet
ecosystem is a very demanding and rapidly changing environment which is new to
most traditional companies. In particular, in order to build and maintain online
applications, companies need a lot of powerful servers and skilled software develop-
ers. As buying and maintaining these servers as well as composing and managing
developer teams are tedious undertakings, more and more companies are choosing to
no longer (fully) develop online applications in house, but instead outsource them to
external parties that are specialized in building such online applications and setting
up online software environments.
Both the trend of building and using applications on the web as well as the idea to
outsource certain IT tasks to web companies are today more commonly referred to as
Cloud Computing, a new service-oriented computing paradigm that can potentially
1.1. CLOUD COMPUTING 3
disrupt the way the computer industry has worked the last few decades.
The following sections will give a more detailed view of what cloud computing
is, by explaining some of the key concepts behind it and discuss some of its primary
benefits and challenges. Note that it is not the author’s intention to provide an
academic in-depth discussion about cloud computing, as this goes beyond the scope
of this thesis. Rather, the following section tries to outline the general ideas and
principles behind it in simple terms. For a more detailed introduction to cloud
computing, refer to [1] and [2]. Readers that are already familiar with the concepts
of cloud computing can safely skip to section 1.3.
1.1 Cloud Computing

1.1.1 Definition and core concepts
Cloud computing is considered by many as the next big thing in the evolution of the
computer. Despite this, the term cloud computing still remains rather abstract to
a lot of people. This vagueness that surrounds cloud computing is probably caused
by the fact that term is predominantly used as a buzz-word by marketeers of large
IT companies. Every website that does more than provide static information is
currently labeled a cloud application while a lot of those websites do not have any
of the specific characteristics that define cloud computing applications. In what
follows, the author will try to take away some of this vagueness by giving a generic
but clear definition of cloud computing and some of its basic concepts.
In its most generic definition, cloud computing represents a new way of think-
ing about computer resources in which privately owned infrastructure and software
are replaced by publicly shared variants. In the cloud computing model, private
companies no longer own expensive servers, nor do they build complex proprietary
software. Instead of following this traditional model, companies now pay for specific
services that are offered by specialized IT vendors. This new service-oriented way
of handling computer resources is perhaps the most essential characteristic of cloud
computing. Besides this service orientation, service delivery through the Internet is
another fundamental property of cloud computing. In other words, in order to use
a cloud computing service, one must be connected to the Internet for as long as the
service is being used1 . The term cloud computing expresses this inherit requirement
by using the word cloud as a metaphor for the Internet.
Although the service-oriented delivery through the Internet is a core characteris-

tic of cloud computing, there are some other traits to the paradigm that distinguish
1
This is technically not entirely correct, as certain cloud computing services keep running while
the user itself is not connected to the web. However, in order to manage or configure these services,
as well as to see their output, the user has to be connected to the Internet.
it from other similar paradigms and general service oriented architectures. That
is, the ideas behind cloud computing try to materialize John McCarthy’s idea2 of
utility-computing by exhibiting some of the same properties that are characteristic
of existing utilities such as electricity, water, gas and telephony. More specifically,
there are 3 key aspects that define cloud computing.
On-demand access to resources
Cloud computing resources share the on-demand features of classic utilities. That
is, when using cloud computing solutions, users do not need plan far ahead for pro-
visioning reasons. Instead, they can quickly and dynamically acquire the resources
they need, for the time that they need them. The term on-demand also refers to
the fact that cloud computing resources are accessible from anywhere at anytime,
as the only real requirement to access resources is an internet connection.
Elasticity
When using cloud services, users are given the illusion of having an infinite amount
of computing resources at their disposal that can easily scale to meet their needs.
By giving users the possibility to quickly scale resources up or down, they have
the flexibility to align IT infrastructure and resources with their business needs in
near real-time. The main benefit of this is that companies no longer need to waste
capacity as they provision for peak loads, nor do they need to worry about losing
customers caused by underprovisioning, as illustrated in figure 1.1.
(a) Provisioning for peak load (b) Underprovisioning
Figure 1.1: Difficulty of provisioning IT infrastructure. (a) Even if peak load can be
correctly anticipated, without elasticity, resources are wasted (shaded area) during
nonpeak times. (b) Potential revenue is sacrificed from users that are not served
(shaded area). Source: [1]
Metered usage
One of the main parallels between cloud computing and existing public utilities is
the payment system. When using a public utility, customers only need to pay for the
amount of resources they have used. Cloud computing suppliers try to adhere to this
model by only letting customers pay for the amount of computer resources that they
have used. This new payment methodology can potentially cause a dramatic shift
2
In 1961, John McCarthy was the first to express the idea that “one day computation may be
organized as a public utility”.
in the way computing resources are dealt with as with such a system, companies no
longer have to make upfront investments in infrastructure. Indeed, a shift to the pay-
what-you-use (or pay-per-use) model, incurs a shift from capital expenses (CAPEX)
to operational expenses (OPEX), which can have far reaching consequences in the
way companies look at and deal with information technology.
Using cloud computing solutions has some other interesting benefits in terms of
cost and flexibility. However, as there is no silver bullet, there are also some key
challenges that remain to be solved. Section 1.1.4 gives an overview of both these
additional benefits and the key challenges.
1.1.2 Cloud computing, a layered model

When more thoroughly defining cloud computing, typically 3 different forms of cloud
computing are recognized. These forms are called Infrastructure-as-a-Service (IaaS),
Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS).
Infrastructure-as-a-Service captures the idea that companies no longer have their

own datacenters but instead rent bare-bone servers from an other, specialized, IT
company. These rented servers are said to be ‘in the cloud’ as they are no longer a
physical part of a company’s computing infrastructure. In this context, the terms
on- and off-premise resource are commonly used to denote servers that are kept
on the premise of a company itself and servers that are hosted by those other IT
companies. Billing for the rented servers is typically done per used server minute or
hour. That is, companies only pay for the amount of computing resources they have
actually used; they are not charged for the time that the servers are unallocated (i.e.
pay-what-you-use).
IaaS providers typically make use of virtualization technology in order to con-
solidate as much customer appliances as possible on a single physical machine. To
make this possible, customers that want to make use of a certain IaaS, will need to
choose from a provider specified set of virtual machine images and types that define
the hardware characteristics, operating system and possibly software packages of
the machine they will get access to. By using such virtualization technology, IaaS
providers are able to get much higher utilization rates on their servers, decreasing
the provider’s costs. Additionally, as IaaS vendors provide services to many cus-
tomers at the same time, they can benefit from economies of scale that no single
company would be able to realize. These economies of scale are not only applicable
to the purchasing of new server hardware - IaaS providers can potentially get sub-
stantial discounts from hardware vendors as they buy servers in large volumes at
once - but also to operational costs such electricity and maintenance staff. A benefit
of this is that, given sufficient competition among IaaS providers, it is interesting for
providers to pass these savings to their customers. Clearly, this makes IaaS solutions
even more appealing from a consumer’s financial point of view.
Besides fully operational virtualized servers, many IaaS providers also offer other
services such as standalone storage services, firewall services, virtual network services
and others. Although the price model for these specific service can be rather different
per provider, virtually all providers apply the pay-what-you-use model for these
services, charging their customers per used storage, bandwidth or time unit.
Platform-as-a-Service (PaaS) builds upon the IaaS idea by adding a software

layer on top of the bare-bone servers. This provided software is not meant for end-
users but rather for software developers, providing building blocks that can be used
to build other, more complex applications. In many cases the software stack provides
a high level of abstraction by providing database managers, programming interfaces
and (vendor specific) software libraries. In doing so, developers no longer need
to worry about the hardware, operating system or low-level software that support
the programming interfaces. Additionally, PaaS solutions typically take care of
the technical issues of scaling up and down applications, parallelizing application
execution or supporting application security services.
When it comes to billing, PaaS products usually have a more complicated billing
method than IaaS solutions. That is, users (developers) are not charged by the time
that they used the platform but rather for the number of database queries made,
program interfaces used or webpages that were accessed.
The third form of cloud computing, Software-as-a-Service, is considered by many

as the most evolved form of cloud computing. With SaaS, complete software solu-
tions are offered directly to end-users. Examples include webmail, office suites, cus-
tomer relationship management (CRM) tools and many others. SaaS products have
become increasingly popular over the last few years as they have rapidly started re-
placing existing desktop applications. Because the size of the online market becomes
larger every day [3], it is becoming increasingly interesting for software vendors to
provide their services online. The result of this is that many software companies are
currently investing heavily in the SaaS market. Of course, as is the case with every
investment, investments in cloud computing need to be profitable. When it comes
to return-on-investment, SaaS vendors commonly use advertisements to monetize
the services that they offer for free to their users or ask additional fees when their
applications are used in corporate environments.
Having discussed the different forms of cloud computing, it is interesting to

briefly compare the different forms in terms of the problems they each try to solve.
(Note that making an exact separation is quite difficult since in many cases different
solutions can be used to solve the same problem. For example, a certain company
may choose a PaaS solution to implement its supply chain management (SCM) sys-
tem while another may choose a SaaS solution to do the same. The final choice is
often determined by the need for specific requirements, company policies or prefer-
ences.) When choosing for IaaS, companies choose the most flexible form of cloud
computing. That is, nearly every IT system can be deployed on the provided bare-
bone server machines. This makes IaaS an excellent choice for companies requiring
very specific (e.g legacy) hard- or software, raw compute power or full control. This
does mean however that building and maintaining solid IaaS solutions still requires
quite a lot of work compared to e.g. a SaaS solution. Such SaaS solutions typically
require only minimal or no development effort but usually offer very limited flexi-
bility. This makes them almost ideal for more widespread applications that handle
common tasks (e.g. email, accounting or human resource management) but less
attractive when more customization is desired. Integration with already existing
systems in particular can be very hard to achieve when using SaaS solutions. Note
that this does not necessarily mean that SaaS solutions are not mature enough, it
only means that SaaS products target a single, specific application. In between IaaS
and SaaS, PaaS often provides a good solution for companies that do want tighter
integration with existing applications or for companies that wish to use specific soft-
ware frameworks without wanting to worry about its deployment and maintenance.
Also, when no (solid) SaaS solution already exists, and there are no specific software
or hardware requirements, PaaS products are often the appropriate choice. Figure
1.2 shows how the different levels of abstraction of IaaS, PaaS and SaaS relate in an
onion model.
Figure 1.2: Onion representation of the cloud computing hierarchy. PaaS builds on
IaaS, while SaaS builds on both PaaS and IaaS.
1.1.3 Deployment models

Public and private clouds
When the term cloud computing or cloud is mentioned, almost always the notion
of a public cloud that is shared among different mutually untrusted consumers is
meant. However, in the context of IaaS, it is also possible to set up cloud-like
environments that can only be used by a specific group of users. The term private
cloud captures this idea, and is used to denote in-house (within a single company or
organization) datacenters that exhibit cloud-like characteristics. The most notable
of these characteristics, is perhaps the use of the pay-what-you-use model to charge
different parties within a single organization. By letting different departments and
teams pay for the amount of computing resources they use from their own budget,
concepts of activity-based costing [4] can be applied to IT-related projects.
As virtualization technology provides interesting benefits - virtual machine iso-

lation and increased server utilization - independent of the cloud deployment model,
it is also commonly used in private clouds. Tools such as Eucalyptus [5], Open-
Nebula [6] and VMWare’s vCloud [7] allow system adminstrators to manage such
a private virtualized datacenter by providing installation, monitoring, migration,
checkpointing and backup tools.
Although private clouds typically lack some of the advantages of public cloud
solutions (“infinite” scaling, economies of scale, etc.), they do give local users a flexi-
ble and agile infrastructure to run service workloads within their own administrative
domains. For many applications this can have clear benefits, as e.g. response times
can be much lower compared to public clouds (due to the locality) and security is
generally a lesser concern as the different parties on the cloud are from the same
organization and thus typically trusted.
When some of the public cloud features are wanted, while also benefiting of
these private cloud features, a so-called hybrid cloud can be set up that combines
the benefits from both.
Hybrid clouds
A hybrid cloud is a cloud environment that contains computing resources from both
private and public clouds. This can for example be done by setting up a virtual LAN
between private and public clouds so that from the user’s perspective it seems as if
the computing resources from the public cloud are part of the private cloud. The
use-case of (almost) transparently expanding in-house computing power with off-
premise capacity is certainly one of the core ideas behind hybrid clouds, as it allows
companies to add extra computing capacity on-demand. Perhaps more importantly
however, is the fact that such a hybrid model allows companies to keep using existing
in-house computing infrastructure, while also gaining some of the advantages of
public clouds. This effectively lets companies avoid having to lose existing (long-
term) investments, when switching to cloud computing solutions. Figure 1.3 gives
an overview of how hybrid clouds relate to private and public clouds.
The use of hybrid clouds can also has some interesting benefits in terms of
security. That is, when certain information requires high protection and security
standards, a company can choose to put the data and software that uses the data on
the private part of the hybrid cloud. By doing so, companies do not have to worry
about the additional security issues that public clouds introduce as they have full
control over the software that runs on the machines that contain the sensitive data.
As a final note on hybrid clouds, it is important to note that although using

hybrid clouds is certainly very interesting, setting them up is not always trivial. This
is partially caused by the lack of standardization between different virtualization
technologies, cloud providers and cloud management tools, but also by the fact that
setting up such a WAN-based computing environment has some inherit difficulties
in terms of network and security configuration.
Figure 1.3: Private, Public and Hybrid clouds (Based on image by Sam Johnston,
CC-BY-SA).
1.1.4 Challenges and Benefits

The main benefits of cloud computing are unmistakably derived from its core prin-
ciples that were discussed in section 1.1.1. On-demand access, elasticity and me-
tered usage are indeed the features that differentiate cloud computing from similar
paradigms. Besides these, cloud computing has some additional benefits of which
the most prominent are listed here.
General advantages of outsourcing Outsourcing activities allow companies to

focus on their core business. This leads to a reduction of capital investments
as well as general increased flexibility. Also, as the contractors to which the
activities are outsourced are specialized in their respective field, they typically
have more expertise and experience about the specifics of the technology that is
being used, leading to better performance and solutions. This is certainly true
in the case of IT outsourcing where skilled software developers and technicians
are needed to develop and maintain software and infrastructure.
Security and certification One of the aspects cloud providers typically invest in
heavily, is security and certification. While security breaches are a risk and
have far reaching consequences for any company, cloud providers risk even
more as an exploit or bug in their software or infrastructure effectively makes
all the companies that use their services vulnerable. Because such failures
of security can lead to customer lawsuits and serious damage to a brand’s
image, cloud providers have a strong incentive to take security very seriously.
Consequently, investing a lot of time and money in security related processes
and certification (also non-security related certification) is a logical thing to
do for many cloud providers. This directly benefits all customers, as making
such heavy investments in security and certification might normally not have
been justifiable for all applications.
Green IT As building and maintaining server infrastructure is a core part of the

business of almost all cloud providers, they have a strong incentive to build this
infrastructure as efficient as possible in order to minimize operating costs. As a
very large part of these costs are attributed to power consumption, minimizing
that power consumption has a strong impact on the cost efficiency of any
datacenter. Reducing power consumption can be achieved by make heavy use
of virtualization technology (as is the case with IaaS solutions) to increase the
average server utilization, lowering the amount of power that is needed per
compute cycle. Additionally, the use of efficient cooling technology can have
a large influence on bringing on those costs further down.
In order to get an idea about the power efficiency of a datacenter the concept
of Power Usage Effectiveness (PUE) is often used. PUE is defined as the ratio
of total amount of power used by a computer data center facility (including
all cooling, lightning, etc. equipment) to the power delivered to computing
equipment. In other words, PUE expresses how much of the power is actually
used by the computing equipment in contrast to cooling and other overhead.
Ideally, a datacenter’s PUE value is 1.0, which corresponds to a data center
where all of the electrical grid power supplied is devoted to IT equipment and
no power is used for cooling and other activities. Currently howevever, most
private datacenters are nowhere close to this ideal value, with most datacenters
having a PUE of around 1.9 [8]. However, by the consolidation of many servers
into large datacenters, cloud computing providers claim to achieve much bet-
ter results, with PUE figures as low as 1.2 [9] and even 1.1 [10]. Besides the
apparent financial benefits bringing down energy consumption incurs, mini-
mizing energy consumption also has a substantial impact on the environment,
hence the reference to “green IT”.
Although using cloud computing solutions has many advantages, there are still
challenges that need to be solved before cloud computing will become a mature
computing paradigm. A good overview of the current challenges that cloud comput-
ing solutions face is given by [1]. In order to be concise, only an overview of these
challenges is given here.
Availability of a Service Organizations may worry about whether cloud comput-

ing services have adequate availability; many cloud providers have yet to prove
that they are able to provide high levels of availability.
Data Lock-in The APIs of cloud computing services itself are still essentially pro-
prietary, and have not been subject of active standardization (although some
initiatives are on the way [11], most notably [12]). In other words, customers
cannot easily extract or export their data and programs from one provider to
run on another. Customer lock-in may be attractive from a cloud provider’s
viewpoint, but by locking users in, they are vulnerable to price increases, to
reliability problems, or even to providers going out of business.
Data Confidentiality and Auditability Most companies are not eager to move
their sensitive corporate data into the public cloud, as by doing so, they move
their data into what is essentially a public network. Consequently, by using
public clouds, companies expose their system to more attacks.
Currently however, most cloud providers give very little guarantees about the
safety of the data of their users. Related to this is the issue of the fact that
many nations set strict rules about moving customer data outside of national
borders (as would typically happen when migrating to a public cloud), or about
having the right to access all stored data (i.e. USA PATRIOT act).
Data Transfer Bottlenecks As applications continue to become more data-inten-

sive, they will require more and more bandwidth when running in the cloud.
Bandwidth is however limited; sending large amounts of data over the Inter-
net is both time-consuming and costly. Given this fact, it is only logical that
these (time and financial) costs incurred by data transfers will be an important
factor in the success or failure of cloud computing.
Performance Unpredictability Although sharing CPUs and main memory work

very well in virtualized environments, sharing I/O is a notable issue. An
example in [1] shows that when multiple VM’s simultaneously write to the
hard disk, the available disk bandwidth for each VM becomes unpredictable.
Scalable Storage As many I/O resources (most notable the harddisk) are typically
a lot slower than their computation counterparts, a possible concern can be
whether the storage resources of cloud providers are able to scale along with the
computation resources. Without special purpose hard- and software, access to
storage systems can potentially become the system’s bottleneck.
Bugs in Large-Scale Distributed Systems A difficult challenge for cloud com-

puting is removing errors in the very large distributed systems as used by cloud
providers. Finding and fixing these bugs can be notoriously difficult as many
of these bugs only occur at the scale of production datacenters, rendering their
reproduction in smaller setups impossible.
Scaling Quickly The speed with which cloud providers are able to scale can be
very important to customers as in many scenarios they require that their appli-
cations can scale according to quickly changing real-time loads. The difficulty
(and opportunity) for providers is to be able to respond quickly to scaling
requests in order to save their customers’ money and income opportunities,
without violating service level agreements.
Reputation Fate Sharing Reputations do not virtualize well. One customers bad
behavior can affect the reputation of the cloud as a whole. For instance, black-
listing of a cloud provider’s IP address range by spam-prevention services may
limit other customers’ connectivity and thus which applications can effectively
be hosted.
1.2. IAAS IN PRACTICE 12
Software Licensing Current software licenses commonly restrict the computers on

which the software can run. Users pay for the software and then pay an annual
maintenance fee. As cloud computing services are based on the pay-what-you-
use model, it is not entirely clear how these software licenses costs should be
incurred to end-users. This has led to the fact that many cloud computing
providers make extensive use of (free) open-source software, although this is
gradually changing now.
Consumer Choice In the current cloud computing ecosystem, consumers do not

have tools that can help them in the search for the cloud computing solu-
tion that best fits their needs. Furthermore, consumers are often constrained
to the use of certain cloud providers because of accounting, administrative,
legislative, policy or other reasons. Also, as there is no direct competition be-
tween providers and consumers, the notion of free choice does not really exist
in the current cloud computing ecosystem. If cloud computing is to evolve
into a mature computing paradigm, there is a need for an open market plat-
form that allows the trade of computing resources between all providers (and
consumers). The importance of the introduction of such a market for IaaS
resources is discussed in detail in section 1.3.3.
1.2 IaaS in Practice

In this section, examples of real-world services will be given to give the reader a better
understanding of how the previously discussed theoretical concepts are applied in
practice. To this end, the product lines of 2 large IaaS providers are discussed in
more detail here. As SaaS and PaaS solutions are of lesser importance in the context
of this thesis, according product examples have been moved to Appendix A.
1.2.1 Amazon Web Services

While there are many IaaS providers, in the current IaaS market, Amazon - the
company that everyone knows from their online book shop [13] - is considered to be
the absolute market leader. Although absolute numbers in terms of market share,
number of customers or sales volume are not available, Amazon is widely regarded
as the company that put IaaS on the map. Since its inception back in 2002, Amazon
Web Services (AWS) has added a whole range of products and services of which the
most important are listed here.
Amazon Elastic Compute Cloud (EC2)
The EC2 service [14] is the most complex solution offered by Amazon. The service
gives users access to a variety of virtual machines with specific hardware character-
istics, which they can use for whatever purpose they want. In order to specify and
manage instances - virtual machines with a specific AMI (Amazon Machine Image)
- Amazon provides users with a set APIs (webservices) that can be accessed directly,
through one of the many provided software libraries, through commandline tools or
through the online AWS Management Console (webinterface). Once launched, users
can access their resources through SSH.
Before launching an instance, users need to make several choices that specify
the different characteristics of the instance. The most important choice perhaps, is
that of the instance type, as it specifies the hardware characteristics of the instance
that is chosen. Table 1.1 gives some examples of different instance types and their
characteristics; a complete overview can be found in [15]. Note the usage of the
EC2 Compute Unit (or ECU for short), which is a hardware agnostic indicator of
the processor’s speed. More information about how the ECU corresponds to various
CPU benchmark results can be found online at [16].
Besides specifying the instance types, users also have to choose from one of
several availability regions, which determines in which of Amazon’s datacenters the
requested resources will be allocated. Currently, Amazon allows users to choose
from 5 different regions from across the globe (US-west, US-east, EU-west, APAC-
Singapore, APAC-Tokyo).
Instance type Hardware characteristics

Small Instance 1.7 GB memory
1 EC2 Compute Unit
(1 virtual core with 1 EC2 Compute Unit)
160 GB instance storage
32-bit platform
I/O Performance: Moderate
Large Instance 7.5 GB memory
4 EC2 Compute Units
(2 virtual cores with 2 EC2 Compute Units each)
850 GB instance storage
64-bit platform
I/O Performance: High
High-Memory 34.2 GB of memory
Double Extra 13 EC2 Compute Units
Large Instance (4 virtual cores with 3.25 EC2Compute Units each)
850 GB of instance storage
64-bit platform
I/O Performance: High
Table 1.1: Examples of Amazon EC2 instance types
When it comes to pricing, Amazon allows customers to choose from three differ-
ent price models. The most commonly used on-demand model, is the most flexible
price model that adheres most to the pay-what-you-use model as it allows users to
start using instances, stop them at any given time and only pay for the amount of
time (in whole hours) they have actually used the instances. For example, at the
time of writing, the cost for a small instance in the EU-West region is $0.095/hour if
a Linux or UNIX based AMI is chosen and $0.12/hour if a Microsoft Windows-based
AMI is chosen. Similarly, the cost for a Double Extra Large High-Memory instance
is $1.14/hour for Linux/UNIX and $1.24 for Windows.
The reserved price model allows customers to get cheaper hourly prices with
guaranteed instance availability by making up-front commitments. That is, if users
choose to pay an upfront fee to reserve a certain instance for a whole year at once,
Amazon offers significant discounts on the hourly price for running that instance,
while also guaranteeing the availability of that instance for that year. In reality, an
upfront payment of $227.50 is required in case of a small instance. Afterwards the
hourly cost of running a Linux- or UNIX-based image on that instance will only
be $0.04, which is less than half of the hourly on-demand price. Similarly, when
reserving a Double Extra Large High-Memory reserved instance for whole year at
once (upfront cost: $2650), the hourly cost will effectively drop to $0.48 per hour,
which is only 42% of the hourly on-demand price. Indeed, when it is known up front
that instances will need to run for at long time at once, using reserved instances
can incur significant savings. Figure 1.4 shows that at the current prices a small
instance should at least run 4137 hours a year before using a reserved instance is
more interesting than using an on-demand instance.
Figure 1.4: Total costs of using a reserved small instance vs. using an on-demand
instance on Amazon’s EC2 platform
Besides on-demand and reserved instances, Amazon’s Elastic Compute Cloud

(EC2) also allows the allocation of so-called spot instances [17]. These instance
types are called so, because they adhere to the volatile pricing characteristics that are
typical of any spot market (see section 2.2.6 for more information on spot markets).
That is, the spot price of the EC2 spot instance is changed on an hourly basis,
according to the current load on EC2. When demand for a specific instance type is
low in a certain availability region, the spot price for that instance type will also be
low for that availability region. Consequently, when demand for an instance type
is high in a certain region, the spot price for that instance type will also be high
within that region.
Since querying the user after each spot price change (after each hour) to deter-
mine whether the user still agrees with the new spot price could potentially generate
a lot of unnecessary traffic, the user is asked to provide a maximum price for the in-
stance before it is started. The spot instance will then keep running until it is either
terminated by the user, or until the spot price exceeds the user specified maximum
price, at which point it is terminated by the EC2 service itself.
Since the spot price of an instance is usually much lower than its on-demand
price, users can potentially save a lot of money by using spot instances instead
of on-demand instances. However, there is always a certain level of uncertainty
about the availability of the spot instances (since the spot price might unexpectedly
peak above the chosen maximum price), which makes using them less interesting for
applications that need guaranteed uptime.
Figure 1.5 shows the the evolution of the spot price for the small instance in
EU-region between 18/01/11 and 23/01/11.
Figure 1.5: Amazon EC2 Spot price of the small instance in the EU-region between
18/01/11 and 23/01/11. Source: [18]
Besides hourly costs associated with running instances, Amazon also charges
customers to transfer data in and out of their datacenters. Incoming data is priced at
$0.1/GB for all availability regions, while prices for outgoing datatransfer are similar
put depend on the region and the amount of data transfer per month ($0.15/GB for
the first 10 TB in the EU region).
In order to build more complete solutions, Amazon also offers additional services
that provide extra functionality to the EC2 platform at an additional cost.
Amazon CloudWatch Allows customers to monitor instances (CPU usage, net-

work traffic, etc).
Auto Scaling Allows customers to start or stop instances automatically, based on

triggers defined in Amazon Cloudwatch.
Elastic IP Addresses Allows customers to allocate fixed IP-addresses for instances.
Elastic Load Balancing Allows customers to balance the load of incoming request
over different instances.
Amazon Elastic Block Store Allows customers to work with block devices (hard-
disks) that can be swapped between different instances.
Details about the extra EC2 services and pricing can be found online [14] [19].
Amazon Simple Storage Service (S3)
The S3 service [20] is a storage service that allows users to store arbitrary data-
objects in so-called buckets that are identified by global unique identifiers. When
using this service, the developer can choose in which of the different availability
regions to create buckets and store data, giving him the freedom to optimize for
latency, minimize costs, or address regulatory requirements. Additionally, Amazon
provides authentication mechanisms that allows developers to ensure that data is
kept secure from unauthorized access. More specifically, objects and buckets can be
made private or public, and rights can be granted to specific users.
Access to the resources is through HTTP or Bitorrent, while the management
of the resources is done through webservices (again, Amazon also provides software
libraries, commandline tools and an online webinterface to do this). Notable is the
fact that Amazon guarantees an availability of 99.99% while guaranteeing a dura-
bility of 99.999999999%, by duplicating all objects over 2 separate facilities. Storing
a GB per month costs $0.14 in the EU-region (for the first 1000GB, afterwards the
price per GB slowly decreases), while $0.01 is paid per 1,000 management requests
(PUT, COPY, POST, LIST) and $0.01 per 10000 access requests (GET).
Amazon Simple Queue Service (SQS)
The SQS service [21] is a scalable hosted queue for storing messages up to 65KB of
text in any format. Such a message queue system can be used to build an automated
workflows in which certain components submit messages (workloads) to the queue
an other components process the messages (workload) by fetching them from the
queue. Access to the queue is done through webservice calls (software libraries are
also provided). Similar to the S3 service, Amazon allows granular access control by
account, IP-address and time-of-day. A price of $0.01 is charged per 10000 queue
requests for all availibility regions. Additionally, $0.1 is charged per GB of inbound
traffic (into the queue) for all regions, while the prices for outbound traffic differ
depending on region and volume (as with EC2, $0.15/GB for the first 10 TB in the
EU region).
Amazon SimpleDB
The SimpleDB service [22] is a scalable and flexible non-relational data store in
which developers can store and query data items. The idea behind SimpleDB is
that developers only need to worry about accessing the database, and not about the
management of the database itself. In practice, this means that the service creates
and manages multiple geographically distributed replicas of the data automatically
to enable high availability and data durability. The service applies the pay-what-
you-use model by charging customers only for the compute and storage resources
actually consumed in serving their requests. In practice, customers pay $0.154 per
used SimpleDB machine hour3 in the EU region, with an additional $0.1 per inbound
GB and $0.15 per outbound GB for the first 10 TB. Also, $0.275 per GB per month
is charged for data that is actually stored in the database.
Other services
Besides these core services, Amazon also offers a variety of additional products and
services that allows customers to build complete solutions on the Amazon Webser-
vices platform more easily.
Amazon CloudFront A content delivery network (CDN) for distributing objects

stored in S3 to so-called “edge locations” near the requester.
Amazon Relational Database Service (RDS) Service providing a scalable MySQL
compatible database server.
Amazon Simple Email Service (SES) Service providing bulk and transactional
email sending.
Amazon Virtual Private Cloud (VPC) Service that allows for the creation of
a logically isolated set of Amazon EC2 instances to be connected to an existing
network using a VPN or regular internet connection.
Amazon Elastic MapReduce Service that provides a hosted Hadoop framework
[23] running on the web-scale infrastructure of EC2 and S3.
Note that some of the services that are provided by Amazon, such as SimpleDB,
RDS, SQS, SES, MapReduce and others should arguably be labeled as PaaS solu-
tions rather than IaaS solutions as those services provide significant abstractions
3
From the SimpleDB website [22]: “Amazon SimpleDB measures the machine utilization of each
request and charges based on the amount of machine capacity used to complete the particular
request (SELECT, GET, PUT, etc.), normalized to the hourly capacity of a circa 2007 1.7 GHz
Xeon processor”
of underlying hardware, operating systems and software libraries. In order to avoid

confusion, the author wishes to note that when the term IaaS is used throughout the
rest of this thesis, the more hardware-related solutions (that is, compute, storage,
network and alike) are meant. More advanced services such as the ones mentioned
here will thus be considered PaaS solutions.
1.2.2 GoGrid
GoGrid [24] is an IaaS provider that offers services that are similar to that of Amazon
Web Services, albeit in a different form and under a slightly different price model.
At the time of writing, GoGrid offers four different infrastructure components [25].
Cloud Servers
The Cloud Servers service is a service very similar to Amazon’s EC2 service and
allows customers to allocate raw compute power by the specifying a server image
and size (similar to Amazon’s instance image and type). Additionally, the customer
needs to specify in which of GoGrid’s datacenters the server should be allocated
(US-West or US-East). The sizes of the servers are characterized by the amount
of memory, CPU cores and internal storage space they have. Table 1.2 shows the
currently available server sizes. Both standard Microsoft Windows and Linux images
are available, while images with more pre-installed software can be found on the
GoGrid Exchange [26]. Access to the cloud servers is through SSH (Linux) or
CPU (cores) RAM (GB) Storage Space (GB)

0.5 0.5 25
1 1 50
2 2 100
4 4 200
8 8 400
8/16* 16 800
Table 1.2: GoGrid Cloud Server sizes. (*Windows Cloud Servers deployed with 16 GB
RAM are assigned 8 cores, Linux cloud servers with 16 GB of RAM have 16 cores.)
Remote Desktop (Windows). Management of the servers is done through either

a web console or by using a RESTful API.
GoGrid tries to differentiate itself from other IaaS providers by guaranteeing
100% uptime for all of its services, while promising a 10000% refund (on a minute
by minute basis) when downtime does occur. Compare this to Amazon, which only
garantuees 99.95% uptime and uses a rather complex refund policy in which 10% of
the monthly bill during the downtime is given as service-credit that can be spend
on AWS services in the next month.
Billing is done per used RAM hour. A RAM hour is expressed as the number
of GBs of RAM that is used in a single hour. For example, using a server with
0.5 GB of RAM incurs the cost of half a RAM hour for every hour the server is
used, while a server with 4GB of RAM incurs a cost of 4 RAM hours per hour it
is used. The price for a RAM hour $0.19 if on-demand access is preferred, but if
stronger up-front commitments are made (if customers pay for a whole month or
year at once), the effective RAM hour cost lays between $0.05 and $0.08. Note that
the price for Windows or Linux-based server images is the same (although some
additional costs apply for the usage of Microsoft SQL Server products). Concerning
bandwidth, $0.29 is paid per GB of outbound transfer if the on-demand model is
chosen, while this can as low as $0.07 per GB when a large upfront commitment is
made. Inbound data-transfer is free.
Dedicated Servers
Besides virtualized hardware, GoGrid also offers their customers access to dedicated
servers, if customers don’t want to run their applications on a multi-customer en-
vironment. Dedicated servers are offered in 3 sizes (standard, advanced and ultra),
offering 4 up to 8 CPU cores and 8 up to 24 GB of RAM. Each server also has
between 500GB and 1TB of RAID storage. Prices range from $300 up to $600 a
month, with extra licensing fees for Windows or Redhat based operating systems.
If an up-front payment for a year is made, customers get a 2-month discount.
A special feature of the Dedicated Servers service is the possibility to setup a
hybrid cloud infrastructure between dedicated servers and cloud servers, combining
the isolation and performance guarantees given by the dedicated servers with the
flexibility and scalability offered by on-demand cloud servers. A possible use-case
could be to run high-performance database servers on dedicated servers while using
cloud webservers that scale according to current demand on a website.
Cloud Storage
The GoGrid Cloud Storage service is a scalable and reliable file-level backup service
for cloud servers running in the GoGrid cloud. Access to the storage is provided
through a variety of protocols such as SCP, FTP, SAMBA/CIFS and RSYNC.
All cloud servers receive a free cloud storage tier that allows customers to save
up to 10GB a month at no additional charges. Once more than this 10GB of storage
is used, $0.15 will be charged per extra GB.
Although as with the Cloud Service service, an 100% uptime (with 10,000% re-
funds) guarantee is given by GoGrid, they do not provide any assurances concerning
backup, nor do they make further commitments in the case of permanent data loss.
Load Balancers
The GoGrid load balancer service is a service allows customers to balance incoming
connections to servers according to the Round Robin or Least Connect4 algorithms.
The Load Balancers service is provided at no additional charge as part of the cloud
server and dedicated server services.
Besides these four core infrastructure components, GoGrid also offers firewall
and content delivery services at additional charges. More information can be found
online [25].
4
The Least Connect algorithm sends incoming connections to the servers with the least amount
of load on them.
1.3. IAAS CLOUD MARKETS 20
1.3 IaaS Cloud Markets

After having discussed the core concepts behind cloud computing, as well as given
some examples of some real world cloud computing products, it is now time to shift
focus to the core subjects of this thesis. The following sections explain the analogies
and differences between IaaS clouds and the power industry and indicate how these
analogies raise the idea of building an IaaS cloud market.
1.3.1 IaaS Clouds and the power industry

When pinpointing the core concepts behind cloud computing in section 1.1.1, it was
mentioned that many of the core principles behind cloud computing are the same
as those behind commoditized energy utilities. Although this is so for all public
utilities, it tends to be particularly true for electricity. The idea of being able to
plug in a network cable to get access to an infinite amount of computing power in
much the same way as power cords can be plugged in the wall to get access to a
seemingly infinite amount of electricity is indeed very appealing. From this point of
view, it is also clear that the cloud utility model applies the most to IaaS clouds,
and less to PaaS or SaaS as the former promises to deliver raw compute power where
the latter two deliver more advanced services on top of that. If IaaS is analogous to
raw electricity, then PaaS should be compared to the different electrical components
that make up household appliances, while SaaS products should be compared to the
appliances themselves.
The idea of having an electricity-like global computing grid that anyone can
access is however not new; it has been the core idea of what is known as Grid
computing [27] for quite some time. As grid computing has been the subject of
extensive research and as explaining the core differences with cloud computing has no
direct value to this thesis, it will not be discussed further here. A good comparison
between cloud and grid computing is given by [28] while the introduction of [2]
contains a good comparison between cloud, grid and cluster computing.
What is interesting in the context of this thesis however, is a more direct compar-
ison between electricity and cloud computing. To that end, a work by Nicolas Carr
provides valuable insight. In [29], Carr tells the story of the birth of the modern
electricity industry and reasons about the similarities and differences between the
revolution in the power industry and that of the computing industry. Specifically,
Carr emphasizes the analogies between the different problems and stages the power
industry had to go through in order to become a mature utility, and the difficulties
and stages the computing industry has gone through in the past and still has to go
through in order to characterized be as a true utility.
Using Carr’s work, as well as the structured comparison between computational

and electrical grids from [30], a systematic overview of the similarities and differences
between IaaS clouds and electrical grids was constructed in Table 1.3.
From this table, it can be learned that most of the similarities between IaaS
clouds and the electrical grid are based on their core idea to deliver a commoditized
utility (electricity vs. compute cycles) through a unified network. That is, leveraging
large economies of scale in order to offer consumers a pay-what-you-use model that
allows for easy and virtually infinite scaling. By doing so, consumers no longer
need to worry about the underlying complexity of setting up and maintaining the
hardware that generates the utility.
The main differences between the electrical grid and IaaS clouds are derived
from the observation that the electricity industry is a far more mature industry
than the cloud computing industry. It is important to note that this immaturity
is not due to a lack of technological advancement, but rather due to the lack of
guaranteed high reliability and quality of service in the current cloud computing
industry. Other issues such as the absence of standardization, regulation or a market
on which providers and consumers can trade resources, also contribute to this. In
fact, most of the differences between IaaS clouds and electrical grids are strongly
related to the current challenges cloud computing faces (see section 1.1.4). As such,
if these challenges can be overcome, cloud computing will become more like a true
utility and consequently, the analogy between IaaS clouds and electrical grids will
become stronger. Of course, it is important not to take the analogy between IaaS
clouds and electrical grids to far as in the end they deliver utilities that address very
different use cases.
1.3.2 Power industry markets

In [31], a detailed analysis of the fundamentals of power system economics is given.
Much of the information that is presented here reflects concepts and ideas explained
in that work.
For most of the twentieth century, when consumers wanted to buy electrical
energy they had no choice but to buy it from the organization that held the monopoly
for the supply of electricity in their region. However, by the 1980s, economists
started arguing that such a model was no longer sustainable. It was argued that the
monopoly status of these organizations removed the incentive to operate efficiently
and encouraged unnecessary investments. Additionally, the economists argued that
the cost of mistakes made by private organizations should be not be passed on to
consumers, as is almost always the case in monopoly economies. In order to solve
this, economists suggested that the supply of electricity should become the object of
market discipline rather than being regulated by a monopoly or government body.
After 20 years of lobbying and making changes in regulations and processes, it is

safe to say that today the liberation of electricity market is a fact in most (western)
countries. Because of this, the general structure of the electricity market has become
far more complex than back in the days of electricity monopolies as ideas, principles
and mechanisms from existing commodity markets have been applied to the open
Feature Electrical power grid IaaS Clouds

Network Transmission lines, underground Internet
cables. Various sophisticated
schemes for line protection.
Ease of use Very simple: plug and play Complex: setup of applications
and software
Reliability High: important lines are dupli- Potentially high: Reliable and
cated. Sophisticated protection durable technology exists. Ac-
schemes exist for power stations. tual long-term reliability remains
to be proven.
Storage Only for low-power DC using No storage of computation power
batteries. possible; data can be stored.
Security Fuses, circuit breakers, . . . System isolation (virtualization).
Firewalls, PKI, . . .
Payment model pay-what-you-use pay-what-you-use
Scaling model “infinite” “infinite”
Regulators In general, managed by au- Non-existent
tonomous bodies and govern-
ment regulators.
Standardization Many widely used standards No standards. Efforts are on the
(sockets, voltage, DC/AC, trans- way. Most notably: OpenStack
mission lines, . . . ) [12].
Driver Economies of scale Economies of scale
Service Level Far reaching SLAs and penalties. Generally very little guarantees.
Agreement
Resource hetero- Many different kind of genera- Many different kind of computa-
geneity tors. Heterogeneity hidden from tional resources. Heterogeneity
consumers. only partially hidden by virtual-
ization technology.
Market scope Exchange (spot) market trade No exchange market for
among providers. No exchange providers nor consumers.
market for consumers.
Time dependency Demand and supply strongly cor- Demand and supply correlated to
of demand/sup- related to time of day. time of day.
ply
Table 1.3: Comparison of the electrical power grid with current IaaS clouds
electricity market. As a result, the structure of the electricity market now has several
layers and levels, involves many different actors, and stimulates active competition
among its participants. To understand better how this market is exactly structured,
consider figure 1.6 which depicts a schematic overview of the most evolved form of
an open electricity market5 .
Figure 1.6: Retail competition model of electricity market based on model from [32].
Source: [31]
The most prominent feature of the diagram is the fact that 2 different sub-
markets are shown: a wholesale market between electricity generating companies
(gencos) and electricity retailers, and a retail market between consumers and retail-
ers. The existence of these separate markets is rather important as it indicates that
the companies that generate power do not need to be the same companies that sell
the power to consumers (although they regularly are). Implicitly, this also means
that it does not need to be case that the companies that distribute generated power
(so-called discos; which can be different companies than the gencos) to the consumers
are the same companies that are being paid for the delivered power. Indeed, in the
market model as shown here, it is possible that independent retailers lease power
lines from discos and sell electricity directly to consumers. Such a market situation
is particularly interesting for consumers as it gives them choice between different
retailers, allowing them to buy electricity from the retailer that is the cheapest in
their region or best fits their needs. Note that figure 1.6 also indicates that large
customers (such as companies that run factories) can buy their electricity directly on
5
Several less competitive (evolved) forms of electricity markets also exist. The discussion of such
markets is however outside the scope of this thesis. For more detailed information, see [32].
the wholesale market, taking out the middle man and margin that retailers typically
take on the energy they resell.
To get the required power for their customers, retailers thus have to buy electric-
ity from the different generating companies through the wholesale market. Buying
of this electricity can be done through bilateral deals between the gencos and the
retailers, or through a managed market in which a third party takes part in the
deal-making process.
When a bilateral deal is made between a genco and a retailer, they typically
agree on a certain price per kilowatt-hour (KWh) that needs to be delivered by the
genco over a certain period of time. Retailers typically make several of such deals
with different gencos in order to acquire the power that they need in order to satisfy
their customers for a certain period. Note that these deals are entirely private; buyer
and seller are in principle the only parties that know that a trade has occurred.
This is different in a managed market, where a third party helps buyers and
sellers find each other more easily. In practice, this is typically done by letting both
buyers (retailers) and sellers (gencos) submit offers with a price and an amount they
wish to sell or buy to the third party which will then find (optimal) matches be-
tween all submitted offers. Using such a managed market system has the potential
advantages of being fair and optimal while guaranteeing anonymity among the mar-
ket participants (besides the involved buyer and seller, only the market itself knows
which deal occured).
As simple as the idea of retailers buying electricity from gencos and selling it
to consumers may sound, there is a specific issue that comes with this way of trad-
ing that deserves extra attention. That is, as the exact consumption behavior of
customers is typically very unpredictable, it is generally very difficult for retailers
to know up-front how much electricity they will exactly need for a given period,
making it thus impossible for them to trade the exact amount of electricity they
will need. Although heuristic-based techniques can help to make better consump-
tion predictions (for example, statistical analysis may yield expected consumption
for a given period), it is typically very hard to make accurate predictions on small
timescales such as a single hour or less. However, making accurate provisioning is
vital to retailers, as not being able to do so may lead to serious financial losses.
More specifically, in case of overprovisioning, a serious financial loss for the retailer
can be incurred as he will be unable to sell already paid for energy to consumers.
Underprovisioning on the other hand, might mean that certain customers are un-
serviced, risking loss of customers and potential penalties or law suites because of
breaking certain service level agreements.
Related to this is the fact that in many cases, it is very difficult for gencos to
quickly respond to requests for additional power as generating extra electricity might
require the startup of additional generators such as windturbines, hydroelectric- or
fossil-fuel plants and so on, which can be both expensive and time-consuming. Sim-
ilarly, shutting down generators might be time-consuming and may incur additional
future costs when they need to be restarted. Also, as gencos and retailers make up
contracts up front, gencos will typically claim penalties when retailers want down-
scale contracts that were previously agreed upon.
In practice this problem is solved by the introduction of a so-called managed

spot market (more about spot markets in section 2.2.6) in which retailers in the
same region can buy and sell electricity to each other on an hourly basis. This
allows retailers to cope with over- or under-capacity on very small timescales. Such
trading is possible due to the fact that the electricity distribution system is in fact
a single grid on which all gencos from which the retailers buy their electricity are
interconnected. Although regulation of such a market is needed to ensure that the
market is balanced - it is important to have demand meet supply up to a certain
extend to ensure the stability of the grid - in practice the market tends to regulate
itself due to the general principles of demand and supply. That is, if the demand for
electricity is high for a certain period, spot prices will automatically go up, making it
more interesting for retailers to buy more electricity from gencos instead of buying
it from other retailers, subsequently lowering demand and thus lowering the spot
price. If demand is low, prices will go down, making it more interesting for retailers
to buy electricity from other retailers, affecting demand, making the prices go up
again.
As in reality, many gencos are also retailers, the provisioning problem becomes
more complex for them, as such retailers can both generate the electricity they
need to fulfill the demand of their customers, as well as buy and sell extra capacity
on the spot market. Producing and trading electricity can no longer be strictly
separated in such scenarios, as the most cost-efficient and profitable way of delivering
electricity becomes a combination of bilateral trading, spot market trading and in-
house electricity production.
1.3.3 IaaS Cloud Market

As the IaaS market continues to grow and the number of IaaS players continues to
increase (as predicted by Gartner [33]), it is likely that in time the IaaS provider
landscape will evolve towards an ecosystem that is similar to that of the power
industry. That is, monopoly positions of large companies such as Amazon will shrink
and direct competition among IaaS providers will increase. As a result, a market
model comparable to that of 1.6, in which different IaaS providers and retailers sell
computing resources to each other and to consumers, is likely to emerge. Figure 1.7
shows such a model.
In this IaaS market model, different IaaS providers sell IaaS resources to each
other and to retailers on the wholesale market, while retailers sell resources to con-
sumers on the retail market. Note that where the single electricity grid made it
possible for providers and retailers to resell electricity to each other in the case of
power markets, the Internet is now the main enabler of the IaaS market. The under-
lying idea of this is that for a whole range of IaaS applications, the actual location
Figure 1.7: Retail IaaS market based on the electricity retail market.
of the physical server on which the resources are allocated is insignificant (and con-
sequently the actual provider that provides these resources is too), as long as the
consumer can access them with a reasonable quality of service6 .
Although the retail market and the wholesale market are depicted similarly in
Figure 1.7, it is important to note that they are actually of a different nature. That is,
as is the case in the electricity industry, the wholesale market should be considered as
a (managed) market on which actual complex trading processes take place between
providers, retailers and large consumers, while the retail market should be observed
as the ad-hoc market that is formed by the free choice that consumers have between
different retailers.
In this context, the function of the retailer can effectively be seen as that of
a broker that hides the complexity of the wholesale market from the consumer.
Consequently, it is clear that the existence of retailers (or retail market) is not a
necessity for an IaaS market to exist, as it is also possible that end consumers take
part in the (wholesale) market directly. As such, the fact whether the wholesale
markets trades only among providers and retailers (inter-provider trading) or also
among providers and (end) consumers is a question of scope that has only limited
influence on the design of the mechanisms that are used in that market. In a less
complex model, consumer and providers thus trade directly with each, without the
6
Of course due to issues surrounding data locality - in many cases programs need to run on a
location (machine) where specific data is present - the statement that any program can run anywhere
is not true in general. However, for a large number of applications that data can be transferred to
the destination along with the program itself. Applications that have more heavy data requirements
can use e.g. distributed file systems to share data both efficient and transparently.
1.4. PROBLEM STATEMENT 27
explicit notion of the retail market.
Although the successful introduction of an IaaS market system has the potential
to materialize the long-lived idea of a truly global computing utility, it is crucial
that this market will be an open market in which many providers and consumers
can participate. That is, only when a market exists that has a sufficiently low entry
barrier that brings together many different consumers and providers, will the price
of computing resources be subject to market discipline rather than being dictated
by hardware vendors or specific IaaS providers.
In order to come to such an open market, there is a need for a exchange markets
that allow different parties to trade more easily by abiding to a predefined set of clear
rules and mechanisms. Only by using such exchange markets will trading become
accessible, fair and transparent.
Actually designing and implementing such an exchange market for IaaS resources
is however no easy feat. The main difficulty with designing an IaaS exchange market
is the fact that the good that is being traded is inherently more complex than in
an other commodity markets such as the electricity market. More specifically, the
problem lies in the fact that the e.g. kilo-watt hour is a one-dimensional homoge-
neous unit, while typical IaaS resources are heterogeneous and multi-dimensional
(e.g. an instance types is characterized by the different hardware components it is
built of). As standardization of resources and APIs among IaaS providers is virtually
non-existent, defining which goods are being traded is non-trivial.
Additionally, in reality many providers use different allocation and price models
(all based on the pay-what-you-use model, but implemented differently) to charge
their customers. Bringing together these different models into a single IaaS exchange
market is not straightforward as it involves searching for a common denominator
between all these models.
The design of a trading language for IaaS resources,the IaaS exchange market
and particularly the exchange’s components and mechanisms are the core subjects
of this thesis, and are progressively discussed throughout the following chapters.
1.4 Problem Statement

As it is important to very clearly state the main problems this thesis tries to solve
in order to outline which related issues are part of its actual subject and which are
not, this section specifies these problems explicitly. In particular, the core question
that this thesis tries to answer is threefold and as follows.
(a) Whether it is theoretically feasible to design an open exchange market for IaaS
cloud resources?
1.4. PROBLEM STATEMENT 28
(b) What would such an IaaS exchange market look like, what are its core compo-
nents and mechanisms?
(c) How can existing web service technology be used to implement such a market?
Of course, determining whether it is possible to design an open market can only

be done by actually trying to design such a market. This effectively means that the
answer to the first question is given by the answer to the second question. However,
to answer that second question, several sub-issues need to be addressed.
First of all, it is important to determine and define the different compo-

nents and actors an IaaS market is made of, which can largely be done by having
a look at how existing (commodity) markets are designed. This involves determin-
ing which actors can interact directly with the market that is being designed, and
which have to use secondary channels to do so. This also involves indicating which
functionalities and responsibilities are part of the core of the market, and
which ones can be taken care of by third parties.
An other important aspect of the design of the market will be the choice of the
market mechanism that determines which provider offers are matched with which
consumer offers. As the matching mechanism effectively determines how consumers
and providers interact with each other, specifying how it this mechanism works
exactly is obviously very important.
The choice of this matching mechanism language is one of the factors that has
a profound impact on the definition on the bidding language that determines
how different market participants should specify their offers. As IaaS resources are
typically very heterogeneous, this bidding language should be powerful enough to
express that heterogeneity.
Additionally, some attention will be spend on the issue of price discovery

which deals with how consumers and providers can get an idea of the value that
the market currently attaches to certain IaaS resources. Price discovery tools are
important as they shape market behavior by giving consumers and providers an idea
of how expensive a given resource is at a given moment, allowing them to act on
that.
Once these theoretical questions have been addressed, focus will be given to the
third question that asks how these proposed ideas can actually be implemented. The
answer to this question will be given by the specification of a software design and
architecture of the market. To verify that this design can be put into practice,
a prototype implementation will also be build and briefly discussed.
As the goal of this thesis is to design an open market for IaaS resources in which
all providers and consumers can participate, the author believes that it is important
1.5. RELATED WORK 29
for the market design to be tailored to today’s IaaS cloud ecosystem to ensure its
real-world applicability. This effectively means that a number of abstractions and
generalizations of IaaS characteristics that are made in other work will not be applied
in this thesis. Doing so ensures that existing IaaS providers can easily take part in
the market. In other words, not forcing providers in a certain price, hardware
or allocation model is a key differentiator of this thesis from other work.
As will become clear in the remainder of this thesis, this choice has a significant
impact on actual market design.
1.5 Related work

In order to give a broader context to the core questions and issues that have just
been outlined, it is important to reference some work that is related to this thesis.
As already mentioned, the idea of introducing a global computing network that

anyone can access has already existed as part of the Grid Computing paradigm
for many years. As such, a lot of related work has already been done in the Grid
Computing context. Some insightful work in this domain includes [34], [35], [36] and
[37]. A good overview of the recent evolution in the grid market area is given in [38].
Most of this work has however focused on grid economy management, utility-
driven scheduling and user value-driven resource allocation, while mostly ignoring
provider profit. However, in the cloud computing context, providers and provider
profit are first citizens. As such, not all ideas and principles from this previous work
are still applicable in the IaaS domain.
Some Grid Computing research that is closer related to the cloud computing
paradigm is given in work by Nicolas Dube and by the GridEcon project.
In [39], Nicolas Dube designs a market for grid computing resources based on the
continuous double auction mechanism. In his work, Dube develops a way to describe
heterogeneous computer resources, and describes how using so-called resource sets
in conjunction with a continuous double auction can be used to develop a market
model for grid computing. Dube also incorporates bidding strategies that lead to a
market equilibrium in his model by using an index-based price discovery mechanism.
The GridEcon project [40] [41] [42] was a “Sixth Framework Programme” Eu-
ropean Community funded project, that “explored the perceived economic barriers
to the adoption of grid and cloud computing”. In particular, the GridEcon project
worked out market matching mechanisms that determine which consumer can have
access to limited provider resources by using virtual machine scheduling algorithms.
A prototype of the proposed market architecture including the matching algorithms
can be found on the GridEcon website.
1.6. THESIS OUTLINE 30
Unrelated to grid computing is the recent launch of the Spotcloud platform [43]
by the US-based company Enomaly. Spotcloud is a spot market for cloud computing
resources that is built on Google’s App Engine which offers consumers “a secure
central platform for buying or selling unused cloud capacity based on location, cost
and quality”. Specifically, SpotCloud allows consumers to choose from a set of
hardware profiles and VM images that have been submitted by providers. Pricing
is determined by the providers and is fixed per appliance (hardware and VM) per
hour; this price also contains all extra costs such as bandwidth or storage costs. In
order to make profit, Enomaly takes a 30% cut from the profit of the providers.
1.6 Thesis outline

Chapter 1: Introduction
This first chapter introduced the concept of Cloud Computing, discussing its core
principles and characteristics. After explaining cloud computing’s layered structure
(IaaS, PaaS, SaaS) and deployment model (public, private and hybrid clouds), some
additional benefits and challenges were discussed. Some real-world examples of cloud
computing products were subsequently given and discussed. After this, the parallels
between IaaS Clouds and the electricity industry were indicated by a summary of
the main similarities and differences between both utilities. This lead to the topic
of electricity markets, and subsequently to the idea of building a similar market for
IaaS Clouds. The last sections clarified this idea by making the relevant research
questions more explicit and discussing related work.
Chapter 2: Market Design Principles
The second chapter provides some background information about markets in general,
which is needed to understand what the important elements are when designing a
market for IaaS resources. To this end, different types of markets, auction models
and the typical components of a commodity market are briefly covered and related
to the design of an IaaS market. Also, a short outline of some desirable market
properties is given. The goal of this chapter is not to provide an in-depth analysis
of each of these topics as a lot of other work already does that, but to outline the
relevant fundamental notions useful to computer scientists.
Chapter 3: IaaS Cloud Computing Market: The Continuous Reverse Auction
The third chapter elaborates on the most critical parts of the IaaS market, namely
its bidding language and matching (auction) mechanism. To do so, it starts with a
Continuous Double Auction based market for grid resources developed by Nicolas
Dube. Once this market model has been properly introduced, some problems this
model faces in the cloud computing setting will be pointed out. Working from the
particularities of these problems, a new bidding language and market mechanism
(the Continuous Reverse Auction) that is able to cope with the specific issues of the
IaaS ecosystem, will be iteratively constructed.
1.6. THESIS OUTLINE 31
Chapter 4: Market Implementation

The design of market mechanisms is mainly theoretical in nature. Nevertheless,
investigating how markets can be implemented in practice is valuable as this imple-
mentation is a complex technological undertaking. In order to give an idea of the
involved complexity, this chapter focuses on the concepts that are introduced in the
previous chapter, and discusses how they can be implemented concretely. Further-
more, the prototype that was implemented as part of this thesis will also be briefly
discussed at the end of this chapter.
Chapter 5: Conclusion
The final chapter of this thesis summarizes the previous chapters and puts forward
the core concepts and ideas that were introduced. To finish, it puts forward some
future work and formulates a final conclusion.
CHAPTER 2
Market Design Principles
This chapter provides some background information about markets in general, which
is needed to understand what the important aspects are when designing a market
for IaaS resources. To this end, different types of markets, auction models and the
typical components of a commodity market are briefly covered and related to the
design of an IaaS market. Also, a short outline of some desirable market properties
is provided. The goal is not to provide an in-depth analysis of each of these topics
as a lot of other work (such as [44]) already does that, but to outline the relevant
fundamental notions useful to computer scientists.
2.1 Introduction to Markets

When designing a market, a very large number of factors need to be taken into
account. Trying to find all of these factors by brainstorming and unstructured
reasoning is destined to lead to a flawed design in which certain important aspects
have not been taken into account. As such, it is imperative to start the market
design process by first taking a step back and having a look at how existing markets
operate. This section starts by giving a generic (yet practical) definition of what a
market is, followed by a oversight of how markets can be categorized.
2.1.1 Market: a definition

In mainstream economics, a market(place) is any system that allows buyers and sell-
ers to exchange goods, services or information. This exchange can be done directly
between the different buyers and sellers or through intermediaries.
The notion of competition is essential in a market as it distinguishes a market
from regular trade. That is, two parties may trade goods, services or information,
but it takes at least a third party to have a market so that there is competition
2.1. INTRODUCTION TO MARKETS 33
on at least one of its two sides. In other words, in a market, forces of demand and
supply are always present. Markets typically vary in size, location and type as well
as in the types of goods and services traded.
Many other formal and informal definitions of markets can be found in various
works [45] [46].
2.1.2 Market Organization

There are many ways to categorize different kinds of markets more specifically. One
way to do this is by looking at how competition among the different market par-
ticipants is structured. Examples include markets of imperfect competition such as
monopolies (and others) and markets of perfect competition in which the behavior
of a single buyer or seller does not affect the market’s outcome. Although a detailed
study each of these market structures (conducted in the field of micro-economics)
can result in valuable insights, it has less added value in the context of this thesis as
the resulting conclusions are mainly of a theoretical nature. For a general overview
of market competition structures and micro-economics, [47] provides a good intro-
ductory work. More in-depth analysis can be found in [48].
A practical way to classify markets, is by the way goods are traded. That is,
in some markets there are intermediates that help buyers and sellers trade good to
each other, while in some markets buyers and sellers trade directly with each other.
Markets in which intermediates have a prominent role are sometimes also referred to
as managed markets or exchange markets. When there is no such party involved, the
term over-the-counter (OTC) or bilateral trading is often used. Where a core aspect
of OTC trading is that no strict form-factor exists, exchanges are typically char-
acterized by transparent pricing, clear basic trading regulations and mechanisms,
and precise transaction costs and fees. Additionally, a typical exchange market in-
cludes mechanisms to determine the price of the traded item, to communicate price
information to its participants and to facilitate deals and transactions.
When studying exchange markets in greater depth, the distinction between a

so-called dealer market and auction market is usually made. In a dealer market,
human market makers manually match deals from sellers and buyers, allowing them
to make profit based on the difference between the bid and the ask price of the
asset that is being sold. The theory is that competition between dealers will pro-
vide the best possible price for investors. In an auction market, a certain type of
auction mechanism is used to facilitate trade among participants. Such an auction
mechanism has the clear advantage that it is unbiased, simple and deterministic,
facilitating its implementation in electronic trading platforms. As auction mecha-
nisms are especially important in commodity markets, they are discussed in more
detail in section 2.3.
A third way to classify goods is by the types of good or services the market
2.2. FINANCIAL MARKETS 34
trades. The most well-known of these are the physical retail markets, such as local
farmers’ markets which are held in town squares on an ongoing or occasional basis.
These markets are characterized by their adhoc nature and the absence of strict
regulation. More regulated and structured are the so-called financial markets, which
is a group of markets that focus on trading either financial securities (such as stocks
and bonds), liquid assets or commodities through exchanges or OTC trading. As
the financial markets are widely considered to be the most mature and sophisticated
trading platforms, having a closer look at them can provide valuable insight. In
the following section, an overview of the most important financial markets is given
together with some considerations of how some ideas and principles of these markets
apply to the IaaS cloud market.
2.2 Financial Markets

2.2.1 Stock markets
A stock market (or equity market) is a public market that trades company shares
and derivatives (see 2.2.5). The stock market can be considered as one of the most
essential pieces of a market economy as it gives companies access to capital and allows
investors to acquire partial ownership of a company. The stock market consists of
two sub markets, called the primary and secondary market. The primary market
deals with issuing the shares of a company when it goes public. The trading of all
other (already public) shares happens on the secondary market. Some of the most
well-known and largest financial markets in the world are stock markets, usually
(but not necessarily) named after their geographical location. Examples are given
in Table 2.1.
Stock Exchange Abbreviation
New York Stock Exchange NYSE
London Stock Exchange LSE
Frankfurt Stock Exchange FSE
Tokyo Stock Exchange TSE
Euronext Paris -
NASDAQ (New York) NASDAQ
Table 2.1: Well-known stock exchanges
As can be seen in the table, stock markets are almost exclusively exchange mar-
kets; OTC trading is almost never used for stocks. The type of exchange mechanism
that is used is typically chosen by the exchange itself, and can thus be different for
each exchange. For example, the NYSE is an auction market that uses an open-
outcry double auction mechanism (see 2.3.5), while the NASDAQ is a dealer market.
As the value of stock market enlisted companies is effectively determined on that

market, and as in reality almost all large companies are enlisted on such a stock
market, it is essentially possible to extract aggregated information about a larger
set of companies at once. Doing so can give buyers and sellers a better idea of how
an industry or even an entire economy is doing. The means of doing this is through
a so-called market index, which shows how a set of publicly owned companies have
traded during a standard trading session in the stock market. By comparing the
value of the index against previous values, economists can get an idea how markets,
industries or economies are forming. Some well-known indices are shown in Table
2.2.
Index Aggregation
Dow Jones 30 large US-based companies
NASDAQ Composite Over 3000 component, mainly technology-
oriented companies
S&P 500 500 large US-based companies that trade
on NYSE or NASDAQ
Nikkei 225 225 large companies listed on the TSE
FTSE 100 100 most highly capitalized UK companies
listed on the LSE
Table 2.2: Well-known stock market indices
Although stock markets and markets to trade IaaS resources seem to have little
in common, there is an important lesson that can be learned from the discussion of
stock markets in the context of this thesis. Namely that even a market that is as large
and mature as the stock market still exists of multiple exchanges that all use different
rules, auction types and have different specializations. In other words, even after
years of experience and change, markets still remain highly heterogeneous. As such,
expecting that a single exchange market will emerge on which all IaaS resources will
be traded is not realistic. Instead, it is more likely that through a combination of
research and commercial incentives, multiple IaaS exchange markets will emerge that
use different mechanisms, rules and practices. As the IaaS industry is particularly
heterogeneous (both in the resources that are offered as in the use of different price
and allocation models), there is certainly room for multiple IaaS exchange markets
that have different characteristics and specialties. Given these remarks, it is clear
that this thesis does not claim to search for or present the solution, but rather to
present a solution that is based on well-founded ideas and principles.
2.2.2 Bond markets

The bond market facilitates the trade of bonds. A bond is a (typically long-term)
debt security by which an investor (who buys the bond) lends money to the issuer of
the bond. The issuer of a bond is then obligated to repay his debt in fixed slices along
with an interest rate. This repayment is do be done at fixed intervals (usually once
a year). When mentioning the bond market, one usually refers to the government
bond market because of its size and liquidity. (Similarly to the stock market, the
bond market can be divided in a primary and secondary market in which bonds are
issued for the the first time or re-traded among market participants). Contrary to
stock markets, most trading is done OTC in bond markets.
2.2.3 Foreign exchange markets

Also known as currency markets or forex markets, foreign exchange markets allow
OTC trading of currencies1 . Although different financial centers can be considered
as different forex markets, most of the time only a single global forex market is
mentioned that depicts the worldwide decentralized market formed by these inde-
pendent geographically dispersed forex markets. Note that this is also regularly
done in the IaaS cloud market context, where the previously mentioned different
decentralized exchange markets are also commonly denoted as a single worldwide
market. In the remainder of this thesis, context will indicate whether with the IaaS
market only a single exchange market is meant, or the worldwide market formed by
the independent IaaS markets.
The primary goal of the global foreign exchange is to make international trade
easier, by allowing businesses to convert one currency to another. The current forex
market took shape during the early 1970s when countries gradually switched from
the fixed exchange rate system as defined by the Bretton Woods System2 to floating
exchange rates.
2.2.4 Commodity Markets

A commodity market is characterized by the fact that it facilitates the exchange
of raw and easily described products. Typical examples of such products include
agricultural commodities such as cotton, wheat and cattle; (precious) metals such
as gold, silver, copper, aluminum and others; and energy commodities such as elec-
tricity, natural gas and uranium. In order to do the actual trading, standardized
contracts that describe good quality and characteristics, volume, delivery date and
price are bought and sold among the market participants.
Rooting from the middle of the nineteenth century in the United States to trade
agricultural products, commodity markets have grown to become the largest of all
financial markets in terms of daily traded monetary volume. Trade can occur both
OTC or through market exchanges. Today about 30 major of such commodity
exchange markets exist, of which the Chicago Mercantile Exchange (CME) is prob-
ably the most well-known. Table 2.3 lists some other examples of large commodity
markets along with the products that are traded on these markets.
As cloud computing, and IaaS in particular, deals with the commoditization of

1
Note that the term exchange is used, while in reality most trade is done OTC.
2
The Bretton Woods system was a system of rules and procedures that was agreed upon by the
allied nations at the end of the second World War, in order to regulate the international monetary
system.
Exchange Location Product Types

Chicago Mercantile Exchange Chicago, US Meats, Currencies, Eurodol-
lars, Equity Index
New York Mercantile Exchange New York, US Energy, Precious Metals, In-
dustrial Metals
European Climate Exchange Europe Emissions
Tokyo Commodity Exchange Tokyo, Japan Energy, Precious Metals, In-
dustrial Metals, Agricultural
Table 2.3: Well-known commodity exchanges
IT products (as already mentioned in the introductory chapter), it is logical that

the study of commodity markets will play an important role when looking into
IaaS market design. In order for an IaaS market to be organized, it is important
that some of the key concepts of commodity markets such as standardized trading
contracts, bidding languages and good delivery are further examined and studied in
the context of IaaS resources. To do so, section 2.5 discusses the core components
and functionalities of commodity markets and explains how these apply to IaaS
markets.
2.2.5 Derivatives markets

Derivative markets trade derivatives, which are agreements between two parties
based on the expected value of an asset (such as a share, bond or currency) at a
specified time in the future. The term derivative is used to denote the fact that
the agreement itself derives its value from the (future) valuation of the underlying
asset. Due to the speculative nature of the derivative markets, they tend to attract
investors that are willing to take more risk in order to leverage profits.
When the trading of a derivative is done using an exchange (as opposed to

OTC), the term futures market or futures exchange are typically used. Futures
markets deal with the exchange of futures contracts (or just a futures), which is an
agreement between two parties to buy or sell a specified asset at a specific date for
a specific price (that is determined on the day the futures contract is negotiated).
All futures contracts have a binding character, as they legally obligate the owner of
the futures contract to buy or sell the specified asset. The so-called option does not
have this obligation, it gives the owner the right to buy or sell the specified asset at
the specified date, at a price that is agreed upon when the option is bought or sold.
Although the term futures market is mostly used to refer to the stock, bond or
forex market, it can also be used to refer to any other market that exchanges con-
tracts that are to be fulfilled at some point in the future. For example, in commodity
markets, a futures contract refers to an agreement to buy or sell a specified volume
of a commodity for a fixed price at some point in the future. Clearly, the idea of fu-
Figure 2.1: Evolution of the electricity spot price for 10th of February 2011 on the
European Energy Exchange. Time is represented on the horizontal axis, price on
the vertical axis. The bold black line represents the average price over 24 hours, the
red bold line the average price during peak hours (between 08.00h and 20.00h).
tures (and options) can also be applied to the IaaS context, where a futures contract
would involve now buying or selling a set of computational resources that are to be
used at some point in the future. By trading futures contracts, both IaaS providers
and consumers can make medium and longterm provisioning decisions more easily.
2.2.6 Spot Markets

Whereas a derivative (futures) market facilitates the trading of contracts that are to
be fulfilled in the future, a spot market accommodates trading assets or commodities
that are to be delivered (almost) immediately. The word spot market refers to the
fact the price negotiation and trading is done on the spot, much as it has been
done for centuries on street markets. Spot markets are unique in the fact that the
so-called spot price for a good or asset is typically quite volatile as it reflects the
direct relation between supply and demand for the good (or asset) within a short
timeframe. This price volatility is caused by the fact that it is generally impossible
to create additional supply of the good that is being traded within the required
short delivery timeframe. Because of this volatility, the less volatile future markets
are regularly used to acquire the bulk of a specific good or asset, while spot markets
are more frequently used to deal with the real-time differences between needed and
already owned goods or assets. Although this inherit volatility of the spot market
can make it difficult to predict the spot price at a given moment in time, historical
analysis of the spot price of many goods frequently shows hourly, daily and monthly
patterns. Successfully speculating on these patterns, can induce large profits within
very small timeframes.
Examples of large spot markets are the forex spot market, in which traded cur-
rency needs to be delivered within 2 days, and various energy spot markets. As
already mentioned in the introductory chapter, in electricity spot markets, different
providers trade electricity on an hourly basis. Doing so allows them to cope with
the mismatch between their provisioned capacity and the capacity they actually
need. In many cases, it is cheaper to buy extra electricity from other providers
than to pay for the startup cost to power on an extra generator (also, depending on
the generator type, starting an extra generator might take a substantial amount of
time). Likewise, in many cases it is cheaper to keep producing electricity and to sell
overcapacity, than to shutdown a generator and risking to have to restart it again
within the next few hours (incurring the startup cost) [49].
Figure 2.1 shows the evolution of the electricity spot price of the 10th of February
2011 of the European Energy Exchange (EEX) [50] - the leading energy exchange
in Central Europe. On the graph, you can clearly see human behavioral patterns.
That is, the spot price is lower during night hours (when demand is lower), and has
peaks in the morning when most people wake up and during the early evening when
people arrive home from work.
In the IaaS cloud context, spot markets can also play an important role to deal
with short timespan volatility in demand and supply of computational resources.
Such a spot market can be global in nature, as by leveraging the fact that many
IaaS applications are essentially location and provider independent, it is possible to
(live) migrate applications according to where computational resources are currently
the cheapest. This can be done among consumers and providers, but also among
providers themselves (inter-provider trading) as is regularly done in the power in-
dustry. By doing so, providers can effectively cope with short-term shortages or
excesses of computational resources in their datacenters.
Note that the computational resources that are acquired on such a global spot
market do not necessarily need to follow the spot allocation model as introduced
by Amazon [17], in which resources are automatically deallocated if the spot price
exceeds a consumer specified maximum price. Also, as the heterogeneity between
different kind of computational resources is large, it will be difficult to specify a
single spot price (if not impossible, see section 3.2.1 for more information about the
problems with a single price component model for IaaS resources) for all types of re-
sources. Instead, it is more likely that in such a spot market, specific predefined sets
of resources (instance types) are traded as that greatly simplifies the introduction
of a single spot price.
The use case in which resources are traded in a spot like manner is however not
necessarily tight to the use of a single spot price. Indeed, when a price for a set of
2.3. AUCTION TYPES 40
computational resources is negotiated on the spot (that is, directly) and allocation
takes place directly afterwards, there are not that much fundamental differences
with spot markets that use a single spot price. As such it is reasonable to call any
IaaS market that adheres to the principles of direct allocation and demand/supply
based price volatility a type of spot market.
2.3 Auction Types

As already noted, many exchanges use auction mechanisms to determine which buyer
is matched with which seller. The use of auctions is however not limited to official
exchanges, they can be used as a general process to sell any good. In general, an
auction refers to any mechanism or set of trading rules for the exchange of goods;
it is the process of buying and selling goods or services by offering them up for bid,
taking bids, and then selling the item to the highest bidder.
In reality, many variations on this basic auction form exists. Differences among
these auctions include time limits, minimum or maximum limits on bid prices, and
special rules for determining the winning bidder(s) and sale price(s). Also, partici-
pants in an auction may or may not know the identities or actions of other partic-
ipants. Depending on the auction, bidders may participate in person or remotely
using telecommunication or computer networks (telephone, Internet, etc.). In order
for the auctioneer (the entity that organizes the auction) to make profit, the seller
usually has to pay a commission that is either fixed or variable (e.g. based on a
percentage of the final sale price).
Below, the most prevalent auction types are discussed in more detail.
2.3.1 English Auction

The English Auction is the most well-known of auction mechanisms. It is often
used in cases where a single seller wishes to sell a single (or multiple) relatively rare
good(s) to one of multiple buyers. During the auction, buyers iteratively bid for the
good that is being auctioned. More specifically, when placing a bid, a buyer should
always bid higher than the highest bid that has been placed until then (called the
standing bid ). The buyer that eventually bids the highest price for the good wins the
auction. Since all bidding is done publicly, all participants know what the highest
bid is (and who placed it) at any given moment. In many cases, the seller is allowed
to set a reserve price - a minimum price at which the auctioneer starts accepting
bids. English auctions end at a specific predetermined time, or when no new bids
have been made for a given length of time. The English auction typically favors
the seller as bidders tend to overbid the good in the heat of the bidding process, an
effect commonly referred to as the winner’s curse.
A bidding system in which all bids are public is typically referred to as an open
outcry auction, as historically bidders cried out their bids (this is still done in some
cases, but alternatives signaling such as raising a bidding paddle also exist).
2.3.2 Dutch Auction

The Dutch auction, also known as an open-outcry descending price auction, uses a
totally different bidding mechanism. Instead of letting buyers compete with each
other directly by increasing the price of the standing bid, the dutch auction starts
with a high asking price (set by the seller) that is iteratively lowered until a buyer is
found that is willing to pay the last announced price. As with the English Auction,
the dutch auction is called an open-outcry auction since all other participants know
who won the auction at which price. The dutch auction gets its name from the
yearly tulip auctions in the Netherlands, in which it is used.
This type of auction is primarily adopted when goods are to be auctioned quickly,
as completing the auction only requires a single bid. Additionally, the descending-
price structure of dutch auctions increases competition among bidders as it makes
acting fast a necessity.
2.3.3 Vickrey Auction

The Vickrey Auction, named after its inventor William Vickrey, is a second-price
sealed-bid auction which means that bids are not made public and that the bidder
that bids the highest price wins the auction, but only needs to pay the price of
the second highest bid. Using this mechanism, it is relatively easy to prove [51]
that the Vickrey Auction is incentive compatible in some situations. Section 2.4.3
provides more details on incentive compatibility, while [52] gives more insight about
the situations in which the Vickrey Auction is incentive compatible.
However, despite the desirable incentive compatibility property in some situ-

ations, there are some disadvantages to using the Vickrey auctioning mechanism.
First of all, it allows for the seller to use so-called shill bids to increase his profit.
That is, if the auctioneer and the seller have shared interests, the auctioneer may
inform the seller of the current highest bid, at which point the seller can make bids
that are only slightly lower than the highest bid, forcing the winning participant to
(almost) pay the price of the winning bid. A second downside to the Vickrey auction
is that is does not allow participants to get an idea of what a certain good is worth
to others3 as the valuations (=bids) of the participants are not disclosed.
When multiple units are being auctioned, the Vickrey auction generalizes to a
uniform price auction (see section 2.3.6) which is not incentive compatible. However,
a variation of the Vickrey auction - the Vickrey-Clarke-Groves (VCG) auction - exists
which remains incentive compatible for multi-unit auctions [53].
3
The process of discovering what the current market value for a certain good is, is called price
discovery and is discussed in more detail in section 2.5.4.
2.3.4 Reverse Auction

In the reverse auction (RA), the roles of buyers and sellers are reversed. That is,
sellers compete to obtain the right to sell to a single buyer. The reverse auction is
thus typically used in cases where a buyer wishes to buy a good or service (as is
common with reverse auctions) and that several parties are interested in selling the
good or service to that buyer. Reverse auctions are typically organized in several
bidding rounds after which the lowest bid price is made public. New bids can only
be lower than the standing bid. This mechanism has the effect that the price of the
good decreases as the auction progresses, in some cases significantly towards the end
of the auction. This makes the reverse auction buyer-oriented, as the buying party
is the one that benefits from the competition among the sellers.
The reverse auction is primarily used in cases where considerable financial stakes
are involved, or when a lot of different providers are available. For example, the
reverse auction is commonly used to procure large service or construction contracts
in the private or public sector (such as road construction or factory maintenance).
2.3.5 Double Auction

The double auction (DA) is used in situations when there are both numerous buyers
and sellers (for example, in commodity markets). In the double auction, sellers offer
the quantity of the good they wish to sell, as well as the price at which they wish
to sell the specified amount. Buyers use the same bidding language, that is, their
offers contain both the amount they wish to buy, as well as the price at which they
wish to buy the specified amount for. In order to avoid confusion, an offer placed
by a seller is commonly known as an ask, while an offer placed by a buyer is referred
to as a bid.
Typically, a double auction uses time frames during which it accepts asks and
bids (e.g. an hour). At the end of such a time frame, a specific matching algorithm
(also called the clearing algorithm) is used to match asks with bids. Since finding
an optimal solution - that is, the matching that maximizes the allocative efficiency
(for more details about allocative efficiency, see section 2.4.1) - might be a very
time-consuming or even intractable task (the number of combinations between asks
and bids becomes rapidly very large), various heuristics are typically applied to find
a reasonable matching within acceptable time.
A common variant of the double auction, the continuous double auction (CDA),
uses such a heuristic to improve matching speed. In the CDA, a bid or ask is, when
possible, immediately matched to an existing ask or bid. If no matching bid or ask
exists, it is added to the pool of asks or bids that have not been matched yet so that
it can be matched at a later time. By matching each offer at the time it arrives,
the CDA avoids the combinatorial explosion that typically occurs at matching time.
Of course the downside of this, is that the CDA can only be maximum allocative
efficient for the incoming bid and ask at that moment in time while the regular double
auction can (theoretically) find a maximum allocative efficient matching under all
the offers that arrive in a time frame4 .
In order to reduce algorithm complexity even more, many CDA implementations

use the double sorted queue algorithm. When using this algorithm, the auctioneer
keeps 2 queues: one for unmatched bids and one for unmatched asks. The queue of
unmatched bids are sorted by decreasing price, the queue of the unmatched asks are
sorted by increasing price. It is the sorting of the queues that results in the reduction
of algorithmic complexity. That is, when a new bid comes in, the algorithm iterates
over the unmatched ask queue and returns the first ask it finds that has a price that
is less or equal to the price of the incoming bid, and that has a matching quantity.
The process for an incoming ask is analogue.
Whether quantities can be split or have to be matched exactly is something that

either has be agreed upon before the start of the auction or has to be specifically
defined for each bid or ask. Since the queue is ordered by increasing price, one is
sure that once a matching ask is found that there is no better ask, as matching an
ask that is further down the queue is always less allocatively efficient than matching
an ask that is higher up the queue. When no matching ask can be found, the
incoming bid is added to the right place in the queue of unmatched bids. Note that
this technique is not specific to continuous double auctions, it can also be used to
decrease the complexity in the matching algorithm of regular double auction.
2.3.6 Uniform Price Auction

The uniform price auction is a multi-unit auction in which bidders bid both a number
of units and a unit-price. Bids are typically sealed. When the auction finishes, the
auctioneer first serves the highest bidder, allotting him the requested number of
units. Then the second highest bidder is served, than the third, and so on until
no more units are available. All bidders pay the same unit price - the lowest price
submitted by one of the buyers that are receiving units - independently of the unit
price they originally bid. Variations exists in which the winners pay the highest
losing bid instead of the lowest winning bid.
2.3.7 Combinatorial Auction

In a combinatorial auction, buyers place bids on combinations of items (called pack-
ages) on sale, instead of bidding on individual items. After the bidding period, the
auctioneer selects the set of bids that result in the most revenue for the seller without
assigning any item to more than one bidder. Obviously, doing this is NP-complete
(this problem can easily be reduced to the well-known NP-complete knapsack prob-
lem [54]), so in many cases the auctioneer uses heuristics to approximate the optimal
4
Note that choosing the size of this time frame is difficult, as it can always be that an offer
arrives just outside of a time frame that would have resulted in a more allocatively efficient match.
The exact size of the time frame is typically dependent of the specific case in which the double
auction is used, as the industry domain itself typically dictates the time constraints.
2.4. DESIRABLE MARKET PROPERTIES 44
solution(s). Combinatorial auctions have been studied extensively, more information

can be found in [55] [56].
2.3.8 Walrasian Auction

The Walrasian auction, named after its inventor Leon Walras, is a uniform price
auction auction in which all participants calculate their demand for a good at every
possible price and submit this to the auctioneer. The actual price is then determined
in such a way that the total demand among all participants is the same as the total
amount of the good on sale. In other words, the Walrasian auction will make sure
that the demand always matches the supply by picking the right combination of
price-quantity points among the participants. Walras suggests that this equilibrium
can be achieved through a process called tâtonnement (French for “groping”), a
form of hill climbing.
2.4 Desirable Market Properties

Before looking in more detail at how a commodity market can be designed, it is
beneficial to address market design from a more theoretical point of view. More
specifically, expressing how an ideal commodity market might look like can provide
good evaluation criteria for any real-world commodity market. Such an ideal market
should conform to a set of principles (have a few key properties), of which the most
important are explained and discussed in this section.
2.4.1 Allocative efficiency

Allocative efficiency is a measure of profit or utility that results from a proposed or
actual choice of a distribution of resources. In a market, the term allocative efficiency
indicates the received profit of the participants for a specific deal compared to the
maximum profit they could receive for any deal. More concretely, the allocative
efficiency of a market gives an idea of how good a market is able to match offers
of buyers and sellers with each other. Usually, the allocative efficiency is directly
influenced by the allocation and pricing rule (see section 2.5.3), specified by the
used market mechanism, that is applied when a deal occurs. It is difficult to make
more specific statements about the effect of pricing rule on the allocative efficiency,
as this is really market specific.
Allocative efficiency is usually less of an issue in situations where there is only

a single seller (or buyer), since the maximum allocative efficient deal is with the
buyer willing to pay the most (seller asking the least). In reality, many such market
mechanisms exist (English auction, Dutch auction, Reverse Auction). In situations
where multiple buyers and sellers compete on the market, it is generally more dif-
ficult to find the match that maximizes the allocative efficiency due to algorithmic
complexity, although some mechanisms - such as the previously discussed Double
Auction - exist. Note that for the maximum allocatively efficient match to be found,
it is imperative that both buyers and sellers submit prices that are truthful to the
actual value they attach to the good they wish to sell or buy (a market is said to be
incentive compatible if its matching mechanism encourages this behavior, see section
2.4.3). Obviously, a market can only find the real maximal allocative efficient match
if it is given the real values participants attach to goods.
2.4.2 Individual Rationality

The individual rationality (IR) property is strongly related to rational choice theory
which provides a framework for understanding and modeling social and economic
behavior. The core idea behind this framework is the assumption that the rational
behavior of any individual in the current microeconomic model is to want more
rather than less of a good. Although this theoretical assumption seems reasonable,
there are many real-world situations in which it is rather difficult for an individual
to actually obtain as many goods as possible. There can be many reasons for this.
For example, it might be difficult for a buyer to find sellers willing to sell a specific
good (or the other way around). It is in such cases that participating in a market
that trades the good makes sense to the buyer and seller. A market is said to
have the individual rationality property if joining the market is the most interesting
choice for all participants. That is, by taking part in the market each participant
will gain more than he would be able obtain on his own. As mentioned, this can
be because the market brings together buyers and sellers, but other reasons also
exists. For example, if the market achieves a better (overall) allocative efficiency
than any other trading mechanism, joining the market is the obvious choice for all
participants. An other example is a situation in which buyer or seller are unsure
about the valuation of the good they wish to buy or sell. In such a case, the market
may provide price discovery mechanisms that allow the participants to discover how
other participants valuate the goods on sale.
In the context of individual rationality, the concept of utility is often used to

express whether joining a market is a logical choice for a buyer or seller. To make
the idea of utility more tangible, it is often expressed as the difference in value a
buyer attaches to a good and the price he needs to pay to acquire the good: u = v−p.
(Analogously for the seller: u = p−v). Joining the market is then individual rational
for each participant i if ui > 0.
Note that this assumes that there is no other way to buy or sell the good, except
by trading on the market. If there is another way to buy or sell the good that yields
a higher utility ui for participant i, then trading the good via that other way that
is clearly the most rational choice for participant i. However, when stating that
a market is individually rational for all participants, often the weaker requirement
that no participant incurs negative utility by taking part in the market is meant.
2.4.3 Incentive compatibility

A market is called incentive compatible (IC) if the combination of all its components,
mechanisms and processes give the result that from a participant’s point of view, the
most interesting strategy is to truthfully reveal private information. More concretely,
participants will only be able to maximize their own profit if they reveal their own
information truthfully. In most cases, this information pertains to the participant’s
valuation for the good. This means that participants cannot gain additional utility
by underbidding or overbidding. Note the subtlety in the fact that although in
many market mechanisms, under- or overbidding will change the outcome of a sale,
it might not actually benefit the participant that did the false bidding. For example,
if a buyer acquires a good by overbidding, he does not actually maximize his own
profit as the good itself does not have the same value as the value the buyer attaches
to the money he needed to buy the good (the buyer thus incurs negative utility).
Similarly, if a buyer is not able to get hold of a good because of underbidding,
opportunity loss occurs, as the buyer might have been able to increase his own
profit by bidding truthfully and possibly acquiring the good that was on sale.
However, there are certainly examples of markets in which underbidding allows

participants to buy goods at prices that are lower than the value participants attach
to the good, or overbidding allows participants to achieve the good at a price that
is within the participant’s valuation for the good. The first situation is the case
in first-price open-outcry auctions, such as the English Auction, while the latter
situation might occur with auction mechanisms where the paid price is not the same
as the bid price, such as in a Uniform Price Auction. Such markets are obviously
not incentive compatible.
Although incentive compatibility is a desirable property for any market, it is

thus not easy to design a market that is incentive compatible, nor easy to verify
that a seemingly incentive compatible mechanism actually is.
2.4.4 Budget Balanced

A market mechanism or auction is said to be (weakly) budget balanced if the total
payments from the buyers, less the revenues of the sellers and the needed transaction
costs is nonnegative. Put differently, a market mechanism is budget balanced if the
auctioneer itself does not have to add extra money to the market in order for a market
clearing to occur. In other words, the payments of the buyers cover the revenues of
the sellers (and any additional transaction costs). Having a budget balanced market
mechanism is clearly important as it is the main incentive for the auctioneer to
organize the auction or exchange of a good. In this context, the market mechanism
is called weakly budget balanced if the auctioneer has a surplus after the matching
process (the auctioneer thus makes profit from the matching process), while the
mechanism is called strongly budget balanced (or just budget balanced ) when there
is neither a surplus nor a deficit.
Although having a budget balanced mechanism is certainly very desirable, it

turns out (as proven in [57]) that it is impossible to create a market mechanism
that is allocatively efficient, individually rational, incentive compatible and budget
balanced at the same time. As such, it is clear that when designing any market
mechanism, certain concessions and choices will need to be made as it is impossible
2.5. DESIGNING A COMMODITY MARKET 47
to have a market mechanism that has all desirable properties. The fact that this is
so emphasizes that there cannot be a single (or multiple) ‘ideal’ market mechanism
or design that fits every situation. Instead, the choice of a certain mechanism is
inherently dependent on the context in which it is used, as that context dictates
which concessions are acceptable and which choices appropriate.
2.4.5 Computational Tractability

Today, many markets and exchanges make extensive use of electronic trading plat-
forms for their speed, increased transparency and reliability. However, making use
of such platforms has the obvious downside that they suffer from the shortcomings
of the current generation of computers. In particular, modern day computers are
known to be notoriously bad at non-deterministic problems (such as determining
which combination of offers is the most alloctively efficient). As such, it is impera-
tive that during the design of any market, thought is given to whether the chosen
market (auction) mechanism can efficiently be implemented using software.
A market mechanism is said to be computationally tractable if it can find matches

in polynomial time. Many auction mechanisms are inherently tractable as they only
need to compare new incoming bids against a single standing bid or reserve price.
Examples include the English, Dutch, Vickrey and Reverse auctions. However, with
Double Auctions and more complex auctions such as the Uniform Price, Combina-
torial and Walrasian Auctions this is not necessarily the case. As these auctions
typically need to compare multiple bids with each other and with a one or multiple
asks, their time-complexity tends to grow exponentially. When the market mecha-
nism is to be used in situations where a lot of simultaneous bids are expected and
matching needs to go fast, it might not be feasible to use those mechanisms with-
out modification. The use of heuristics to find a pseudo-optimal match is in many
cases a good alternative. Additionally, a parallelizable matching mechanism has the
clear benefit that multiple servers can be used to work on the matching problem
simultaneously.
2.5 Designing a Commodity Market

After having discussed financial markets, auction mechanisms and desirable market
properties, it is now time to have a closer look at how commodity exchange markets
are typically designed. This section will explore this topic in more detail by dis-
cussing the various components and concepts that are typically part of a commodity
exchange market.
2.5.1 Market actors

In any (commodity) market, there are several different market actors. With the
term market actor, any party that interacts with the market is meant. Sometimes
the word market player is also used to instead of market actor. The most prominent
of these actors are the buyers and sellers that trade goods on the exchange. Buyers
are also referred to as consumers, while sellers are also called providers. Although
both terms can be used interchangeably, the terms ‘consumer’ and ‘provider’ lay
more weight on the consumption aspects of the buyers and sellers. Note that it does
not necessarily need to be the case that the seller is the same party that actually
provides the good that is being sold, nor does the buyer needs to be the party that
actually consumes the acquired good. Throughout the remainder of this thesis, the
terms ‘buyer’ and ‘seller’ might occasionally be mentioned, but most of the time the
terms ‘consumer’ and ‘provider’ will be used.
The market or exchange is most of the time also considered as a market actor.
If the exchange is auction-based, it is sometimes also referred to as the auctioneer.
Again, both terms can be used interchangeably. Throughout this thesis, the words
‘market’, ‘market mechanism’, ‘auction’, ‘auction mechanism’ and ‘exchange’ are
also used to denote the same entity. For clarity; this entity is the component that
accepts offers from consumers and providers and finds matches between them. Ad-
ditionally, this component may provide extra functionality such as price discovery
mechanisms (section 2.5.4) and contract enforcement (section 2.5.6).
The role of brokers
Although consumers and providers regularly take part in the market themselves, in
many situations they use also brokers to participate in their place. In this context
brokers are also called bidding-agents, as they place bids on behalf of providers
and consumers. The use of such bidding-agents can have many benefits for both
providers and consumers. Such a key benefit is that brokers can hide some (possibly
all) of the complexity of the market by translating specific requests and constraints
of consumers and providers to the market’s - possibly complex - bidding language
(section 2.5.2). Additionally, brokers can hide much of the different steps that
typically occur in a bidding process and only present the end-result to the consumer
or provider. Such a service can be especially helpful to consumers and providers when
electronic trading platforms are used that require complex hardware and software
setups to interact with. Examples include brokers that hide all complexity behind
simple user interfaces or implement certain bidding strategies in order to maximize
profit.
Another more general role of bidding-agents can be that of the retailer that
effectively hides the market from the consumer or provider. That is, a retailer sells
goods to consumers or buys goods from a provider using a custom price model (fixed
price per good, fixed part and variable price, etc.) that is apparently independent
from what is going on the market itself. Doing so is advantageous to the customer
(either consumer or provider) as using such a retailer effectively allows him to protect
himself against price fluctuations in the market.
Retailers make profit by leveraging the mid- to long-term differences between
the actual market price at which they buy or sells goods and the price they offer to
their customers. Note that the broker-retailer principle presented here is essentially
the same as in the retailer model used in the power industry as discussed in the
introduction (section 1.3.2).
Besides their roles as bidding-agents or retailers, brokers can also perform func-
tions on behalf of the market itself. For example, in order to reduce the complexity
of the market design and implementation, as well as to give market participants
the freedom to use multiple alternatives, the market may outsource functions such
as price discovery, contract enforcement and accounting to external broker parties.
Doing so has the advantage that the market becomes more self-contained and au-
tonomous as certain responsibilities are no longer part of it. In the case of electronic
trading platforms, this has the additional advantage that the platform becomes more
modular and the risk of introducing single points of failure is reduced.
Figure 2.2 shows an overview of how different market actors interact.
Figure 2.2: Market actors in an commodity market
IaaS cloud market actors

When considering the IaaS cloud market in particular, the following actors can be
identified.
Provider In the IaaS market, the IaaS providers are the organizations offering IaaS
resources for sale on the market. These will typically be virtualized instance-
like resources oriented towards computational use-cases. Other products such
as non-virtualized hardware and software, firewall or load balancing services
or persistent storage solutions can also be part of the products offered by

the providers. However, providers that charge similar to IaaS providers (e.g.
per compute hour) but only offer more advanced platform services such as
specific database or programming framework deployments are not considered
IaaS providers, as these kind of services are clearly part of the PaaS spectrum.
Examples of IaaS providers are specialized companies such as Amazon, GoGrid

or Rackspace, but also non-specialized companies, research centers or univer-
sities that want to earn some money by selling their excess computational or
storage capacity. In the future, groups of end-users might also want to take
part in the market as a provider by selling the idle resources of their desktop
computers.
Consumer In the IaaS market, the consumer is the party that wants to lease com-
putational resources for a certain time. Possible use-cases for these resources
include hosting webapplications, performing (scientific) calculations (so-called
High Performance Computing (HPC) [58]), deploying development and test
environments and many others. Note that the role of the consumer can also
be taken by an IaaS provider that wishes to buy extra capacity from other
providers to cope with temporary capacity shortages. Such inter-provider
trading was already mentioned in the discussion of spot markets in section
2.2.6.
Market (exchange) In the IaaS cloud context, the role of the exchange is to match
IaaS resources offered by IaaS providers with requests for certain resources as
demanded by the consumers.
Broker The role of brokers in the IaaS market is very similar to that of the brokers
as depicted in Figure 2.2; they provide services that simplify market inter-
action for the consumers and providers. These include providing easy-to-use
front-ends, implementation of bidding strategies, protection against rapid price
fluctuations and others. Additionally, brokers can be used to provide market
related services such price discovery or any of the auxiliary services described
in section 2.5.6.
2.5.2 Bidding language

Throughout the previous sections, it was already mentioned that the bidding lan-
guage is a core component of each market. As the word suggests, the bidding
language of a market defines the language or protocol that is to be used to specify
and describe the contents of a bid. In order to avoid confusion, the term ask is com-
monly used to denote a bid that is submitted by a provider as the provider is asking
to sell a certain item. A bid by a consumer keeps the term bid as the consumer
actually bids a price to buy something. Note that these terms were first mentioned
in the context of the double auction (section 2.3.5), but they can easily also be used
here in a more general context.
The main goal of specifying a clear bidding language is to avoid confusion or

discussion during and after trading. That is, if trading is to occur, each party that
is involved must have a clear understanding of what other parties are offering, as well
as understand under which conditions the offers are valid. Moreover, in exchange
markets, having a clear bidding language is an absolute requirement as asks and
bids can only be compared and matched to each other if they are specified in the
same language. In this light, the importance of a formally defined bidding language
has only increased with the introduction of electronic trading platforms.
Although it is difficult to make general statements about bidding languages with-

out ruling out some specific use-cases, there are some similarities between the bidding
languages of most commodity markets. That is, a typical offer (both bid and ask)
contains at least a volume, a price and an expiration date. To represent such offers
more concisely, a tuple-based representation such as [volume, price, bid] can be
used. In this notion, the tuple [5, 129, 12/07/11 13:14:28 GMT-2] thus repre-
sents an offer with a volume of 5 and a price of 129, which expires on the 12th
of July 2011 at the specified time. If this offer would be an ask (thus made by a
provider), the volume would indicate the number of units of a good the provider
wishes to sell, while the price is the price at which the provider would wish to sell
the specified volume. The expiration date indicates until when the ask is valid (the
market should no longer match this ask after it has expired). Similarly, when the
offer would be a bid (thus made by a consumer), the volume would indicate the
number of units the consumer wishes to buy, while the price indicates the price at
which the consumer would be willing to buy the specified volume.
Note that up until now, it has been assumed that an offer cannot be split,
meaning that the volume specified in the bid or ask needs to be traded at once
at the specified price or not at all. However, many commodity markets allow for
so-called partial matches in which only a part of the original offer is matched, by
splitting offers into several sub offers. In such a case, the price that is part of the
offer specifies the unit price instead of the price of the whole volume. Of course,
variations in which multiple prices are given (one for the whole volume, one for a
single unit, one for x units, etc) can also be thought of. In such cases, the bidding
language will need to be adjusted to accommodate this functionality.
Importance of the unit definition in a commodity market
Strongly related to the specification of the bidding language is the specification of
the good under sale itself. An important aspect in this context is the specification
of the quantity of a good that is defined as a single unit. Doing so is an absolute
requirement as without it, trading of a certain volume of the good has no mean-
ing at all as different actors might use different unit sizes. In existing commodity
markets, these good units are therefore standardized. Examples include the oil bar-
rel (158.9873 liter), the ounce (when trading gold) and the wheat bushel (although
today replaced by standard weights).
Besides determining the quantity the unit represents, there should also be a
description of the properties and characteristics of the single unit of trade as in many
cases there exists substantial qualitative differences between different units. For
example, in the oil industry, a barrel of oil is categorized according to its crudeness5 .
Similar unit categorization exist for almost all commodities.
Although similar units of different categories are in many cases traded on different
markets (as is the case with oil), it is also possible to trade these units on the same
market, as long as the bidding language is expressive enough to denote the differences
between the units of different categories.
Without going into to much detail for now, it is apparent that in the context of
cloud computing, the definition of the IaaS unit will be far from straightforward as
the goods that are being traded on such a market will be a lot less homogeneous
than the goods that are traded on more classic commodity markets. Indeed, when
comparing different offers of different IaaS providers (see section 1.2), it is clear that
there exists considerable differences in both the resources that are being offered, as
well as in the way these resources are described. Additionally, extra information such
as the price for certain resources, the time that resources need to be allocated, extra
quality of service constraints or the time a certain offer is valid, should also be easily
describable. As such, a vital step in the design of any market for IaaS resources is
the specification of a clear bidding language that is able to express both the inherit
resource heterogeneity as well as any additional constraints and properties of an
offer. In the context of this thesis, this topic is revisited in detail in section 3.1.
2.5.3 Market mechanism

In terms of choices that need to be made when designing an exchange market, the
choice of the market mechanism is certainly one of the most important ones as it
effectively determines how consumers and providers will trade with each other. In
effect, the market mechanism is defined by the specification of two important rules:
the so-called allocation and pricing rule.
Allocation rule
The allocation rule (also called the matching or clearing rule or mechanism) de-
termines which goods are allocated to which consumer or provider. Although not
strictly necessarily, in many cases, the allocation mechanism is chosen to be a vari-
ation of one of the auction types that were discussed in section 2.3. The benefits of
choosing from among these is that all of these auctions have been studied extensively
and consequently their benefits and shortcomings are well-known. The exact choice
between the different auctions is dependent on several factors. A very important
factor is the relation between the different market actors. In the commodity market
context, where there are typically both many consumers and providers and where
the auctioneer is a priori not biased to one of both sides, the (continuous) double
auction is a logical choice as it is designed to accept multiple bids and asks and
5
A measure indicating how much processing the oil has been through. The term crude oil denotes
raw, unprocessed oil.
match them without favoring either consumers or providers.
Given that the auction mechanism determines when and how consumers and
providers can place offers, it is logical that it has a strong influence on the entire
bidding process, and consequently the bidding strategy that is used by consumers
and providers. Likewise, the auction mechanism has a large influence on the bidding
language as depending on this mechanism, the bidding language should to be able
to express the particularities of the bidding process. For example, when a Dutch
Auction is used (section 2.3.2), the bidding language only needs to be able to express
an ‘accept’, while in double auctions, the bidding language is far more complex as it
also needs to be able to express what the consumer or provider is exactly offering.
Pricing rule
The pricing rule is closely related to the allocation rule as it determines the price
that is to be paid by the buyer once an allocation is found. Sometimes, this pricing
rule is entirely determined by the allocation rule itself (as is the case in second-
price auctions), but with other allocation rules, it is possible to use different pricing
rules with the same allocation rule. For example, in double auctions many different
pricing rules can be used. The rather straightforward rule is to let the consumer
pay the provider the price he actually bid, while another rule might be to let the
consumer only pay the price that was asked by the provider. The first choice favors
the provider as he effectively receives all surplus that the consumer was willing to
pay above the provider’s requested price. The second choice than does the opposite;
it favors the consumer by not letting him pay for the surplus he was willing to pay
for the good. Note that this second pricing rule requires sealed bids, as making
bids public will heavily affect the consumers’ bidding strategy. That is, if consumers
know that they only need to pay for the price that is requested by the provider, they
will start overbidding to acquire a good knowing that they will not need to pay the
price they are actually bidding.
More complex pricing rules also exist. For example, some auctions use a pricing
rule that lets consumer and providers share the surplus. In such a case, the clearing
price p that needs to be paid by the consumer is equal to (a + b)/2 where a is the
provider’s ask price and b the consumer’s bid price. A generalization of this rule,
the so-called k-pricing rule parameterizes this rule with a parameter k by setting
the clearing price p to kb + (1 − k)a. Clearly, the basic case (all surplus is equally
split between both parties) is given by setting k = 0.5, while any k > 0.5 favors the
seller and any k < 0.5 favors the buyer.
The choice of the pricing rule has a large influence on both the bidding strategy
of the participants and the allocative efficiency of the market. More information
about the pricing rule in double auctions can be found in [59].
IaaS cloud market mechanism
Given the similarities between the power industry and IaaS clouds as indicated in
the introductory chapter, trying to design an IaaS cloud market using a double
auction seems to be a valid starting point, as it is the primary market mechanism

for electricity trading. However, given the fact that the traded goods (services)
of the IaaS market are totally different than that of the power industry markets,
it should not come as a surprise that it will turn out that using a classic double
auction will not suffice in the IaaS cloud scenario. In fact, due to the potential
infinite number of combinations between different hardware resources and service
constraints as well as pricing model heterogeneity among providers, it will turn out
that using a double auction is just not feasible at all in the current IaaS market
without leaving out a large number of use cases. The next chapter will elaborate on
this, and will put forward a new market mechanism that aims to solve some of these
problems. Specifically, section 3.2 will explain the problems with the (continuous)
double auction in the IaaS cloud setting, while section 3.3 introduces the new auction
mechanism that attempts to solve these issues.
2.5.4 Price discovery

The price discovery process describes the mechanisms that are used by buyers and
sellers to determine or approximate the price of a good in the marketplace. Price
discovery is often associated with asset valuation, which denotes the mechanisms
used by the buyer or seller to determine the value they attach to the good (asset)
for sale. Note the subtle difference between the two: the term price discovery denotes
the bi-directional process in which buyers and sellers arrive at a transaction price for
a specific item at a given time, while asset valuation is a process that every buyer
or seller does for themselves. Obviously, both processes are closely related to each
other as the price at which a trade occurs is heavily influenced by the value that
both parties attach to the good for sale. Because of this tight coupling between
price discovery and asset valuation, the terms are frequently used interchangeably
in literature. In this thesis however, the term ‘price discovery’ will exclusively be
used. The context in which the term is used should make its exact meaning clear.
The use of price discovery mechanisms has two distinct advantages. First of all,
it allows new buyers or sellers to discover the price at which the market is currently
clearing offers. For example, in a double auction, a good price discovery mechanism
allows consumers and providers to discover a price at which they are likely to buy
or sell goods. This can for example be the average price at which units are currently
being traded (in which case bidding a price that is higher than the average price
increases the buyer’s chances of actually making a trade, while lowering the price
decreases that chance; the opposite is true from the seller’s point of view).
The ability to do price discovery is also attractive because it allows consumers

to make an upfront trade-off between the price they will need to pay on the market
and the resources and quality of service they will receive for that price. Providers
on the other hand can use price discovery techniques to get an idea of the type of
resources and quality of service they will need to deliver for a certain price. This
ability to make trade-offs without having to make commitments effectively leads to
the second advantage of price discovery: it allows existing participants to determine
a bidding strategy that optimizes their own profit or utility by choosing the moment
at which they buy or sell certain resources. Such a bidding strategy can be tailored
to short term periods, as well as medium to long term periods. For example, a short
term bidding strategy might describe the bidding behavior needed to quickly obtain
a large amount of a good for the lowest price possible, while a medium or long term
bidding strategy might involve deciding a which point in time to buy certain goods
by carefully monitoring the prices of past trades, and looking for patterns. A good
example of such a pattern can be found in electricity markets, in which the price
for a Kilowatt/hour is usually lower at night when general power usage is lower (see
Figure 2.1 in section 2.2.6). Consequently, it is cheaper to run non-critical operations
that consume substantial amounts of electricity at night than during the daytime6 .
When considering price discovery in an IaaS cloud context, it is clear that the
same remarks hold. Both the ability to get an idea of the price at which computa-
tional resources are traded for new providers and consumers as well as the possibility
for existing providers and consumers to make better provisioning decisions are valu-
able tools in the cloud computing setting. Note that this process of making good
provisioning decisions is however more complex in the IaaS cloud context, as opti-
mality is no longer only determined by the choice of provider or consumer and time,
but also by the choice of the resources itself. That is, by choosing different types
of resources (that is, different hardware or software components), different levels of
performance for different applications will be obtained. Additionally, by allocating
the same resources via different allocation models (such as on-demand, spot, future
or reserved - detailed in section 2.5.5), the price, performance and behavior for the
same set of resources during the same time period can be totally different. As such,
far more complex methods and heuristics will be needed to intelligently make the
aforementioned trade-off between price and received or delivered quality of service.
The discussion of such techniques is however beyond the scope of this thesis, and
will as such not be further considered.
The issue of price discovery itself is however an important topic in the context
of this thesis. However, due to the heterogeneity of the resources that are being
traded, giving a good indication of what certain computational resources or services
will actually cost at a given time is far from trivial and closely related to the market
mechanism and bidding language that are being used. Because of this, it is hard
to make more explicit statements about price discovery in an IaaS market without
first having a closer look at its matching mechanism and bidding language. As such,
exact considerations about the price discovery mechanism in the IaaS cloud market
are postponed until section 3.3.3, after the discussion of these other components.
6
Given that electricity providers pass this price reduction to their consumers, as is the case with
the well-known night-tariffs.
2.5.5 Market Settlement

When designing a commodity market, it is not only important to clearly indicate
which good is being traded by giving a good unit definition that both describes the
quality as the quantity of a single unit; it is also important to make a decision on
the delivery time of the goods that are being traded on the market (the so-called
settlement of the trade). In this light, the choice between a futures market (see
section 2.2.5), in which goods need to be delivered at some specified point in the
future, and a spot market (see section 2.2.6), in which goods are to be delivered
directly, needs to be made.
Although the choice of whether a market is a spot or futures market foremost

determines which providers and consumers will take part in the market, it is possible
that this choice also has a profound impact on other parts of the market such as the
matching mechanism. For example, in immediate spot markets it is important that
matches are found quickly as delivery needs to take place as fast as possible while
in a futures market, it is acceptable if the matching process takes a lot longer. As
such, it is clear that in some spot markets, finding matches quickly has priority over
finding the best matches, while the opposite is generally true in futures markets.
As the choice of the matching mechanism has a profound impact on almost every
other part of the market (such as the bidding language, bidding process, actors’
strategies and price discovery mechanism), it is clear that the choice of the delivery
time of the market is an important one.
In the IaaS cloud market, the situation is however a bit different. That is, as the
goods that are being traded are very heterogeneous in nature, the delivery time of
these goods can easily be considered as yet another characteristic that differentiates
it from other goods. When studying the different offers of IaaS providers, it can
be observed that this is exactly what is done in practice: different IaaS providers
offer computational resources to their customers and allow them to allocate these
resources using a variety of different models such as on-demand, spot or reserved.
When choosing a different allocation model, customers get the exact same resources,
but get different guarantees in terms of price and resource availability.
Because delivery time is considered as any other property of an IaaS offer, it

is possible to introduce a single market in which IaaS resources with different al-
location models are all traded together. Note that this is generally not done in
other commodity markets as delivery time is usually one of the few characteristics
on which goods do differentiate. Because of this, it is usually decided to create sep-
arate markets in order to keep the market, market mechanism and bidding language
simple. In the IaaS market however, heterogeneity between goods is a core aspect
that cannot be ignored, making it easier to incorporate the aspect of delivery time
into the market itself7 .

IaaS resource allocation models
The terms on-demand, futures/reserved and spot instance will further be used to
indicate the resource (instance) behavior and price models similar to those specified
by Amazon. In a more generic form, these different models are defined as follows.
On-demand When using the on-demand model, resources are available immedi-
ately and can be used by a consumer for a non-specified amount of time.
Consumers are billed at a fixed rate per time-unit (hour, minute, ...). Figure
2.3 shows a timeline of how the on-demand model works.
Figure 2.3: On-demand resources, resources are allocated at a time t and stay allo-
cated for an indefinite length of time (until deallocated by the consumer).
Futures When using the futures model, resources are available for a specified point
in the future and for a specified amount of time. Consumers pay an up-front
fixed price. Figure 2.4 shows a timeline of how the futures model works.
Figure 2.4: Futures market timeline, resources are allocated from a time t + x until
t + y.
Reserved The reserved pricing model offered by many IaaS providers is a special
futures model that borrows principles from the on-demand model. Specifically,
in the reserved price model an upfront fixed price is paid to make sure resources
are available in a certain period, and additionally a (lowered) price is paid per
time-unit the resources are actually used. Figure 2.5 shows a timeline of how
the reserved model works.
Figure 2.5: Reserved market timeline, resources are reserved from time t until t + y
and allocated at various intervals (chosen by the consumer) within the specified
boundaries.
7
Note that this does not mean that IaaS resources with different delivery times must be traded
on a single market, it is just easier to do so in the context of IaaS markets.
Spot When using the spot model, the model as introduced by Amazon is used.
That is, consumers get access to resources immediately and specify a maximum
spot price. Every time-unit (hour, minute), the spot price is changed by the
provider. The consumer always pays the spot price for that time unit. As long
as the spot price is lower or equal than the maximum spot price specified by the
consumer, the resources will remain available. When the spot price is higher
than the spot price, the consumer should be either notified or the resources
should be released immediately (this can be a provider specific policy). Figure
2.6 shows a timeline of how the spot model works.
Figure 2.6: Spot market timeline, resources are allocated from a time t and stay
allocated until t + 3 when the spot price of the provider exceeds the maximum price
of the bid.
2.5.6 Auxiliary services

After the market has found a match between a consumer and the provider and
determined the price that needs to be paid by the consumer, its task is usually
over. However, this does not mean that the actual trade between the consumer
and provider has taken place. To facilitate the actual trade between both parties,
several mutually trusted parties that can deliver the necessarily auxiliary services
are typically used. Listed here are some of these auxiliary services.
Money transfer service Service that allows two parties that do not entirely trust
each other to exchange money. Note that the function is different as that of
a bank as the service knows the details of the deal made by the market, and
can thus verify that the consumer has paid the right amount of money at the
right time. Additionally, the service can cope with different currencies.
Good delivery service Service that delivers the goods to the consumer at a pre-
determined time and place. Service is either paid for by the consumer of
provider, depending on market rules.
Contract enforcement service Service that verifies whether the quality of the
delivered good is adequate (e.g. purity of metal, crudeness of oil) and whether
the good was delivered in time and on the right location. It is quite common
that a mutually trusted third party (such as an government official or bailiff)
is used to do the actual verification.
In the IaaS cloud context, the issue of contract enforcement is particularly

challenging, as it involves verifying that a provider actually supplies resources
that deliver the promised quality of service. Verifying this is not easy, as to do
this accurately, a trusted third party must be able to run a potentially large
number of tests and benchmarks at regular randomized intervals in such a way
that the results cannot be influenced by the consumer or provider. Putting
this into practice this is a rather difficult administrative and technical challenge
that requires advanced monitoring and analysis tools. Today, some companies
such CloudHarmony [16] and CloudSleuth [60] already provide rather detailed
analyses of the performance of well-known IaaS providers; integrating such
tools as broker services into the market has significant value and can be the
subject of future research.
Authentication and Authorization service Service that provides advanced and

secure user management. When making use of electronic trading platforms,
security and trust is always an important topic. The correct and secure fa-
cilitation of authentication and authorization services is however a complex
issue, especially when many different parties and organizations with different
security requirements and implementations interact with each other. As such,
outsourcing such services to third parties can take away a lot of this burden
from the market.
CHAPTER 3
IaaS Cloud Computing Market: The Continuous Reverse Auction
After having discussed the importance of introducing a market for IaaS resources
in the first chapter, as well as given an overview of how such a market should be
built in the second chapter, it is now time to elaborate on the most critical parts
of the market, namely its bidding language and matching (auction) mechanism.
This chapter will start with the introduction of a Continuous Double Auction based
market for grid resources that was developed by Nicolas Dube in [39]. Once this
market model has been properly introduced, the author will point out some problems
this model faces in the cloud computing setting. Working from the particularities of
these problems, a new bidding language and market mechanism that is able to cope
with the specific issues of the IaaS ecosystem will be iteratively constructed.
The ratio behind the choice to start from an existing CDA model is twofold.
First of all, given the resemblances between the power industry and IaaS clouds as
discussed in section 1.3.1, using a double auction based market is a reasonable first
choice as this market mechanism is already being used in power markets. Indeed,
without studying the problem in more detail, it seems that once the appropriate
bidding language is introduced that can express the heterogeneity of IaaS resources,
the CDA mechanism is a viable market mechanism as it is able to quickly match
bids and asks of multiple providers and consumers.
A second reason to start from an existing market model is the fact that a market
design is something that needs to grow steadily and is not something that just
sprouts into the mind; this chapter starts from an existing solution before deviating
from it to reflect the iterative process the author went through while designing the
market model detailed throughout this chapter.
By the end of this chapter, a bidding language for cloud computing resources
3.1. CONTINUOUS DOUBLE AUCTION FOR IAAS RESOURCES 61
will be proposed that uses so-called tag sets to describe resources and constraint sets
to specify consumer and provider constraints on these resources. Additionally, a
new auction mechanism called the Continuous Reverse Auction is introduced which
attempts to solve the problems of the Continuous Double Auction in the cloud
setting.
3.1 Continuous Double Auction for IaaS Resources

Because of the many resemblances between the IaaS industry and power industry,
looking at the market mechanisms used in the power industry is a logical first step.
As discussed before, the Continuous Double Auction is used in multiple countries
as the primary market mechanism to do spot trading for electricity. Indeed, the
CDA market model has already earned its name as a fast (both in terms of market
clearing speed, as in algorithmic tractability) and reliable mechanism. As such, it is
interesting to have a look at how the CDA mechanism could used as the matching
component of an IaaS market.
3.1.1 Dube’s CDA-based market

To study a CDA-based matching mechanism for IaaS resources, the PhD thesis of
Nicolas Dube [39] acts as a good starting point as it introduces such a market for
grid systems. In particular, Dube’s thesis contains some good ideas about how to
represent computer resources (using so-called resource sets), how to match bids and
asks of consumers and providers, and makes an interesting contribution on how to
introduce a price discovery mechanism for such a market model1 .
Dube’s market exists of a central market component that accepts bids from
consumers and asks from providers. As with every CDA, the market maintains two
queues, one for the bids and one for the asks. The queue containing the bids is
sorted by decreasing price, the queue containing the asks is sorted by increasing
price. When a new bid or ask comes in that cannot be matched against an existing
offer it is added to its according queue. A bid is matched against an ask if the
resource set attached to the bid that describes the requested computing resources
matches that of the ask, and if the price of the bid is higher or equal to that of the
ask. Similarly, an ask is matched against an existing (not previously matched) bid
if the resource set attached to the ask fulfills the bid’s resource requirements and
the price attached to the ask is lower than that of the bid.
The actual matching algorithm for an incoming bid works by iterating over the
queue containing the asks and trying to match each of the asks in the queue to the
new bid. Once a match is found, the mechanism removes the matching ask from
1
Dube’s price discovery mechanism will not be discussed in detail here; more information about
it can be found in [39].
its queue and notifies both parties (consumer and provider) that a match has been
found. Matching an incoming ask with an existing bid works similarly.
Although it might seem that so far, there are no apparent differences between
the classic continuous double auction as used in the power industry and Dube’s
version that was just described, the introduction of the resource sets do introduce a
significant change to the model. That is, rather than only comparing bids an asks
based on price, they are now also compared using the resource sets with which they
are both annotated. This adds algorithmic complexity to the matching process.
The key point is however, that in effect, Dube has extended the double auction
mechanism so that it can cope with very heterogeneous commodities. It is clear
that this change can have a significant impact on the characteristics of the double
auction, as optimal matching is no longer only determined by price, but also by the
optimality of resource set matching. To better understand how this change alters
the matching mechanism, the following section describes the resource sets in more
detail, by explaining how they are actually used to describe the different components
of a computational resource.
3.1.2 Resource sets in detail

A resource set is (as the words suggest) a description of computer resources that is
either requested by the consumer in case of a bid or offered by the provider in case
of an ask. When considering a set of resources that is requested by the consumer,
Dube uses the term requirement set, while the term component set is used to denote
resources that are offered by a provider. Without going into to much detail, it is
intuitively clear that the resource set S in 3.1 represents a computer which has an
x86 system architecture, 2 core processor clocked at 2.4 Ghz, 4GB of RAM and
16GB of hard disk space.
S = {x86proc arch , 2proc cores , 2.4proc clock , 4mem size , 16disk size } (3.1)
Obviously, the used unit as well as the type and domain of each of the items
in the set should be clearly defined. Dube does this by specifying a discrete, finite
domain set for each of the types of resources, along with the used unit (if applicable).
Some examples of such domain sets can be seen in 3.2.
Φproc arch = {x86, itanium, intel, amd, power, bleugene, sparc, nec}
Φproc clock = {1.6, 1.8, 2.0, 2.2, 2.4, 2.6, 2.8, 3.0, 3.2, 3.4, 3.6, 3.8, 4.0} (Ghz)
Φproc cores = {1, 2, 4, 6, 8, 16, 32}
Φproc L2 = {1, 2, 4, 6, 8, 12, 16} (MB)
Φmem size = {1, 2, 4, 6, 8, 16, 32, 64, 128, 256} (GB)
Φdisk size = {1, 2, 4, 6, 8, 16, 32, 64, 128, 256, 512, 768, 1024} (GB)
Φnet tech = {100T, GigE, 10GigE, IB SDR, IB DDR, Myrinet, NumaLink4}
Φnet bw = {10, 100, 1000, 2000} (MB/s)
(3.2)
Although resource sets contain hardware requirements or components most of

the time, they can contain software (such as operating systems, software libraries) or
benchmarks results as well. Again, it is clear that the resource set S in 3.3 represents
a computer with 4 GB of ram, a 32 GB harddisk, a processor capable of reaching
12 GFlops as measured by the Linpack benchmark [61], and specific versions of the
Globus [62] toolkit software package and Infinipath driver installed2 .
S = {4mem size , 32disk size , 12proc linpack , GLOBUS 4.0.6soft package ,

InfiniPath 2.1soft driver } (3.3)
The domains of these software packages and benchmarks also need to be defined,
as is done in 3.4.
Φsoft package = {BLAST 2.2.14, GROMACS 3.3.3, FLUENT 6.3, GLOBUS 4.0.6}
Φsoft driver = {OFED 1.2.5, InfiniPath 2.1}
Φproc linpack = {4, 8, 12, 16, 20, 24, 32, 40, 48, 64, 96, 128} (GFlops)
Φproc specint = {20, 40, 60, 80, 100, 120, 140, 160, 180, 200} (2006, rate, base)
(3.4)
Matching resource sets

Theoretically, Dube states that a match between a requirement set and a compo-
nent set occurs if the intersection between the two results in the same set as the
requirement set, as indicated by 3.5. In this equation, Rb represents a requirement
set for a bid b and Co represents the component set for an offer (ask) o.
Rb matches Co ⇐⇒ Co matches Rb ⇐⇒ Rb ∩ Co = Rb (3.5)

2
The specified example is Grid Computing oriented as it is based on the examples given in
Dube’s thesis. However, it is clear that any software package, library or benchmark can be used.
In practice, an algorithmic approach is needed to determine whether two resource

sets actually match. That is, to check whether a given requirement set matches a
given component set, the requirement set is iterated, and for each of the items in
the requirement set an item of the same type (e.g. proc arch) in the component set
is searched. If such an item exists, the two items are compared more closely to see
if they actually match. If no such item is present in the component set, the result
of the matching procedure is negative (there is no need to check the other items).
Although not explicitly mentioned in his thesis, Dube actually considers two
different types of item matching: exact matching and what can be called comparative
matching. An exact match occurs when two resources have the exact same value
(e.g.: [proc arch = x86]), a comparative match occurs when the resources have
the required relation (e.g.: [RAM = 1GB] < [RAM = 2GB]).
Intuitively, it is thus clear that 3.5 expresses that a requirement set matches a
component set if all items in the requirement set have a matching item in component
set. Consequently, a component set matches a requirement set if it contains a
matching item for each of the items in the requirement set. Note that a component
set can thus match a requirement set with less items in it, while requirement sets
can never match component sets that have fewer items.
Although matching bids and asks by matching their respective resource sets
as just described is computationally intensive (due to the possibly high number
of comparisons between item sets), the number of bids and asks that are actually
compared with each other should be relatively low due to the continuous nature of
the matching algorithm and the fact that the queues are sorted by price. The fact
that the auction is continuous guarantees that a single offer is matched immediately
when it arrives, compared to a regular double auction where all bids are compared
to all asks at the end of the offer submission period. The use of the sorted queues
on the other hand, ensures that once a match is found there is no need to search for
an other match since the found match will be maximally efficient (as discussed in
2.3.5).
Furthermore, the matching itself can be made more efficient by implementing

the item lookup using hash sets. Also, by comparing the sizes of the requirement
set and component set prior to comparing their items, the result of the match is
already known (that is, negative) when the component set contains less items than
the requirement set.
3.1.3 Using resource sets to describe IaaS resources

Given the similarity between grid resources and IaaS resources, it seems reasonable
to try and use the resource sets as introduced in the previous section to describe
IaaS resources. Indeed, as with grid resources, IaaS resources are typically described
in terms of their hard- and software components and can thus be expressed by
resource sets, as shown in example 3.6 which shows a resource set description of a
small instance [15] as offered by Amazon on its EC2 platform.critical) patches in
the provider’s management software (e.g. VM management) are applied.
small instance = {1ECU , 1.7mem size , 160disk size , 32platform } (3.6)

However, although resource sets provide a viable way to specify many of the
hardware and software characteristics of IaaS cloud resources; they lack the expres-
siveness and flexibility that is needed to express some other more complex constraints
that are inherently part of IaaS solutions. In what follows, these shortcomings of re-
source sets will be outlined, followed by the introduction of a more generic constraint
specification language that can deal with these issues.
When comparing IaaS resources against grid resources, there are besides hard-
ware and software constraints almost always additional constraints - such as price,
QoS and security constraints - that need to be taken into account. The fact that
this is the case is based on the observation that in the cloud computing setting these
additional constraints are first citizens, while they are far less prevalent in the grid
computing setting. While most existing compute grids still only offer publicly funded
best-effort services that are oriented towards scientific workloads, cloud computing
services aim to deliver general purpose, metered usage solutions with guaranteed
quality of service. This broader and more commercial view means that providers
and consumers consider a far larger and more complex set of criteria when specifying
a particular resource. The use of these extra criteria and requirements will become
even more pervasive with the advent of the so-called cloud 2.0, which promises to
offer value beyond the provision of raw hardware by focusing on delivering high per-
formance, highly available and secure computing infrastructure for business-critical
production applications [63].
In order to get a more concrete idea of some of these criteria, consider Table 3.1
which expresses them as constraints.
Although Table 3.1 is by no means meant to be exhaustive - a lot of additional

constraints can easily be thought off - it does contains some interesting examples
that are difficult or even impossible to express using Dube’s resource sets.
For example, a constraint such as the Refund policy constraint will typically
include very provider specific conditions under which a refund can occur, as well as
what this refund actually includes. As such a constraint is typically very complex, it
cannot be expressed by a single value, and consequently does not fit in the current
resource set model. Instead, a more elaborate model that allows resources and
constraints to be characterized by multiple values is needed to express such complex
constraints.
Other constraints, such as the Region and the Software version range constraints
are also difficult to express using Dube’s resource sets as the resource sets do not
provide the more elaborate matching semantics that are needed by those constraints.
In particular, to match a Region constraint, knowledge of geographical location is
Constraint Description
Quality of Service (SLAs)
Uptime Provider guaranteed hardware uptime.
Refund policies Policy used by the provider when SLAs are violated.
Delivered performance Actual delivered performance. May include many pa-
rameters: CPU performance (flops), memory perfor-
mance, disk performance.
Network performance Average and maximum access times (network delay),
available up and downstream bandwidth.
Price
Hourly price Price that is paid per hour for the resources.
Bandwidth price Price that is paid for the used bandwidth.
Future price Stability Provider guarantees about price future stability.
Historic price information Information about the price of the resource in the past.
Security
Resource access Supported methods to access resources (SSH, FTP,
Remote Desktop, etc.).
Certification Security certificates held the provider (e.g. SAS70).
Patch policy How fast (critical) patches in the provider’s manage-
ment software (e.g. VM management) are applied.
Misc
Region Geographical region in which the resources are lo-
cated.
Allocation model Whether on-demand, future, reserved or spot re-
sources are used.
Software version range Support for different versions of the same software
package.
API Software API used to access and manage resources.
Additional services Other optional services (firewall, load balancer, . . . )
Table 3.1: Examples of additional constraints that are regularly used when describ-
ing IaaS resources.
needed to know which regions are included or overlap with others. This kind of
matching is not supported by the resource sets. Similarly, the possibility to specify
a range of versions of a particular software suite (by either listing the supported
versions or by specifying a min and max version) cannot be done by Dube’s “exact
matching” or “comparative matching” semantics.
This lack of expressiveness in the matching mechanism of the resource sets be-
comes even more apparent when trying to combine different simple constraints into
new, more complex constraints. Such situations can for example occur when con-
sumers are willing to make compromises if the compensation is large enough. That
is, it is not unthinkable that a consumer is willing to accept lower uptime guaran-
tees if more interesting refund policies come with it. Similarly, customers might be
willing to pay higher prices if the resources they can acquire will be able to get the
job done faster. Expressing these kinds of constraints is just not possible using the
matching semantics of resource sets. Instead, in the case of a CDA, several different
bids that express these subtle trade-offs in requirements should be submitted to the
market. Note that doing so is typically problematic as the consumer only wishes to
match a single of all those bids. In other words, giving consumers a language that
is able to express complex constraints such as trade-offs between simple constraints
has substantial value as it gives them the ability to model their bids much closer to
their actual requirements, increasing the overall usefulness of the market.
Besides the fact that when using resource sets certain constraints cannot be
expressed, there is a potentially other important issue that is of a more practical
nature. That is, when using resource sets, a market is needed that has specific
knowledge about each of the types of resources that are used as it needs to know
how to match these resources, i.e. it needs to know the different matching semantics
of each of the resources. Although using such a mechanism was still feasible when
considering a fixed set of resources with fixed domain sets and only two types of
matching (that is, exact matching and comparative “greater than” matching); in a
situation where more and more complex constraints are used with complex matching
semantics, using such a system is no longer viable. Indeed, given the heterogeneity
of IaaS solutions today, using a matching mechanism in which the market must be
aware of all possible resources and constraints is not manageable nor future proof.
Instead, a bidding language is needed in which offers can be compared and matched
to each other without the market needing to have specific knowledge about each
of the constraints and resources that are part of the offers. The following section
introduces such a market-agnostic resource specification language by letting the
consumers and providers specify constraint matching semantics.
3.1.4 Introducing tag and constraint sets

To solve the previously stated problems, the concept of the resource set will be
replaced by the notions of tag sets and constraint sets, which enable the specification
of more complex constraints and remove the need for the market to have knowledge of
each type of item in the resource sets. In particular, the specification of a constraint
on a resource that a consumer is requesting or a provider is offering will be separated
from the specification of the values that characterize that resource. By doing so, the
properties that characterize these resources are decoupled from the semantics that
are used to match them with each other during the matching process. In other words,
by letting the consumer and provider specify the properties of a resource, as well
as the semantics that determine how these resource are to be matched, the market
itself no longer needs to have knowledge about the matching semantics of each kind
of resource. Additionally, by letting providers and consumer specify the matching
semantics, the market is no longer constrained to a different few types of matching
(as is the case in Dube’s model), but can instead support any kind of matching.
Finally, by splitting the values from the matching semantics, the specification of
complex constraints that can express constraints that span multiple resources (such
as the QoS or consumer trade-off constraints that were mentioned before) can be
introduced relatively easily.
When a consumer or provider wants to describe a certain resource, requirement,

service or feature they can do that using tag sets that contain tags that describe the
different characteristics of the resource or feature that is being considered. Formally,
a tag T in a tag set T S is defined as the tuple in 3.7. It contains a name and a set
of attributes that define the tag.
T = [NT , AT ] (3.7)
The name NT of the tag can be any regular string3 , but has to be unique for
each type of tag and within a tag set (there can not be two tags with the same name
in the same tag set). The attributes AT associated with the tag (see 3.8) is a set of
key-value pairs.
AT = {(x, f (x))}
(3.8)
f : x ∈ string 7→ value
These attributes will typically contain the values that further characterize a
given tag. For example, in the case of a Disk tag (NT =“Disk”), the tag attributes
will contain both the unit in which the hard disk size is specified (typically GB)
as well as the actual size of the disk, i.e. [Disk, {(unit, GB), (value, 160)}].
Similarly, an OperatingSystem tag may contain both the name as well as the version
of an operating system, as in [OperatingSystem, {(name, Windows), (version,
7)}]. The values of attributes do not have to be singular values, they can also be of
a compound nature. That is, it is also possible that these values are lists or nested
tags. For example, if multiple operating systems are supported by a provider, he
can list them all as follows.
[SupportedOperatingSystems, {(list, {
[OperatingSystem, {(name, Windows), (version, 7)}],
[OperatingSystem, {(name, Ubuntu), (version, 11.04)}]}
)}]
The most apparent difference between the tag sets as defined here and Dube’s
resource sets is that using tag sets multiple properties (or attributes as they are
called here) can be under grouped under a single tag, allowing for a more hierarchical
approach to resource specification. A less visible difference is that tag sets in itself
have no significance during matching, they are just values that describe an offer. The
3
When implementing the tag, this can just be the string type provided by the programming
language of choice.
real matching semantics of tags are given by so-called constraint sets that accompany
tag sets.
These constraint sets contain constraints that define a certain matching restric-
tion on an offer. Formally, a constraint (3.9) is a function that defines such a
matching restriction on an offer O1 , by expressing restrictions between the tags of
O1 and the tags of an other offer O2 against which O1 is matched.
C : (T S O1 , T S O2 ) 7→ {true, f alse} (3.9)

Note that constraints are defined on an offer and not only on a bid, implying
that constraints can also be defined on the tag sets that accompany a provider’s
ask. By allowing this, providers effectively get the possibility to set constraints on
the bids they are willing to match with. Although this might not seem important
at first, allowing providers to set constraints does allow them to refuse certain bids
based on specific conditions such as e.g. the consumer’s country of origin4 or e.g.
the lack of certain other information in the consumer’s tag set.
Given the generic definition in 3.9, it is clear that in practice the outcome of
a constraint function can be influenced by a large number of factors. To better
understand how constraints work in practice, consider a consumer specified memory
constraint as given by Algorithm 1.
Algorithm 1 MemoryConstraint(T S 1 , T S 2 )
T1 ← T S 1 [“Memory”];
T2 ← T S 2 [“Memory”];
if T1 6= null and T2 6= null then
if (AT1 [“unit”] = AT2 [“unit”]) and (AT1 [“value”] ≤ AT2 [“value”]) then
return true;
end if
end if
return false;
As already noted, a constraint function always takes two arguments: the tag set
(T S 1 ) that belongs to the same offer to which the constraint belongs and the tag
set (T S 2 ) that belongs to the offer against which the constraint is verified. The
Memory constraint works as follows. First, the Memory tag is looked for in both tag
sets. In practice, this can either be done by iterating over both sets and comparing
the names of each tag, or by using a hash table like structure to store tag sets that
maps tag names to tags, in which case a lookup can happen in a single step. If one
of both sets does not contain the Memory tag than the result of the constraint is
negative. If both tag sets do contain the Memory tag, the algorithm first compares
4
Certain providers apply restrictions on the countries of origin of their consumers. Most notably,
all US-based providers may not do business with consumers from a certain number of countries on
which the US government has set a trading restrictions.
the “unit” attributes to verify whether both tags are specified in the same unit, and
then checks whether “value” specified in T1 is less than or equal to that of T2 (note
that the efficiency remark concerning hash table lookup of attributes also applies
here). If this is the case, the result of the function is positive (the constraint is thus
fulfilled), in all other cases the result is negative.
In this example, the Memory constraint was used to define matching semantics.
Besides the Memory constraint, many other constraints obviously exist. Some of these
operations should be standardized by the market to avoid confusion among the mar-
ket participants, while others can remain consumer- or provider-specific. Examples
of constraints that should be standardized are simple hardware-related constraints
such as the Memory, Disk and CPUClock constraints. More complex standardized
constraints may include the aforementioned Region and SoftwareVersions con-
straints.
A good example of a situation where e.g. the consumer would want to specify
a custom constraint is the previously given case where a trade-off between resource
uptime and refund policy is required. A conceptual overview of how such a complex
constraint can be defined is given by Algorithm 2.
Algorithm 2 UptimeRefundConstraint(T S 1 , T S 2 )
Tu1 ← T S 1 [“uptime”];
Tr1 ← T S 1 [“refund”];
Tu2 ← T S 2 [“uptime”];
Tr2 ← T S 2 [“refund”];
if Tu1 6= null and Tr1 6= null and Tu2 6= null and Tr2 6= null then
delta ← 0
if Tu1 [“value”] > Tu2 [“value”] then
delta ← (Tu1 [“value”]− Tu2 [“value”])
end if
if Tr2 [“value”] ≥ (Tr1 [“value”] + delta * Tr1 [“compensation”]) then
return true;
end if
end if
return false;
Now, although up until now it may have seemed that verifying whether these
constraints are fulfilled or not is straightforward as it only involves applying the given
algorithms, in practice actually verifying the constraints is a bit more difficult. This
is primarily caused by the fact that in reality, the consumer and provider must have
a way to transfer the algorithm that embodies the constraint to the market. The
most obvious way to do this, is by allowing consumers and providers to run custom
code that can verify these constraints within the market. Despite the fact that the
action of running custom programming code on a remote location itself is relatively
straightforward using some software libraries, doing so introduces a whole new set
of technical challenges as this remote code must be verified to be safe prior to being
executed on the market. That is, in order to guarantee the market’s integrity, it
must be verified whether the consumer’s or provider’s code that verifies a constraint
does not contain malicious code. While introducing a system that actually does this
is certainly possible, there is also a way to avoid the technical hassle of having to
examine the code of each incoming constraint.
By giving the market a database of standard constraints for which it knows the
matching semantics and leaving the matching of non-standardized constraints to the
consumer and provider themselves, the technical difficulties that remote code intro-
duces are effectively avoided. In practice, such a system would work by letting the
consumers and providers specify the name of the constraint as part of the constraint
set when a standard constraint is used and specify the endpoint of a webservice
that can verify custom constraints in the other cases. Note that by doing this, the
market still remains agnostic of the matching semantics of resources themselves, as
the relation between a resource (tag) and its matching semantics is still determined
by the consumer or provider.
When elaborating on this idea of using a set of standard constraints for which
the market knows the matching semantics, it is interesting to introduce a set of
more generic constraints in order to maximize the number of constraints that can
be matched by the market itself. The idea being that such constraints should be
able to verify whether the properties in any given tag in both the provider and
consumer set have a certain relation. For example, having a constraint such as the
LessThan constraint as defined in Algorithm 3, allows consumers and providers to
express that the bid or ask against which their bid or ask is matched should have a
specific tag and that the value of a specific attribute in that tag should be less than
the value of the attribute with the same name in the same tag that accompanied
the bid or ask that specified the constraint. Other generic constraints such as the
GreaterThan, Equal, LessThanOrEqual and GreatThanOrEqual would have similar
semantics. Constraints such as the EqualAll constraint defined in Algorithm 4 could
similarly be used to express that two tags sets should both contain a certain tag and
that all attributes in the tag of the first set should have an attribute with the same
name and value in the tag of the second set5 .
To further indicate the usefulness of such constraints, consider the Resource con-
straint in Algorithm 5 which is a generalization of the Memory constraint of Algorithm
1 that can also be used to implement many other hardware related constraints (such
as Disk or CpuClock).
In order to express which tag is checked in a generic constraint, the notation
Constraint[Parameters...] (as in Resource[Memory] or Equal[AllocationType,
value]) will be used in the remainder of this thesis.
5
Note that this is different from requiring that both tags have the exact same attributes, as the
second tag might have more attributes than the first tag. Obviously, the market could also choose
to define the semantics to be so that both tags must have the exact same attributes.
Algorithm 3 LessThanConstraint(T S 1 , T S 2 , tagName, attributeName)

T1 ← T S 1 [tagName];
if AT1 [attributeName] < AT2 [attributeName] then
return true;
end if
end if
return false;
Algorithm 4 EqualAll(T S 1 , T S 2 , tagName)

if T1 = null or T2 = null then
return false;
end if
for all (key, value) ∈ AT1 do
if value 6= AT2 [key] then
return false;
end if
end for
return true;
Algorithm 5 ResourceConstraint(T S 1 , T S 2 , tagName)

if (AT1 [“unit”] = AT2 [“unit”]) and (AT1 [“value”] ≤ AT2 [“value”]) then
return true;
end if
end if
return false;
The technicalities of how this system with standard and custom constraints (us-
ing web services) would work exactly go beyond the scope of this chapter, but will
be explained in detail in the next chapter, as part of the software architecture of the
IaaS market.
3.1.5 The CDA using tag and constraint sets

Using the tag and constraint sets, it is now possible to introduce a CDA based
market, similar to Dube’s CDA described in section 3.1, that can cope with the
constraints as needed in an IaaS setting. Figure 3.1 gives a schematic overview of
how such a market would work.
Figure 3.1: Overview of the Continuous Double Auction for IaaS Resources
From this figure it can be seen that the market process can be split in three
distinct phases. In the first phase, consumers submit bids and providers submit
asks to the market. The bidding language that is used to describe a consumer’s bid
and a provider’s ask is technically the same and can be represented as the following
5-tuple.
[Volume, Price, ExpirationDate, TagSet, ConstraintSet]
The different components of this tuple should be interpreted according to whether

they represent a bid or an ask, similar to the explanation given in section 2.5.2. That
is, in case the tuple represents a bid, the Volume indicates the number of specified
goods the consumer is willing to buy, the Price indicates the price at which the con-
sumer is willing to buy a single unit and the ExpirationDate indicates until when
the consumer’s bid is valid. The Tagset contains the tags that describe the resources
the consumer is requesting, possibly complemented with some additional (personal)
information about the consumer and the request as to increase the likelihood that
any provider imposed constraints can be fulfilled. The ConstraintSet contains the
actual constraints the consumer is imposing on the tag set of the provider’s offer.
If the offer is an ask, the Volume indicates the number of specified units the
provider is willing to sell, the Price indicates the price at which the provider
is willing to sell a single unit and the ExpirationDate indicates until when the
provider’s ask is valid. The Tagset contains the tags that describe the resources the
provider is offering as well as some information about the provider itself, while the
ConstraintSet contains the constraints the provider is imposing on the tag set of
the consumer.
As noted before (in section 2.5.2), in order to determine which bids can be
matched with which asks, it is important that the market knows whether asks and
bids with a Volume larger than 1 can be split into several sub bids with smaller
volumes or not. Although this decision could be made market-wide (that is, a
decision is made whether splitting offers is allowed or not, and this decision applies
to all offers), using the tagsets it is also possible to let consumers and providers decide
for themselves whether they allow their offer to be split. Concretely, this is done
by adding the [Splittable, (value, x)] tag, which specifies that the number of
units of an offer that has to be matched at once must be a multiple of x. The fact
that this kind of consumer and provider specific behavior can be incorporated into
the market without needing to change the bidding language again demonstrates the
flexibility of the tag and constraint based approach.
The second phase that is shown in Figure 3.1, is the matching phase within the
market itself. For this, the sorted queue based matching mechanism that has been
described before is used. An incoming bid matches an existing ask in the price
ascending ask queue if the price of the incoming bid is higher than that of the ask
and if the constraints in the constraint sets of both the bid and ask are fulfilled (as
shown formally in 3.10 for a bid B and ask A).
∀ C ∈ CS A : C(T S A , T S B ) = true ∧ ∀ C ∈ CS B : C(T S B , T S A ) = true (3.10)
If a match is found, the third phase of the market process is entered in which
the consumer and provider are notified of the found match. If no match is found,
the consumer’s bid is added to the bid queue that is sorted according to decreasing
price. The process for an incoming ask is completely analogous. If no match can
be found before the expiration date of an offer, it is automatically removed from
its queue. Whether the consumer or provider that submitted the offer is notified of
this or not is of little overall importance and can be considered an implementation
detail.
3.1.6 Practical example

Now that tag and constraint sets, as well as a CDA-based market that uses these sets
have been properly introduced, it is crucial to verify whether these concepts would
really function in the current IaaS environment by trying to map some existing
IaaS solutions to the proposed model. To do so, this section will give a rudimentary
example of how the process of matching an Amazon EC2 small instance would work.
Consider a market with 4 different consumers and a single provider (that is, Ama-
zon), and let the bidding process proceed as follows.
1. Amazon submits an ask, specifying a volume of 5 small instances at the price

of $0.10 per hour.
Ask1 = [5, 0.10, 12/07/11 13:00:00 UTC-7, {

[Memory, {(unit, MB), (value, 1700)}], [EC2_ECU, {(value, 1)}],
[Disk, {(unit, GB), (value, 160)}], [Platform, {(value, x86)}],
[EC2_API, {(value, m1.small)}], [SLA_Uptime, {(value, 99.95)}],
[SupportedOperatingSystems, {(list, ...)}],

[SupportedEC2_AMI_IDS, {(list, ...)}],
[Splittable, {(value, 1)}],
[AllocationType, {(value, OnDemand)}],
[Time, {(value, 1), (unit, hour)}],
[Region, {(value, USA/Virginia)}]
[Provider, {(name, Amazon AWS), (url, http://aws.amazon.com)}] },
{ ConsumerLocationExclusion["Cuba"],
ConsumerLocationExclusion["Libya"], ... }
]
In this ask, the hardware aspects of the small instance are described using
various tags (Memory, Disk, etc). Additionally, the provider also added a list
of supported operating systems, a list of supported AMIs (Amazon Machine
Image), a Splittable tag indicating that the ask can be split into single units,
a tag describing the region in which the resources are located, etc. Note among
these tags the use of the AllocationType tag to indicate that the resources are
offered according to the on-demand allocation model. Other values for this tag
could be Spot, Future or Reserved as discussed in section 2.5.5. When the
chosen allocation model is on-demand, future or reserved, the price specified
in the ask should be considered as a price that needs to be paid periodically
according to the time unit specified in the Time tag6 . When Spot is chosen, this
Time tag keeps it’s meaning as the periodical length of the time after which
the consumer needs to pay, but the price in the ask should be interpreted
as the current spot price rather than the price that needs to be paid every
time unit. As this price changes after each period, the ask’s expiration date
should consequently be set to expire after the passing of the current period. If
the provider wishes to offer spot resources for another period, he should then
resubmit the ask with an updated price.
Besides tags, Amazon also specifies multiple ConsumerLocationExclusion

constraints to make sure that the consumers it is matched with are from coun-
tries with which it is willing (or allowed) to do business with.
2. A consumer submits a bid with a price of $0.08 and some hardware constraints.
Bid1 = [1, 0.08, 09/07/11 14:10:00 UTC+1, {

[Memory, {(unit, MB), (value, 1200)}],
[Consumer, {(name, "John Smith"),
(location, "Paris, France, EU")}] },
{ Resource["Memory"], Equal["AllocationType", "value"],
EqualAll["Time"] }
]
6
If the reserved or future allocation model is used, an additional Duration tag that specifies how
long the resources will actually need to be allocated or reserved should also be specified as part of
a bid.
As the price of this bid is lower than that of Amazon’s ask in the ask queue,
it is not matched and consequently added to the bid queue.
3. A consumer submits a bid with a price of $0.12 and some hardware constraints.
Bid2 = [2, 0.12, 05/06/11 02:00:00 UTC+1, {

[Disk, {(unit, GB), (value, 100)}],
[Consumer, {(name, "Jan Modaal"),
(location, "Brussels, Belgium, EU")}] },
{ Resource["Disk"], Resource["Memory"],
Equal["AllocationType", "value"], EqualAll["Time"] }
]
As the price of this bid is higher than that of the ask in the ask queue and both
provider and consumer constraints are fulfilled, this bid is matched against the
ask.
4. A consumer submits a bid with a price of $0.20 and the following tags and
constraints.
Bid3 = [2, 0.20, 15/06/11 14:00:00 UTC-2, {

[CpuClock, {(unit, Ghz), (value, 2.2)}]
[CpuCores, {(value, 2)}],
[Consumer, {(name, "Jane Smith"),
(location, "Boston, Massachusetts, USA")}] },
{ Resource["CpuClock"], LargerThanEqual["CpuCores", "value"],
Resource["Memory"], Equal["AllocationType", "value"],
EqualAll["Time"] }
]
As this bids contains constraints on CpuClock and CpuCores tags that are not
present in Amazon’s ask, it can not be matched and will thus be added to the
bid queue.
As a side note, notice that the consumer is requesting specific processor hard-
ware here, which is not supported by most IaaS providers. That is, most of the
time, a high level of abstraction is made when it comes to processor power. By
using processor type agnostic units such as the ECU, cloud providers can hide
heterogeneity among there own infrastructure while at the same time allow
consumers to specify a processor based on its delivered performance rather
than on its specific hardware characteristics.
A few providers (such as Newservers [64] and CloudSigma [65]) do however
allow consumers to express more specific processor requirements.
5. A consumer submits a bid with a price of $0.14 and the following tags and
constraints.
Bid4 = [4, 0.14, 05/06/11 02:00:00 UTC+2, {

[EC2_ECU, {(value, 1)}],
[Splittable, {(value, 1)}],
[Consumer, {(name, "Mohammed Naser"),
(location, "Tripoli, Libya, Africa")}] },
Equal["AllocationType", "value"], EqualAll["Time"] }
]
Although the offered price of this bid is higher than the price in Amazon’s ask
and the consumer constraints are all met, no match will occur. This is due
to the fact that this bid comes from someone from Libya, which means that
the ConsumerLocationExclusion constraint set by the provider will not be
fulfilled and consequently the bid and ask will cannot be matched with each
other.
After these bids and asks, the bid and ask queue are as follows.
Bid Queue Ask Queue

Price ($) Volume Bid Price ($) Volume Ask
0.08 1 Bid1 0.10 3 Ask1
0.14 4 Bid4
0.20 2 Bid3
Although this example shows that it is indeed possible to use a CDA-based

market to trade IaaS resources, it neglects the fact that if different providers and
consumers use different time units to price their offers, any price-based sorting of
offers in queues is effectively meaningless. That is, the prices that are attached to
offers can only be used to meaningfully compare offers with each other if the specified
resource allocation times of the offers are the same.
While it is possible to solve this problem by only accepting timeframes of a

specific size (e.g. a single hour) or by normalizing offers and prices to the same time
unit before putting them in the appropriate queue, the presence of this problem
actually suggests the existence of a far more fundamental issue with using the CDA
for IaaS resources. The next section will discuss this problem is in detail and present
the first steps towards a solution.
3.2. ISSUES WITH THE CDA MODEL FOR IAAS RESOURCES 78
3.2 Issues with the CDA model for IaaS resources

Although tag and constraint sets allow for increased flexibility, when using them in
conjunction with the CDA mechanism they still fail to address a more fundamental
issue related to the IaaS pricing model. That is, as cloud computing is based on the
pay-what-you-use pricing model, the price that consumers pay for a set of resources
is not only determined by the characteristics that define these resources, but also by
how these resources are used. More specifically, in the cloud setting, providers do
not only charge consumers for the allocation of a specific set of resources, but also for
how these resources are actually used once they are allocated. To implement such a
model, the total cost for the use of a specific set of resources cannot be determined
by a single price but needs to be influenced by variety of different price components
that get different weights according to the actual use of the resources. Because of
this, using the classic CDA that only takes into account a single price component
(such as the mechanism described in section 3.1.5) is no longer feasible.
3.2.1 IaaS price models

In practice, most providers implement the multi-price-component model by offering
their customers a number of basic resource configurations for fixed price per time-
frame and charging extra for the use of specific resources, features or services. A
good example of such a resource is the use of network bandwidth, which is typically
measured in transferred bytes and charged separately from the time-based cost of
the allocation of the resources themselves. Table 3.2 gives an overview of how dif-
ferent cloud providers charge bandwidth, while Table 3.3 lists some other resources
and services that are often charged for separately by providers. Note that some of
the entries in this table are optional services for some IaaS providers (most notably,
those marked with an asterisk ∗), but are part of standard IaaS packages of other
providers.
Provider Bandwidth Price Model

Amazon (EU-West) $0.10/GB inbound traffic
price outbound traffic varies (1 GB free, $0.15/GB for up to
10 TB/month, $0.11/GB for the next 40 TB/month, . . . )
GoGrid inbound traffic is free
$0.29/GB for outbound traffic (cheaper prices for larger up-
front commitments)
NewServers 3GB of traffic (inbound or outbound) per hour included.
$0.10 per extra GB
Flexiscale 5 units per GB (1000 units = $17.95)
same price for inbound and outbound traffic
Table 3.2: Bandwidth prices of different IaaS providers
As already mentioned, the fact that IaaS providers use a multi-price-component

Cost Commonly Used Price Model

Bandwidth pay per amount of inbound and outbound traffic
Disk I/O pay per number of used disk reads and writes
Storage pay per timeframe and/or per stored GB
Operating systems pay per timeframe for properietary OSes
Software licenses pay per timeframe, per user or by an other measure
IP address allocation* pay per allocated (block of) IP address(es)
Firewall services* pay per timeframe and/or per amount of processed traffic
LoadBalancer services* pay per timeframe and/or per amount of processed traffic
Table 3.3: Resources and services that are commonly charged separately by IaaS
providers
model to charge their customers poses a real problem to the use of the CDA mech-
anism in the IaaS cloud setting, as the CDA model can only take into account a
single price. When looking at this problem for the first time, it might seem that
there is a rather simple solution to this problem. That is, letting consumers and
providers specify a price as part of their offer for the allocation of a set of resources
and letting them use constraints that define restrictions on the minimum or maxi-
mum price of the extra price components. For example, a consumer can specify that
he is offering to pay 10ct per hour for a certain set of resources and additionally
specify an OutboundBandwidthCost constraint that verifies that the price that he
specified in an OutboundBandwidthCost tag accompanying his bid is lower than that
of the provider. Analogous, a provider can specify that he is asking a price for a
set of resources and additionally specify a constraint that checks that the price in
the consumers tag is higher or actual to the price the provider wants for a GB of
outgoing bandwidth.
Although at first it thus may seem that using constraints that impose maximum
or minimum values solves the problem of the extra price components, it actually
does not. There are several reasons for this. First of all, it might be very difficult
for the consumer to determine what a realistic price is for each of the different cost
components (although this can probably be solved using some (advanced) price dis-
covery mechanisms). Related to this, is the fact that many consumers are probably
only interested in the total cost of using a set of resources, and not in the exact
allotment of all different price components.
Secondly, when a minimum or maximum price constraint is set, it is not entirely
clear what the actual unit-price for use of the particular resource or service should
be once a match is made. That is, should the price specified by the consumer be
used, the price specified by the provider, or is a more complex pricing rule needed?
Although this is in essence not really a problem, but rather a decision that needs
to be made, the fact that such decisions potentially need to be made on a per price
component and per provider basis does add more complexity to this method.
The most important issue however, is the fact that when this kind of constraints
are used, the market can no longer guarantee optimal matches using the information
that is currently provided. This comes directly from the fact that minimum/maxi-
mum price constraints do not take the actual usage of the specific resource or feature
into account, only its unit price. To understand why this is an issue, consider Figure
3.2, which depicts the total cost of 2 instances7 (at different providers) that use an
increasing amount of bandwidth within a single hour. It is assumed that a fixed
hourly price of 10 ct is paid for instance A, while the fixed hourly price for instance
B is 13 ct. Additionally, the provider of instance A charges 10 ct per used GB of
network bandwidth, while the provider of instance B charges 7 ct per used GB. It
is intuitively clear, and shown on the graph, that when no bandwidth is used, using
instance A is the cheapest. However, when bandwidth is consumed, the difference
in total cost between instance A and B becomes smaller. If more than 1 GB is
consumed, using instance B will minimize total costs. Figure 3.3 shows the same
Figure 3.2: Graph showing the importance of taking into account the usage of
additional costs components. As long as less than 1 GB of data is transfered, using
instance A is the cheapest, otherwise using instance B is the cheapest.
result, but varies both the used time (in hours) and the used bandwidth (in GB).
7
The term instance in this context denotes a basic virtualized computing environment, in the
same way Amazon uses this term to denote the virtualized servers it offers on its EC2 platform.
Obviously, the provider corresponding to the lowest plane for any time-bandwidth
combination is the cheapest.
It is clear that this example can be generalized to incorporate n providers and
m price components. Finding the cheapest provider for a specific set of usage values
is then the same as searching through n hyperplanes in an m + 1 dimensional space,
and finding the hyperplane that has the smallest cost value at the point defined by
the m given usage values.
Figure 3.3: Graph showing the importance of taking into account the usage of
additional costs components. Both time and bandwidth vary, the cheapest instance
for a given time and bandwidth combination is given by the lowest (smallest total
cost) plane.
In other words, figures 3.2 and 3.3 indicate that ordering bids and asks in queues
according to a single price-component as is done in the CDA-based market makes
no sense for IaaS resources as this ordering does not reflect the actual price that
will need to be paid or will be charged to make use of these resources. If a CDA
mechanism based on a single price component would still be used, it is destined to be
gamed by providers that advertise very low prices for that single price component,
but charge expensive prices for the actual use of resources. That is, in this mechanism
a provider might be ranked at the top of the ask queue by advertising an hourly
price of $0.001 for a set of computational resources, while charging a whole dollar for
1 MB of network bandwidth. Almost surely, this makes it a very expensive provider
for a whole range of consumers compared to other providers that advertise a higher
hourly price. Consequently, this provider should be at the bottom of the ask queue
and not on top. Obviously, such an IaaS market that only incorporates a single-price
component system has limited applicability.
Usage and price tags
It is thus clear that if any IaaS market is to make acceptable matches, it needs to
have consumer usage data for all of the different price components as well as the
actual prices as determined by the providers. As such, it is imperative that con-
sumers specify their usage of different resources as part of their bid and providers
specify the different price components as part of their ask. Luckily, this can be
done rather easily using the previously introduced tag sets. For example, ex-
pressing that 3.6 GB of outgoing bandwidth will be needed is as easy as defining
[OutgoingBandwidthUsage, {(unit, GB), (value, 3.6)}] as part of the tagset
of a bid. Similarly, specifying that the price for that bandwidth is $0.10 per GB
is as easy as adding [OutgoingBandwidthPrice, {(unit, GB), (value, 0.10)}]
to the tagset of the ask.
Using these usage values and prices, it is theoretically possible for the market to
determine the actual price for the use of a given set of resources. However, it will turn
out that due to the heterogeneity of price models that are used by different providers,
actually determining this price is still unfeasible without more information about
how the different price components relate to each other. Before explaining this in
more detail, a remark concerning the importance of the accurateness of the consumer
specified usage values must be made. That is, as the choice of which consumer
is matched with which provider is determined by the total cost of using a set of
resources, it is clear that the accurateness of the values in the usage tags is crucial
as they directly influence the total cost for the use of these resources. However,
in reality, determining these usage values is not always that easy. To accurately
determine the bandwidth usage or the disk I/O behavior of a certain application,
advanced analysis is often necessary. As doing such analyses is time-consuming
and requires a certain expertise, many customers might not be able or willing to
determine accurate (or at least sensible) usage values. To solve this, brokers should
be used that benchmark certain applications or use-cases for consumers in order to
determine good usage values. Alternatively, brokers might help consumers estimate
these values based on the consumer’s use-case (e.g. small website, scientific CPU
intensive application, etc).
3.2.2 Using a multi price component model

Given the usage and price information as specified by the usage and price tags, it is
now seems possible for the market to determine the actual price that will charged
by provider for a specific consumer bid. In order to do so, the market could use a
model similar to the one given in 3.11.
X
P (a, b) = v(T ) pA (T ) (3.11)
T ∈ (T S a ∪ T S b )
In this formula, the exact price P (a, b) for a certain bid b that is matched with
an ask a of provider A is determined by the weighted sum (according to the usage
of the consumer) of the different price components of the provider. The weights are
given by v(T ), which is defined to be the value of the usage-tag that is used by the
price-component for the resource specified by tag T . The term pA (T ) is then the
unit price specified by provider A for usage of the resource specified by tag T .
In practice, the formula in 3.11 would be translated to a combination of an
OutgoingBandwidthUsage tag with and OutgoingBandwidthPrice tag for network
bandwidth or to a combination of a TimeUsage with a TimePrice for a memory
resource (specified by a Memory tag). The total cost for a given bid is then determined
by the sum of all these different combinations8 .
As simple as the idea behind the formula in 3.11 might be, actually incorporat-
ing it into the market mechanism and specifically the market’s bidding language is
non-trivial. This comes from the fact that the relation between the usage tags and
the price tags must be specified by the provider as the market itself does not have
this provider-specific knowledge. The way of expressing this must be flexible enough
to allow providers to indicate which prices belong to which resource and usage tags
as well as indicate that certain combinations of tags only incur a single cost. For
example, when the consumer specifies a Memory tag and constraint, it should be
possible for the provider to indicate that the combination of the value in the con-
sumer’s TimeUsage tag and the value in the provider’s MemoryPrice determine the
actual cost of the memory. Similarly, a provider should have the means to indicate
that when when both CpuClock and Memory tags and constraints are present, the
consumer should only be charged once for both resources by using the values in the
consumer’s TimeUsage and the provider’s TimePrice tags.
Finally, note that it is very well possible that the usage is not specified in the
tags attached to the bid but that the matching ask imposes a standard usage. In
practice, this is particularly true when certain hardware resources are specified (e.g.:
Memory) but no usage tags (such as TimeUsage) are given (the usage function should
than return e.g. a single hour for those hardware constraints).
A possible way to define the relations between the usage tags that define v(T )
and the unit price tags that define pA (T ) is by using a table such as Table 3.4 that
specifies which resource tags should by measured by which consumer usage tags as
well as the unit and unit price that should be used to determine the actual price for
the given resources.
8
Note that the formula in 3.11 implicitly assumes that the usage unit in the usage and price
tags are the same (e.g. hour, MB, etc.). It is certainly also possible that only compatible units
are required (e.g. minutes in the usage tag and hours in the price tag), and that v(T ) and pA (T )
automatically convert between compatible units.
Resource Tag Usage Tag Usage unit Unit price

Memory TimeUsage hour 0.10
CpuClock
CpuCores
Disk
SLA
OS (Windows based) TimeUsage hour 0.03
LoadBalancing TimeUsage hour 0.10
LoadBalancing OutgoingBandwidthUsage GB 0.02
LoadBalancing IncomingBandwidthUsage GB 0.02
OutgoingBandwidth OutgoingBandwidthUsage GB 0.13
Disk I/O DiskAccessUsage million accesses 0.05
Table 3.4: Provider specified table that defines the relations between usage and price
tags
Note that some resource types such as Memory and CpuClock are grouped to-
gether while other resources, such as LoadBalancing occur multiple times in the
table. By grouping the resources, providers can indicate that the consumer should
only be charged once for those resources. Similarly, by adding a resource multiple
times with different usage tags, the provider can indicate that certain resources in-
cur costs that are determined by multiple factors (in the case of the load balancing
service, both per hour and per in- or outgoing GB). Note that if a certain resource
is not specified in the table, no cost will taken into account for that resource. Also
note that there might be tags in the first column that are not really resources but
rather requirements that also incur a cost. For example, Table 3.4 indicates that
if an SLA tag is specified but no Memory, CpuClock, CpuCores or Disk tags, a cost
dependent on the TimeUsage tag will still be applied.
Although a solution using a model similar to 3.11 and the price table specified in
3.4 would probably work, such a solution is still flawed in two important ways. First
of all, it assumes that when certain tags are present in a bid, there are also constraints
specified in the same bid that enforce the use of these tags. More concretely, the
model assumes that when a Memory tag is present in a bid, the consumer should
automatically be charged for that (typically according to the TimeUsage tag), even
though it is perfectly possible that no Memory constraint is attached to the bid that
actually enforces any memory requirement. In order to solve this, a more complex
bidding language, based on a more evolved form of Table 3.4, should be introduced
that also takes into account the relation between tags and constraints.
A second, more problematic issue with the proposed price model is that it is
linear in nature as it makes a linear combination of the consumer’s usage values and
the provider’s price components. However, in reality many providers do not use such
3.3. THE CONTINUOUS REVERSE AUCTION FOR IAAS RESOURCES 85
a simple linear price model, but use more complex models to charge their customers.
For instance, many providers use stepped pricing rules in which consumers pay less
per unit if they buy more units9 . Besides such stepped prices, it is not unthinkable
that certain providers may wish to use more complex (non linear) functions or even
a piece of software code to determine their exact quotation for a given bid. A good
example of such a use case is given by NewServers [64], which provides 3 GB of free
outgoing bandwidth per hour, but charges $0.10 per additional GB. Expressing such
complex pricing rules that take several units into account can only be done using
programmed code.
Towards a flexible solution
The only real way of solving all price model related problems as well as the con-
straint enforcement issue is by letting providers run custom code - that determines
a quotation for each matching bid - within the market itself. However, one can ask
whether the term continuous double auction is still applicable in such case. In fact,
such a market mechanism looks more like a reverse auction (since each provider
provides a new quotation for each matching bid based on custom code) than it looks
like a double auction. When further exploring this idea, one can ask whether taking
the difficult step of allowing remote code to run within the market (which intro-
duces a lot of security issues that need to be taken care of) can not be replaced by
letting that code run on the providers itself, similar to how the problem of custom
constraint code was avoided by the use of web service calls as put forward in section
3.1.4.
Indeed, by contacting each provider and asking for a quotation for each incoming
bid, the market can determine exact prices for bids without needing to sacrifice price
model flexibility. The next section introduces a new market mechanism, called the
Continuous Reverse Auction, that does exactly that.
3.3 The Continuous Reverse Auction for IaaS Resources

The Continuous Reverse Auction (CRA) is a new type of auction in which ideas of
the Reverse Auction and the Continuous Double Auction are combined to trade (het-
erogeneous) commodities among multiple buyers and sellers. A systematic overview
of how the CRA works, is provided in Figure 3.4.
3.3.1 Mechanism and Principles

In the Continuous Reverse Auction, bids made by the consumers are replaced by
so-called requests, and asks are replaced by what will be called quotes. The reason
why the words request and quote are used instead of bid and ask is due to the way
the CRA functions. In the Continuous Reverse Auction, it is no longer the case
9
This is the case for e.g. Amazon which charges a different price per outgoing GB depending on
your total outgoing monthly bandwidth usage, as already indicated by Table 3.2.
Figure 3.4: Schematic overview of the workings of the Continuous Reverse Auction
that consumers and providers specify prices for a predefined set of resources they
are requesting or offering. Instead, consumers submit requests to the market and
providers provide quotes for these requests through the market. Specifically, the
bidding process is started with a consumer submitting a request to the market. The
market will subsequently find the cheapest provider for that request by querying
a group of previously registered providers and ask them for a quote for the given
consumer request. Providers answer to this request by either sending a quote to the
market specifying the price they will charge to fulfill the request or by indicating
that they are unable to match the given request. After all providers have delivered
their quote, the market is able to select the cheapest of all the quotes and offer it as
a match to the consumer who can then either accept or decline the match.
By using a system in which providers no longer submit prices to the market

but are asked to give a price on a per request basis, the problems discussed in the
previous section are circumvented as providers are free to use the price model they
want without needing to resort to remote code. Additionally, as providers now
determine whether they can match a request with one of their offers or not, the
market no longer needs to know how to match constraints, nor does it need to have
constraint specific knowledge. Indeed, when using the CRA mechanism, the market
is participant and constraint agnostic, and acts only as an intermediate.
Although the reasons why a quote-based approach for providers is used in the
CRA have been explained extensively, the reason why consumers no longer submit
prices in the CRA has not yet been covered. The rationale behind this choice stems
from the fact that in the CRA, it is no longer necessary for the consumer to do so.
To understand this, consider the reasons why a consumer does submit a price in the
CDA.
First of all, in the CDA, the price specified in a bid is used to determine which
provider asks need to be considered for matching and which do not. That is, the
price specified in a bid is in fact the maximum price the consumer is willing to
pay for the resources he is requesting. Any asks that are more expensive than the
price specified in the bid will not be considered when trying to match the bid, even
though it is very well possible that only those asks fulfill the necessary constraints.
By doing this, the market is able to match asks to bids more efficiently as it no
longer needs to perform computationally intensive constraint checks for asks that
are too expensive. In the CRA however, this is no longer an issue as the market
no longer needs to check provider constraints itself. As all quotes returned to the
market fulfill the consumer’s constraints per definition, there is no need for a price
to perform a selection among the provider offers.
A second, arguably more important, reason why a price is specified in a bid when
using the CDA is to create an ordering among consumers. That is, if a bid is not
matched immediately, it is placed in the bid queue according to descending price
(bids with lower prices are placed further down the queue) in order to be matched
at a later time. The reasoning behind this ordering is that consumers that are
willing to pay more for their resources should get preference above consumers that
are paying less as this increases the overall allocative efficiency. By first considering
the bids at the top of the queue when matching an incoming ask, this preference
is effectively created among the consumers. Concretely, this means that if two
consumers are requesting the same resources, the consumer that is willing to pay
most gets matched first as this is maximizes the overall profit. That is, by matching
with the consumer that specified the higher price, the provider will be able to get
more money for the resources and the consumer that attaches to most value to the
resources gets to actually allocate them.
However, in the IaaS ecosystem, this is no longer important as the principle of
“infinite provider capacity” leads to the fact that providers currently do not differ-
entiate among consumers, and will always serve both consumers at the same price.
As such, there is no need for a direct ordering between consumers, and consequently
the price specified as part of a bid is also no longer necessary. Section 3.3.5 makes
some further considerations about this in the context of allocative efficiency.
Now, although it is theoretically still possible that a consumer specifies a maxi-

mum price as part of his request to the CRA, not requiring this frees the consumer
from the difficult task of having to determine an accurate price estimation for a
request without actually making it. By allowing the consumer to accept or decline a
quote (match) himself given its price, the consumer can do accurate price discovery
(this is discussed in more detail in section 3.3.3) without having to resort to more
difficult and less accurate techniques.
Also, observe that modifying the CRA model as to allow consumers to submit
prices to the market in order to preserve the surplus between the price a consumer
is willing to pay and the price the provider is asking, is not that difficult. Doing so
only involves letting the market check whether the consumer’s price is higher than
that of the cheapest provider and apply a certain pricing rule as to determine which
price will actually need to be paid by the consumer10 . Although there is currently no
direct need to implement this kind of matching due to the infinite capacity model,
it is theoretically not impossible that th view towards the infinite capacity model
will change in the future and that providers will increasingly start differentiating
among different consumers by using price and allocation models that follow demand
and supply more closely. The EC2 spot market is already a good example of such a
model within a single company. The further study of these kinds of market model
is an interesting subject for future research.
Relation to the Reverse Auction
As the name suggest, the Continuous Reverse Auction is based on principles of the
Reverse Auction. Although there consequently are a lot of resemblances between
both auction mechanisms, there are also some notable differences between them.
The main differences between the CRA and the regular RA can be found in the
facts that the CRA is a continuous auction and deals with multiple buyers, while
the RA is not continuous and also considers multiple sellers but only a single buyer.
As with the CDA, the word continuous in Continuous Reverse Auction refers to the
fact that the market tries to find a match immediately after a request is submitted,
as compared to a regular double auction in which matches are made in a specific
matching phase or a regular reverse auction in which multiple price request rounds
are used. In short, by using the word continuous in the name of the CRA, it is thus
implied that the CRA uses only a single price request round and that matches are
returned directly.
The reason why the CRA only uses a single price round is based on the observa-
tion that for virtually all requests such multiple price rounds are just not beneficial
as in the IaaS ecosystem providers are not interested in competing with each other
for every single consumer request. That is, assuming an IaaS market that will
have to match a very large number of requests at any given time (in the order of
thousands per hour or even minute), the matching of a single additional request is
financially insignificant compared to the total number of requests that are matched
by a provider; even if that specific request involves a considerable amount of money.
As such, from a providers point of view, trying to ‘win’ a single additional request is
just not effective. Instead, in order to make significant financial gains, the provider
should adjust its general matching strategy as to ‘win’ a whole range of similar re-
quests. How a provider can do this is briefly explained in section 3.3.3, which deals
with price discovery in the CRA.
Note that the insight that the multiple price rounds are no longer needed in the
10
Recall the pricing rule examples in section 2.5.3.
CRA is in fact derived from the reason why multiple rounds are needed in the regular
reverse auction. That is, in the normal RA, multiple price rounds are used because
large amounts of money are involved and only a single (typically special) good or
service is being bought by the consumer. Because of this, winning the auction
is crucial to providers in order to do any trading at all. In the IaaS ecosystem
however, the goods that are being bought are cheap and abundant (in other words,
commodities). If a provider is unable to win a particular request, nothing is lost,
as there are plenty of opportunities to do win requests in the future. As such, the
multiple price rounds as used by the RA are no longer needed as providers do not
need to compete over a single request in order to trade resources.
Of course, only using a single price request round also has some benefits in terms
keeping the system’s complexity limited. That is, there is no need for providers
to maintain state as each price request is independent from any previous request.
Additionally, as only a single message needs to be exchanged between the market
and providers for a match to occur, the load on the entire system is reduced, as well
as the time that is needed to find a match.
3.3.2 Bidding Language

Now that an overview of how the CRA functions as well as the core ideas behind it
have been discussed, it is time to elaborate on the bidding language of the mecha-
nism. As Figure 3.4 provided a good overview of the different steps of the CRA’s
bidding process, it will be reused as a basis to discuss the CRA’s bidding language.
The bidding process of the CRA begins with a consumer that wants to allocate
a set of IaaS resources. In order to do so, the consumer submits a request to the
market. The bidding language that is used to specify this request, is based on the
tag and constraint sets that have been introduced before.
Request = [Volume, TagSet, ConstraintSet]
As mentioned before, a consumer only has to accept or decline an offered price,

meaning that it is no longer necessary for him to specify a price as part of his
request. Also, as a request in the CRA is either matched immediately or not at all
(if no provider is able to fulfill the consumer specified constraints), there is no need to
specify a expiration date as part of the request. In other words, the only components
of a request are a Volume indicating how many units a consumer wishes to allocate,
and a TagSet and Constraint specifying the resource he wishes to allocate.
Once the request has arrived at the market, the market will send out a price
request to all registered providers, to which the providers respond with either a
negative answer meaning that they cannot match the request or with a quote that
specifies a price for the request.
Quote = [Price, Volume, Identifier]

Besides the Price, the quote also contains a Volume that indicates how many
units the provider can or is willing to match (this is only important if the consumer’s
request is splittable, that is, tagged with the Splittable tag). The Identifier in
the quote allows a consumer to actually allocate resources after a match has been
found on the market; it allows the consumer to proof the commitment a provider
made when it handed out a certain quote. How this works exactly is detailed in the
following chapter.
After all registered providers have returned a quote (or have replied that they
cannot match the request), the market returns the cheapest quote as a match to
the consumer. If bids can be split, the market can match different units of a con-
sumer’s request with different providers and return multiple matches. Naturally, this
matching occurs according to the quote prices specified by the providers (cheaper
providers are matched first). Algorithm 6 shows how the market could implement
such a matching procedure. In order to be concise, Algorithm 6 does not incorporate
splittable bids.
Algorithm 6 submitRequest(r: Request): Match

bestQuotes ← {}
for p in providers do
bestQuote ← bestQuotes.f irst();
currentQuote ← p.getQuote(r);
if bestQuote = null or currentQuote.getP rice() < bestQuote.getP rice() then
bestQuotes ← {currentQuote}
else if bestQuote.getP rice() = currentQuote.getP rice() then
bestQuotes.add(currentQuote)
end if
end for
matchingQuote ← selectRandomElement(bestQuotes);
return new Match(matchingQuote);
After the consumer has received the match (or multiple matches), he only needs
to confirm or decline this match to the market in order for the matching process to
complete.
Compound Requests
This section detailed the bidding language and process of the CRA mechanism.
Among other things, it defined which components are part of a consumer’s request.
However, the given definition implicitly assumes that the application for which the
consumer is submitting a request has a clear and single behavioral profile. This does
not necessarily need to be the case. That is, there exist several applications that
can be split in several distinct phases that each have different behavioral profiles
and resource requirements. A good example of such an application is a scientific
batch queue application in which a large amount of data is fetched in a first phase,
calculations occur in a second phase and results are reported back in a final phase.
In order to deal with these kinds of applications, it is interesting to introduce a

more elaborate bidding language that is able to cope with these kind of requirements.
Appendix B does so by introducing the concept of compound requests.
3.3.3 Price Discovery

Although the problems that were discussed in section 3.2 have been effectively ad-
dressed with the introduction of the CRA, the issue of price-discovery has not been
discussed yet. Recall from section 2.5.4 that price discovery is the process that de-
scribes the mechanisms that are used by buyers and sellers to determine or approx-
imate the price of a good in the marketplace. The use of price discovery techniques
allows consumers and providers to discover the price at which the market is currently
trading goods. This can be useful for consumers or providers that are new to the
market to get an idea of the price at which resources are being traded. Consumers
in particular can make use of price discovery techniques to make an upfront trade-off
between the price they will need to pay on the market and the resources and quality
of service they will receive for that price.
Additionally, price discovery techniques allow consumers and providers to deter-
mine bidding strategies by monitoring the price for a specific resource in the market
and choosing the time at which they buy or sell certain resources intelligently as to
maximize their personal profit.
In the CRA market, price discovery is in fact already built-in for consumers
as they have the possibility to request a quote from the market at any given time
without any further obligation. As such, both new as existing consumers can always
discover the price for any request and make decisions based on that.
The situation is a bit different for providers as they have no idea at which price
and rate certain resources are being traded except when they are the provider that
actually gets to trade the resources for a given request. The best providers can do
is get an estimation about which resources are in popular demand according to the
price requests they are receiving from the market.
In order to solve this lack of price discovery tools for providers, the CRA model is
extended with a billboard-like mechanism on which the market publishes anonymized
consumer accepted matches. This is done in real-time; after a match is found, it is
directly published on the billboard. Because of this real-time behavior, the billboard
effectively represents the current market situation. Whether the billboard contains
only the last match, or the last x matches is an implementation detail and does not
change its applicability. Figure 3.5 shows an updated design of the CRA that takes
the billboard into account.
Using the billboard, providers can analyze the anonymized matches and adjust
Figure 3.5: Price discovery in the CRA market using a billboard to publish
anonymized bids
their pricing strategy accordingly. For example, if a provider is able to learn from the
matches that the price of matches in a specific category of requests is consistently
lower than the price it is offering, it might be interesting for the provider to adjust
its strategy and lower its quote price in order to get more matches. Analogously, if a
provider learns from the billboard that it is the only provider that has matches for a
specific category of requests, it might be interesting for the provider to increase the
quote price for that category of requests, in order to increase its profit. This notion
that providers will adjust their strategy according to the prices of the matches they
observe on the billboard introduces a form of indirect competition among providers
which is desirable in terms of market dynamics. Section 3.3.5 briefly revisits this
idea in the context of allocative efficiency.
Decision Support Systems
Decision Support Systems (DSS) are computer-based information systems that sup-
ports business or organizational decision-making activities. In the context of markets
and resource allocation, such systems often make heavy use of available price discov-
ery techniques to determine optimal bidding strategies or trade-offs. As this involves
contacting the market (and through the market the providers) very frequently in case
of the CRA, the question whether the use of such systems put to much strain on
the market becomes very relevant. Although the CRA market mechanism itself
is computationally inexpensive, the amount of communication per request between
consumer, market and providers is considerable (see section 3.3.5). In order to relieve
some of the strain from the market, it can consequently be interesting to allow con-
sumers to have access to the billboard, so that they can do analysis on the matches
that are posted there. This would allow consumers to approximate the price of a
set of resources, based on e.g. the average or normalized price for these resources.
In other words, giving consumers a different way to do price discovery that puts no
additional burden on the market.
Note that doing this kind of analysis can become very complex as it involves
extracting price information from matches that rarely contain exactly the same
resources and usage values. Nonetheless, because of the potential benefits of doing
such analysis, its further study is an interesting topic for future research.
3.3.4 Practical Example

To demonstrate the exact workings of the CRA, and to clarify how it differentiates
from the previously discussed CDA model, it is interesting to consider an example
similar to that of section 3.1.6 and see how the CRA finds matches for the consumers.
Assume that 3 providers (Amazon AWS, GoGrid, Rackspace [66]) are registered
with the market prior to the arrival of any request and consider the arrival of fol-
lowing requests.
1. A consumer submits a simple request to the market for that contains a Memory
constraint.
Request1 = [1, {
[TimeUsage, {(value, 1), (unit, hour)}],
[Consumer, {(name, "John Smith"),
(location, "Paris, France, EU")}] },
{ Resource["Memory"], Equal["AllocationType", "value"],
EqualAll["TimeUsage"] }
]
After relaying this request to the providers, they will all respond with a quote
that contain the following prices11 .
Provider Price ($)

Amazon 0.085
GoGrid 0.38
RackSpace 0.12
In order to return these prices, the different providers first have to make sure
that they had an offer that actually matches all constraints that are specified
in the request. To do this, providers should model their own offers using the
11
In the following examples, prices are based on the actual prices of these providers on April 25th,
2011.
tag and constraint sets and match the requests against these offers by verifying
whether both the consumer’s and the provider’s constraints are satisfied. Note
that it is assumed here that providers understand the tags and constraints as
specified by the consumer. In reality this will only be the case if providers
and consumers share a set of common tags and constraints. The next chapter
(section 4.2 in particular) elaborates on how this sharing of tags and constraints
can be achieved.
Only when providers have determined whether they can match a request with a
certain offer, they should determine the actual price for that offer based on the
usage values specified in the consumer’s bid. Note that providers should specify
their quote in the same time unit as the time unit specified in the request. This
is enforced by the TimeUsage constraint in the consumer’s request. If no such
constraint is present, then the quotes returned by the provider are in fact
meaningless. As such, it is the consumer’s responsibility to make sure that
such a TimeUsage constraint is always present.
As was the case with the Time tag in the example of section 3.1.6, the mean-
ing of the TimeUsage tag should be interpreted differently according to the
AllocationType tag that is part of the same request as the TimeUsage tag.
That is, if the specified allocation type is on-demand or spot, the TimeUsage
tag should be interpreted as the size of a single time-frame, while in case of a
future allocation (or if no AllocationType tag is present), the TimeUsage tag
should be interpreted as the total time the resources are requested. In case the
allocation model is reserved, there will be a need for an additional tag (e.g.
ReservationTimeFrame) that specifies what the reservation timeframe is.
In case of Request1, the market will return Amazon’s quote to the consumer
as it has the lowest price. In reality, this quote corresponds to a small instance
with 1.7GB of RAM, 1 ECU and 160GB of hard disk space.
2. A consumer submits a more complex request to the market that contains some
hardware, software and bandwidth descriptions.
Request2 = [1, {
[OperatingSystem, {(name, Windows), (version, Server 2008)}],
[OutgoingBandwidthUsage, {(unit, GB), (value, 3)}],
{ Resource["Disk"], Resource["Memory"], EqualAll["TimeUsage"]
Equal["AllocationType", "value"], EqualAll["OperatingSystem"] }
]
Note that the specified OutgoingBandwidthUsage tag should be interpreted

as the usage within the timeframe defined by the TimeUsage tag. In other
words, in this request the consumer has specified that he will be needing 3GB
of outgoing bandwidth on an hourly basis. Note that if the consumer does
not specify such a TimeUsage tag, the OutgoingBandwidthUsage is effectively
meaningless.
After relaying this request to the providers, the market receives the following
prices back.
Provider Price ($)

Amazon 0.78
GoGrid 1.25
RackSpace 0.70
As Rackspace offers the cheapest solution, the market will return that quote
to the consumer. In reality, this quote corresponds to a Windows 2008 Server
instance with 2GB of RAM, 4 virtual CPU cores and 80GB of hard disk space.
3. A consumer submits a request for a one-time allocation of 2 servers for a fixed
time (in other words, a Future request).
Request3 = [2, {
[OutgoingBandwidthUsage, {(unit, GB), (value, 50)}],
[IncomingBandwidthUsage, {(unit, GB), (value, 10)}],
[AllocationType, {(value, Future)}],
[StartTime, {(value, 15/06/11 14:00:00 UTC-2)}]
EqualAll["AllocationType"],
Equal["StartTime", "value"], Equal["TimeUsage", "unit"]
GreaterThanEqual["TimeUsage", "value"] }
]
The market once again relays the request to the providers, of which only two
respond that they can match the request (GoGrid does not have an offer with
more than 8GB of RAM).
Provider Price ($)

Amazon 776.70
GoGrid -
RackSpace 969.80
As Amazon is the cheapest of the two, it will be returned as match to the

consumer.
3.3.5 Desirable Market Properties

Now that the Continuous Reverse Auction has been properly introduced, it is inter-
esting to review the desirable market properties that were discussed in section 2.4
and see how they apply to a CRA-based market. Note that the intent is not the
provide an in-depth evaluation of these properties (as doing so requires advanced
game-theoretic [67] and complexity analysis [57]), but rather to address the different
properties in a more intuitive, ad-hoc manner.
Allocative Efficiency
Remember that the term allocative efficiency indicates the measure in which a mar-
ket is able to find the most optimal matches between consumers and providers. As
consumers are always price takers in the CRA model (meaning that there is no such
thing as matching the consumer that is willing to pay most with the provider that
asks the most) is is very difficult to make definite statements about the allocative
efficiency in a CRA-based market.
From a consumer’s point of view, it is clear that a CRA-based market always

finds the best match (that is, the cheapest match); as the CRA was specifically
designed to do so. However, when looking at the market as a whole, things are more
complicated. As direct competition - based on the prices specified in bids - between
consumers as it existed in the double auction is replaced by indirect competition
in the continuous reverse auction - based on different maximum prices different
consumers are willing to accept for a given set of resources - the idea of one match
being better than another no longer exists for a provider. Consequently, making
statements about which match is the most optimal is impossible.
The fact that this is impossible, is a result of the “infinite capacity” model cloud
providers currently convey to their customers. In this model, computing capacity is
abundant and consequently direct competition for capacity among consumers does
not exist as there is plenty of capacity for all. In reality however, forces of demand
and supply do exist, and there are certainly times at which capacity is a lot scarcer
than at other times. In order to cope with this, while at the same time keeping
the illusion of infinity capacity, providers increasingly let their customers engage in
forms of indirect competition by increasing the price of their resources according to
demand. The Amazon EC2 spot market is a good example of this: it let’s consumers
specify a maximum price they are willing to pay for a certain set of resources, thereby
giving priority to consumers that are willing to pay higher prices.
By letting the consumer decide what the maximum price is at which he accepts
a certain match the CRA does in fact the same. As such, the CRA does have an
implicit mechanism that let’s providers differentiate among different consumers.
The use of this mechanism in itself does not make the market allocative effi-
cient on a short timeframe as no difference is made between consumers that accept
matches at the same point in time. However, this indirect competition between
consumers does give the market allocative efficient-like characteristics when viewed
over a longer period of time. That is, by the fact that in the CRA-based market
the price for a set of resources varies according to current demand, the market will
iteratively converge to a situation where scarcer resources are allotted to consumers
with higher valuations for those resources (as consumers with lower valuations will
no longer accept the proposed price for those resources). As such, given enough com-
petition among providers (which makes sure that the price for resources reflects their
demand), a CRA-based market will exhibit optimal-like matching characteristics.
Individual Rationality
In its essence the term Individual Rationality conveys the question whether taking
part in the market is the most rational choice for each of the markets participants. In
case of the CRA, this is certainly so for consumers as the CRA’s matching mechanism
guarantees that consumers get the lowest available price. Additionally, as consumers
can always decide to not accept a match, they are never forced to incur negative
utility.
The situation is a bit different from the provider’s point of view, as the CRA
introduces a form of increased competition among the providers by the use of the
billboard. In reality this will mean that large well-known providers (such as Amazon)
will have the least added value by joining a CRA-based market as doing so increases
competition among them and others. However, as these providers are large, they
typically enjoy better economies of scale than the smaller providers, meaning that
the cost of their resources should be lower than that of other providers, allowing
them to have larger profit margins for the resources they sell on the market.
The small providers will however also benefit from entering the market, as the
direct competition allows them to directly target a very large audience. This is
also true for providers that are specialized in particular IaaS solutions, as their
specialization usually allows them to offer their solutions at cheaper prices than
similar solutions from other non-specialized providers, meaning that they will be
able to ‘win’ most requests that fall under their specialization.
In any case, a provider does not gain negative utility by entering a CRA-based
market as he is never required to sell below the cost of production (or service). Note
that the fact that the provider might have to pay a fee to take part in the market
is ignored here. If a provider has to pay such a fee to take part in the market, he
might incur a negative utility (if this fee to take part in the market is larger than
the profit he makes from trading resources on the market). The same is obviously
true for consumers.
Incentive compatibility
As mentioned in section 2.4.3, a market is called incentive compatible if its match-
ing mechanism forces the participants to bid truthfully in order to maximize their
personal utility. When considering whether the CRA model is incentive compatible
or not, it is clear that little can be said regarding the consumers, as in the CRA
mechanism, consumers themselves do not submit prices. As the consumer is a price
taker in the CRA, he can never directly influence the price of his match. Also, as
the auction is continuous, the consumer cannot influence the price of his match by
taking the role of a provider and submitting shill bids to try and influence the price
of the other providers.
While there is not much to say about incentive compatibility for the consumers,
some remarks do hold for the providers. That is, as all providers have perfect infor-
mation about the price of the resources that are being traded due to the introduction
of the billboard and given that there is enough competition among providers, there
will be an incentive among providers to send quotes to the market that contain
prices that are close to the actual value the provider attaches to offered resources at
that moment. This comes from the fact that the only way a provider can increase
its chance to match certain resources is to decrease its price. As such, it seems
reasonable to suggest that in general providers will have a tendency to lower their
prices to the point at which they have reached their current marginal value. In such
a scenario, the actual price for a given set of resources on the market will exhibit
spot market behavior as it will reflect the current demand for those resources.
Budget Balanced
Recall that a market is called budget balanced if the party that organizes the market
does not have to add extra money to the market in order for a market clearing to
occur. When considering a CRA-based market, it is clear that it is budget balanced
as the consumer always pays to full costs that are asked by the provider when a match
is accepted. As such, the market never has to intervene to make sure that a trade is
financially balanced. However, the market does have the additional cost of having
to run the electronic trading platform. To cover these costs, any real-world market
will probably ask participants to pay a fee to take part in the market. Although
requesting a flat subscription based fee is surely possible, a more fair model is based
on the pay-what-you-use principles and charges participants based on the number
of interactions they have with the market. That is, consumers pay for each request
they make, while providers pay based on the number of price requests they receive
(and responses they send back). A more fine-grained model may charge participants
based on the number of web service or even HTTP (GET, POST) requests. Whether
the market is then weakly or strongly budget balanced is clearly dependent on the
exact details of the payment model and of no further importance here.
Computational Tractability
When considering the subject of computational tractability in the light of the Con-
tinuous Reverse Auction, it is intuitively clear that algorithmic complexity of the
matching mechanism itself is rather low as the only real task the market has to do
is determine which of the quotes for a specific request contains the lowest price. By
letting the providers determine whether they match a certain request as well as de-
termine the price for that specific request, the market has effectively distributed the
computational complexity that is inherent in the matching of the heterogeneous IaaS
resources. Furthermore, as no bids or asks need to be stored in queues as is the case
in the double auction, no state needs to be maintained between subsequent matches.
As such, parallelizing the market’s matching mechanism over different servers is pos-
sible, adding to attractiveness of the CRA mechanism from a computational point
of view.
Figure 3.6: Hierarchical market setup to reduce CRA communication bottleneck.

Different providers register with different leaf market components. Each leaf market
component sends out n price requests to its registered providers and only sends the
cheapest match to its parent market component. The top level market component
then picks the cheapest of the matches it received from its children as the actual
match for the consumer.
A downside to the CRA, is that the amount of communication between all in-
volved actors will be larger than in other market mechanisms (particularly, the dou-
ble auction) as the market sends out price requests to all providers (which send back
replies) for each consumer request. Concretely, if there are n registered providers
that can all fulfill a certain consumer request, 2n + 3 messages will be needed for a
match to occur. That is:
• 1 initial request from the consumer to the market
• n price requests from the market to the providers
• n responses from the providers to the market
• 1 match message from the market to the consumer
• 1 match acceptance message from the consumer to the market
When comparing this to the Continuous Double Auction in which no additional
messages with providers need to be exchanged for an incoming consumer bid, it is
clear that the flexibility the CRA offers comes at a considerable communication cost.
In order to keep this cost manageable, providers might use brokers that have very
good connectivity to the market. However, in the end, it is of course the network
connectivity of the market itself that becomes the bottleneck. In order to cope
with this, a hierarchy of market components can be setup that each determine the
cheapest match and forward that to the their parent component. An overview of
how this could work is depicted in Figure 3.6.
3.3.6 The Continuous Reverse Auction for other commodities

Although the Continuous Reverse Auction has been introduced to deal with spe-
cific issues concerning IaaS solutions, the general principles behind the CRA are
not bound to the IaaS or cloud computing ecosystem. By taking a step back from
the cloud computing context of this thesis, it becomes apparent that the proposed
CRA model tries to solve the more general problem of trading heterogeneous com-
modities in an environment with a seemingly infinite supply, many consumers and
providers, and many different price models. In fact, as tag and constraint sets are
not specifically tailored to the IaaS environment, but have the flexibility to describe
any requirement or commodity, the CRA based market as described in the previous
sections needs almost no adjustments to be used in a different context. An example
of a market in which the CRA could be used is given by the following use case.
Use Case 1: Commodity market for Rental Cars
In the rental car market, many different cars are rented by many consumers at any
given time. Because of the varied requirements of consumers, car rental companies
use a pay-what-you-use model to charge users according to their specific needs. How-
ever, the exact details of these price models vary wildly among different providers.
Some providers charge their customers per day, others per hour. Some charge extra
according to mileage, or if the car is returned at another location than it was picked
up. Extra charges may also apply for young drivers, (extra) insurance, returning the
car with a empty gas tank, the use of gps systems, etc. Some providers allow the
car to leave the country where the car was picked up, others charge extra or don’t
allow it. Virtually all providers charge more for larger or more luxurious cars, etc.
Without going in further detail, it is clear that the car rental market exhibits
the same heterogeneity (both in the used price models as in the description of the
commodity) as the IaaS market. As such, using the CRA to trade rental cars seems
to be a viable option.
CHAPTER 4
Market Implementation
The design of market mechanisms is mainly theoretical in nature. Nevertheless,

investigating how markets can be implemented in practice is valuable as this imple-
mentation is a complex technological undertaking. This stems from the fact that
a market is inherently a distributed system in which many different parties with
different interests interact with each other. As such, any electronic trading platform
that brings those parties together will be distributed in nature, and will have to deal
with computer protocols, quality of service constraints and security.
In order to give an idea of the involved complexity, this chapter focuses on the
previously introduced concepts of tag and constraint sets as well as the Continuous
Reverse Auction, and discusses how they can be implemented concretely. Further-
more, a prototype that was implemented as part of this thesis to show the real world
applicability of the CRA based market will be briefly discussed at the end of this
chapter.
4.1 Implementation of the Continuous Reverse Auction

As any market for cloud computing resources is inherently distributed, specifying
the language with which the involved parties interact is a key architectural deci-
sion. As most actors (consumers, providers, the market and different brokers) are
loosely coupled, using service-oriented architecture design principles [68] will keep
the overall design simple, elegant and modular. Also, as the platform on which cloud
computing is based (that is, the Internet) is an inherently heterogeneous ecosystem
in terms of operating systems, used programming languages and application soft-
ware, it is compulsory that the communication mechanism used by the different
market components is independent of these specific details. Furthermore, as one of
4.1. IMPLEMENTATION OF THE CONTINUOUS REVERSE AUCTION 102
the goals of this thesis is to design a market model for IaaS cloud resources that can
be used today, it is imperative that the chosen communication technology is mature
and in wide-spread use. Based on these requirements, the choice for a WSDL/SOAP
based web service architecture seems valid as web service technology was specifically
designed with these requirements in mind. As a discussion about the exact work-
ings and principles of this technology is outside the scope of this thesis, readers are
referred to [69] for more information about web services.
Based on webservice technology, it is possible to deploy an electronic CRA-based

market, by implementing the different market components as depicted in Figure 4.1.
Figure 4.1: Implementation of a CRA-based market using webservices
The core components of the market itself are the webservices that allow the
consumers to make requests and providers to register in order to respond to price
requests.
The ConsumerIaaSMarket service is a rather simple service that allows con-

sumers to make requests and accept or decline previously matched requests. Ad-
ditionally, the service allows the consumer to retrieve some information (such as a
name, description, certificates, etc.) about the market. Note that although interface
of the ConsumerIaaSMarket is rather simple, its implementation is more complex as

the service needs to contact all registered providers for the submitted request and
store the request’s match for a certain amount of time (the service is thus stateful),
awaiting the potential acceptance of the match. Once the match is accepted by the
consumer, the service can anonymize it, publish it to the billboard and remove it
from its internal data structures. If the match is declined by the consumer, the
service can also safely discard it. If the consumer does not answer within a specified
timeframe, the market should also be allowed to discard the match as to avoid need-
ing to permanently store matches that are never accepted or declined. Note that
the size of this timeframe effectively determines the lifetime of a match (that is, how
long a match given by a provider is valid). Clearly, it is technically also possible
that the lifetime of a match is specified by the provider on a per match basis; at
the cost of extra complexity in both the bidding language and the implementation
of the market.
The ProviderIaaSMarket service is the webservice that allows providers or their

representative (brokers) to register a PriceRequestService and an Execution-
Service. It should also allow providers to retrieve some extra information about
the market itself (as is the case with the ConsumerIaaSMarket service).
The PriceRequestService is a standardized webservice (standardized inter-

face) implemented by all providers that allows the market to request prices for
consumer requests. By using such a standardized interface, the actual implemen-
tation of the matching algorithm (as given in Algorithm 6 in section 3.3.2) for the
ConsumerIaaSMarket service is almost trivial. Listing 4.1 shows a (slightly simpli-
fied) implementation of the ConsumerIaaSMarket’s submitRequest() operation in
the Java programming language. Note that through the use of the JAX-WS library
[70], it is relatively straightforward to adjust the given code to use webservices1 .
In practice however, it is important that some additional remarks are taken into
account. That is, the implementation of Listing 4.1 should be modified to contact
the different providers in parallel as this is obviously far more efficient. Additionally,
it is important that the market enforces some quality of service rules when query-
ing providers. Concretely, in order to guarantee that matches are found within a
considerable amount of time (that is, within the order of a few seconds at most)
the market requires the PriceRequestServices to answer requests within a certain
amount of time (e.g. 500 ms). If a specific PriceRequestService does not answer
within this timeframe, the market should be allowed to continue the matching pro-
cess without waiting for a response, effectively ignoring this service for the request in
question. When a PriceRequestService fails to meet the QoS constraints multiple
times, the market may apply more severe penalties. For example, it can decide to
only contact violating services for certain (randomly selected) consumer requests in
1
Actually, the code in Listing 4.1 is very similar to the code that is actually used in the prototype
that accompanies this thesis; the only noteworthy additions are a few JAX-WS annotations on some
interfaces and classes.
1 p u b l i c Match submitRequest ( ConsumerRequest consumerRequest ) {

2 List < Quote > bestQuotes = new ArrayList < Quote >() ;
3 Iterator < Provider > itor = providers . iterator () ;
4 while ( itor . hasNext () ) {
5 Provider provider = itor . next () ;
6 P r ic e R eq u e st S e rv i c e service = Service . createService (
provider . g e t P r i c e R e q u e s t S e r v i c e D e s c r i p t i o n () ,
P r ic e R eq u e st S e rv i c e . c l a s s ) ;
7 PriceRequest request = cr ea teP ri ce Re qu es t ( consumerRequest )
8 Quote quote = service . getQuote ( request ) ;
9
10 i f ( quote != null ) {
11 i f ( bestQuotes . isEmpty () ) {
12 bestQuotes . add ( quote ) ;
13 } else {
14 Quote bestQuote = bestQuotes . get (0) ;
15 i f ( quote . getPrice () < bestQuote . getPrice () ) {
16 bestQuotes . clear () ;
18 } e l s e i f ( quote . getPrice () == bestQuote . getPrice () )
{
20 }
21 }
22 } // else null : s e r v i c e cannot handle r e q u e s t e d bid
23 }
24
25 i f (! bestQuotes . isEmpty () ) {
26 RandomSelector < Quote > randomSelector = new RandomSelector
< Quote >( bestQuotes . toArray (new Quote []) ) ;
27 Match match = new Match ( UUID . randomUUID () . toString () ,
quote . getPrice () , quote . getProviderInfo () , quote .
g e t E x e c u t i o n S e r v i c e D e s c r i p t i o n () ) ;
28 pendingMatches . put ( match . getIdentifier () , new MarketMatch
( consumerRequest , match , quote . getTicket () , new Date ()
));
29 return match ;
30 }
31 return null ;
32 }
Listing 4.1: Java implementation of the CRA matching mechanism of the
ConsumerIaaSMarket (Simplified for readability).
order to lower the workload on that service. In extreme cases, the market may ban
the service entirely.
The ExecutionService is a provider-specific service that allows consumers to

access their acquired resources once they have accepted a match. When looking
into this service in more detail, it becomes clear very quickly that the reoccurring
problem of standardization (or lack thereof), is again a serious issue when trying to
define this service in a uniform way. That is, as there is currently no standard way
in how different cloud providers let customers access their resources (although some
initiatives to solve this are on their way, most notably the OpenStack [12] initiative),
it is impossible to define a single standardized interface for the ExecutionService.
In this respect, the only service the market really can provide is passing a
provider- and match-specific identifier from the PriceRequestService to the con-
sumer once he accepts a match, so that the consumer can identify the accepted
match to the provider’s ExecutionService. Note that no further details on how a
consumer needs to do this are given here, as this is entirely provider specific. It is
possible that this process happens fully automatically through web service calls or
through brokers, or the consumer could manually provide the given identifier through
the provider’s website. The further issues in actually acquiring the resources (which
can involve authenticating, passing e.g. certificates or tokens, installing tools, etc.)
are not further discussed here.
On a side note, instead of passing an identifier, it is probably more interesting to

pass a more generic ticket-like datastructure that can contain more information than
only a match specific identifier, such as an expiration date or extra provider specific
initialization data. Furthermore, by passing an encrypted hash that describes the
matched resources instead of passing the identifier or ticket itself, providers can
completely separate their PriceRequestService and ExecutionService. That is,
by passing an encrypted hash between both services via the consumer, the need for
any direct communication between both services is eliminated. This is particularly
interesting when the PriceRequestService service is operated by a broker rather
than the provider itself. By using asymmetric (public key) encryption, providers
can be sure that hashes cannot be altered with. Figure 4.2 shows how this would
work.
Besides the ProviderIaaSMarket and ConsumerIaaSMarket services, the market

should also provide a secured MarketAdminstration webservice to allow easy in-
spection and management of the market by administrators.
The Billboard webservice should allow the market to publish anonymized matches.
Additionally, the operator of the Billboard webservice should also host a website
4.2. IMPLEMENTING TAGS AND CONSTRAINTS 106
Figure 4.2: Passing encrypted tickets between the PriceRequestService and the
ExecutionService via the consumer
on which the published matches can be found by the providers2 .
An overview of all the operations of the different webservices can be seen in Figure
4.3, which shows them in UML representation. Note that the PriceRequestService
and ExecutionService parameters of the register() method of the Provider-
IaaSMarket represent descriptions (which contain among others, the WSDL loca-
tion) of actual deployments of these standardized services.
4.2 Implementing Tags and Constraints

Now that is has been explained how the different components of a CRA-based market
can be implemented using webservices, some light will be shed on the implementation
of the tag and constraint sets that are part of a consumer’s request.
4.2.1 Implementing tags

The tags that were first introduced in section 3.1.4, can be implemented quite lit-
erally as objects containing a name and a set of attributes. Figure 4.4 shows class
2
This information can also be delivered in a more structured syndication format such as RSS or
Atom to allow providers to process this information automatically.
Figure 4.3: UML diagram of the webservices of a CRA-based IaaS market
diagrams of the implementation of the Tag and TagSet in the Java programming
language.
Figure 4.4: UML diagrams of a Tag and TagSet
As already mentioned before, the name of the tag can just be of the regular
string type of the language (in this case, the native java.lang.String type). The
tag attributes are easily implemented using associative array like structures, such as
Java’s java.util.Map. By using a HashMap, constant lookup times are guaranteed.
The implementation of the TagSet also uses the HashMap to have fast Tag lookup
within the TagSet.
4.2.2 Implementing constraints

The implementation of constraints is a bit more complicated as there exists a division
between custom constraints and standardized constraints. Recall that standardized
constraints are constraints of which the semantics are standardized and available
to all involved parties, while custom constraints allow customers and providers to
express participant specific matching semantics through the use of webservices.
Implementing standardized constraints
The implementation of standardized constraints is straightforward as it only involves
specifying the name of the constraint and optionally some parameters that initialize
the constraint (this can again be implemented with the use of associative arrays). Fig
4.5 shows an UML representation of how this can be done in the Java programming
language.
Figure 4.5: UML diagrams of a ConstraintDescription and Contraint-

Parameters that specify a standard constraint
Although the implementation of a constraint specification is thus quite simple,

there are some remarks about the distribution of the semantics of the constraints.
That is, in order to guarantee correct matching behavior, constraints should be
precisely defined so that there is a clear and unique understanding among all par-
ticipants of what the constraint exactly specifies. As it is a core characteristic of the
CRA that the market itself is unaware of constraint contents, an additional entity
will be needed that provides this exact definition. A separate (market independent)
lookup service that clearly specifies how constraint names map to actual matching
semantics can solve this problem fairly easy. In order to specify these constraint
semantics, the lookup service can make use of a set of (algebraic) rules, pseudocode
or a specific programming language.
It is clear that multiple such services can exist, possibly specializing in different
categories of constraints. A good way to avoid constraint name clashes in the case of
multiple lookup services is by using the Reverse DNS naming convention as regularly
applied when using the Java programming language.
Figure 4.6 shows how the consumers and providers would interact with such a
lookup service. Note that the lookup table in the figure contains a column with an
expiration date for each given constraint name. This expiration date indicates how
long the given constraint definition is valid; that is, the date indicates until when the
semantics for a certain constraint name are guaranteed to be unchanged. Using this
expiration date, both consumers and providers can effectively cache the semantics
of constraints, drastically reducing the amount of communication between them
and the lookup service. It is clear that many constraints, such as the Resource-
Constraint or EqualityConstraint, are unlikely to change at all, and can thus
have expiration dates that lay several months in the future.
Since lookup services define a unique binding between the name and the meaning
of a constraint, it is important that lookup services are officially recognized and
accredited by a mutually trusted third party, as constraints are a core part of the
contract between provider and consumer when a match is accepted. This extra
Figure 4.6: Using a constraint lookup service to determine constraint semantics
third party (which could be the market, the lookup service or another broker), can
then be trusted to objectively verify whether the contract is met by both parties or
not. This includes making sure the consumer has paid for the resources, as well as
verifying that provided resources actually meet the constraints that were specified
in the consumer’s request3 .
Note that the concept of the lookup service does not necessarily need to exist
as a webservice; as long as there is a way for providers and consumers to share the
semantics of particular constraints. In many cases it is thus sufficient that a provider
states the meaning of a certain constraint on his website. A good example of such a
constraint would be the ECUConstraint that can be used to model Elastic Compute
Units.
In order to lower the entry-barrier for providers (this is especially important in

a bootstrap phase), a standard set of the most common constraints can be provided
as a software library. This way, providers will not need to download code from the
lookup service or implement the constraints themselves based on the lookup service’s
definition. If providers internally model their offers using tag and constraint sets,
3
Note that doing so is actually non trivial as it involves thoroughly examining and monitoring
the allocated resources.
they can even use the software library to determine whether a given request matches
one of their offers or not.
Additionally, providing such a library will limit the proliferation of constraints
that have similar or the same semantics. The market should also embrace a default
lookup service that contains the definition of all the constraint operations in the
software library as a backup for customers and providers that do not have direct
access to the software library.
Using the reverse DNS naming convention that is used to identify constraints and
reflection it is easy to implement such a standard constraint library. Figure 4.7 shows
the class diagrams of the command pattern that could be used to implement such a
library in an elegant way. Specifically, the Constraint interface specifies a validate
method that is used at the provider side to actually validate the constraint. The
MyConstraint diagram represents a concrete constraint implementation (such as
the ResourceConstraint) that is provided as part of the library. When a consumer
specifies a constraint, he will specify the qualified name of the class that implements
the desired constraint as part of a ConstraintDescription that he attaches to his
request. This will make validating the constraint easy for the provider, as he then
only needs to create an instance of the specified constraint class using reflection, and
call the validate method on the resulting object, passing the consumers tags, the
tags used to model the provider’s offer and the constraint parameters as specified
by the consumer.
Listing 4.2 shows how a consumer would specify such a constraint as part of his
request specifically, while 4.3 shows the Java code that the provider can use to verify
whether the constraint is met.
Figure 4.7: Implementing a standard constraint library using reflection
1 ConsumerRequest request = new ConsumerRequest () ;

2 Tag tag = new Tag ( " Memory " ) ;
3 tag . setProperty ( " unit " , " GB " ) ;
4 tag . setProperty ( " value " , 1.7) ;
5 C o n s t r a i n t P a r a m e t e r s params = new C o n s t r a i n t P a r a m e t e r s () ;
6 params . addParameter ( " tag " ," Memory " ) ;
7 C o n s t r a i n t D e s c r i p t i o n conDescr = new C o n s t r a i n t D e s c r i p t i o n () ;
8 constr . se tConst raintN ame ( Res ou rc eC on st ra in t . c l a s s . getName () ) ;

9 conDescr . setParameters ( params ) ;
10 request . attachConstraint ( conDescr ) ;
11 // attach other constraints
12 market . submitRequest ( request ) ;
Listing 4.2: Consumer-side code specifying a Memory Tag and ResourceConstraint
as part of a request
1 // the variable constraintDescription is a constraint from

the consumer ’s request
2 Class <? > constraintClass = Class . forName (
c o n s t r a i n t D e s c r i p t i o n . getName () ) ;
3 Constraint constraint = ( Constraint ) constraintClass .
newInstance () ;
4 // the variable offer is the provider ’s offer
5 Set < ConstraintViolation > violations =
6 constraint . validate ( request . getTags () , offer . getTags () ,
c o n s t r a i n t D e s c r i p t i o n . g e t C o n s t r a i n t P a r a m e t e r s () ) ;
7 i f ( violations . isEmpty () ) {
8 // constraint met
9 } else {
10 // constraint not met
11 }
Listing 4.3: Provider-side code that verifies whether a constraint is fulfilled
Although using constraints with well specified semantics is the only real require-
ment for market participants, it is quite unlikely that many matches will occur if
the tags that consumers and providers use to model resources, requirements, services
and features are not standardized. That is, most (if not all) constraints require that
there are tags with specific names or attributes in the tag set that accompanies the
constraint. For example, when using the ResourceConstraint, a tag with a specific
name that has value and unit attributes is required in both the consumer and the
provider tag set in order for a match to occur.
To address this, lookup services should also define tags (names and associated
attributes) besides constraint semantics. The Reverse DNS convention can again be
used to avoid name clashes. The standard constraint library should also be extended
to include set of standard tags (and the default lookup service should consequently
also contain these tags). Figure 4.8 shows class diagrams of some of the tags that
are part of the standard library.
Implementing custom constraints
Besides the standardized constraints, consumers also have the possibility to specify
custom constraints that allow them to express specific matching behavior that is
not available as part of the standard library or in any of the lookup services. To
do this, the consumer should setup a webservice (or use a broker that does this for
him) that is able to check whether a given constraint is met given a tag set that was
Figure 4.8: Standardized tags in the standardized constraint library
Figure 4.9: Implementation of custom constraints using webservices

4.3. IMPLEMENTATION OF THE PROTOTYPE 113
specified by the consumer and a tag set that describes a provider’s offer. Figure 4.9
shows how such a system would work.
When the consumer specifies a customConstraint as part of his request, he

should also specify an URL at which a webservice can be found that is able to verify
whether the particular constraint is met. When the provider encounters such a con-
straint, he should then make a webservice call to the standardized CheckConstraint
interface at the specified URL, passing the name of the custom constraint, the con-
sumer specified tag set, a tag set representing the offer against which the provider
is validating the particular request and any additional ConstraintParameters that
were specified by the consumer. Given this information, the webservice should be
able to determine whether the constraint is met or not and return a boolean response
to the provider.
Note that from an efficiency standpoint, it is more interesting that the provider
first verifies all standardized constraints as no time consuming webservice calls are
needed to check these constraints. When the provider finds out that one of the
standard constraints is violated (and that a specific offer can thus not be matched
with the consumer’s request), there is no longer a need to check the other constraints.
If it turns out that certain custom constraints are often used by consumers, it is
obviously beneficiary to add an implementation of the constraint directly to a lookup
service so that it can be downloaded by providers and consumers. Note however,
that there may be situations in which the verification of the constraint involves
the usage of proprietary code or contacting additional web services or databases.
Submitting such code to a lookup service may not be desirable or even possible. In
these cases, the use of the webservice-based custom constraint mechanism based is
an appropriate alternative.
4.3 Implementation of the prototype

In order to prove the real-world applicability of the Continuous Revere Auction,
a prototype was implemented as part of this thesis. The prototype implements
the CRA-based market using the design from Figure 4.1, effectively implementing
the ConsumerIaaSMarket, ProviderIaaSMarket, MarketAdminstration and Bill-
board web services according to the interfaces as defined by Figure 4.3. The price
models of four different providers (Amazon, GoGrid, CloudSigma, Rackspace) were
implemented in order to show that the CRA can indeed cope with the heterogeneity
in resource specification and price models of different cloud providers. Web appli-
cations were build to interact with the various web services. Screenshots of some of
these web applications are shown in Figures 4.11 and 4.12.
To do the actual implementation, a variety of technologies - listed in Table 4.1 -

were used. A real-world deployment was made on Amazon’s EC2 platform, using 4
small instances, as shown in figure 4.10.
Implementing a full production-level market is of course a far more vast and

complex endeavor as it involves the implementation of many additional subsystems
such as user management and security, accounting, contract enforcement and others.
To implement such a system, a whole developer team, as well as far more elaborate
system design is needed. As such, these components were not implemented in the
prototype.
Additionally, some more advanced functionality such as compound requests,

splittable requests, custom constraints and some allocation types have also not been
implemented as part of the prototype as doing so is largely an implementation ef-
fort4 that does not directly contribute to the notion that the CRA mechanism can
indeed be implemented in practice.
The complete source code of the prototype can be found at http://jorisroovers.

com/files/IaaSMarketPrototype.zip.
Figure 4.10: Prototype deployment
4
Although implementing some of this functionality is actually non-trivial.
Technology Explanation
Java EE The Java programming language was chosen due the
author’s personal experience and affinity with the lan-
guage. Furthermore, as Java is in widespread use, de-
ployment on different platforms is easy.
JAX-WS The JAX-WS library allows developers to expose reg-
ular Java classes as web services by adding a few an-
notations. JAX-WS effectively hides all complexity
involved with interpreting WSDL, as well as marshal-
ing SOAP messages.
Spring Framework The Spring framework provides developers with ex-
tensions for building web applications on top of the
Java EE platform more easily. Specifically, the Model-
View-Controller model, dependency injection library
and some of web service libraries were used in the pro-
totype.
HTML, CSS, JSP, JSTL Various technologies that can be used for the imple-
mentation of the representational aspect of websites.
JSP is the server-side technology that can bring to-
gether front-end markup and backend business logic.
Javascript Javascript (AJAX) allows asynchronous communica-
tion between a website front-end and backend, allow-
ing the server to interaction asynchronously with the
user when it is performing time-consuming tasks. This
is used in the prototype to notify the consumer of the
progress of the matching process.
EAR The Enterprise ARchive allows the deployment of mul-
tiple web services (packaged in Web ARchives) as a
single application. The prototype uses EARs to de-
ploy the various grouped web services as single appli-
cations.
Glassfish application server Java EE Application server used to host the different
EARs.
Table 4.1: Technologies used to implement the prototype

Figure 4.11: Screenshot of the resource specification user interface as offered by a

broker to a consumer
Figure 4.12: Screenshot of the user interface offered to administrators to monitor

and manage the market
CHAPTER 5
Conclusion
This final chapter summarizes the previous chapters and puts forward the core con-
cepts and idea that were introduced in this thesis. To finish, it puts forward some
future work and formulates a final conclusion.
5.1 Summary
This thesis started with an introduction to the topic of cloud computing which
is materializing the long-lived dream of a general-purpose utility-like computing
platform that can be accessed from anywhere at anytime. To do so, cloud computing
introduces core concepts from existing utilities such as quick on-demand provisioning
and the pay-what-you-use model to allow users to focus on their core activities while
leaving the task to build and maintain IT solutions to expert companies. During
the study of cloud computing in the first chapter of this thesis, it was established
that the most hardware oriented layer of cloud computing, offered by the so-called
Infrastructure-as-a-Service solutions, is the one that most resembles existing utilities;
power utilities in particular. From this observation it became apparent that in order
to have a genuine global computing utility, there is a need for an open market in
which consumers and providers, as well as providers among each other, have the
possibility to trade computing resources on an immediate basis. Although attempts
have recently been made to introduce such a market (most notably [43]), current
solutions are still in its infancy and require providers to fit strict service and price
models as well as implement specific APIs or use particular frameworks. Given
the simplicity and imposed restrictions of these products, the question still remains
whether it is feasible to create a truly open and accessible IaaS market. As such, the
core question of this thesis has consequently been how such an open market, that is
able to cope with the heterogeneity of the current IaaS landscape, can be built.
5.1. SUMMARY 118
In order to tackle this question appropriately, the second chapter looked further
into the principles behind market design. After looking into the different kinds
of markets, existing auction mechanisms as well as theoretically desirable market
properties were discussed. Towards the end of the second chapter, focus turned
to the design of commodity markets in particular; identifying core components,
functionality and responsibilities. During this discussion, special attention was given
to how these components and functionalities are applicable when designing an open
IaaS market.
In the third chapter the two most important of these components, the market’s
bidding language and matching mechanism, were discussed in further detail. To do
this, the author started from a Continuous Double Auction-based market for grid
computing resources introduced by Nicolas Dube in [39]. In Dube’s model, so-called
resource sets are used by consumers and providers to describe the hardware and
software components they require or offer. Based on the observation that in the
IaaS cloud computing setting, a more generic constraint mechanism is needed as
in the commercial cloud context many other factors such as price and quality of
service play a key role, the concepts of tag and constraint sets were introduced. By
using tag and constraint sets, both consumers and providers have the possibility
to express complex constraints on the resources they are requesting or offering.
This is achieved by separating the specification of a matching constraint on an
offer from the properties that characterize that offer. Specifically, so-called tags
that contain attributes (key-value pairs) are used to describe a given offer, while
additional constraint operations attached to the offer specify matching restrictions
based on these tags.
Using this new tag and constraint mechanism, a market based on the Continuous
Double Auction was presented. Although this CDA based market is simple and
elegant, studying it in more detail revealed that because of the heterogeneity of the
price models used by IaaS providers today, the CDA mechanism cannot be used
without making considerable assumptions and forcing providers in a tight service
and price model. As one of the goals of this thesis was to avoid forcing consumers
or providers to adhere to a strict set of rules (based on the observation that if
an IaaS market is to be successful, it needs to accommodate current practices), a
different solutions was needed. In the search for a solution, a new kind of auction
mechanism - the Continuous Reverse Auction (CRA) - was introduced, which applies
principles from the Reverse Auction to commodity markets by modifying its behavior
to cope with both multiple consumers and providers. By letting providers determine
themselves whether they wish to match a certain consumer bid, the CRA takes away
this computational intensive task from the market and allows providers to use custom
price models. Matching bids is greatly simplified by the use of the tag and constraint
sets that were introduced before.
The issue of price discovery is solved by the CRA by the non-obligatory char-
acter of the mechanism from a consumer point of view and by the introduction of
5.2. CONCLUSION 119
a billboard to allow providers to monitor prices for specific resources more closely.
When studying the Continuous Reverse Auction in more detail, it became clear that
the mechanism can also be used in other commodity markets that deal with com-
plex heterogeneous goods and price models. Also, from a first examination, it seems
that the CRA (partially) holds some desirable market properties such as individ-
ual rationality, maximum allocative efficiency, budget balance and computational
tractability. However, more in-depth work is needed to verify this.
The one but last chapter of this thesis dealt with the more practical side of
building an IaaS market, by considering how the previously introduced concepts can
be molded into a software architecture. It was explained that a lookup service will
be needed to easily share the semantics of tags and constraints among providers and
consumers. Additionally, it was argued that a software library containing a set of
standard tags and constraints is desirable for efficiency and consistency reasons. To
implement the market itself, it was pointed out that several different components are
needed. Interfaces for these components were defined and discussed in detail. Extra
attention was given to the (simple) implementation of the CRA matching algorithm
and to the issues concerning the lack of standardization in APIs to access acquired
resources.
To proof the real-world applicability of the tag and constraint sets as well as
the CRA based market, a web-service based prototype was implemented in the Java
programming language with the help of the Spring Library. In this prototype, the
offers and pricing models of different providers were implemented to show that the
proposed solutions are indeed capable of dealing with the heterogeneity of current
IaaS solutions.
5.2 Conclusion
Based on the previous summary, the main conclusion of this work is that it is possible
to design a market for IaaS Cloud resources. However, to deal with the heterogeneity
of the current IaaS landscape, a complex bidding language and market mechanism
is needed. To this end, this thesis argues that the (Continuous) Double Auction
lacks the possibility to deal with this heterogeneity and proposed the Continuous
Reverse Auction combined with a bidding language based on the concept of tag and
constraint sets as a solution.
Future work may include an in-depth study of the CRA mechanism to validate
its ideas and possible significance. To this extent, the use of simulations can prob-
ably provide valuable insights. Furthermore, the study of IaaS markets that are
oriented towards specific use-cases such as high-performance or scientific computing
can be interesting as in these use-cases many assumptions can be made. Dube’s
thesis already proved that given a simple price model in which only a single price
component needs to be taken into account, it is possible to create a CDA-based
5.2. CONCLUSION 120
market for grid resources. Based on this, the author believes that using constraint
sets, use-case specific CDA-based IaaS markets can be build in which the use of only
use a single price component is justifiable.
Additional future work may include the impact of allowing consumer to submit
prices to the CRA mechanism as discussed in section 3.3.1 and the research of ad-
vanced price discovery techniques as discussed in the section about Decision Support
Systems in the context of the CRA (section 3.3.3).
In the end, this thesis is an attempt to contribute to the development of a global

platform in which providers and consumers come together to trade the abstract
notion of computing resources. Although the idea of a market in which IaaS resources
can be traded is a natural evolution in the vision of cloud computing itself, actually
building such a market turns out to be a complex task. The author hopes that this
thesis may contain some insights and ideas that can help to undertake this task, as
to contribute to the realization of a global utility-like computing model.
Appendices
APPENDIX A
Cloud Computing in Practice: SaaS and PaaS
This appendix contains some examples of real-world Software-as-a-Service (SaaS)

and Platform-as-a-Service (PaaS) products.
A.1 Software-as-a-Service
There are a plethora of SaaS applications on the Internet, ranging from personal
photo-sharing services such as SmugMug [71] to more enterprise oriented applica-
tions such as the HreForce [72] HRM application. In order to be brief, only 2
well-known SaaS suites are discussed here; a more complete listing of different SaaS
applications can be found online [73].
A.1.1 Google Apps

Google Apps [74] is a suite of several different online office applications. The most
prominent are listed here.
Gmail Complex online email application.
Google Calendar Online calendar application.
Google Docs Online word-processor, spreadsheet editor, powerpoint-editor and

file storage facilties.
Google Sites Online tool to quickly build and integrate simple webpages.
Google Reader Web-based aggregator, capable of reading Atom and RSS feeds.
Picasa Web Albums Online photo-sharing service.
Google Adwords Online advertisement tools.

A.2. PLATFORM-AS-A-SERVICE 123
Users can choose to either make use of the Google-branded versions of these
applications that have advertisements, or choose to use custom branded versions
that are hosted by Google but can be tightly integrated into a corporate website.
A free tier with a limited amount of accounts and support exist for the corporate-
version of Google Apps, but is mainly focused on very small businesses. Larger
corporations pay $50 per user per year, and receive more storage, better security
and SLAs (99.9 percent uptime) for that price.
As of 2010, Google also has an Apps Marketplace [75] on which it allows inde-
pendent developers and software vendors to offer free or subscription-based online
applications that integrate with the Google Apps application suite.
A.1.2 Salesforce
Salesforce [76] is well-known for its online Customer Relationship Management
(CRM) products that include marketing, sales, analytics and service & support
applications.
Sales Cloud The Sales Cloud application provides a wide set of tools that allows
sales representatives to track customer profiles and account history, allows
them to manage marketing campaigns and track other information unique to
the representative’s company sales process.
Service Cloud The Service Cloud application is a customer support application

that provides companies with a call center-like view that allows them to create
and track cases and automatically route and escalate important information.
Chatter Chatter is a real-time collaboration platform, dubbed by Salesforce as

“Facebook for the enterprise”, that allows employees to access a secure, private
social environment.
JigSaw Jigsaw is a community-driven contacts database that allows Salesforce cus-

tomers to quickly find full and correct contact information about each other.
Salesforce uses a subscription-based pricing model for all of its service that re-
quires customers to pay a certain fee per user per month. The exact prices depend
on the which of the different editions of the services is used; more expensive editions
offer more features and increased reliability and support. Details can be found online
[77].
A.2 Platform-as-a-Service
Although the number of SaaS products is seemingly endless, PaaS products are a
lot more scarce; only a handful large IT vendors offer mature PaaS solutions today.
Two of those, offered Google and Microsoft, are discussed here.
A.2.1 Google App Engine

Google App Engine (GAE) [78] is a platform for developing and hosting web appli-
cations on top of Google-managed data centers. In practice, developers can write
applications in the Java or Python programming languages which are subsequently
uploaded to Google servers. Once uploaded, developers do not need to worry about
the deployment of their applications; provisioning, scaling and loadbalancing is taken
care of by Google itself. Besides these services, Google also provides developers with
a set of software libraries and tools that allows them to simplify common tasks. The
most important of these tools are listed here.
Google Web Toolkit Java-based development toolkit for building and optimizing
complex browser-based applications.
App Engine Memcache Fast, transient distributed storage for caching the results
of datastore queries and calculations.
App Engine URL Fetch Allows access to resources over the web, and to com-
municate with other hosts using the HTTP and HTTPS protocols.
App Engine Images The Images service lets applications transform and manip-
ulate image data in several formats.
App Engine Mail Allows App Engine applications to send email messages on be-
half of the app’s administrators, and on behalf of users with Google accounts.
App Engine XMPP Allows App Engine applications to send and receive instant
messages to and from users of XMPP-compatible instant message services.
Google provides a rather substantial free tier for App Engine usage (exact quotas
depend on the used services, details can be found online [79]), but once more than the
free quota is used, Google charges the prices according to the amount of bandwidth
used, data stored and CPU time used by their servers. Details can be found on the
AppEngine billing webpage [80].
A.2.2 Microsoft Azure Services

Microsoft Azure Services Platform [81] is an online software platform that allows
developers to run and host applications in Microsoft’s datacenters. Although some
of the services that are delivered can arguably also be categorized as IaaS solutions,
Microsoft markets the Azure product as a PaaS because of its tight integration with
other Microsoft (cloud) products. More concretely, on the lowest level Microsoft
provides developers with a “cloud operating system” called Windows Azure that
serves as a runtime for applications that can be written in any of the programming
languages supported by the ṄET framework [82] or a few other programming lan-
guages such as Java, Ruby or PHP. More advanced functionality can be obtained
by making use of the AppFabric middleware that provides pre-built, higher level
services by means of a pre-built class library, simplifying the development of cloud
applications. These services include advanced access control services, caching ser-
vices, messaging services and integration with the Microsoft Bizztalk server [83] that
provides Enterprise Application Integration (EAI) features. Additionally, applica-
tions can integrate with the SQL Azure Database service which is a cloud computing
oriented online database service based on Microsoft SQL Server technology [84]. On
an even higher level, Microsoft allows developers to integrate their applications with
some of the SaaS products Microsoft offers as part of their Windows Live brand,
which includes online photo, email, contact and office applications.
For billing, Microsoft uses a model that is similar to that of IaaS providers by
charging developers by the hour depending on the instance type they choose to run
the Windows Azure operating system on (see section 1.2). Additional charges for
bandwidth, storage and the usage of the AppFabric and SQL Azure services also
apply. Details can be found online [85].
APPENDIX B
Compound Requests
Besides the normal requests, the market should also have a mechanism that allows
for a different kind of request; the so-called compound request. The compound
request is a consumer request that is composed of several normal requests and can
be used in cases where the usage of the requested resources can be partitioned into
several distinct phases or stages. For example, it is quite common (especially with
scientific workloads) that the time used by cloud resources can be partitioned into
fetching input data, subsequently processing that input data, and finally writing
back the output data. Similarly, there are applications that only periodically need
to transfer a lot of data, while transmitting very few to no data at other moments.
This occurs most typically in scenarios where periodical backups are taken.
In other words, there are multiple applications that exhibit this behavior of
having different distinct workload profiles at different points in time. Although such
profiles are typically characterized by being either bandwidth- or compute-focused,
there are certainly different scenarios possible in which e.g. an application might
periodically need a lot of memory or CPU cycles to perform heavy tasks, while most
of the time only needing a fraction of those resources.
It is clear that by only using the normal requests that were defined before, ex-
pressing such behavior is impossible. However, giving consumers a mechanism to do
so, can have distinct advantages for both themselves as for the cloud providers sup-
plying the resources. To better understand this, consider an example of a scientific
workload that exists of 3 distinct stages:
1. Transfer 30 GB of raw data to the cloud.
2. Process data in the cloud. No simultaneous data-transfer (except for occasional
heart-beat message), at least 10 computing hours necessary on requested re-
sources.
127
3. Transfer 1GB of result data back.
When trying to specify such a workload using a normal request, it would be

impossible to express the fact that the bulk of the datatransfer needs to happen at
the start of the task. Instead, one would have to specify that the 30GB of inbound
data would be transferred across the whole (10+ hours) of processing. Additionally,
all the requested compute resources would have to be present from in the beginning,
while the consumer would only be using them after all the input data has been
received (and until the result data is sent back).
The main disadvantage of this discrepancy between allocating resources and

actually using them is that the consumers will need to pay for the allocation of all the
resources, even if they are not currently using them. Although this is partially solved
IaaS solutions by the usage of a multi-price component model as discussed in section
3.2, when the difference between the allocated resources and the resources that are
being used is too large, there will almost always be a significant financial penalty
associated with that. Although at first it may also seem that another disadvantage
is the fact that it could be that certain prices such as e.g inbound data transfer
change during the time the workload is running1 , consequently changing the total
cost for the consumer based on when certain resources are used, this is not an
issue since a consumer and provider seal a contract about the price at the time of
matching. In other words, a provider is never allowed to change the price of any
price component for a given match. Allowing this would nullify the optimality of any
matching mechanism since in such cases, providers can potentially game the system
by initially offering low prices while raising them after a short period of time.
The allocation discrepancy can be solved by using compound requests, as in such

a case, providers are able to allocate resources that match the resources required by
the workload much more closely. In the example above, this could mean that in the
first and third stage, the consumer would not have to pay for the computing resources
that are used in the second stage, while no bandwidth costs will be incurred in the
second stage. Additionally, as the provider knows that the task in the second stage
will run for at least 10 hours, he might apply prices of reserved resources (instances)
for 10 hours, while using an on-demand price model for the following hours. Finally,
if the job that is running in second stage would transfer data, it could be priced
differently than in the first and third stage. The ratio behind this is that from a
provisioning point of view, it is reasonable to expect that (in the future) providers
might apply lower bandwidth rates when they know that a specific amount of data
needs to be transferred in a small timeframe, then when they know that that data
is to be transfered somewhere within a much larger timeframe. Put differently, it
is not to unlikely to expect that in time certain providers will use different price
1
Such scenarios are not very likely at the time of writing (providers typically use fixed prices per
GB), but it is not unthinkable that in the future, more spot like behavior will emerge in which the
prices for data-transfer will vary according to current bandwidth demand.
128
models based on the flexibility that is requested by the consumer2 .

Specifying Compound Requests
As far as the bidding language is concerned, a compound request is not that different
from a regular request.
Compound Request = [Volume, TagSet*, ConstraintSet]
The main difference with regular requests is however that the tagset may contain
tags that have attributes that itself are requests. By doing so, sub requests can
effectively be nested inside other requests, and constraints can be specified between
these sub requests as part of the ConstraintSet that is part of the top-level request.
Good examples of such constraints are for example a RequestDependency constraint
that specifies in which order the resources in the subrequests will have to be allocated
or a DataLocality constraint that specifies that the requests should fulfilled on a
single physical machine (or at least that the local data should be preserved between
switching between resources of sub requests).
To fully understand how this could be done, consider the the compound request
that specifies the previously mentioned three-phase application.
Compound Request = [1,

{[SubRequests,
{(SubRequest1,
[1, { [Memory, {(value, 1500), (unit, 1)}],
[TimeUsage, {(unit, hour), (value, 1)}],
[IncomingBandwidthUsage, {(unit, GB), (value, 30)}]
},
{ [Resource["Memory"], EqualAll["TimeUsage"] }
]
),
(SubRequest2,
[MinDuration, {(unit, hour), (value, 10)}]
},
{ [Resource["Memory"], EqualAll["TimeUsage"],
Resource["MinDuration"] }
]
),
(SubRequest3,
[OutgoingBandwidthUsage, {(unit, GB), (value, 1)}]
2
Actually, this already happens today. Many providers charge more expensive prices for the
allocation of on-demand resources compared to the allocation of future or reserved resources.
129
},
{ [Resource["Memory"], EqualAll["TimeUsage"]}
]
)
}
]},
{ RequestDependency["SubRequest2", "SubRequest1"],
RequestDependency["SubRequest3", "SubRequest2"],
DataLocality["SubRequest1", "SubRequest2"],
DataLocality["SubRequest2", "SubRequest3"]
}
]
Note that no allocation type has been specified in the different subrequests, which
means that the provider can decide itself which kind of resources to allocate (this can
even be Spot !)3 . The second request does contain a MinDuration request specifying
the minimum duration the consumer will be using the resources. As already noted,
the use of the RequestDependency and DataLocality ensure the desired behavior
of the resources once they are acquired.
Provider matching behavior should be slightly adjusted as to recognize sub re-

quests within requests and match them first before trying to match the entire re-
quest. However, besides this minor adjustment, no real changes are needed in the
market itself. As such, the ability to nest requests inside of other requests using
tags, as well as the possibility to define relations between these nested request using
constraint sets, shows once again that the introduction of the tag and constraint
sets add considerable flexibility to the market.
Additional remarks
This section contains some additional remarks concerning the use of compound
requests.
The effects of using compound requests cannot be achieved by using multiple

normal requests. That is, it might seem that by chunking a request in multiple
different request and matching each of these requests separately, the same result
can be achieved. This is not the case, as it could be that the market matches
the different normal requests with different providers as it could very well be that
different providers are cheaper than each other for the different individual request.
This is a real problem as only a single workload is considered; it is imperative that
this workload is matched with a single provider as data locality is almost always
important. The concept of the compound request clearly expresses the fact that
these different individual requests should be matched by the same provider and as a
whole. When a compound request is matched with a certain provider, it is thus that
3
Note that the consumer has not specified any additional bandwidth speed constraints, which
together with the lack of an allocation type specification may lead to the fact that the acquired
resources are deallocated after an hour without the data transfer being complete.
provider that offers the cheapest price for that whole compound request, although
it might be that that provider is more expensive than all other providers for all
individual sub requests.
Despite the fact that implementing compound request as part of the market is
not very difficult, there are some technical difficulties to support the different parts
(stages) of the compound request and the transition between them. Most impor-
tantly, it is not trivial for cloud providers to provide the possibility to dynamically
change the allocated resources (vertical scaling), or to only pay for transfered data
without needing to pay for other cloud resources (instances) that are storing the
transfered data. However, some providers are already making (significant) efforts
to allow for such increased flexibility. Indeed, certain cloud providers such as Ama-
zon and GoGrid are already offering cloud storage facilities (Amazon S3 [20] and
GoGrid Cloud Storage [86]) that allow customers to store data without needing to
run instances. Similarly, Amazon allows users to dynamically change instance types
by using Elastic Block Devices [87].
Of course, as was the case before, the market only acts as an intermediate be-
tween the consumer and provider. Actually giving the consumer access to the ac-
quired cloud resources, helping the consumer setting up his requested environment
and coordination the transition between the different stages is an different matter
entirely, and should be handled by separate broker services. Section 4.1 touches this
subject in a little more depth with the introduction of the ExecutionService.
Bibliography
[1] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. H. Katz, A. Konwinski,

G. Lee, D. A. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “Above the
clouds: A berkeley view of cloud computing,” Tech. Rep. UCB/EECS-2009-28,
EECS Department, University of California, Berkeley, February 2009.
[2] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud comput-

ing and emerging it platforms: Vision, hype, and reality for delivering comput-
ing as the 5th utility,” Future Gener. Comput. Syst., vol. 25, no. 6, pp. 599–616,
2009.
[3] “World internet users and population statistics.” http://www.

internetworldstats.com/stats.htm, 2011. Last accessed on 24/03/2011.
[4] R. S. Kaplan and W. Bruns, Accounting and Management: A Field Study Per-
spective. Harvard Business School Press, 1987.
[5] D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff, and

D. Zagorodnov, “The eucalyptus open-source cloud-computing system,” in in
Proceedings of Cloud Computing and Its Applications, (Chicago, Illinois), Oc-
tober 2008.
[6] “OpenNebula Virtual Infrastructure Manager.” http://www.opennebula.com.
[7] VMWare, “vCloud.” http://www.vmware.com/products/vcloud/overview.

html, 2009. Last accessed on 24/03/2011.
[8] Environmental Protection Agency, “Report to congress on server and data en-
ergy efficiency,” 2007.
[9] Datacenter Knowledge, “Microsoft: PUE of 1.22 for data center

containers.” http://www.datacenterknowledge.com/archives/2008/10/20/
microsoft-pue-of-122-for-data-center-containers/, 2011.
[10] Google, “Data center efficiency measurements.” http://www.google.com/
corporate/datacenter/efficiency-measurements.html, 2011.
[11] “Cloud standards overview.” http://cloud-standards.org, 2011.
[12] “OpenStack.” http://www.openstack.org, 2010. Last accessed on

18/03/2011.
[13] Amazon, “Online Bookshop.” http://amazon.com, 2011.
[14] Amazon, “Elastic Compute Cloud.” http://aws.amazon.com/ec2, 2008. Last

accessed on 24/03/2011.
[15] Amazon, “EC2 Instance Types.” http://aws.amazon.com/ec2/

instance-types/, 2011. Last accessed on 24/03/2011.
[16] CloudHarmony, “Cloudharmony, benchmarking in the cloud.” http://www.

cloudharmony.com/. Last accessed on 22.05.2011.
[17] Amazon Web Services LLC, “Amazon EC2 Spot Instances,” 2009. Last accessed
on 24/03/2011.
[18] “Cloudexchange.” http://cloudexchange.org/.
[19] Amazon Web Services LLC, “Amazon EC2 Pricing.” http://aws.amazon.com/

ec2/#pricing, 2010. Last accessed on 24/03/2011.
[20] Amazon, “Simple Storage Service.” http://aws.amazon.com/s3, 2008. Last

[21] Amazon, “Simple Queue Service.” http://aws.amazon.com/sqs/, 2008. Last

[22] Amazon, “SimpleDB.” http://aws.amazon.com/simpledb/, 2008. Last ac-

cessed on 24/03/2011.
[23] Apache, “Hadoop.” http://hadoop.apache.org/.
[24] “GoGrid.” http://gogrid.com, 2008. Last accessed on 24/03/2011.
[25] GoGrid, “Cloudhosting.” http://www.gogrid.com/cloud-hosting, 2008.

Last accessed on 24/03/2011.
[26] GoGrid, “GoGrid Exchange.” http://exchange.gogrid.com, 2010. Last ac-

cessed on 24/03/2011.
[27] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infras-
tructure. Morgan Kaufmann Publishers, Inc, first ed., 1998.
[28] I. Foster, Y. Zhao, I. Raicu, and S. Lu, “Cloud computing and grid computing
360-degree compared,” in Proceedings of the Grid Computing Environments
Workshop (GCE 08), 2008.
[29] N. Carr, The Big Switch: Rewiring the world, from Edison to Google. New
York, NY, USA: W.W. Norton, 2008.
[30] M. Chetty and R. Rajkumar Buyya, “Weaving computational grids: How anal-
ogous are they with electrical grids,” Computing in Science and Engineering,
vol. July/agust, pp. 61–71, 2002.
[31] D. S. Kirschen and G. Strbac, Fundamentals of Power System Economics. West

Sussex, England: John Wiley & Sons, Ltd, 2004.
[32] S. Hunt and G. Shuttleworth, Competition and Choice in Electricity. Chich-

ester, England: John Wiley & Sons, Ltd, 1996.
[33] L. Leong, “Adopting cloud infrastructure as a service in the ’real world’,” tech.
rep., Gartner, 2011.
[34] B. Schnizler, Resource Allocation in the Grid – A Market Engineering Approach.

PhD thesis, University of Karlsruhe, 2007.
[35] G. Stuer, K. Vanmechelen, and J. Broeckhove, “A commodity market algorithm

for pricing substitutable grid resources,” Future Generation Computer Systems,
vol. 23, no. 5, pp. 688–701, 2007.
[36] K. Vanmechelen, Economic Grid Resource Management using Spot and Futures
Markets. PhD thesis, University of Antwerp, 2009.
[37] K. Vanmechelen, W. Depoorter, and J. Broeckhove, “Market-based grid re-

source co-allocation and reservation for applications with hard deadlines,” Con-
currency and Computation: Practice and Experience, vol. 21, p. 22702297, 2009.
[38] J. Broberg, S. Venugopal, and R. Buyya, “Market-oriented grids and utility

computing: The state-of-the-art and future directions,” Journal of Grid Com-
puting, vol. 6, no. 2, pp. 255–276, 2008.
[39] N. Dubé, SuperComputing Futures: the Next Sharing Paradigm for HPC Re-
sources. PhD thesis, Université Laval, 2008.
[40] “GridEcon Project.” http://www.gridecon.eu/, 2006. Last accessed on

26/03/2011.
[41] J. Altmann, C. Courcoubetis, G. D. Stamoulis, M. Dramitinos, T. Rayna,

M. Risch, and C. Bannink, “GridEcon: A Market Place for Computing Re-
sources,” 2006.
[42] C. Courcoubetis, M. Dramitinos, T. Rayna, S. Soursos, and G. D. Stamoulis,

“Market mechanisms for trading grid resources,” 2006.
[43] Enomaly, “SpotCloud.” http://www.spotcloud.com, 2011.

[44] J. Kalagnanam and D. Parkes, “Auctions, bidding and exchange design,” in
Supply Chain Analysis in the eBusiness Area (D. Simchi-Levi, S. Wu, and
M. Shen, eds.), Int. Series in Operations Research and Management Science,
ch. 5, Kluwer Academic Publishers, 2003.
[45] P. Hardwick, B. Khan, and J. Langmead, An Introduction to Modern Eco-

nomics. Financial Times / Prentice Hal, 1999.
[46] A. Sullivan and S. M. Sheffrin, Economics: Principles in action. New Jersey:

Pearson Prentice Hall, 2003.
[47] R. Bade and M. Parkin, Foundations of Microeconomics. Addison Wesley, 2003.
[48] A. Mas-Colell, M. D. Whinston, and J. R. Green, Microeconomic Theory. Ox-

ford University Press, 1995.
[49] D. S. Kirschen and G. Strbac, Fundamentals of Power System Economics,

ch. Markets for electrical energy (The Need for a Managed Spot Market),
pp. 51–52. West Sussex, England: John Wiley & Sons, Ltd, 2004.
[50] European Energy Exchange. http://www.eex.com/. Last accessed on

18/03/2011.
[51] Wikipedia, the free encyclopedia, “Proof of incentive compatibilty of vick-

rey auction.” http://en.wikipedia.org/w/index.php?title=Vickrey_
auction&oldid=419008059#Proof_of_optimality_of_truthful_bidding.
Permalink (verified correctness), Last accessed on 27/03/2011.
[52] T. Sandholm, “Limitations of the Vickrey auction in computational multiagent

systems,” in Proceedings of the First International Conference on Multi–Agent
Systems (V. Lesser, ed.), (Cambridge, MA), pp. 299–306, MIT Press, 1996.
[53] P. Cramton, Y. Shoham, and R. Steinberg, Combinatorial Auctions, ch. The

Lovely but Lonely Vickrey Auction. MIT Press, 2006.
[54] M. Silvano and P. Toth, Knapsack Problems: Algorithms and Computer Imple-
mentations, ch. Knapsack Problem, pp. 13–15. John Wiley and Sons, 1990.
[55] P. Cramton, Y. Shoham, and R. Steinberg, Combinatorial Auctions. MIT Press,

2006.
[56] R. V. Sven de Vries, “Combinatorial auctions: A survey.” December 2000.
[57] R. B. Myerson and M. A. Satterthwaite, “Efficient mechanisms for bilateral

trading,” Journal of Economic Theory, vol. 29, no. 2, pp. 265–281, 1983.
[58] C. Vecchiola, S. Pandey, and R. Buyya, “High-performance cloud computing:

A view of scientific applications,” in Proceedings of the 10th International Sym-
posium on Pervasive Systems, Algorithms and Networks (I-SPAN 2009, IEEE
CS Press, USA), 2009.
[59] S. Parsons, M. Marcinkiewicz, J. Niu, and S. Phelps, “Everything you wanted
to know about double auctions but were afraid to (bid or) ask,” tech. rep., in
preparation.
[60] CloudSleuth, “Cloudsleuth, decoding the mysteries of the cloud.” https://

www.cloudsleuth.net. Last accessed on 22/05/2011.
[61] “High-Performance Linpack,” 2006. University of Tennessee.
[62] I. Foster, “Globus toolkit version 4: Software for service-oriented systems,”

in Proceedings of the IFIP International Conference on Network and Parallel
Computing (H. Jin, D. Reed, and W. Jiang, eds.), vol. 3779 of Lecture Notes in
Computer Science, (Heidelberg), pp. 2–13, Springer-Verlag, 2006.
[63] D. Durkee, “Why cloud computing will never be free,” Communications of

ACM, vol. 53, pp. 62–69, May 2010.
[64] “NewServers.” http://www.newservers.com/, 2009. Last Accessed on

23/04/11.
[65] “CloudSigma.” http://cloudsigma.com, 2009. Last Accessed on 23/04/11.
[66] “Rackspace.” http://rackspace.com, 1998. Last Accessed on 23/04/11.
[67] D. Fudenberg and J. Tirole, Game Theory. Cambridge: MIT Press, 1991.
[68] T. Erl, Service-Oriented Architecture - Concepts, Technology, Architecture.

Prentice Hall Service Oriented Computing Series, Upper Saddle River, NJ,
USA: Prentice Hall PTR, 6 ed., 2006.
[69] K. Gottschalk, S. Graham, H. Kreger, and J. Snell, “Introduction to web ser-

vices architecture,” IBM Systems Journal, vol. 41, no. 2, pp. 170 –177, 2002.
[70] “JAX-WS (JSR 224).” http://jax-ws.java.net. Last accessed on

22/05/2011.
[71] “SmugMug,” 2011. Last accessed on 24/03/2011.
[72] “HreForce.” http://www.hreforce.com, 2011. Last accessed on 24/03/2011.
[73] “SaaS Directory,” 2011. Last accessed on 24/03/2011.
[74] Google, “Apps.” http://www.google.com/apps, 2006. Last accessed on

24/03/2011.
[75] Google, “Apps MarketPlace.” https://www.google.com/enterprise/

marketplace, 2010. Last accessed on 24/03/2011.
[76] “Salesforce.” http://www.salesforce.com, 1999.
[77] “Salesforce pricing.” http://www.salesforce.com/eu/crm/

editions-pricing.jsp, 2011.
[78] Google, “Appengine.” http://code.google.com/appengine, 2008. Last ac-
cessed = 24/03/2011.
[79] Google, “AppEngine Quotas.” http://code.google.com/appengine/docs/

quotas.html, 2011. Last accessed on 24/03/2011.
[80] Google, “AppEngine Billing.” http://code.google.com/appengine/docs/

billing.html, 2011. Last accessed on 24/03/2011.
[81] Microsoft, “Azure Services.” http://www.microsoft.com/windowsazure,

2010. Last accessed on 24/03/2011.
[82] Microsoft, “.NET framework.” http://www.microsoft.com/net, 2002. Last

[83] Microsoft, “BizTalk Server.” http://www.microsoft.com/biztalk, 2000. Last

[84] Microsoft, “SQL Server.” http://www.microsoft.com/sqlserver, 1989. Last

[85] Microsoft, “Azure Services Pricing.” http://www.microsoft.com/

windowsazure/pricing, 2011. Last accessed on 24/03/2011.
[86] GoGrid, “CloudStorage.” http://www.gogrid.com/cloud-hosting/

cloud-storage.php, 2008. Last accessed on 24/03/2011.
[87] Amazon, “Elastic Block Storage.” http://aws.amazon.com/ebs/, 2008. Last


A Market Design For IaaS Cloud Resources

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

A Market Design For IaaS Cloud Resources

Uploaded by

Copyright:

Available Formats

A Market Design for

IaaS Cloud Resources

Proefschrift voorgelegd op 30 mei 2011 tot het behalen van

Research Group Computational

2 Market Design Principles 32

3 IaaS Cloud Computing Market: The Continuous Reverse Auction 60

4 Market Implementation 101

Appendix A Cloud Computing in Practice: SaaS and PaaS 122

Appendix B Compound Requests 126

1.1 Difficulty of accurately provisioning IT infrastructure . . . . . . . . . 4

3.1 Overview of the Continuous Double Auction for IaaS Resources . . . 73

4.1 Implementation of a CRA-based market using webservices . . . . . . 102

1.1 Examples of Amazon EC2 instance types . . . . . . . . . . . . . . . 13

2.1 Well-known stock exchanges . . . . . . . . . . . . . . . . . . . . . . . 34

4.1 Technologies used to implement the prototype . . . . . . . . . . . . . 115

When it comes to acknowledgements, I first and foremost wish to thank my

Finally, I wish to express very sincere gratitude and appreciation to my fellow

The so-called Infrastructure-as-a-Service (IaaS) providers materialize the cloud

As the online community continues to grow, it is only logical from a company’s

1.1 Cloud Computing

Although the service-oriented delivery through the Internet is a core characteris-

(a) Provisioning for peak load (b) Underprovisioning

1.1.2 Cloud computing, a layered model

Infrastructure-as-a-Service captures the idea that companies no longer have their

Platform-as-a-Service (PaaS) builds upon the IaaS idea by adding a software

The third form of cloud computing, Software-as-a-Service, is considered by many

Having discussed the different forms of cloud computing, it is interesting to

1.1.3 Deployment models

As virtualization technology provides interesting benefits - virtual machine iso-

As a final note on hybrid clouds, it is important to note that although using

1.1.4 Challenges and Benefits

General advantages of outsourcing Outsourcing activities allow companies to

Green IT As building and maintaining server infrastructure is a core part of the

Availability of a Service Organizations may worry about whether cloud comput-

Data Transfer Bottlenecks As applications continue to become more data-inten-

Performance Unpredictability Although sharing CPUs and main memory work

Bugs in Large-Scale Distributed Systems A difficult challenge for cloud com-

Software Licensing Current software licenses commonly restrict the computers on

Consumer Choice In the current cloud computing ecosystem, consumers do not

1.2 IaaS in Practice

1.2.1 Amazon Web Services

Instance type Hardware characteristics

Table 1.1: Examples of Amazon EC2 instance types

Besides on-demand and reserved instances, Amazon’s Elastic Compute Cloud

Amazon CloudWatch Allows customers to monitor instances (CPU usage, net-

Auto Scaling Allows customers to start or stop instances automatically, based on

Elastic IP Addresses Allows customers to allocate fixed IP-addresses for instances.

Amazon CloudFront A content delivery network (CDN) for distributing objects

of underlying hardware, operating systems and software libraries. In order to avoid

CPU (cores) RAM (GB) Storage Space (GB)

Remote Desktop (Windows). Management of the servers is done through either

1.3 IaaS Cloud Markets

1.3.1 IaaS Clouds and the power industry

Using Carr’s work, as well as the structured comparison between computational

1.3.2 Power industry markets

After 20 years of lobbying and making changes in regulations and processes, it is

Feature Electrical power grid IaaS Clouds

In practice this problem is solved by the introduction of a so-called managed

1.3.3 IaaS Cloud Market

explicit notion of the retail market.

1.4 Problem Statement

Of course, determining whether it is possible to design an open market can only

First of all, it is important to determine and define the different compo-