IaaS Building Guide v1

Infrastructure-as-a-Service BuiIder's Guide
Network Edition: The Case for Network VirtuaIization

v1.0.4 - O4 2010
n ussocuton vth
CClYRlCH1 2009-20ll, 1he Coudscung Croup, lnc
ALL RlCH1S RLSLRVLD RLLSL CNLY \l1H lLRMlSSlCN 2
TABLE OF CONTENTS
lntroduction 3
fxccutivc Summary 3
fvoIution of CIoud Computing 5
Coud Computng 1oduy 6
Coud Computng und the Netvork 7
Nctwork ScaIabiIity & Nctwork VirtuaIization 9
laycr 3 Routing Vs. laycr 2 Switching Ovcrvicw 11
Coud Netvorkng: 1he Luyer 3 Approuch 13
I! ln C|oud Oa|a Cen|ets 13
I! Itos and Cons 14
Coud Netvorkng: 1he Luyer 2 Approuch 18
I2 |n C|oud Oa|a Cen|ets 18
I2 Itos and Cons 19
Summurzng L2 Vs. L3 20
CIoud Nctworking 21
Cpenlov Netvorkng 22
Netvork Vrtuuzuton n Coud Dutu Centers 24
CIoudscaIing Rccommcndations 25
Netvork Vrtuuzuton Cptons 25
!IC 25
!|c|ta !e|votls 25
C|oud.com 25
Cpen!ebu|a and Iuca|yp|us 25
Summary 26
About CIoudScaIing 26
INTRODUCTION
Coud computng oers u truy revoutonury purudgm sht n the mode or hov l1
servces ure consumed und munuged. Much more thun u set o technooges or even
un outsourcng mode, coud computng provdes u nev vuy or l1 to do busness.
1hs vhtepuper ocuses on hepng coud provders bud lnrustructure-us-u-Servce
(luuS) couds. ln purtcuur, ths edton o the luuS uder's Cude expores hov to
dever coud scue netvorkng or l0,000+ physcu servers.
As u techncu gude ths puper mukes un uttempt to strke u buunce betveen u
purey ucudemc resource, u prugmutc reerence, und enough busness normuton
to uov u broud runge o reuders to derve vuue. 1hereore, the turget uudence
or ths document s ussumed to be netvork urchtects, coud engneers, C1Cs, und
techncu busness euders ookng to understund hov to bud netvorks nsde dutu
centers ut coud scue.
Some normuton here s ntentonuy smped to reuch u brouder uudence, but
there s more thun enough detu to pont experenced netvork engneers und urch-
tects n the rght drecton vhen tryng to understund hov to bud coud scue luuS
netvorks.
EXECUTIVE SUMMARY
Coud netvorkng s not u trvu tusk. 1he lnternet tse s u pece o nrustructure
desgned to run ut mussve scue. ln ts hstory und evouton, u urge number o chu-
enges hud to be overcome to run u urge, gobu netvork o mons upon mons
o connected devces. Modern dutu centers desgned to provde coud servce oer-
ngs, such us lnrustructure-us-u-Servce (luuS) uce smur chuenges to budng the
lnternet tse due to ther sze. At ts smpest, vhen provdng vrtuu muchnes on
demund, ke Amuzon's LC2, ve ure tukng ubout dutu centers thut need to provde
us much us l monund potentuy much more over tmenetvorked devces n
u snge ucty
l
.
udng u dutu center netvork thut cun munuge one mon netvorked devces
meuns usng technques very smur to those used n urge lSl buckbones und the ln-
ternet ut urge. ln uct, muny o the urgest coud provders, such us Amuzon, Cooge,
Yuhoo!, und lucebook, preer Luyer 3 (L3) netvorkng technques over typcu Luyer 2
(L2) netvorkng technques becuuse o ther proven scuubty. L2 netvorkng mod-
es, prevuent n typcu enterprse dutu centers, ure knovn to huve sgncunt scung
probems und most enterprse dutu centers ure n the l0,000-server runge, not the
l00,000- or even l,000,000-server runge.
1 | |he ||me o| |h|s vt|||ng mazon IC2s s|ze |s es||ma|ed a| ~666,666+ v|t|ua| setvets.
ln uddton to the need to netvork mussve numbers o physcu servers together, the
rse o hurdvure vrtuuzuton meuns thut the number o connected devces n u dutu
center, purtcuury those o luuS couds, s grovng exponentuy. lor exumpe, our
ounder Rundy us estmuted thut Amuzon's LC2 hud us muny us 40,000 physcu
servers und perhups 300,000 vrtuu servers n }une 2009 vth un uveruge o 8 vrtuu
muchnes per physcu server. Never couds ure seeng denstes o 40:l, vhch euds
us to concude thut u smur szed depoyment coud contun over l.6 mon vrtuu
servers n the neur uture
2
. Cny L3 netvorkng technques ure desgned or ths scue.
\he L3 netvorkng technques ure exceent rom the pont o vev o the coud
provder, they restrct hov coud customers cun use ther vrtuu netvorkng betveen
servers. lor exumpe, Amuzon's LC2 does not eusy uov the use o broudcust truc,
mutcust netvorkng, or uov customers to pck ther ovn ll uddress runge. Muny
use cuses, ncudng support o egucy uppcutons, requre ths unctonuty, vhch
s provded by uovng euch customer to huve ther ovn L2 netvork(s).
An deu urrungement, thereore, voud be to uov coud provders to bud und
operute ther netvorks usng L3 netvorkng technques, vhe uovng customers to
use L2 or ther ovn purposes. 1hs s nov possbe usng dutu center netvork vrtu-
uzuton. As the methodoogy und technoogy mutures, t v become the prevuent
mode or budng coud scue netvorks.
2 \e lnov o| a| |eas| one c|oud |ool|ng a| dens|||es even h|ghet, such as 06:1.
EVOLUTION OF CLOUD COMPUTING
Coud computng s u nev vuy o deverng l1 servces thut s ony u ev yeurs od,
but s seeng rupd udopton. 1hs nev methodoogy ncudes chunges to technoog-
cu, urchtecturu, operutonu, und servce modes. Coud computng cun be thought
o us Computng 3.0. lt v dspuce the current computng purudgm, cent-server
Lnterprse computng (Computng 2.0), vhch dspuced bg ron munrume comput-
ng (Computng l.0). 1hs dugrum depcts the uspects o the evouton to ths pont:
ln ths dugrum you cun see u ceur evouton rom munrume to cent-server und
nuy to coud. 1here s u very ceur chunge n vuues or both those budng l1
servces und those consumng them. lor the buder, concerns huve urgey moved
tovurds usng dstrbuted systems technques ut u mussve scue through uutomuton,
decentruzuton, more und cheuper hurdvure, und vuung sotvure soutons over
hurdvure soutons. Compunes ke Cooge, Amuzon, und Mcrosot typy ths trend
toduy.
lor the consumer, there s nov u dstnct derence n hov they experence l1 ser-
vces they consume ut home (e.g. CMu) und those they consume ut vork (e.g.
Lxchunge). At home, the experence s ov rcton, eusy to use, und requres no
nterventon or hep rom un l1 person, vhe ut vork, t s very much the opposte.
Smutuneousy, onne veb servces ure steudy gunng n terms o eutures und
unctonuty.
Another vuy to thnk ubout coud computng s to consder t us the l1 equvuent
to urge scue robotcs uctores or munuucturng curs. Snce the l970s, vhen n-
troduced nto the uutomotve ndustry, robotcs und uutomuton huve sgncunty
trunsormed thut ndustry. luy uutomuted uutomobe uctores provde sgncunt
ecency mprovements through ncreuses n quuty und producton output or curs.
ln eect, robotcs und uutomuton uoved the cur ndustry to scue' more eectvey.
Lnortunutey, one sde eect o ncreused ecency n ths cuse s thut t requres
retrunng und udustment o vorkorces. Coud computng v huve u smur m-
puct on the l1 ndustry us t provdes mussve uutomuton und ecency guns or l1
servces.
CLOUD COMPUTING TODAY
Coud computng evoved rom the vuy thut Cooge, Amuzon, Mcrosot, und Yu-
hoo! operute ther onne l1 servces. 1he mussve scue mped n deverng these
servces to huge uudences, necesstuted budng l1 n competey nev vuys. 1hese
nev technooges, technques, und operutonu chunges uov busnesses to ucheve
coud scue. lor exumpe, echte's ClC detus the derence betveen stung eves
or echte und Cooge n u recent ClC.com urtce:
!ex|, Bech|e| s|ud|ed hov Coog|es setvets opeta|ed. Coog|e had one sys|em
adm|n|s|ta|ot |ot evety 26,666 setvets, vh||e Bech|e| had one |ot evety 166
setvets. \ha| ve |eatned |s |ha| you have |o s|andatd|ze ||le ctazy and s|mp||
|y |he env|tonmen|, Ram|e|h says. Coog|e bas|ca||y bu||ds |he|t ovn setvets
by |he |housands ot ge|s |hem bu||| |n a s|m||at |ash|on, and |hey tun |he same
so||vate on ||. So, ve had |o ge| mote s|mp|||ed and s|andatd|zed.
Cny mussve uutomuton, stundurdzuton, und coud computng technques cun
provde ths knd o everuge, u derence o 200 tmes. Coud poneers ke Cooge
und Amuzon get smur everuge or u operutonu und cuptu expenses: hurdvure,
pover, ubor, spuce, coong, und deveopment. 1hs s the eve o ecency thut
types coud computng.
\e vtnessed u smur progresson vhen cent-server computng dspuced the
munrume mode. 1vo deveopments rom ths eurer sht ure notevorthy. lrst, u
ong-term l0- to 20-yeur sht produced u undumentu mpuct n the vuy thut l1 vus
everuged nsde o busnesses. Second, munrumes never ded, they smpy becume
souted nto u smuer nche o the l1 undscupe. Lven toduy, munrume computng
contnues to grov ut u steudy cp.
lt s sue to suy thut the evouton nto coud computng rom cent-server (und mun-
rume) v be much the sume vth most nev uppcutons und servces beng de-
poyed usng coud computng technques n the coud, vhe cent-server contn-
ues on u much sover grovth truectory. As vth munrume to cent-server, muny
uppcutons und servces v mgrute to usng coud computng over tme.
\th the proeruton o mobe computng devces, bud-outs o very urge scue
coud dutu center cupuctes, und the euse o entry nto muss murkets mude pos-
sbe vth putorms such us lucebook, ve ure enterng u nev eru. \e v see muny
more uses o l1 technoogy thun ever beore, und ths v urgey drve the udopton
o coud computng. ln eect, ve v see u Cumbrun exposon o uppcutons
und servces, much us cent-server creuted u nev opportuntes or busnesses. 1hs
meuns ve're ut the very begnnng o u ong term chunge, us shovn n the dugrum
beov.
CLOUD COMPUTING AND THE NETWORK
Lke ndustruzed robotc uutomobe uctores, bgger s better or coud dutu
centers. Lurger dutu centers cun ucheve economes o scue thut ure mpossbe to
ucheve othervse. Lverythng rom spuce to pover, coong, servers, storuge, und
netvorks cun be eectvey desgned to muxmze ecency. Servce provders vho
bud pubc utty coud servces ure drven to muxmze ecency becuuse every l1
dour spent or suved drecty reutes to prots eurned. 1he netvorks nsde these dutu
centers ure no excepton.
Coud netvorks, much ke the lnternet becuuse o sze, huve specu concerns or
tvo key groups: provders und consumers. lrovders o coud scue dutu centers must
be ube to ucheve ecency und scuubty n ther netvorks. An ncreuse o ust
one percentuge pont n ecency cun suve mons o dours. Scuubty drecty
mpucts u servce provder's ubty to grov eectvey n the uce o demund vhe
keepng operutonu costs ov.
Meunvhe, consumers nd themseves vth tvo knds o uppcutons they must
support on coud provder nrustructure: egucy (cent-server or munrume) und
coud. As expected, nev coud uppcutons cun be desgned or the specu envron-
ments o coud provders to muxmze ther ovn scuubty und eustcty. Smutu-
neousy, hovever, coud consumers huve huge numbers o egucy uppcutons thut
muke ussumptons ubout operutng n trudtonu netvorkng envronments ound n
cent-server-bused dutu centers toduy.
ecuuse toduy's couds muny support etherbut not bothtypes o uppcutons,
uny compete souton or coud netvorkng needs to uddress these tvo very sepurute
concerns:
l. Scuubty und ecency or coud uppcutons und coud provders
2. Support or egucy enterprse uppcutons
lor exumpe, LC2, the murket euder, vus urgey desgned or nev coud uppcu-
tons und optmzed or scuubty und ecency. y usng u Luyer 3 (L3) netvorkng
desgn, they ure ube to grov to u mussve sze. Amuzon's success n ths regurd s
ureudy ve knovn, vth un estmuted 500,000+ vrtuu servers runnng on 60,000+
physcu servers toduy.
Cther couds, such us Suvvs or Hostng.com, ocus on enterprse support or egucy
uppcutons. 1hese ure typcuy VMvure-bused vCoud provders vhose urchtec-
tures ure desgned or supportng egucy cent-server uppcutons. 1her netvorks
ure u Luyer 2 (L2) netvorkng desgn, vhch smpes consumer ssues vth egucy
uppcutons but ncreuses compexty, cost, und scuubty or the provder.
1he need or both soutons cun be nerred rom Amuzon's Vrtuu lrvute Coud
(VlC) servce, vhch provdes u smpstc L2 cupubty on top o ther exstng L3
netvork.
NETWORK SCALABILITY & NETWORK VIRTUALIZATION
lt s ve understood toduy thut Luyer 3 (L3) orented netvork desgns cun scue
to mussve sze. 1hs s the desgn o the lnternet. Luyer 2 (L2) netvork desgns ure
knovn to huve sgncunt scuubty ssues. Most enterprse dutu centers ure orented
on L2 desgn prncpes, vhe most lnternet Servce lrovders (lSls) und urge coud
dutu centers use the L3 mode. esdes Amuzon, lucebook s knovn to use the L3
mode nsde ts dutu center und t s vdey beeved thut Cooge und Mcrosot do
us ve.
A number o technques und technooges uttempt to sove some o the scuubty,
perormunce, und securty ssues n L2 netvork desgns. lor exumpe, VLANs ure u
mechunsm used to provde both scuubty und securty (souton). Lse o VLANs
uovs or souton betveen netvork segments vthout usng routng (L3). Never
technques, such us Rrdges und SLA11LL, uttempt to provde uddtonu mechu-
nsms or scung L2 orented netvorks. Lnortunutey, these technques contnue u
tme-honored trudton o beng boted-on to the exstng L2 methodooges.
An emergng technque thut oers ur more promse s thut o ne|votl v|t|ua||za||on.
Netvork vrtuuzuton s u combnuton o technoogy und methodoogy thut uovs
or the use o L3 netvork desgns or the servce provder, but L2 netvork desgns or
the consumer o the coud. 1hs upprouch s sometmes reerred to us L2 over L3
or L2oL3. 1he udvuntuge o ths s thut t uses ve understood netvork desgns or
both the provder und consumer.
Ne!vork Vir!ua|iza!ion xp|ained
Netvork vrtuuzuton comes nto puy vhen un ubstructon uyer s creuted be-
tveen the provder netvork und the consumer netvork(s), provdng u sepuruton
o concerns. 1he servce provder cun
operute the L3 netvork topoogy very
ecenty, vhe provdng u trud-
tonu enterprse netvork dutu cen-
ter vev or consumers. Addtonuy,
the L2oL3 mode uovs customers to
pck betveen the L2 und L3 modes.
Customers vth u nev uppcuton
thut cun be desgned n the coud
computng mode, by choosng to use
un L3 netvork desgn or thut upp-
cuton und eschevng ess scuube
L2 netvorkng, gunng the scuubty
nherent n the ormer desgn.
Netvork vrtuuzuton uso ocuses on provdng progrummubty o netvork con-
guruton, purtcuury the vrtuuzed L2 netvork uyer thut coud consumers v
see n u typcu lnrustructure-us-u-Servce (luuS) depoyment. 1hs progrummubty
s key to uovng or scuubty on the provder sde. Lurge couds ke Amuzon
Amuzon's Lustc Compute Coud (LC2), Ruckspuce Coud, und CoCrd, ure hghy
uutomuted systems. Luch nev customer depoyment requres thut netvorkng s uo-
cuted upproprutey. \e desgned L2oL3 soutons provde un All thut cun ntegrute
to provsonng und schedung systems.
ln the oovng sectons, ve v dscuss L3 und L2 netvork desgns us depoyed
n coud provder dutu centers toduy, ooved by u deeper dve nto some detu on
hov L2oL3 vorks.
lnuy, ve' muke recommendutons on emergng provders und technoogy stucks
thut cun uov you to everuge netvork vrtuuzuton.
aaS Ne!vorking Approaches
1here ure three muor upprouches to budng luuS netvorks toduy: L3, L2, und net-
vork vrtuuzuton. \e v rst provde un overvev, then ook ut euch o the three
technques, hov they ure used, ther pros und cons, und vhere coud netvorkng s
heuded n the uture.
LAYER 3 ROUTING VS. LAYER 2 SWITCHING OVERVIEW
l you ure not umur vth the 1Cl/ll stuck (uku mode), ve recommend thut you
umurze yourse vth the buscs.
1he 1Cl/ll und other netvorkng modes (e.g. CSl Reerence Mode) ure stucks,
vhere euch uyer o the stuck provdes derent unctonuty. 1he oovng dugrum
shovs both 1Cl/ll und CSl stucks sde-by-sde:
1he tvo uyers o the stuck ve cure ubout ure r2 und r3. ln most modern dutu
centers, Luyer 2 (the Dutu Lnk Luyer) s Lthernet. 1here ure u vurety o uternutves
to Lthernet, but most ure used n \de Areu Netvorks (\ANs), not Locu Areu Net-
vorks (LANs), vhere Lthernet s kng. Drecty ubove Luyer 2 s Luyer 3, the lnternet
lrotoco (ll) Luyer. Lvery computer on the lnternet uses un ll uddress, vhch uovs
you to communcute vth thut computer. \hen ve dscuss movng dutu ut Luyer 2
n Lthernet-bused envronments, ve tuk ubout svtchng, vhe n Luyer 3, ve tuk
ubout routng. As mped by the dugrum ubove, svtchng (L2) und routng (L3) ure
not entrey ndependent. Cne depends on the other.
ln ths document, vhen ve suy un L2 orented netvork desgn, ve meun one n
vhch the ocus o eort s on scuubty, perormunce, und securty or the svtchng
uyer. \hen ve suy un L3 orented netvork desgn, ve meun the ocus o eort s
on the routng uyer.
1he reuson to use un L2 upprouch nsteud o un L3 upprouch s mproved smpcty
und usubty. At the over end, un L2 netvork desgn s much, much smper thun un
L3 netvork desgn. Lsubty s uso much euser. \th Lthernet, euch devce hus u
unque Lthernet uddress. \hen movng betveen physcu ocutons n the sume dutu
center or cumpus, the Lthernet uddress tes vhere thut devce cun be ound. 1hs
meuns thut the ll uddress doesn't huve to be chunged vhen the ocuton s chunged.
\th ll, hovever, most devces must huve upduted ll uddresses vhen ther ocutons
ure chunged.
Cn the other hund, L2 netvork desgns smpy don't vork on urger scues, such us
the lnternet. 1he need or every svtch to understund the ocuton o every Lthernet
uddress on the netvork mukes ths mpossbe.
Somevhere n the mdde s u urge gruy ureu vhere L2 netvork desgns huve u num-
ber o bot-on technooges und protocos thut uttempt to uov them to scue up.
Athough there huve been vuryng degrees o success, L2 st does not scue us mght
be desred. lerhups more mportunty, these L2 scung technques do not sucent-
y ucknovedge undumentu customer
requrements: L2 netvorks ure usuuy
desgned uround u snge uppcuton or
customer und hence ndvduuy don't
need to be very urge. VLANs thut pro-
vde souton und protocos ke Spun-
nng 1ree lrotoco (S1l), Rupd Spunnng
1ree lrotoco (RS1l), und Mutpe Spun-
nng 1ree lrotoco (MS1l) u uttempt to
munuge very urge muttenunt netvorks,
n vhch euch ndvduu netvork s mun-
ugeube, but the uggregute s not.
ln ths regurd, netvork vrtuuzuton ts
the requrements snce t s desgned so thut euch tenunt receves ts ovn set o L2
netvork domuns, vhe the underyng physcu uyer s scued bused on L3not
L2netvork scuubty.
CLOUD NETWORKING: THE LAYER 3 APPROACH
L3 netvorkng s the upprouch used nsde urge coud provders toduy such us Amu-
zon, Cooge, und lucebook. As u proven upprouch to coud scue nrustructure, L3
cun operute ut urbtrury scue. ln other vords, vhen but propery, t shoud be pos-
sbe to bud u L3 netvork o uny sze. 1hs s hov the lnternet s desgned toduy.
lut smpy, L3 netvorkng uovs or uny scue netvork by hdng u o the detus o
the netvork dependng on the topoogcu ocuton. y uggregutng route normu-
ton, usng u herurchcu uddressng scheme, und ony storng vhut s necessury to
understund vhether dutu s ocu or remote, euch system n the L3 netvork cun muke
reutvey smpe decsons on u smu dutuset. ln contrust, L2 netvorkng requres
thut core systems knov u ot ubout every system n the netvork.
lor exumpe, ve uttempted to use L2 netvorkng to run the lnternet toduy, the
buck-bone routers o lSls voud huve to hod hundreds o mons o entres n ther
routng tubes und muke decsons ucross urge dutusets. Not ony voud hurdvure
cupube o ecenty hundng such dutu be prohbtvey expensve, but the ever
ncreusng sze o the lnternet tse voud eventuuy muke t mpossbe to keep up.
L3 netvorkng technques sove ths probem eegunty.
L3 ln C|oud Data Centers
1he oovng dugrum shovs hov L3 netvorkng vorks n pructce n coud dutu
centers toduy. Luch node or system n the nrustructure ucts us un L3 router und hus
ony enough normuton to muke u ocu routng decson, puntng the dutu up to the
next ter o router us necessury.
1he most obvous derence n un L3 routed netvork topoogy or u coud dutu cen-
ter s thut ts routng protocos ure run u the vuy dovn to euch ndvduu server
(coud node). Routng protocos ure generuy broken nto tvo knds: lnteror Cute-
vuy lrotocos (lCls) und Lxteror Cutevuy lrotocos (LCls). lCls ure used nsde o
u snge busness netvork. Most L3 soutons thut run netvorkng dovn to the coud
nodes probuby use Cpen Shortest luth lrst (CSll) us the lCl, but lntermedute
System to lntermedute System (lS-lS) s uso possbe
3
. LCls ure used excusvey or
lnternet buckbones und ure not reevunt here.
1he next thng you muy notce s thut euch vrtuu server s drecty connected ony
to ts deuut gutevuyn ths cuse, the physcu coud nodeund there ure no other
vrtuu servers on the sume L2 netvork us the vrtuu server. 1hs s vhy broudcust
truc s mpossbe, mutcust netvorkng s hurd, und vhy uppcutons thut muke
ussumptons ubout servers beng physcuy co-ocuted cun become conused.
L3 Pros and Cons
1here ure pros und cons to the L3 upprouch und, vhe there coud be u vgorous
debute ubout vhether ths upprouch s better or vorse, ve knov or certun thut L3
routng hus severu key udvuntuges:
ManagcabIc nctworking: Route uggreguton uovs L3 to scue. ecuuse euch router
needs ony u subset o u netvork normuton to muke u decson vhere to send
dutu, urge netvorks ure extremey munugeube.
ffcicnt nctwork utiIization and improvcd bandwith: Lquu Cost Mut-luthng
(LCMl) meuns thut dutu truveng rom one pont to unother cun use mutpe puths
smutuneousy, uovng or very ecent netvork utzuton und greuter umounts
o uggregute bundvdth.
lncrcascd utiIization and cfcicncy: Shortest-luth lrst (Sll) routng meuns thut n
urger netvorks dutu v tuke the shortest und quckest puth to ts destnuton, resut-
ng n ncreused utzuton und ecency.
1he prmury dsudvuntuge o L3 netvorkng s thut ocuty mutters. Lnke L2 Lther-
net, t's not possbe to smpy move u devce rom one L3 netvork to unother vthout
udustments. Movng u server requres chungng ts ll uddress to un uddress on the
nev netvork. Despte ths dsudvuntuge, L3 s ceury the technque o choce or
urge scue netvorkng, so et's tuke u ook ut euch o the udvuntuges n some detu.
Fou!e Aggrega!ion
lor L3 netvorkng to scue propery, route normuton s uggreguted ut euch uyer. ln
the prevous dugrum, or exumpe, the egress routers huve smpe routes to u smu
! ln |ac| mos| |o|ls use CSII. 1he no|ab|e excep||on |s Coog|e vho uses lSlS ||le many 1|et1 ln|etne|
Setv|ce Itov|dets (lSIs).
number o netvorks thut pont to the next uyer dovn (uggregute routers). 1he ug-
gregute routers then huve normuton ubout the netvorks n euch ruck thut pont to
euch top-o-ruck (1oR) svtch, but do not necessury knov vhere euch vrtuu server
s ocuted nsde the ruck. lnuy, the 1oR svtches knov vhch servers huve vhch
netvorks thut vrtuu servers resde on.
1hs tube shovs hov the netvorkng mght be uggreguted ut euch uyer usng un
exumpe netvork bock (l0.l.l.0/24
4
):
SWTCH LAYEP NETWOPK/MASK NUMBEP OF NETWOPK POUTES
Egress 10.1.0.0/16 1 65,536-P address heIwork (/16)
cohIaihihg 256 256 /24 heIworks
AggregaIe 10.1.1.0/24 1 256-P address heIwork (/24)
Top-oI-Pack 10.1.1.0/26 - 10.1.1.192/26 4 64-P address heIworks (/26)
Cloud Nodes 10.1.1.0/30 - 10.1.1.60/30 16 4-P address heIworks (/30)
VirIual Machihes 10.1.1.0/30 1 4-P local area heIwork wiIh 2 Ps
usable, ohe Ior Ihe VM ahd ohe Ior Ihe
gaIeway (i.e. cloud hode)
Route uggreguton meuns thut vhen und u host n the netvork needs to move ocu-
tons, ts ll uddress uso must chunge. lnsde u construned und controed envron-
ment such us lucebook, or exumpe, the uppcuton tse cun be tod ubout chunges
to host ll uddresses. Hovever, most luuS customers v be more umur vth un L2
purudgm, vhere they smpy move ther servers uround rom puce to puce vthout
uny thought to ll uddressng. Amuzon's LC2 s but predomnunty uround L3 net-
vorkng, vhch expuns vhy u nevy reuocuted vrtuu muchne thut s runnng on
the sume physcu muchne usuuy hus the sume ll uddress.
ln uddton to ssues vth ll uddressng, L3 netvorkng does not uov broudcust tru-
c. roudcust netvorkng s nherent to L2 netvorkng, but s not uvuube n un L3
envronment. Smury, mutcust truc cun uso be chuengng, uthough t s pos-
sbe to provde mutcust n un L3 netvork. ln muny modern uppcutons, broudcust
und mutcust truc ure used or dscovery. An exumpe o ths s Mcrosot's NetlCS
protoco. \ndovs servers ook or euch other on the sume netvork usng NetlCS.
\thout L2 connectvty betveen servers, u derent method hus to be used to uov
\ndovs servers to dscover euch other.
4 n exp|ana||on abou| hov C|ass|ess ln|etdoma|n Rou||ng (ClOR) no|a||on and ne|votl segmen|a
||on votls may be |oo |echn|ca| |ot some aud|ences and |s beyond |he scope o| |h|s documen|. I|ease
see |he \|l|ped|a at||c|e |ot a mote |ndep|h exp|ana||on.
CP
Lquu-cost mut-puth (LCMl) routng s u technque used to muxmze bundvdth
betveen tvo routers over u snge hop by usng mutpe equu-cost nks. lt s u
oud-buuncng technque, und comes vth nutve support n both the CSll und lS-
lS routng protocos. Ceneruy, ndvduu ovs ure restrcted to u snge puth snce
per-pucket decsons cun creute probems vhen muxmum trunsmsson unts (M1L)
or utences der ucross puths. Nevertheess, LCMl oers u strughtorvurd vuy to
ncreuse uggregute throughput betveen routers vhen u snge nk cun not uccom-
modute the truc voume.
Shor!es!-Pa!h Firs!
lor uny gruph vth nodes, und veghted nks betveen nodes, ndng the shortest
veghted puth betveen u snge source und uny destnuton node on the gruph s un
mportunt probem. lmugne u typcu roud or truve mup vth ctes, rouds, und ds-
tunces. A shortest-puth rst (Sll) ugorthm
5
v te you vhch rouds to choose vhen
drvng rom one cty to unother mnmzng the totu meuge drven. 1hs eegunt
ugorthm vus dscovered by Ldsger Dkstru n l959 und underes the domnunt
lCl nk stute protocos lS-lS (lntermedute System to lntermedute System) und CSll
(Cpen Shortest luth lrst).
Sll s ust und reube n seectng the shortest puth rom u source node to uny node
on the gruph. Routng protocos mpementng Sll ugorthms uso provde operutors
the ubty to udust the veght o euch nk or uctors such us utency, throughput,
reubty, und cost, thus gruntng strong contro over truc engneerng ucross the
netvork. ln pructce, ths provdes hghy tunube, ust convergng, und reube oop
ree routng. 1hs s n contrust to L2 svtchng, vhch n pructce uses Spunnng 1ree
lrotoco (S1l) to remove oops und the ubty to shut dovn mutpe puths und the
use o LCMl. ln order or S1l to be truy ust convergng ke Sll-bused protocos, u
sgncunt umount o tunng must huppen ut u svtches. Munuuy teng the svtch
ubrc vhut ts topoogy s nsteud o dscoverng dynumcuy (us n L3) s error prone
und brtte.
1he reutvey mnor dovn sde s thut these L3 nk stute protocos requre un uccu-
rute mup o the gruph to muke these decsons, und u routers n the sume ureu need
to vork rom dentcu mups n reu tme to uvod routng oops.
S |so ca||ed Dkstru's ugorthm.
Another chuenge urses rom the uct thut every router n the ureu must knov ubout
every other router und the nks thut connect them. 1hs creutes sze construnts,
vhch these protocos munuge by dvdng the nternu netvork nto ureus or nsttut-
ng herurches. 1he urgest snge CSll ureu ve ure uvure o contuned l000 rout-
ers, but ths s generuy consdered u poory desgned und needessy urge netvork
desgn. 1hese dovnsdes ure consdered ucts-o-e or nk stute protocos, und ure
ve munuged by u combnuton o the protoco mpementutons und reusonube
netvork desgn.
CLOUD NETWORKING: THE LAYER 2 APPROACH
Lnke L3 netvorkng, L2 svtchng s very eusy to understund und use ut u smu
scue. Smpy uttuch uny tvo servers to the sume L2 netvork segment und they cun
nd euch other nstunty, vthout nvovement rom L3 routers. lnsde most dutu cen-
ters, L2 netvorkng s the domnunt method or connectng devces. \he not ube
to operute ut coud scue, L2 does vork ve or smu und medum depoyments.
\hen operutng ess thun l,000 servers togethermost uppcutons und dutu cen-
ter needs ure ur ess thun l,000 serversL2 netvorkng s eusy to use, congure,
depoy, und munuge. As prevousy mentoned, muny modern uppcutons muke us-
sumptons ubout beng ube to use L2 protocos or dscovery und eusy netvorkng.
lt shoud be uppurent ut ths tme thut ut the uppcuton eve L2 s u desrube net-
vorkng technque. lrom u customer perspectve, t's eusest to smpy huve ots o L2
netvorks, one or euch uppcuton or unctonu ureu o u dutu center. 1he oovng
dugrum depcts hov L2 netvorks ure creuted nsde enterprse dutu centers toduy:
Lnortunutey, L2 netvorkng hus sgncunt compexty und scung ssues once you
reuch u certun sze.
L2 in C|oud Data Centers
ln contrust to L3, L2 netvorkng meuns thut euch host knovs hov to tuk to ocu
hosts on the sume netvork segment. A desgnuted router (uku de|au|| tou|et or de|au||
ga|evay) s usuuy ussgned or the host to send u dutu thut s not ocu.
lt s convenent or u dstrbuted uppcuton on mutpe servers to huve the other
servers thut muke up the uppcuton on the sume L2 netvork segment. 1hs mukes
dscovery euser, reduces the chunces o un mpuct rom u router uure, und mpes
thut securty devces such us revus ure not ntererng. ln uct, there s un mpct
contruct betveen netvork operutors und ther customers. \hen un L2 netvork bock
(e.g. l92.l68.0.0/24) s ussgned to u customer, t s ussumed thut the customer cun
do vhut they vunt vth the ussgned netvork. 1hs ncudes uddng nev muchnes or
chungng ll uddresses. 1he ony requrement n ths contruct s thut the svtch sends
u nonocu netvork truc to ther deuut router.
1he oovng dugrum depcts un L2 orented netvorkng topoogy:
1hs s u smped dugrum, but t shovs hov L2 svtchng hdes the topoogy. lor
euch L2 domun u o the devces uppeur to be ocu, meunng here the top-o-ruck
svtch (1oR) und the vrtuu svtch (vS\) ure nvsbe to the vrtuu muchnes. 1hs
smpes netvorkng. lt uso meuns thut u vrtuu muchne s moved rom one
coud node to u derent coud node n the sume ruck, t ust vorks. Cn the other
hund, u vrtuu muchne hud to move betveen these tvo L2 domuns, ts ll uddress
voud huve to chunge und t voud nov be routed (L3), requrng updutes to ensure
t cun contnue to speuk vth the other devces on ts od netvork.
You v uso notce thut no netvork cun be competey L2. 1hs s becuuse L2 net-
vorks ure used ony or ocu netvorks. L3 netvorkng s requred or reuchng re-
mote netvorks or the lnternet. ln other vords, L3 netvorkng cun never go uvuy.
L2 Pros and Cons
L2 netvorkng (prmury Lthernet) hus sgncunt udvuntuges over L3 (ll) n ts sm-
ped uddressng und euse o use.
1he shortcomngs or urge scue netvorks, hovever, ure gurng. ln purtcuur, L2's
ut uddressng, vhch mukes smped uddressng possbe uso mukes huvng urge
scue netvorks dcut. ll's herurchcu uddressng, n compurson, uovs smpy-
ng the probem through route uggreguton.
\he L2 hus u ut uddress scheme, ths uso enorces u very strct tree topoogy
vhere euch svtch must ook or oops. 1hs s the probem thut Spunnng 1ree lroto-
co (S1l) vus desgned to sove. Lnortunutey, S1l must shut dovn nks n order to
uvod oops, vhch meuns thut LCMl s not possbe. lt uso meuns thut n L2 depoy-
ments most truc tukes the ongest route possbe, movng rom the bottom o the
tree, to the top o the tree und buck dovn.
lor exumpe, n u urge L2 dutu center netvork usng ggubt Lthernet, t s not un-
usuu to see core nks betveen svtches ut the top o the tree thut ure l0 or l00
tmes the sze o edge nks (to hosts). 1hs mukes core nks expensve us they ure
usuuy ut ppes such us l0Cg, 40Cg, und l00Cg Lthernet becuuse u truc
must trunst these nks betveen core svtches. 1hs puts un undue burden on those
svtches, mukng them more expensve, und mukng L2 dcut to munuge und scue
or urge netvorks.
ecuuse L2 svtchng s nherenty hurd to scue, most uttempts to muke t more scu-
ube uppeur to be bot-ons or requre the uddton o L3 netvorkng technques. lor
exumpe, Rrdges (ormery 1RlLL) s un uttempt to provde better topoogy und L2
svtchng normuton betveen svtches usng lS-lS. Cther compunes' uttempts to
muke L2 netvorkng scue, such us SLA11LL und Csco's Luyer 2 Mutputh lrotoco
(L2Ml), shure the sume churucterstc o stretchng L2 netvorks to behuve ke L3
routed netvorks. An deu souton, o course, voud be L3's scung propertes vth
L2's euse o use.
Despte ts ssues, u number o urge coud provders toduy do uttempt to use L2 net-
vorkng to provde euch customer ther ovn L2 netvork (uku VLAN). Most o these
uttempts vork sucenty ve ut smu or medum sze, but begn to u upurt ruther
qucky once sgncunt sze s ucheved.
SUMMARIZING L2 VS. L3
1hs tube summurzes the tvo upprouches:
L3 L2
FasI cohvergehce Iimes FasI cohvergehce Iimes ohly wheh Iuhed
properly
Use all available bahdwidIh (via ECMP) Simple Io cohIgure
Proveh scalabiliIy Tree Iopology works aI small Io medium
scale ohly
The hIerheI is L3 A Iypical daIa cehIer is L2
CLOUD NETWORKING
Netvork vrtuuzuton uttempts to med the L2 und L3 upprouches. Nev upprouches
to coud netvorkng overcome the L2 scung ssues by ether creutng vrtuu L2
netvorks ucross L3, sometmes reerred to us L2 over L3
6
or by emnutng the L2
scung ssues utogether.
\th these upprouches, ve provde u uyer o ubstructon betveen the physcu und
vrtuu netvork topooges. Netvork customers muy huve us muny L2 netvorks us
they ke, vred hovever they ke, vhe the underyng physcu netvorkng s run
us coud scue L2 netvork or mutpe L2 netvorks connected ucross L3 netvorks.
1hs uovs the best o both vords. 1he trudeo s some uddtonu compexty und
concerns uround perormunce, most o vhch ure beng uddressed toduy.
1he dugrum beov shovs hov netvork vrtuuzuton vorks. A key churucterstc o
true netvork vrtuuzuton s the ubstructon uyer betveen the physcu und the
vrtuu. 1hs s ukn to the hypervsor n u vrtuuzed server. 1he hypervsor ucts us
un ubstructon, hdng the physcu hurdvure und provdng u vrtuuzed set o hurd-
vure or the guest operutng systems.
6 Iayet! |s ovet Iayet2 |n |he 1CI/lI s|acl.
1hs s st u bt ubstruct. Let's tuke u ook ut u more detued netvork topoogy thut
shovs hov L2 over L3 uctuuy vorks. 1he oovng dugrum shovs hov euch coud
tenunt's vSvtch cun provde u dedcuted CRL tunne to every other tenunt, thereby
creutng u uy vrtuuzed (und dedcuted) L2 domun, much ke u VLAN:
Seen here s thut nvsby to u o the customer vrtuu servers truc thut s not on the
ocu coud node (shovn us u vhte box) s tunneed over L3 usng the CRL protoco.
OPENFLOW NETWORKING
Cpenlov s u nev upprouch to provdng scue-out vrtuuzed L2 netvorks n the
dutucenter. ln un Cpenlov netvork the svtches do not use trudtonu L2 or L3
protocos to determne the nks truversed or routes ooved by truc. lnsteud, un
Cpenlov controer congures the svtches, expcty budng L2 orvurdng puths
by drecty popuutng the pucket mutchng und orvurdng tubes n the svtch hurd-
vure. 1he controer hus u knovedge o the physcu und vrtuu netvork topoogy
und orchestrutes the provsonng o u vrtuu netvorks ucross the compete set o
connected Cpenlov devces.
Cpenlov enubed svtches ure essentuy dumb congess devces vhch rey
on the externu controer to bud vrtuu netvorks. 1hs contro s perormed usng
the Cpenlov protoco, un open stundurd. 1he protoco denes u mechunsm or
denng ovs by munpuutng the orvurdng tubes o svtches und or perorm-
ng uctons on these ovs. luckets thut mutch tube entres ure orvurded drecty
n hurdvure, vhe the rst pucket o un unmutched ov s sent to the controer,
enubng nev ovs to be nstued dynumcuy.
1he oovng dugrum ustrutes u smpe exumpe vhere Cpenlov hus congured
u orvurdng puth rom u VM n one coud node to u VM n unother:
ln ths exumpe the VMs thut ure purt o the sume vrtuuzed Lthernet cun trunsmt
L2 rumes to euch other us on u smpe svtch. Cpenlov dynumcuy buds the
necessury orvurdng entres to puss the rumes rom VM to VM ucross the netvork.
An u-Cpenlov netvork buds on the scue-out udvuntuges thut the L3 upprouch
hus over L2. \th routed L3 netvorks the conguruton s typcuy stutc, vth rout-
ng und oud buuncng punned und congured n udvunce und stored n non-vo-
ute memory on euch L3 devce. 1he puth tuken by L3 truc v ony chunge n
response to nk or devce uure vhen the Sll ugorthm s trggered to recucuute
puths. ln un Cpenlov netvork the netvork eements need to huve very tte stutc
conguruton, euvng the controer to muke ntegent oud buuncng decsons ut
the tme ovs ure creuted. 1he Cpenlov cun pot optmu ov puths bused upon
the current stute o the chungng vrtuu netvork conguruton.
A stundurd ut tree topoogy cun be used or depoyment hovever u utter topoogy
or mesh v uso vork ve snce Cpenlov v muke use o u nks ut over ev-
es. lut topooges cun reduce the requrement or very ut expensve nks ut the top
o the tree.
NETWORK VIRTUALIZATION IN CLOUD DATA CENTERS
Netvork vrtuuzuton s beng converged on by muny. Mcrosot hus vrtten vhte-
pupers detung the L2oL3 technque rom the perspectve o u urge scue dutu cen-
ter operutor, vhe oks ke NLC und Ncru huve been vorkng on the progrummutc
und munugement uspects. lor more normuton, ve recommend reudng Mcrosot's
orgnu vhtepuper on Vuunt Loud uuncng, und ts updute on VL2'.
1here ure u number o vuys to mpement netvork vrtuuzuton, und uthough t's
reutvey eury duys, ve' certuny see more. Lxumpes ncude:
- Cenerc Routng Lncupsuuton (CRL)
- Vrtuu Dstrbuted Lthernet (VDL)
- Custom soutons (e.g. L2 tunneng over LDl)
- Cpenlov (stunduone)
- Cpenlov + CRL tunneng
A o these technques uov the servce provder to bud u hghy scuube netvork
vhe gvng euch customer ts ovn set o smpe to use und eusy congured L2 net-
vorks. ln uddton, becuuse customers ure competey souted rom one unother, t s
possbe to run overuppng ll uddress runges und provde udvunced eutures such us
quuty o servce (oS) und dstrbuted revus.
1hs s vhy ve purtcuury ke usng Cpenlov, ether stunduone or uong vth
CRL or LDl tunneng. CRL or LDl hundes tunneng und Cpenlov cun be used
us u progrummutc nteruce to svtches (physcu und vrtuu) und or udvunced eu-
tures ke oS.
OpenF|ov
A bre usde on Cpenlov. Athough Cpenlov s very nev technoogy, t s ureudy
uppeurng n soutons uvuube nov or n the neur uture. 1he deuut vSvtch n the
upcomng Ctrx XenServer 6.0 reeuse uses CpenVSvtch, un Cpenlov enubed
vrtuu svtch, NLC s nov shppng Cpenlov enubed svtches, und the Ncru
teum s vorkng on provdng Cpenlov enubed soutons.
esdes the udvunced netvorkng cupubtes mentoned ubove, Cpenlov cun be
ouded us rmvure onto physcu svtches, somethng the NLC teum hus deveoped,
und others ure deveopng. y runnng the sume protoco or conguruton ucross
both physcu und vrtuu svtches t s uso possbe to run mxed coud envronments
n vhch the tenunt hus both vrtuu und physcu servers. 1hs v be un mportunt
concept n the uture, us some ntensve vorkouds, such us dutubuses und e serv-
ers, muy not be vrtuuzed.
CLOUDSCALING RECOMMENDATIONS
\e ure u vendor ugnostc consutuncy, but ve voud be remss ve ddn't pont
out u number o soutons thut provde netvork vrtuuzuton thut ure ureudy n use
toduy. Some o these ure soutons ocused soey on netvork vrtuuzuton, vhe
others ure more ke u coud stucks thut ncorporute netvork vrtuuzuton n them.
NETWORK VIRTUALIZATION OPTIONS
NEC
NLC s u S35 on gobu euder n netvorkng, oerng u broud portoo o l1
und communcutons soutons. 1hey ure mukng sgncunt nvestments n ths
spuce, uctvey vorkng on netvork vrtuuzuton soutons thut combne ther phys-
cu svtches runnng Cpenlov vth nnovutve contro und munugement systems.
lurtnerng vth Stunord und other eudng orgunzutons ncudng commercu
enterprses - on eury Cpenlov mpementutons, NLC s urge, but movng ust.
\e consder NLC to be u good choce or enterprses ookng to be eury udopters n
netvork vrtuuzuton vhe mnmzng rsk.
Nicira Netvorks
Cne o the poneers und euders n ths spuce, the Ncru teum s u ust-movng sturtup
thut s munuged by the peope vho deveoped Cpenlov ut Stunord, us ve us teum
members vho vorked on vrtuu svtchng ut both VMvure und Csco. Ncru hus
been runnng eury pots vth urge couds thut use netvork vrtuuzuton, mukng
t un uttructve choce.
C|oud.com
Coud.com's
7
CoudStuck s one o the most productzed o the lnrustructure-us-u-
Servce (luuS) sotvure puckuges. CoudStuck uses netvork vrtuuzuton to provde
L2 domuns betveen cents. Coud.com chose u custom LDl tunneng souton
to optmze perormunce. y vrtng ther ovn souton, ntegrutng t tghty vth
the hypervsor, und removng certun unctonuty (e.g. 1Cl checksummng vhch
s usuuy hunded by NlCs), they but u hgher perormng netvork vrtuuzuton
souton. CoudStuck uso provdes u vuy to do L2 VLAN netvorkng usng physcu
hurdvure, uthough ve don't generuy recommend ths technque or urge provders
due to the nherent ssues uround scuubty und uck o exbty.
OpenNebu|a and Euca|yptus
oth o these oer strong open source proects und both support VDL. \e gener-
uy preer CpenNebuu over Lucuyptus or ts euse o use und moduur nuture, und
becuuse Lucuyptus pued uvuy rom ts open source roots, vhe CpenNebuu hus
7 Iotmet|y ca||ed VMCps
embruced them. l you vunted to ro your ovn usng CRL or Cpenlov+CRL, ve
recommend sturtng vth CpenNebuu.
SUMMARY
lrovders need scuubty und coud users need the euse und umurty o smpe
Lthernet netvorkng. Netvork vrtuuzuton provdes ths. lt s truy vrtuuzuton n
thut t provdes u ceun ubstructon uyer thut creutes u sepuruton o concerns be-
tveen the coud provder und coud user. 1here ure potentuy some dovnsdes n
terms o throughput ecences, but vth l0CL on the horzon, technooges such
us Cpenlov und the tremendous umount o pover nov uvuube n u snge lL
ruckmount server, smu necences ure more thun overrdden by muor guns n
scuubty.
\e beeve netvork vrtuuzuton s the vuy o the uture o urge coud dutu cen-
ters. lt smpes netvorkng or u und uvods compex bot-on technooges. est
o u, t puys nherenty to the strengths o commodty systems. lnsteud o buyng
ncreusngy expensve netvorkng geur, t s much ess expensve to scue-out usng
L3 netvorkng technques on cheup equpment und then use the ubstructon uyers
to hde t u beneuth.
Coudscung s uvuube or engugements to ussst you vth your coud dutu center
needs. leuse nqure urther you huve questons.
ABOUT CLOUDSCALING
Coudscung buds the vord's urgest lnrustructure-us-u-Servce couds or teecom
compunes, servce provders, und enterprses. \e oer u compete sute o servces
rom ntu strutegy und punnng, through mpementuton und support. 1he Coud-
scung teum s comprsed o coud veteruns rom ndustry eudng compunes such
us Amuzon, CoCrd, RghtScue, Cpscode, Cunoncu, Reductve Lubs, Lngne Yurd,
t1orrent, VMvure, euy und Mcrosot.
\unt to eurn more: Here's hov you cun reuch us:
\ebste coudscung.com
1vtter coudscung
Lmu nocoudscung.com
lhone (877) 636-8589
lnt' +l (4l5) 508-3270

IaaS Building Guide v1

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

IaaS Building Guide v1

Uploaded by

Copyright:

Available Formats

Infrastructure-as-a-Service BuiIder's Guide

Network Edition: The Case for Network VirtuaIization

You might also like