MCA-308

1311748

ABSTRACT

The RAIN project is research collaboration between Caltech and NA A-!"# on distrib$ted co%p$tin& and data stora&e s'ste%s (or ($t$re space-borne %issions) The &oal o( the project is to identi(' and de*elop +e' b$ildin& bloc+s (or reliable distrib$ted s'ste%s b$ilt with ine,pensi*e o((-the-shel( co%ponents) The RAIN plat(or% consists o( a hetero&eneo$s cl$ster o( co%p$tin& and-or stora&e nodes connected *ia %$ltiple inter(aces to networ+s con(i&$red in (a$lt-tolerant topolo&ies) The RAIN so(tware co%ponents r$n in conj$nction with operatin& s'ste% ser*ices and standard networ+ protocols) Thro$&h so(tware-i%ple%ented (a$lt tolerance. the s'ste% tolerates %$ltiple node. lin+. and switch (ail$res. with no sin&le point o( (ail$re) The RAIN technolo&' has been trans(erred to RAIN(init'. a start-$p co%pan' (oc$sin& on creatin& cl$stered sol$tions (or i%pro*in& the per(or%ance and a*ailabilit' o( Internet data centers)

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 1

MCA-308

1311748

1. INTRODUCTION
RAIN technolo&' ori&inated in a research project at the Cali(ornia Instit$te o( Technolo&' 2Caltech3. in collaboration with NA A4s !et "rop$lsion #aborator' and the 5e(ense Ad*anced Research "rojects A&enc' 25AR"A3) The na%e o( the ori&inal research project was RAIN. which stands (or Reliable Arra' o( Independent Nodes) The %ain p$rpose o( the RAIN project was to identi(' +e' so(tware b$ildin& bloc+s (or creatin& reliable distrib$ted applications $sin& o((-the-shel( hardware) The (oc$s o( the research was on hi&h-per(or%ance. (a$lt-tolerant and portable cl$sterin& technolo&' (or space-borne co%p$tin&) #ed b' Caltech pro(essor h$+i 0r$c+. the RAIN research tea% in 1668 (or%ed a co%pan' called Rain(init') Rain(init'. located in Mo$ntain 7iew. Cali(). is alread' shippin& its (irst co%%ercial so(tware pac+a&e deri*ed (ro% the RAIN technolo&'. and co%pan' o((icials plan to release se*eral other Internet-oriented applications) The RAIN project was started (o$r 'ears a&o at Caltech to create an alternati*e to the e,pensi*e. special-p$rpose co%p$ter s'ste%s $sed in space %issions) The Caltech Researchers wanted to p$t to&ether a hi&hl' reliable and a*ailable co%p$ter s'ste% b' distrib$tin& processin& across %an' low-cost co%%ercial hardware and software components. To tie these co%ponents to&ether. the researchers created RAIN so(tware. which has three co%ponents8 1) A co%ponent that stores data across distrib$ted processors and retrie*es it e*en i( so%e o( the processors (ail) 9) A co%%$nications co%ponent that creates a red$ndant networ+ between %$ltiple processors and s$pports a sin&le. $ni(or% wa' o( connectin& to an' o( the processors) 3) A co%p$tin& co%ponent that a$to%aticall' reco*ers and restarts applications i( a processor (ails)

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 9

MCA-308 Application M"I-"7M

1311748

RAI N TC"-I " @thernet M'rinet ATM er*ernet

Networ+ Connections :i&$re18 RAIN o(tware Architect$re Myrinet switches pro*ide the hi&h speed cl$ster %essa&e passin& networ+ (or passin& %essa&es between co%p$te nodes and (or I-;) The M'rinet switches ha*e a (ew co$nters that can be accessed (ro% an ethernet connection to the switch) These co$nters can be accessed to %onitor the health o( the connections. cables. etc) The (ollowin& in(or%ation re(ers to the 1<-port. the clos-<4 switches. and the M'rinet9000 switches) ServerNet is a switched (abric co%%$nications lin+ pri%aril' $sed in proprietar' co%p$ters %ade b' Tande% Co%p$ters. Co%pa=. and >") Its (eat$res incl$de &ood scalabilit'. clean (a$lt contain%ent. error detection and (ailo*er) The er*erNet architect$re speci(ication de(ines a connection between nodes. either processor or hi&h per(or%ance I-; nodes s$ch as stora&e de*ices) Tande% Co%p$ters de*eloped the ori&inal er*erNet architect$re and protocols (or $se in its own proprietar' co%p$ter s'ste%s startin& in 1669. and released the (irst er*erNet s'ste%s in 166?) @arl' atte%pts to license the technolo&' and inter(ace chips to other co%panies (ailed. d$e in part to a disconnect between the c$lt$re o( sellin& co%plete hardware - so(tware %iddleware co%p$ter s'ste%s and that needed (or sellin& and s$pportin& chips and licensin& technolo&') A (ollow-on de*elop%ent e((ort ported the 7irt$al Inter(ace Architect$re to er*erNet with "CI inter(ace boards connectin& personal co%p$ters) In(iniband directl' inherited %an' er*erNet (eat$res) A(ter 9? 'ears. s'ste%s still ship toda' based on the er*erNet architect$re) M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 3

MCA-308

1311748

2. ORIGIN
1) Rain Technolo&' de*eloped b' the Cali(ornia Instit$te o( technolo&'. in collaboration with NA AAs !et "rop$lsion laborator' and the 5AR"A) 9) The na%e o( the ori&inal research project was RAIN. which stands (or Reliable Arra' o( Independent Nodes) 3) The RAIN research tea% in 1668 (or%ed a co%pan' called Rain(init')

3. ARCHITECTURE
The RAIN technolo&' incorporates a n$%ber o( $ni=$e inno*ations as its core %od$les8 Reliable transport ens$res the reliable co%%$nication between the nodes in the cl$ster) This transport has a b$ilt-in ac+nowled&e%ent sche%e that ens$res reliable pac+et deli*er') It transparentl' $ses all a*ailable networ+ lin+s to reach the destination) Bhen it (ails to do so. it alerts the $pper la'er. there(ore ($nctionin& as a (ail$re detector) This %od$le is portable to di((erent co%p$ter plat(or%s. operatin& s'ste%s and networ+in& en*iron%ents) Consistent &lobal state sharin& protocol pro*ides consistent &ro$p %e%bership. opti%iCed in(or%ation distrib$tion and distrib$ted &ro$p-decision %a+in& (or a RAIN cl$ster) This %od$le is at the core o( a RAIN cl$ster) It enables e((icient &ro$p co%%$nication a%on& the co%p$tin& nodes. and ens$res that the' operate to&ether witho$t con(lict) Alwa's on I" %aintains pools o( Dalwa's-a*ailableD *irt$al I"s) This *irt$al I"s is nothin& b$t the lo&ical addresses that can %o*e (ro% one node to another (or load sharin& or (ail-o*er) 1s$all' a pool o( *irt$al I"s is created (or each s$bnet that the RAIN cl$ster is connected to) A pool can consist o( one or %ore *irt$al I"s) Alwa's on I" &$arantees that all *irt$al I" addresses representin& the cl$ster are a*ailable as lon& as at least one node in the cl$ster is operational) In other words. when a ph'sical node (ails in the cl$ster. its *irt$al I" will be ta+en o*er b' another health' node in the cl$ster) #ocal and &lobal (a$lt %onitors %onitor. on a contin$o$s or e*ent-dri*en basis. the critical reso$rces within and aro$nd the cl$ster8 networ+ connections. Rain(init' or other applications residin& on the nodes. re%ote nodes or applications) It is an inte&ral part o( the RAIN technolo&'. &$aranteein& the health' operation o( the cl$ster) M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 4

MCA-308

1311748

4. FEATURES OF RAIN
1. Comm ni!"tion. i3 ii3 iii3 0$ndled inter(ace) #in+ %onitorin&) :a$lt-tolerant interconnects topolo&ies) The "roble% A NaE*e Approach 5ia%eters Constr$ction dcF9 2. D"t" Stor"#e. 3. Gro $ Mem%er&'i$. To+en Mechanis%) A&&ressi*e :ail$re 5etection) Conser*ati*e :ail$re 5etection) 1ni=$eness o( To+ens) 611 Mechanis%s To+en Re&eneration 5'na%ic calabilit' #in+ :ail$res and Transient :ail$res

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 ?

MCA-308

1311748

1 ( Comm ni!"tion
As the networ+ is (re=$entl' a sin&le point o( (ail$re. RAIN pro*ides (a$lt tolerance in the networ+ thro$&h the (ollowin& %echanis%s8 i3 B n)*e) inter+"!e&, Nodes are per%itted to ha*e %$ltiple inter(ace cards) This not onl' adds (a$lt tolerance to the networ+. b$t also &i*es i%pro*ed bandwidth) ii3 -in. monitorin#, To correctl' $se %$ltiple paths between nodes in the presence o( (a$lts. we ha*e de*eloped a lin+ state %onitorin& protocol that pro*ides a consistent histor' o( the lin+ state at each endpoint) iii3 F" *t(to*er"nt inter!onne!t& to$o*o#ie&, Networ+ partitionin& is alwa's a proble% when a cl$ster o( co%p$ters %$st act as a whole) Be ha*e desi&ned networ+ topolo&ies that are resistant to partitionin& as networ+ ele%ents (ail) T'e /ro%*em8 Be loo+ at the (ollowin& proble%8 Gi*en n switches o( de&ree ds connected in a rin&. what is the best wa' to connect n co%p$te nodes o( de&ree dc to the switches to %ini%iCe the possibilit' o( partitionin& the co%p$te nodes when switch (ail$re occ$rH :i&$re3 ill$strates the proble%)

:i&$re 3 A N"0ve A$$ro"!'8 At a (irst &lance. :i&$re 4a %a' see% a sol$tion to o$r proble%) In this constr$ction we si%pl' connect the co%p$te nodes to the nearest switches in M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 <

MCA-308

1311748

re&$lar (ashion) I( we $se this approach. we are rel'in& entirel' on (a$lt tolerance in the switchin& networ+) A rin& is 1-(a$lt-tolerant (or connecti*it'. so we can lose one switch witho$t $pset) A second switch (ail$re can partition the switches and th$s the co%p$te nodes. as in (i&$re 4b) this pro%pts the st$d' o( whether we can $se the %$ltiple connections o( the co%p$te nodes to %a+e the co%p$te nodes %ore resistant to partitionin&) In other word. we want a constr$ction where the connecti*it' o( the nodes is %aintained e*en a(ter the switch networ+ has beco%e partitioned)

Di"meter& Con&tr !tion )!122

2 ( D"t" Stor"#e
:a$lt tolerance in data stora&e o*er %$ltiple dis+s is achie*ed thro$&h red$ndant stora&e sche%es) No*el error-correctin& codes ha*e been de*eloped (or this p$rpose) These are arra' codes that encode and decode $sin& si%ple I;R operations) Traditional RAI5 codes &enerall' allow %irrorin& or parit' as options) Arra' codes e,hibit opti%alit' in the stora&e re=$ire%ents M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 7

MCA-308

1311748

as well as in the n$%ber o( $pdate operations needed) Altho$&h so%e o( the ori&inal %oti*ations (or these codes co%e (ro% traditional RAI5 s'ste%s. these sche%es appl' e=$all' well to partitionin& data o*er dis+s on distinct nodes or e*en partitionin& data o*er re%ote &eo&raphic locations)

3 ( Gro $ Mem%er&'i$
Toleratin& (a$lts in an as'nchrono$s distrib$ted s'ste% is a challen&in& tas+) Reliable &ro$p Me%bership ser*ice ens$res that processes in a &ro$p %aintain a consistent *iew o( the &lobal %e%bership) In order (or a distrib$ted application to wor+ correctl' in the presence o( (a$lts. a certain le*el o( proble%s in an as'nchrono$s distrib$ted s'ste% s$ch as consens$s. &ro$p %e%bership. co%%it and ato%ic broadcast that ha*e been e,tensi*el' st$died b' researchers) In the RAIN s'ste%. the &ro$p %e%bership protocol is the critical b$ildin& bloc+) It is a di((ic$lt tas+ especiall' when chan&e in %e%bership occ$rs. either d$e to (ail$res or *ol$ntar' joins and withdrawals) In (act $nder the classical as'nchrono$s en*iron%ent. the &ro$p %e%bership proble% has been pro*en i%possible to sol*e in the presence o( an' (ail$res) The $nderl'in& reason (or the i%possibilit' is that accordin& to the classical de(inition o( as'nchrono$s en*iron%ent. processes in the s'ste% share no co%%on cloc+ and there is no bo$nd on the %essa&e dela') 1nder this de(inition it is i%possible to i%ple%ent a reliable (a$lt detector. (or no (a$lt detector can distin&$ish between a crashed %ode and a *er' slow %ode) ince the establish%ent o( this theoretic res$lt researchers ha*e been stri*in& to circ$%*ent this i%possibilit') Theorists ha*e %odi(ied the speci(ication while practitioners ha*e b$ilt a n$%ber o( real s'ste%s that achie*e a le*el o( reliabilit' in their partic$lar en*iron%ent) To.en Me!'"ni&m

The nodes in the %e%bership are ordered in a lo&ical rin&) A to+en is a %essa&e that is bein& passed at a re&$lar inter*al (ro% one node to ne,t node in the rin&) The reliable pac+et co%%$nication la'er is $sed (or the trans%ission o( the to+en. and &$arantees that the to+en will e*ent$all' reach the destination) The to+en carries the a$thoritati*e +nowled&e o( the %e%bership when a node recei*es a to+enJ it $pdates its local %e%bership in(or%ation accordin& to the to+en) The to+en is also $sed (or (ail$re detection) There are two *ariants (or (ail$re detection protocol in this to+en %echanis%) The a&&ressi*e detection protocol achie*es M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 8

MCA-308

1311748

(ast detection ti%e b$t is %ore prone to incorrect decisions *iC. it %a' te%poraril' e,cl$de a node onl' in the presence o( lin+ (ail$res) The conser*ati*e detection protocol e,cl$des a node onl' when its co%%$nication has (ailed (ro% all nodes in the connected co%ponent) The conser*ati*e (ail$re detection protocol has slower detection ti%e than the other detection protocol)

2a3

2b3

2c3

:i&8 2a3 To+en %o*e%ent with no lin+ (ail$re) 2b3 To+en %o*e%ent with one lin+ (ail$re and a&&ressi*e (ail$re detection) 2c3 To+en %o*e%ent with one lin+ (ail$re and conser*ati*e (ail$re detection) A##re&&ive F"i* re Dete!tion

Bhen the a&&ressi*e (ail$re detection protocol is $sed. a(ter a node (ails to send a to+en to the ne,t node. the (or%er node i%%ediatel' decides that the latter node has (ailed or disconnected. and re%o*es in(or%ation and passes the to+en to the ne,t li*e node in the rin&) This protocol does not &$arantee that all nodes in the connected co%ponent are incl$ded in the %e%bership at all ti%es) I( a node looses a connection to part o( the s'ste% beca$se o( lin+ (ail$re. it co$ld be e,cl$ded (ro% the %e%bership) The e,cl$ded node will a$to%aticall' rejoin the s'ste%. howe*er. *ia the 611 %echanis%. which will describe in the ne,t section) :or e&). (or the sit$ation in (i&$re2b3. the lin+ between A and 0 is bro+en) A(ter node A (ails to send the to+en to node 0. the a&&ressi*e (ail$re detection protocol e,cl$des node 0 (ro% the %e%bership) The rin& chan&es (ro% A0C5 to AC5 $ntil node 0 rejoins the %e%bership when the 611 %echanis% is acti*ated)

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 6

MCA-308

1311748

Con&erv"tive F"i* re Dete!tion

In co%parison when conser*ati*e (ail$re detection protocol is $sed. partiall' disconnected nodes will not be e,cl$ded) Bhen a node detects that another node is not respondin&. the (or%er node does not re%o*e the latter node (ro% the %e%bership instead it chan&es the order o( the rin&) In (i&$re 2c3. a(ter node A (ails to send the to+en to node 0. it chan&es the order o( the rin& (ro% A0C5 to AC05) Node A then sends the to+en to node C. and C to node 0) in the case when a node is indeed bro+en. all the nodes in the connected co%ponent (ail to send the to+en to this node) Bhen a node (ails to send a to+en to another node twice in a row. it re%o*es that node (ro% the %e%bership) Uni3 ene&& o+ To.en&

The to+en %echanis% is the basic co%ponent o( the %e%bership protocol) It &$arantees that there e,ists no %ore than one to+en in the s'ste% at an' ti%e) This sin&le to+en detects the (ail$res. records the %e%bership and $pdates all li*e nodes as it tra*els aro$nd the rin&) A(ter a (ailed node is deter%ined. all li*e nodes in the %e%bership are $na%bi&$o$sl' in(or%ed within one ro$nd o( to+en tra*el) Gro$p %e%bership consens$s is there(ore achie*ed) 411 Me!'"ni&m&

>a*in& described the to+en %echanis%. (ew =$estions re%ain) Bhat i( a node (ails when it processes the to+en and conse=$entl' the to+en is lostH Is it possible to add a new node to the s'ste%H >ow does the s'ste% reco*er (ro% the transient (ail$resH All o( these =$estions can be answered b' the 611 %echanis%) To.en Re#ener"tion

To deal with the to+en loss proble%. a ti%e o$t has been set on each node in the %e%bership) I( a node does not recei*e a to+en (or a certain period o( ti%e. it enters the TAR7ING %ode) The node s$spects that the to+en has been lost and sends o$t a 611 %essa&e to the ne,t node in the rin&) The 611 %essa&e is a re=$est (or a ri&ht to re&enerate the to+en. and is to be pro*ided b' all M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 10

MCA-308

1311748

the li*e nodes in the %e%bership) It is i%perati*e to allow one and onl' one node to re&enerate the to+en when a to+en re&eneration is needed) To &$arantee this %$t$al e,cl$si*it'. we $tiliCe the se=$ence n$%ber on the to+en) @*er' ti%e a to+en is bein& passed (ro% one node to another. the se=$ence n$%ber on it is increased b' one) The pri%ar' ($nction o( the se=$ence n$%ber is to allow the recei*in& node to discard the o$t o( se=$ence to+ens) The se=$ence n$%ber also pla's an i%portant role in the to+en re&eneration %echanis%) @ach node %a+es a local cop' o( the to+en e*er' ti%e that the node recei*es it) Bhen a node needs to send a 611 %essa&e to re=$est the re&eneration o( to+en. it adds this %essa&e to the se=$ence n$%ber that is on its last local cop' o( the to+en) This se=$ence n$%ber will be co%pared to all the se=$ence n$%bers on the local copies o( the to+en on the other li*e nodes) The 611 re=$ests will be denied b' an' node. which possesses a %ore recent cop' o( the to+en) In the e*ent that the to+en is lost. e*er' li*e node sends o$t a 611 re=$est a(ter its TAR7ING ti%eo$t e,pires) ;nl' the node with the latest cop' o( the to+en will recei*e the ri&ht to re&enerate the to+en) Dyn"mi! S!"*"%i*ity

The 611 %essa&e is not onl' $sed as a to+en re&eneration re=$est. b$t also as a re=$est to join the &ro$p) Bhen a new node wishes to participate in the %e%bership. it sends a 611 %essa&e to an' node in the cl$ster) The recei*in& node notices that the ori&inatin& node o( this 611 is not a %e%ber o( the distrib$ted s'ste%. and there(ore. treats it as a join re=$est) The ne,t ti%e that it recei*es the to+en. it adds the new node to the %e%bership. and sends the to+en to the new node) The new node beco%es a part o( the s'ste%) -in. F"i* re& "n) Tr"n&ient F"i* re&

The $ni(ication o( the to+en re&eneration re=$est and the join re=$est (acilitates the treat%ent o( the lin+ (ail$res in the a&&ressi*e (ail$re detection protocol) 1sin& the e,a%ple in (i&$re 2b3. node 0 has been re%o*ed (ro% the %e%bership beca$se o( the (ail$re between A and 0) node 0 does not recei*e the to+en (or a while and it enters the TAR7ING %ode and sends o$t a 611 %essa&e to node C) node C notices that node 0 is not a part o( the %e%bership and there(ore treats the 611 as a join re=$est) The rin& is chan&ed to A0C5 and node 0 joins the %e%bership) M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 11

MCA-308

1311748

Transient (ail$res are treated with the sa%e %echanis%) Bhen a transient (ail$re occ$rs a node is re%o*ed (ro% the %e%bership) A(ter the node reco*ers it sends o$t a 611 %essa&e) The 611 %essa&e is treated as a join re=$est and the node is added bac+ into the cl$ster) In the sa%e (ashion. wron& decisions %ade in a local (ail$re detector can be corrected. &$aranteein& that all no (a$lt' nodes in the pri%ar' connected co%ponent e*ent$all' sta' in the pri%ar' %e%bership) "$ttin& to&ether the to+en and 611 %echanis%s. we ha*e a reliable &ro$p %e%bership protocol) 1sin& this protocol it is eas' to b$ild the (a$lt %ana&e%ent ser*ice) It is also possible to attach to the to+en application dependant s'nchroniCation in(or%ation.

5. AD6ANTAGES
1) RAIN Technolo&' is the %ost scalable so(tware cl$ster technolo&' (or the Internet %ar+et place toda') 9) There is no li%it on the siCe o( a RAIN cl$ster) 3) All nodes are acti*e and can participate in load balancin&) 4) This so(tware onl' technolo&' is open and hi&hl' portable)

7. A//-ICATIONS
Be consider se*eral applications i%ple%ented on RAIN plat(or% based on the co%%$nication. (a$lt %ana&e%ent and data stora&e b$ildin& bloc+s8 a *ideo ser*er 2RAIN 7ideo3. a web ser*er 2 N;B3. and a distrib$ted chec+ pointin& s'ste% 2RAIN Chec+3) Hi#' "v"i*"%i*ity vi)eo &erver, There has been considerable research in the areas o( (a$lt tolerant internet and %$lti%edia ser*ers) @,a%ples are the $n CA#R project at $n Micros'ste%s K1?L. :or this RAIN 7ideo application. a collection o( *ideos are written and encoded to all n nodes in the s'ste% with distrib$ted store operations) A(ter this @ach node r$ns a client application that atte%pts to displa' a *ideo. as well as a ser*er application that s$pplies encoded *ideo data) M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 19

MCA-308

1311748

Hi#' "v"i*"%i*ity 8e% &erver, N;B is %eant (or tron& Networ+ o( Beb er*ers) It i%ple%ents the concept project that de%onstrates the (eat$res o( the RAIN s'ste%) The %ain p$rpose is to de*elop a hi&hl' a*ailable :a$lt-Tolerant 5istrib$ted Beb er*er Cl$ster that %ini%iCes the ris+ o( down ti%e (or %ission critical Internet and intranet applications) The N;B project $ses se*eral +e' b$ildin& bloc+s o( the RAIN technolo&') :irst. it considers the reliable co%%$nication la'er is $sed to handle all o( the %essa&es. which passes between the ser*ers in the N;B s'ste%) econdl'. the to+en-based (a$lt %ana&e%ent %od$le is $sed to establish the set o( ser*ers participatin& in the cl$ster)

Di&tri% te) !'e!. $ointin# me!'"ni&m, A chec+point and rollbac+-reco*er' %echanis% on the RAIN plat(or% based on the distrib$ted store and retrie*e operations) The sche%e r$ns in conj$nction with a leader election protocol) This protocol ens$res that there is a $ni=$e node desi&nated as leader in e*er' connected set o( nodes) As each job e,ec$tes. a chec+point o( the state is ta+en periodicall') The state is encoded and written to all accessible nodes with a distrib$ted store operation) I( a node (ails or beco%es inaccessible. the leader assi&ns the nodeAs job to other nodes) M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 13

MCA-308

1311748

9. CONC-USION The &oal o( the RAIN project has been to address (a$lt %ana&e%ent. co%%$nication and stora&e in a distrib$ted en*iron%ent) 0$ildin& bloc+s that we consider i%portant are those pro*idin& reliable co%%$nication. &ro$p %e%bership and reliable stora&e) i%pl'. RAIN allows (or the &ro$pin& o( an $nli%ited n$%ber o( nodes. which can then ($nction as one sin&le &iant node. sharin& load or ta+in& o*er i( one or %ore o( the nodes ceases to ($nction correctl') The ($t$re direction o( this wor+ is. 5e*elop%ent o( A"I4s (or $sin& the *ario$s b$ildin& bloc+s) The i%ple%entation o( a real distrib$ted (ile s'ste%

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 14

MCA-308

1311748

:.

REFERANCES

K1L) www)wi+ipedia)co% K9L) www)searchdatacenter)techtar&et)co% K3L) www)ca%p$s(e*er)co% K4L) www)&oo&le)co% K?L) www)se%inarti%e)co%

M)M) Instit$te o( Co%p$ter Technolo&' / 0$siness Mana&e%ent. Maharishi Mar+andeshwar 1ni*ersit'. M$llana 2A%bala3 1?