11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

nginx(pronounced"enginex")isafreeopensourcewebserverwrittenbyIgorSysoev,aRussiansoftware
engineer.Sinceitspubliclaunchin2004,nginxhasfocusedonhighperformance,highconcurrencyandlow
memoryusage.Additionalfeaturesontopofthewebserverfunctionality,likeloadbalancing,caching,access
andbandwidthcontrol,andtheabilitytointegrateefficientlywithavarietyofapplications,havehelpedto
makenginxagoodchoiceformodernwebsitearchitectures.Currentlynginxisthesecondmostpopularopen
sourcewebserverontheInternet.

14.1.WhyIsHighConcurrencyImportant?
ThesedaystheInternetissowidespreadandubiquitousit'shardtoimagineitwasn'texactlythere,aswe
knowit,adecadeago.Ithasgreatlyevolved,fromsimpleHTMLproducingclickabletext,basedonNCSA
andthenonApachewebservers,toanalwaysoncommunicationmediumusedbymorethan2billionusers
worldwide.WiththeproliferationofpermanentlyconnectedPCs,mobiledevicesandrecentlytablets,the
Internetlandscapeisrapidlychangingandentireeconomieshavebecomedigitallywired.Onlineservices
havebecomemuchmoreelaboratewithaclearbiastowardsinstantlyavailableliveinformationand
entertainment.Securityaspectsofrunningonlinebusinesshavealsosignificantlychanged.Accordingly,
websitesarenowmuchmorecomplexthanbefore,andgenerallyrequirealotmoreengineeringeffortstobe
robustandscalable.
Oneofthebiggestchallengesforawebsitearchitecthasalwaysbeenconcurrency.Sincethebeginningof
webservices,thelevelofconcurrencyhasbeencontinuouslygrowing.It'snotuncommonforapopular
websitetoservehundredsofthousandsandevenmillionsofsimultaneoususers.Adecadeago,themajor
causeofconcurrencywasslowclientsuserswithADSLordialupconnections.Nowadays,concurrencyis
causedbyacombinationofmobileclientsandnewerapplicationarchitectureswhicharetypicallybasedon
maintainingapersistentconnectionthatallowstheclienttobeupdatedwithnews,tweets,friendfeeds,and
soon.Anotherimportantfactorcontributingtoincreasedconcurrencyisthechangedbehaviorofmodern
browsers,whichopenfourtosixsimultaneousconnectionstoawebsitetoimprovepageloadspeed.
Toillustratetheproblemwithslowclients,imagineasimpleApachebasedwebserverwhichproducesa
relativelyshort100KBresponseawebpagewithtextoranimage.Itcanbemerelyafractionofasecondto
generateorretrievethispage,butittakes10secondstotransmitittoaclientwithabandwidthof80kbps(10
KB/s).Essentially,thewebserverwouldrelativelyquicklypull100KBofcontent,andthenitwouldbebusy
for10secondsslowlysendingthiscontenttotheclientbeforefreeingitsconnection.Nowimaginethatyou
have1,000simultaneouslyconnectedclientswhohaverequestedsimilarcontent.Ifonly1MBofadditional
memoryisallocatedperclient,itwouldresultin1000MB(about1GB)ofextramemorydevotedtoserving
just1000clients100KBofcontent.Inreality,atypicalwebserverbasedonApachecommonlyallocates
morethan1MBofadditionalmemoryperconnection,andregrettablytensofkbpsisstilloftentheeffective
speedofmobilecommunications.Althoughthesituationwithsendingcontenttoaslowclientmightbe,to
someextent,improvedbyincreasingthesizeofoperatingsystemkernelsocketbuffers,it'snotageneral
solutiontotheproblemandcanhaveundesirablesideeffects.
Withpersistentconnectionstheproblemofhandlingconcurrencyisevenmorepronounced,becausetoavoid
latencyassociatedwithestablishingnewHTTPconnections,clientswouldstayconnected,andforeach
connectedclientthere'sacertainamountofmemoryallocatedbythewebserver.
Consequently,tohandletheincreasedworkloadsassociatedwithgrowingaudiencesandhencehigherlevels
ofconcurrencyandtobeabletocontinuouslydosoawebsiteshouldbebasedonanumberofvery
efficientbuildingblocks.Whiletheotherpartsoftheequationsuchashardware(CPU,memory,disks),
networkcapacity,applicationanddatastoragearchitecturesareobviouslyimportant,itisinthewebserver
softwarethatclientconnectionsareacceptedandprocessed.Thus,thewebservershouldbeabletoscale
nonlinearlywiththegrowingnumberofsimultaneousconnectionsandrequestspersecond.

Isn'tApacheSuitable?
Apache,thewebserversoftwarethatstilllargelydominatestheInternettoday,hasitsrootsinthebeginning
ofthe1990s.Originally,itsarchitecturematchedthethenexistingoperatingsystemsandhardware,butalso
thestateoftheInternet,whereawebsitewastypicallyastandalonephysicalserverrunningasingleinstance
http://www.aosabook.org/en/nginx.html

1/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

ofApache.Bythebeginningofthe2000sitwasobviousthatthestandalonewebservermodelcouldnotbe
easilyreplicatedtosatisfytheneedsofgrowingwebservices.AlthoughApacheprovidedasolidfoundation
forfuturedevelopment,itwasarchitectedtospawnacopyofitselfforeachnewconnection,whichwasnot
suitablefornonlinearscalabilityofawebsite.EventuallyApachebecameageneralpurposewebserver
focusingonhavingmanydifferentfeatures,avarietyofthirdpartyextensions,anduniversalapplicabilityto
practicallyanykindofwebapplicationdevelopment.However,nothingcomeswithoutapriceandthe
downsidetohavingsucharichanduniversalcombinationoftoolsinasinglepieceofsoftwareisless
scalabilitybecauseofincreasedCPUandmemoryusageperconnection.
Thus,whenserverhardware,operatingsystemsandnetworkresourcesceasedtobemajorconstraintsfor
websitegrowth,webdevelopersworldwidestartedtolookaroundforamoreefficientmeansofrunningweb
servers.Aroundtenyearsago,DanielKegel,aprominentsoftwareengineer,proclaimedthat"it'stimeforweb
serverstohandletenthousandclientssimultaneously"andpredictedwhatwenowcallInternetcloud
services.Kegel'sC10Kmanifestspurredanumberofattemptstosolvetheproblemofwebserver
optimizationtohandlealargenumberofclientsatthesametime,andnginxturnedouttobeoneofthemost
successfulones.
AimedatsolvingtheC10Kproblemof10,000simultaneousconnections,nginxwaswrittenwithadifferent
architectureinmindonewhichismuchmoresuitablefornonlinearscalabilityinboththenumberof
simultaneousconnectionsandrequestspersecond.nginxiseventbased,soitdoesnotfollowApache'sstyle
ofspawningnewprocessesorthreadsforeachwebpagerequest.Theendresultisthatevenasload
increases,memoryandCPUusageremainmanageable.nginxcannowdelivertensofthousandsof
concurrentconnectionsonaserverwithtypicalhardware.
Whenthefirstversionofnginxwasreleased,itwasmeanttobedeployedalongsideApachesuchthatstatic
contentlikeHTML,CSS,JavaScriptandimageswerehandledbynginxtooffloadconcurrencyandlatency
processingfromApachebasedapplicationservers.Overthecourseofitsdevelopment,nginxhasadded
integrationwithapplicationsthroughtheuseofFastCGI,uswgiorSCGIprotocols,andwithdistributed
memoryobjectcachingsystemslikememcached.Otherusefulfunctionalitylikereverseproxywithload
balancingandcachingwasaddedaswell.Theseadditionalfeatureshaveshapednginxintoanefficient
combinationoftoolstobuildascalablewebinfrastructureupon.
InFebruary2012,theApache2.4.xbranchwasreleasedtothepublic.AlthoughthislatestreleaseofApache
hasaddednewmultiprocessingcoremodulesandnewproxymodulesaimedatenhancingscalabilityand
performance,it'stoosoontotellifitsperformance,concurrencyandresourceutilizationarenowonparwith,
orbetterthan,pureeventdrivenwebservers.ItwouldbeverynicetoseeApacheapplicationserversscale
betterwiththenewversion,though,asitcouldpotentiallyalleviatebottlenecksonthebackendsidewhichstill
oftenremainunsolvedintypicalnginxplusApachewebconfigurations.

AreThereMoreAdvantagestoUsingnginx?
Handlinghighconcurrencywithhighperformanceandefficiencyhasalwaysbeenthekeybenefitofdeploying
nginx.However,therearenowevenmoreinterestingbenefits.
Inthelastfewyears,webarchitectshaveembracedtheideaofdecouplingandseparatingtheirapplication
infrastructurefromthewebserver.However,whatwouldpreviouslyexistintheformofaLAMP(Linux,
Apache,MySQL,PHP,PythonorPerl)basedwebsite,mightnowbecomenotmerelyaLEMPbasedone(`E'
standingfor`Enginex'),butmoreandmoreoftenanexerciseinpushingthewebservertotheedgeofthe
infrastructureandintegratingthesameorarevampedsetofapplicationsanddatabasetoolsarounditina
differentway.
nginxisverywellsuitedforthis,asitprovidesthekeyfeaturesnecessarytoconvenientlyoffloadconcurrency,
latencyprocessing,SSL(securesocketslayer),staticcontent,compressionandcaching,connectionsand
requeststhrottling,andevenHTTPmediastreamingfromtheapplicationlayertoamuchmoreefficientedge
webserverlayer.Italsoallowsintegratingdirectlywithmemcached/Redisorother"NoSQL"solutions,to
boostperformancewhenservingalargenumberofconcurrentusers.
Withrecentflavorsofdevelopmentkitsandprogramminglanguagesgainingwideuse,moreandmore
http://www.aosabook.org/en/nginx.html

2/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

companiesarechangingtheirapplicationdevelopmentanddeploymenthabits.nginxhasbecomeoneofthe
mostimportantcomponentsofthesechangingparadigms,andithasalreadyhelpedmanycompaniesstart
anddeveloptheirwebservicesquicklyandwithintheirbudgets.
Thefirstlinesofnginxwerewrittenin2002.In2004itwasreleasedtothepublicunderthetwoclauseBSD
license.Thenumberofnginxusershasbeengrowingeversince,contributingideas,andsubmittingbug
reports,suggestionsandobservationsthathavebeenimmenselyhelpfulandbeneficialfortheentire
community.
ThenginxcodebaseisoriginalandwaswrittenentirelyfromscratchintheCprogramminglanguage.nginx
hasbeenportedtomanyarchitecturesandoperatingsystems,includingLinux,FreeBSD,Solaris,MacOSX,
AIXandMicrosoftWindows.nginxhasitsownlibrariesandwithitsstandardmodulesdoesnotusemuch
beyondthesystem'sClibrary,exceptforzlib,PCREandOpenSSLwhichcanbeoptionallyexcludedfroma
buildifnotneededorbecauseofpotentiallicenseconflicts.
AfewwordsabouttheWindowsversionofnginx.WhilenginxworksinaWindowsenvironment,theWindows
versionofnginxismorelikeaproofofconceptratherthanafullyfunctionalport.Therearecertainlimitations
ofthenginxandWindowskernelarchitecturesthatdonotinteractwellatthistime.Theknownissuesofthe
nginxversionforWindowsincludeamuchlowernumberofconcurrentconnections,decreasedperformance,
nocachingandnobandwidthpolicing.FutureversionsofnginxforWindowswillmatchthemainstream
functionalitymoreclosely.

14.2.OverviewofnginxArchitecture
Traditionalprocessorthreadbasedmodelsofhandlingconcurrentconnectionsinvolvehandlingeach
connectionwithaseparateprocessorthread,andblockingonnetworkorinput/outputoperations.Depending
ontheapplication,itcanbeveryinefficientintermsofmemoryandCPUconsumption.Spawningaseparate
processorthreadrequirespreparationofanewruntimeenvironment,includingallocationofheapandstack
memory,andthecreationofanewexecutioncontext.AdditionalCPUtimeisalsospentcreatingtheseitems,
whichcaneventuallyleadtopoorperformanceduetothreadthrashingonexcessivecontextswitching.Allof
thesecomplicationsmanifestthemselvesinolderwebserverarchitectureslikeApache's.Thisisatradeoff
betweenofferingarichsetofgenerallyapplicablefeaturesandoptimizedusageofserverresources.
Fromtheverybeginning,nginxwasmeanttobeaspecializedtooltoachievemoreperformance,densityand
economicaluseofserverresourceswhileenablingdynamicgrowthofawebsite,soithasfollowedadifferent
model.Itwasactuallyinspiredbytheongoingdevelopmentofadvancedeventbasedmechanismsina
varietyofoperatingsystems.Whatresultedisamodular,eventdriven,asynchronous,singlethreaded,non
blockingarchitecturewhichbecamethefoundationofnginxcode.
nginxusesmultiplexingandeventnotificationsheavily,anddedicatesspecifictaskstoseparateprocesses.
Connectionsareprocessedinahighlyefficientrunloopinalimitednumberofsinglethreadedprocesses
called workers.Withineach workernginxcanhandlemanythousandsofconcurrentconnectionsand
requestspersecond.

CodeStructure
Thenginx workercodeincludesthecoreandthefunctionalmodules.Thecoreofnginxisresponsiblefor
maintainingatightrunloopandexecutingappropriatesectionsofmodules'codeoneachstageofrequest
processing.Modulesconstitutemostofthepresentationandapplicationlayerfunctionality.Modulesreadfrom
andwritetothenetworkandstorage,transformcontent,dooutboundfiltering,applyserversideinclude
actionsandpasstherequeststotheupstreamserverswhenproxyingisactivated.
nginx'smodulararchitecturegenerallyallowsdeveloperstoextendthesetofwebserverfeatureswithout
modifyingthenginxcore.nginxmodulescomeinslightlydifferentincarnations,namelycoremodules,event
modules,phasehandlers,protocols,variablehandlers,filters,upstreamsandloadbalancers.Atthistime,
nginxdoesn'tsupportdynamicallyloadedmodulesi.e.,modulesarecompiledalongwiththecoreatbuild
stage.However,supportforloadablemodulesandABIisplannedforthefuturemajorreleases.Moredetailed
informationabouttherolesofdifferentmodulescanbefoundinSection14.4.
http://www.aosabook.org/en/nginx.html

3/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

Whilehandlingavarietyofactionsassociatedwithaccepting,processingandmanagingnetworkconnections
andcontentretrieval,nginxuseseventnotificationmechanismsandanumberofdiskI/Operformance
enhancementsinLinux,SolarisandBSDbasedoperatingsystems,like kqueue, epoll,and event
ports.Thegoalistoprovideasmanyhintstotheoperatingsystemaspossible,inregardstoobtaining
timelyasynchronousfeedbackforinboundandoutboundtraffic,diskoperations,readingfromorwritingto
sockets,timeoutsandsoon.TheusageofdifferentmethodsformultiplexingandadvancedI/Ooperationsis
heavilyoptimizedforeveryUnixbasedoperatingsystemnginxrunson.
AhighleveloverviewofnginxarchitectureispresentedinFigure14.1.

Figure14.1:Diagramofnginx'sarchitecture

WorkersModel
Aspreviouslymentioned,nginxdoesn'tspawnaprocessorthreadforeveryconnection.Instead, worker
processesacceptnewrequestsfromashared"listen"socketandexecuteahighlyefficientrunloopinside
each workertoprocessthousandsofconnectionsper worker.There'snospecializedarbitrationor
distributionofconnectionstothe workersinnginxthisworkisdonebytheOSkernelmechanisms.Upon
startup,aninitialsetoflisteningsocketsiscreated. workersthencontinuouslyaccept,readfromandwrite
tothesocketswhileprocessingHTTPrequestsandresponses.
Therunloopisthemostcomplicatedpartofthenginx workercode.Itincludescomprehensiveinnercalls
andreliesheavilyontheideaofasynchronoustaskhandling.Asynchronousoperationsareimplemented
throughmodularity,eventnotifications,extensiveuseofcallbackfunctionsandfinetunedtimers.Overall,the
keyprincipleistobeasnonblockingaspossible.Theonlysituationwherenginxcanstillblockiswhen
there'snotenoughdiskstorageperformancefora workerprocess.
Becausenginxdoesnotforkaprocessorthreadperconnection,memoryusageisveryconservativeand
extremelyefficientinthevastmajorityofcases.nginxconservesCPUcyclesaswellbecausethere'sno
ongoingcreatedestroypatternforprocessesorthreads.Whatnginxdoesischeckthestateofthenetwork
andstorage,initializenewconnections,addthemtotherunloop,andprocessasynchronouslyuntil
completion,atwhichpointtheconnectionisdeallocatedandremovedfromtherunloop.Combinedwiththe
carefuluseof syscallsandanaccurateimplementationofsupportinginterfaceslikepoolandslabmemory
allocators,nginxtypicallyachievesmoderatetolowCPUusageevenunderextremeworkloads.
Becausenginxspawnsseveral workerstohandleconnections,itscaleswellacrossmultiplecores.
Generally,aseparate workerpercoreallowsfullutilizationofmulticorearchitectures,andpreventsthread
http://www.aosabook.org/en/nginx.html

4/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

thrashingandlockups.There'snoresourcestarvationandtheresourcecontrollingmechanismsareisolated
withinsinglethreaded workerprocesses.Thismodelalsoallowsmorescalabilityacrossphysicalstorage
devices,facilitatesmorediskutilizationandavoidsblockingondiskI/O.Asaresult,serverresourcesare
utilizedmoreefficientlywiththeworkloadsharedacrossseveralworkers.
WithsomediskuseandCPUloadpatterns,thenumberofnginx workersshouldbeadjusted.Therulesare
somewhatbasichere,andsystemadministratorsshouldtryacoupleofconfigurationsfortheirworkloads.
Generalrecommendationsmightbethefollowing:iftheloadpatternisCPUintensiveforinstance,handling
alotofTCP/IP,doingSSL,orcompressionthenumberofnginx workersshouldmatchthenumberof
CPUcoresiftheloadismostlydiskI/Oboundforinstance,servingdifferentsetsofcontentfromstorage,or
heavyproxyingthenumberof workersmightbeoneandahalftotwotimesthenumberofcores.Some
engineerschoosethenumberof workersbasedonthenumberofindividualstorageunitsinstead,though
efficiencyofthisapproachdependsonthetypeandconfigurationofdiskstorage.
Onemajorproblemthatthedevelopersofnginxwillbesolvinginupcomingversionsishowtoavoidmostof
theblockingondiskI/O.Atthemoment,ifthere'snotenoughstorageperformancetoservediskoperations
generatedbyaparticular worker,that workermaystillblockonreading/writingfromdisk.Anumberof
mechanismsandconfigurationfiledirectivesexisttomitigatesuchdiskI/Oblockingscenarios.Mostnotably,
combinationsofoptionslikesendfileandAIOtypicallyproducealotofheadroomfordiskperformance.An
nginxinstallationshouldbeplannedbasedonthedataset,theamountofmemoryavailablefornginx,andthe
underlyingstoragearchitecture.
Anotherproblemwiththeexisting workermodelisrelatedtolimitedsupportforembeddedscripting.For
one,withthestandardnginxdistribution,onlyembeddingPerlscriptsissupported.Thereisasimple
explanationforthat:thekeyproblemisthepossibilityofanembeddedscripttoblockonanyoperationorexit
unexpectedly.Bothtypesofbehaviorwouldimmediatelyleadtoasituationwheretheworkerishung,
affectingmanythousandsofconnectionsatonce.Moreworkisplannedtomakeembeddedscriptingwith
nginxsimpler,morereliableandsuitableforabroaderrangeofapplications.

nginxProcessRoles
nginxrunsseveralprocessesinmemorythereisasinglemasterprocessandseveral workerprocesses.
Therearealsoacoupleofspecialpurposeprocesses,specificallyacacheloaderandcachemanager.All
processesaresinglethreadedinversion1.xofnginx.Allprocessesprimarilyusesharedmemory
mechanismsforinterprocesscommunication.Themasterprocessisrunasthe rootuser.Thecache
loader,cachemanagerand workersrunasanunprivilegeduser.
Themasterprocessisresponsibleforthefollowingtasks:
readingandvalidatingconfiguration
creating,bindingandclosingsockets
starting,terminatingandmaintainingtheconfigurednumberof workerprocesses
reconfiguringwithoutserviceinterruption
controllingnonstopbinaryupgrades(startingnewbinaryandrollingbackifnecessary)
reopeninglogfiles
compilingembeddedPerlscripts
The workerprocessesaccept,handleandprocessconnectionsfromclients,providereverseproxyingand
filteringfunctionalityanddoalmosteverythingelsethatnginxiscapableof.Inregardstomonitoringthe
behaviorofannginxinstance,asystemadministratorshouldkeepaneyeon workersastheyarethe
processesreflectingtheactualdaytodayoperationsofawebserver.
Thecacheloaderprocessisresponsibleforcheckingtheondiskcacheitemsandpopulatingnginx'sin
memorydatabasewithcachemetadata.Essentially,thecacheloaderpreparesnginxinstancestoworkwith
filesalreadystoredondiskinaspeciallyallocateddirectorystructure.Ittraversesthedirectories,checks
cachecontentmetadata,updatestherelevantentriesinsharedmemoryandthenexitswheneverythingis
cleanandreadyforuse.
http://www.aosabook.org/en/nginx.html

5/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

Thecachemanagerismostlyresponsibleforcacheexpirationandinvalidation.Itstaysinmemoryduring
normalnginxoperationanditisrestartedbythemasterprocessinthecaseoffailure.

BriefOverviewofnginxCaching
Cachinginnginxisimplementedintheformofhierarchicaldatastorageonafilesystem.Cachekeysare
configurable,anddifferentrequestspecificparameterscanbeusedtocontrolwhatgetsintothecache.
Cachekeysandcachemetadataarestoredinthesharedmemorysegments,whichthecacheloader,cache
managerand workerscanaccess.Currentlythereisnotanyinmemorycachingoffiles,otherthan
optimizationsimpliedbytheoperatingsystem'svirtualfilesystemmechanisms.Eachcachedresponseis
placedinadifferentfileonthefilesystem.Thehierarchy(levelsandnamingdetails)arecontrolledthrough
nginxconfigurationdirectives.Whenaresponseiswrittentothecachedirectorystructure,thepathandthe
nameofthefilearederivedfromanMD5hashoftheproxyURL.
Theprocessforplacingcontentinthecacheisasfollows:Whennginxreadstheresponsefromanupstream
server,thecontentisfirstwrittentoatemporaryfileoutsideofthecachedirectorystructure.Whennginx
finishesprocessingtherequestitrenamesthetemporaryfileandmovesittothecachedirectory.Ifthe
temporaryfilesdirectoryforproxyingisonanotherfilesystem,thefilewillbecopied,thusit'srecommended
tokeepbothtemporaryandcachedirectoriesonthesamefilesystem.Itisalsoquitesafetodeletefilesfrom
thecachedirectorystructurewhentheyneedtobeexplicitlypurged.Therearethirdpartyextensionsfor
nginxwhichmakeitpossibletocontrolcachedcontentremotely,andmoreworkisplannedtointegratethis
functionalityinthemaindistribution.

14.3.nginxConfiguration
nginx'sconfigurationsystemwasinspiredbyIgorSysoev'sexperienceswithApache.Hismaininsightwas
thatascalableconfigurationsystemisessentialforawebserver.Themainscalingproblemwasencountered
whenmaintaininglargecomplicatedconfigurationswithlotsofvirtualservers,directories,locationsand
datasets.Inarelativelybigwebsetupitcanbeanightmareifnotdoneproperlybothattheapplicationlevel
andbythesystemengineerhimself.
Asaresult,nginxconfigurationwasdesignedtosimplifydaytodayoperationsandtoprovideaneasymeans
forfurtherexpansionofwebserverconfiguration.
nginxconfigurationiskeptinanumberofplaintextfileswhichtypicallyresidein /usr/local/etc/nginxor
/etc/nginx.Themainconfigurationfileisusuallycalled nginx.conf.Tokeepituncluttered,partsofthe
configurationcanbeputinseparatefileswhichcanbeautomaticallyincludedinthemainone.However,it
shouldbenotedherethatnginxdoesnotcurrentlysupportApachestyledistributedconfigurations(i.e.,
.htaccessfiles).Alloftheconfigurationrelevanttonginxwebserverbehaviorshouldresideina
centralizedsetofconfigurationfiles.
Theconfigurationfilesareinitiallyreadandverifiedbythemasterprocess.Acompiledreadonlyformofthe
nginxconfigurationisavailabletothe workerprocessesastheyareforkedfromthemasterprocess.
Configurationstructuresareautomaticallysharedbytheusualvirtualmemorymanagementmechanisms.
nginxconfigurationhasseveraldifferentcontextsfor main, http, server, upstream, location(and
also mailformailproxy)blocksofdirectives.Contextsneveroverlap.Forinstance,thereisnosuchthing
asputtinga locationblockinthe mainblockofdirectives.Also,toavoidunnecessaryambiguitythere
isn'tanythinglikea"globalwebserver"configuration.nginxconfigurationismeanttobecleanandlogical,
allowinguserstomaintaincomplicatedconfigurationfilesthatcomprisethousandsofdirectives.Inaprivate
conversation,Sysoevsaid,"Locations,directories,andotherblocksintheglobalserverconfigurationarethe
featuresIneverlikedinApache,sothisisthereasonwhytheywereneverimplementedinnginx."
Configurationsyntax,formattinganddefinitionsfollowasocalledCstyleconvention.Thisparticularapproach
tomakingconfigurationfilesisalreadybeingusedbyavarietyofopensourceandcommercialsoftware
applications.Bydesign,Cstyleconfigurationiswellsuitedfornesteddescriptions,beinglogicalandeasyto
create,readandmaintain,andlikedbymanyengineers.Cstyleconfigurationofnginxcanalsobeeasily
automated.
http://www.aosabook.org/en/nginx.html

6/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

WhilesomeofthenginxdirectivesresemblecertainpartsofApacheconfiguration,settingupannginx
instanceisquiteadifferentexperience.Forinstance,rewriterulesaresupportedbynginx,thoughitwould
requireanadministratortomanuallyadaptalegacyApacherewriteconfigurationtomatchnginxstyle.The
implementationoftherewriteenginedifferstoo.
Ingeneral,nginxsettingsalsoprovidesupportforseveraloriginalmechanismsthatcanbeveryusefulaspart
ofaleanwebserverconfiguration.Itmakessensetobrieflymentionvariablesandthe try_filesdirective,
whicharesomewhatuniquetonginx.Variablesinnginxweredevelopedtoprovideanadditionalevenmore
powerfulmechanismtocontrolruntimeconfigurationofawebserver.Variablesareoptimizedforquick
evaluationandareinternallyprecompiledtoindices.Evaluationisdoneondemandi.e.,thevalueofa
variableistypicallycalculatedonlyonceandcachedforthelifetimeofaparticularrequest.Variablescanbe
usedwithdifferentconfigurationdirectives,providingadditionalflexibilityfordescribingconditionalrequest
processingbehavior.
The try_filesdirectivewasinitiallymeanttograduallyreplaceconditional ifconfigurationstatementsin
amoreproperway,anditwasdesignedtoquicklyandefficientlytry/matchagainstdifferentURItocontent
mappings.Overall,the try_filesdirectiveworkswellandcanbeextremelyefficientanduseful.Itis
recommendedthatthereaderthoroughlycheckthe try_filesdirectiveandadoptitsusewhenever
applicable.

14.4.nginxInternals
Aswasmentionedbefore,thenginxcodebaseconsistsofacoreandanumberofmodules.Thecoreofnginx
isresponsibleforprovidingthefoundationofthewebserver,webandmailreverseproxyfunctionalitiesit
enablestheuseofunderlyingnetworkprotocols,buildsthenecessaryruntimeenvironment,andensures
seamlessinteractionbetweendifferentmodules.However,mostoftheprotocolandapplicationspecific
featuresaredonebynginxmodules,notthecore.
Internally,nginxprocessesconnectionsthroughapipeline,orchain,ofmodules.Inotherwords,forevery
operationthere'samodulewhichisdoingtherelevantworke.g.,compression,modifyingcontent,executing
serversideincludes,communicatingtotheupstreamapplicationserversthroughFastCGIoruwsgiprotocols,
ortalkingtomemcached.
Thereareacoupleofnginxmodulesthatsitsomewherebetweenthecoreandthereal"functional"modules.
Thesemodulesare httpand mail.Thesetwomodulesprovideanadditionallevelofabstractionbetween
thecoreandlowerlevelcomponents.Inthesemodules,thehandlingofthesequenceofeventsassociated
witharespectiveapplicationlayerprotocollikeHTTP,SMTPorIMAPisimplemented.Incombinationwiththe
nginxcore,theseupperlevelmodulesareresponsibleformaintainingtherightorderofcallstotherespective
functionalmodules.WhiletheHTTPprotocoliscurrentlyimplementedaspartofthe httpmodule,thereare
planstoseparateitintoafunctionalmoduleinthefuture,duetotheneedtosupportotherprotocolslike
SPDY(see"SPDY:Anexperimentalprotocolforafasterweb").
Thefunctionalmodulescanbedividedintoeventmodules,phasehandlers,outputfilters,variablehandlers,
protocols,upstreamsandloadbalancers.MostofthesemodulescomplementtheHTTPfunctionalityofnginx,
thougheventmodulesandprotocolsarealsousedfor mail.EventmodulesprovideaparticularOS
dependenteventnotificationmechanismlike kqueueor epoll.Theeventmodulethatnginxuses
dependsontheoperatingsystemcapabilitiesandbuildconfiguration.Protocolmodulesallownginxto
communicatethroughHTTPS,TLS/SSL,SMTP,POP3andIMAP.
AtypicalHTTPrequestprocessingcyclelookslikethefollowing.
1. ClientsendsHTTPrequest.
2. nginxcorechoosestheappropriatephasehandlerbasedontheconfiguredlocationmatchingthe
request.
3. Ifconfiguredtodoso,aloadbalancerpicksanupstreamserverforproxying.
4. Phasehandlerdoesitsjobandpasseseachoutputbuffertothefirstfilter.
5. Firstfilterpassestheoutputtothesecondfilter.
6. Secondfilterpassestheoutputtothird(andsoon).
http://www.aosabook.org/en/nginx.html

7/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

7. Finalresponseissenttotheclient.
nginxmoduleinvocationisextremelycustomizable.Itisperformedthroughaseriesofcallbacksusing
pointerstotheexecutablefunctions.However,thedownsideofthisisthatitmayplaceabigburdenon
programmerswhowouldliketowritetheirownmodules,becausetheymustdefineexactlyhowandwhenthe
moduleshouldrun.BoththenginxAPIanddevelopers'documentationarebeingimprovedandmademore
availabletoalleviatethis.
Someexamplesofwhereamodulecanattachare:
Beforetheconfigurationfileisreadandprocessed
Foreachconfigurationdirectiveforthelocationandtheserverwhereitappears
Whenthemainconfigurationisinitialized
Whentheserver(i.e.,host/port)isinitialized
Whentheserverconfigurationismergedwiththemainconfiguration
Whenthelocationconfigurationisinitializedormergedwithitsparentserverconfiguration
Whenthemasterprocessstartsorexits
Whenanewworkerprocessstartsorexits
Whenhandlingarequest
Whenfilteringtheresponseheaderandthebody
Whenpicking,initiatingandreinitiatingarequesttoanupstreamserver
Whenprocessingtheresponsefromanupstreamserver
Whenfinishinganinteractionwithanupstreamserver
Insidea worker,thesequenceofactionsleadingtotherunloopwheretheresponseisgeneratedlookslike
thefollowing:
1. Begin ngx_worker_process_cycle().
2. ProcesseventswithOSspecificmechanisms(suchas epollor kqueue).
3. Accepteventsanddispatchtherelevantactions.
4. Process/proxyrequestheaderandbody.
5. Generateresponsecontent(header,body)andstreamittotheclient.
6. Finalizerequest.
7. Reinitializetimersandevents.
Therunloopitself(steps5and6)ensuresincrementalgenerationofaresponseandstreamingittotheclient.
AmoredetailedviewofprocessinganHTTPrequestmightlooklikethis:
1. Initializerequestprocessing.
2. Processheader.
3. Processbody.
4. Calltheassociatedhandler.
5. Runthroughtheprocessingphases.
Whichbringsustothephases.WhennginxhandlesanHTTPrequest,itpassesitthroughanumberof
processingphases.Ateachphasetherearehandlerstocall.Ingeneral,phasehandlersprocessarequest
andproducetherelevantoutput.Phasehandlersareattachedtothelocationsdefinedintheconfigurationfile.
Phasehandlerstypicallydofourthings:getthelocationconfiguration,generateanappropriateresponse,
sendtheheader,andsendthebody.Ahandlerhasoneargument:aspecificstructuredescribingtherequest.
Arequeststructurehasalotofusefulinformationabouttheclientrequest,suchastherequestmethod,URI,
andheader.
WhentheHTTPrequestheaderisread,nginxdoesalookupoftheassociatedvirtualserverconfiguration.If
thevirtualserverisfound,therequestgoesthroughsixphases:
1. serverrewritephase
2. locationphase
3. locationrewritephase(whichcanbringtherequestbacktothepreviousphase)
http://www.aosabook.org/en/nginx.html

8/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

4. accesscontrolphase
5. try_filesphase
6. logphase
Inanattempttogeneratethenecessarycontentinresponsetotherequest,nginxpassestherequesttoa
suitablecontenthandler.Dependingontheexactlocationconfiguration,nginxmaytrysocalledunconditional
handlersfirst,like perl, proxy_pass, flv, mp4,etc.Iftherequestdoesnotmatchanyoftheabove
contenthandlers,itispickedbyoneofthefollowinghandlers,inthisexactorder: randomindex, index,
autoindex, gzip_static, static.
Indexingmoduledetailscanbefoundinthenginxdocumentation,butthesearethemoduleswhichhandle
requestswithatrailingslash.Ifaspecializedmodulelike mp4or autoindexisn'tappropriate,thecontent
isconsideredtobejustafileordirectoryondisk(thatis,static)andisservedbythe staticcontent
handler.ForadirectoryitwouldautomaticallyrewritetheURIsothatthetrailingslashisalwaysthere(and
thenissueanHTTPredirect).
Thecontenthandlers'contentisthenpassedtothefilters.Filtersarealsoattachedtolocations,andtherecan
beseveralfiltersconfiguredforalocation.Filtersdothetaskofmanipulatingtheoutputproducedbya
handler.Theorderoffilterexecutionisdeterminedatcompiletime.Fortheoutoftheboxfiltersit's
predefined,andforathirdpartyfilteritcanbeconfiguredatthebuildstage.Intheexistingnginx
implementation,filterscanonlydooutboundchangesandthereiscurrentlynomechanismtowriteandattach
filterstodoinputcontenttransformation.Inputfilteringwillappearinfutureversionsofnginx.
Filtersfollowaparticulardesignpattern.Afiltergetscalled,startsworking,andcallsthenextfilteruntilthe
finalfilterinthechainiscalled.Afterthat,nginxfinalizestheresponse.Filtersdon'thavetowaitforthe
previousfiltertofinish.Thenextfilterinachaincanstartitsownworkassoonastheinputfromtheprevious
oneisavailable(functionallymuchliketheUnixpipeline).Inturn,theoutputresponsebeinggeneratedcanbe
passedtotheclientbeforetheentireresponsefromtheupstreamserverisreceived.
Thereareheaderfiltersandbodyfiltersnginxfeedstheheaderandthebodyoftheresponsetothe
associatedfiltersseparately.
Aheaderfilterconsistsofthreebasicsteps:
1. Decidewhethertooperateonthisresponse.
2. Operateontheresponse.
3. Callthenextfilter.
Bodyfilterstransformthegeneratedcontent.Examplesofbodyfiltersinclude:
serversideincludes
XSLTfiltering
imagefiltering(forinstance,resizingimagesonthefly)
charsetmodification
gzipcompression
chunkedencoding
Afterthefilterchain,theresponseispassedtothewriter.Alongwiththewriterthereareacoupleofadditional
specialpurposefilters,namelythe copyfilter,andthe postponefilter.The copyfilterisresponsiblefor
fillingmemorybufferswiththerelevantresponsecontentwhichmightbestoredinaproxytemporary
directory.The postponefilterisusedforsubrequests.
Subrequestsareaveryimportantmechanismforrequest/responseprocessing.Subrequestsarealsooneof
themostpowerfulaspectsofnginx.WithsubrequestsnginxcanreturntheresultsfromadifferentURLthan
theonetheclientoriginallyrequested.Somewebframeworkscallthisaninternalredirect.However,nginx
goesfurthernotonlycanfiltersperformmultiplesubrequestsandcombinetheoutputsintoasingle
response,butsubrequestscanalsobenestedandhierarchical.Asubrequestcanperformitsownsub
subrequest,andasubsubrequestcaninitiatesubsubsubrequests.Subrequestscanmaptofilesonthehard
disk,otherhandlers,orupstreamservers.Subrequestsaremostusefulforinsertingadditionalcontentbased
http://www.aosabook.org/en/nginx.html

9/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

ondatafromtheoriginalresponse.Forexample,theSSI(serversideinclude)moduleusesafiltertoparse
thecontentsofthereturneddocument,andthenreplaces includedirectiveswiththecontentsofspecified
URLs.Or,itcanbeanexampleofmakingafilterthattreatstheentirecontentsofadocumentasaURLtobe
retrieved,andthenappendsthenewdocumenttotheURLitself.
Upstreamandloadbalancersarealsoworthdescribingbriefly.Upstreamsareusedtoimplementwhatcanbe
identifiedasacontenthandlerwhichisareverseproxy( proxy_passhandler).Upstreammodulesmostly
preparetherequesttobesenttoanupstreamserver(or"backend")andreceivetheresponsefromthe
upstreamserver.Therearenocallstooutputfiltershere.Whatanupstreammoduledoesexactlyisset
callbackstobeinvokedwhentheupstreamserverisreadytobewrittentoandreadfrom.Callbacks
implementingthefollowingfunctionalityexist:
Craftingarequestbuffer(orachainofthem)tobesenttotheupstreamserver
Reinitializing/resettingtheconnectiontotheupstreamserver(whichhappensrightbeforecreatingthe
requestagain)
Processingthefirstbitsofanupstreamresponseandsavingpointerstothepayloadreceivedfromthe
upstreamserver
Abortingrequests(whichhappenswhentheclientterminatesprematurely)
Finalizingtherequestwhennginxfinishesreadingfromtheupstreamserver
Trimmingtheresponsebody(e.g.removingatrailer)
Loadbalancermodulesattachtothe proxy_passhandlertoprovidetheabilitytochooseanupstream
serverwhenmorethanoneupstreamserveriseligible.Aloadbalancerregistersanenablingconfigurationfile
directive,providesadditionalupstreaminitializationfunctions(toresolveupstreamnamesinDNS,etc.),
initializestheconnectionstructures,decideswheretoroutetherequests,andupdatesstatsinformation.
Currentlynginxsupportstwostandarddisciplinesforloadbalancingtoupstreamservers:roundrobinandip
hash.
Upstreamandloadbalancinghandlingmechanismsincludealgorithmstodetectfailedupstreamserversand
toreroutenewrequeststotheremainingonesthoughalotofadditionalworkisplannedtoenhancethis
functionality.Ingeneral,moreworkonloadbalancersisplanned,andinthenextversionsofnginxthe
mechanismsfordistributingtheloadacrossdifferentupstreamserversaswellashealthcheckswillbegreatly
improved.
Therearealsoacoupleofotherinterestingmoduleswhichprovideanadditionalsetofvariablesforuseinthe
configurationfile.Whilethevariablesinnginxarecreatedandupdatedacrossdifferentmodules,thereare
twomodulesthatareentirelydedicatedtovariables: geoand map.The geomoduleisusedtofacilitate
trackingofclientsbasedontheirIPaddresses.Thismodulecancreatearbitraryvariablesthatdependonthe
client'sIPaddress.Theothermodule, map,allowsforthecreationofvariablesfromothervariables,
essentiallyprovidingtheabilitytodoflexiblemappingsofhostnamesandotherruntimevariables.Thiskindof
modulemaybecalledthevariablehandler.
Memoryallocationmechanismsimplementedinsideasinglenginx workerwere,tosomeextent,inspiredby
Apache.Ahighleveldescriptionofnginxmemorymanagementwouldbethefollowing:Foreachconnection,
thenecessarymemorybuffersaredynamicallyallocated,linked,usedforstoringandmanipulatingtheheader
andbodyoftherequestandtheresponse,andthenfreeduponconnectionrelease.Itisveryimportantto
notethatnginxtriestoavoidcopyingdatainmemoryasmuchaspossibleandmostofthedataispassed
alongbypointervalues,notbycalling memcpy.
Goingabitdeeper,whentheresponseisgeneratedbyamodule,theretrievedcontentisputinamemory
bufferwhichisthenaddedtoabufferchainlink.Subsequentprocessingworkswiththisbufferchainlinkas
well.Bufferchainsarequitecomplicatedinnginxbecausethereareseveralprocessingscenarioswhichdiffer
dependingonthemoduletype.Forinstance,itcanbequitetrickytomanagethebufferspreciselywhile
implementingabodyfiltermodule.Suchamodulecanonlyoperateononebuffer(chainlink)atatimeandit
mustdecidewhethertooverwritetheinputbuffer,replacethebufferwithanewlyallocatedbuffer,orinserta
newbufferbeforeorafterthebufferinquestion.Tocomplicatethings,sometimesamodulewillreceive
severalbufferssothatithasanincompletebufferchainthatitmustoperateon.However,atthistimenginx
http://www.aosabook.org/en/nginx.html

10/11

11/20/2014

The Architecture of Open Source Applications (Volume 2): nginx

providesonlyalowlevelAPIformanipulatingbufferchains,sobeforedoinganyactualimplementationa
thirdpartymoduledevelopershouldbecomereallyfluentwiththisarcanepartofnginx.
Anoteontheaboveapproachisthattherearememorybuffersallocatedfortheentirelifeofaconnection,
thusforlonglivedconnectionssomeextramemoryiskept.Atthesametime,onanidlekeepaliveconnection,
nginxspendsjust550bytesofmemory.Apossibleoptimizationforfuturereleasesofnginxwouldbetoreuse
andsharememorybuffersforlonglivedconnections.
Thetaskofmanagingmemoryallocationisdonebythenginxpoolallocator.Sharedmemoryareasareused
toacceptmutex,cachemetadata,theSSLsessioncacheandtheinformationassociatedwithbandwidth
policingandmanagement(limits).Thereisaslaballocatorimplementedinnginxtomanagesharedmemory
allocation.Toallowsimultaneoussafeuseofsharedmemory,anumberoflockingmechanismsareavailable
(mutexesandsemaphores).Inordertoorganizecomplexdatastructures,nginxalsoprovidesaredblacktree
implementation.Redblacktreesareusedtokeepcachemetadatainsharedmemory,tracknonregex
locationdefinitionsandforacoupleofothertasks.
Unfortunately,alloftheabovewasneverdescribedinaconsistentandsimplemanner,makingthejobof
developingthirdpartyextensionsfornginxquitecomplicated.Althoughsomegooddocumentsonnginx
internalsexistforinstance,thoseproducedbyEvanMillersuchdocumentsrequiredahugereverse
engineeringeffort,andtheimplementationofnginxmodulesisstillablackartformany.
Despitecertaindifficultiesassociatedwiththirdpartymoduledevelopment,thenginxusercommunityrecently
sawalotofusefulthirdpartymodules.Thereis,forinstance,anembeddedLuainterpretermodulefornginx,
additionalmodulesforloadbalancing,fullWebDAVsupport,advancedcachecontrolandotherinteresting
thirdpartyworkthattheauthorsofthischapterencourageandwillsupportinthefuture.

14.5.LessonsLearned
WhenIgorSysoevstartedtowritenginx,mostofthesoftwareenablingtheInternetalreadyexisted,andthe
architectureofsuchsoftwaretypicallyfolloweddefinitionsoflegacyserverandnetworkhardware,operating
systems,andoldInternetarchitectureingeneral.However,thisdidn'tpreventIgorfromthinkinghemightbe
abletoimprovethingsinthewebserversarea.So,whilethefirstlessonmightseemobvious,itisthis:thereis
alwaysroomforimprovement.
Withtheideaofbetterwebsoftwareinmind,Igorspentalotoftimedevelopingtheinitialcodestructureand
studyingdifferentwaysofoptimizingthecodeforavarietyofoperatingsystems.Tenyearslaterheis
developingaprototypeofnginxversion2.0,takingintoaccounttheyearsofactivedevelopmentonversion1.
Itisclearthattheinitialprototypeofanewarchitecture,andtheinitialcodestructure,arevitallyimportantfor
thefutureofasoftwareproduct.
Anotherpointworthmentioningisthatdevelopmentshouldbefocused.TheWindowsversionofnginxis
probablyagoodexampleofhowitisworthavoidingthedilutionofdevelopmenteffortsonsomethingthatis
neitherthedeveloper'scorecompetenceorthetargetapplication.Itisequallyapplicabletotherewriteengine
thatappearedduringseveralattemptstoenhancenginxwithmorefeaturesforbackwardcompatibilitywith
theexistinglegacysetups.
Lastbutnotleast,itisworthmentioningthatdespitethefactthatthenginxdevelopercommunityisnotvery
large,thirdpartymodulesandextensionsfornginxhavealwaysbeenaveryimportantpartofitspopularity.
TheworkdonebyEvanMiller,PiotrSikora,ValeryKholodkov,ZhangYichun(agentzh)andothertalented
softwareengineershasbeenmuchappreciatedbythenginxusercommunityanditsoriginaldevelopers.

http://www.aosabook.org/en/nginx.html

11/11