node's event loop from the inside out - sam roberts, ibm
TRANSCRIPT
The event loop fromthe inside out
Sam Roberts <[email protected]>, IBMgithub: @sam-github, twitter: @octetcloud
Weshouldbeabletoanswerthesequestions:
• Whatistheeventloop?(Hint:itsnotanEventEmitter)• Isnodesingle- ormulti-threaded?(Bonusquestion:When?)• WhyisNode.js saidto"scalewell"?
AprimerinUnixsystemprogramming
Warning:pseudo"C"codeliesahead!
Networkconnectionsuse"sockets",namedafterthesystemcallused:
int s=socket();
Socketdescriptorsareoftenreferredtoas"filedescriptors",confusingly,sincefiledescriptorsarenotnecessarilyreferencestothefilesystem.Sorry.
FiledescriptorsareO/S"objectorientation",theypointtoobjectsinthekernelwithavirtual"interface"(read/write/close/poll/etc.).
Scaleproblem:thread-per-connection
int server=socket();bind(server,80)listen(server)
while(int connection=accept(server)){pthread_create(echo,connection)
}
voidecho(int connection){charbuf[4096];while(int size=read(connection,buffer,sizeof buf)){
write(connection,buffer,size);}
}
Scalesolution:epoll - setup
int server=...//likebefore
int eventfd =epoll_create1(0);
struct epoll_event ev ={.events=EPOLLIN,.data.fd =server};
epoll_ctl(epollfd,EPOLL_CTL_ADD,server,&ev);
…
epoll isusedonLinux,kqueue onBSDsissimilar,WindowshasquiteadifferentAPI,butwiththesameendresult.
Scalesolution:epoll - loop
…struct epoll_event events[10];
//Thisis the"eventloop",eachloopcouldbecalleda"tick”oftheloop,butthenitwouldbeconfused//withprocess.nextTick().
while((int max=epoll_wait(eventfd,events,10))){…
Scalesolution:epoll – handleserversocket
…while((int max=epoll_wait(eventfd,events,10))){
for(n=0;n< max;n++){if(events[n].data.fd.fd ==server){
//Serversockethasconnection!int connection=accept(server);ev.events =EPOLLIN;ev.data.fd =connection;epoll_ctl(eventfd,EPOLL_CTL_ADD,connection,&ev);
}else…
Scalesolution:epoll – handleconnectionsocket
…while((int max=epoll_wait(eventfd,events,10))){
for(n=0;n< max;n++){…}else{
//Connectionsockethasdata!charbuf[4096];int size=read(connection,buffer,sizeof buf);write(connection,buffer,size);
}}}
Whatisthenodeeventloop?
Asemi-infiniteloop,pollingandblockingontheO/Suntilsomeinasetoffiledescriptorsareready.
Whendoesnodeexit?
Itexitswhenitnolongerhasanyeventstowaitfor,atwhichpointtheloopmustcomplete.
Note:.unref()markshandlesthatarebeingwaitedonintheloopas"notcounting"towardskeepingtheeventloopalive.
CanwepollforallNode.js events?
Yesandno.Therearebasicallythreeclassesofthings:
1. Pollable filedescriptors:canbedirectlywaitedon
2. Time:nexttimeoutcanbedirectlywaitedon
3. Everythingelse:musthappenoffloop,andsignalbacktotheloopwhendone
Pollable:sockets(net/dgram/http/tls/https/child_process pipes/stdin,out,err)
Classic,wellsupported.
Pollable:time(timeoutsandintervals)
poll(...,int timeout)kqueue(...,struct timespec*timeout)epoll_wait(...,int timeout,...)
timeout resolutionismilliseconds,timespec isnanoseconds,butbothareroundedup tosystemclockgranularity.
Onlyonetimeoutatatimecanbewaitedon,butNode.js keepsalltimeoutssorted,andsetsthetimeoutvaluetothenextone.
Notpollable:filesystem
Everythinginfs.* usestheuv threadpool(unlesstheyaresync).
Theblockingcallismadebyathread,andwhenitcompletes,readinessissignaledbacktotheeventloopusingeitheraneventfdoraself-pipe.
Aside:self-pipe
Apipe,whereoneendiswrittentobyathreadorsignalhandler,andtheotherendispolledintheloop.Traditionalwayto"wakeup"apollingloopwhentheeventtowaitforisnotdirectlyrepresentableasafiledescriptor.
Sometimespollable:dns
• dns.lookup() callsgetaddrinfo(),afunctioninthesystemresolverlibrarythatmakesblockingsocketcallsandcannotbeintegratedintoapollingloop.Likefs.*,it’scalledinthethreadpool.
• dns.<everythingelse> callsfunctionsinc-ares,anon-blockingDNSresolver,andintegrateswiththeloop,not thethreadpool.
Docsbendoverbackwardstoexplainhowthesetwodiffer,butnowthatyouknowthatblockinglibrarycallsmustbeshuntedofftothethreadpool,whereasDNSqueriesuseTCP/UDPsocketsandcanintegrateintotheeventloop,thedistinctionshouldbeclear.
ImportantnotesabouttheUVthreadpool
Itissharedby:• fs,• dns,• http.get/request() (ifcalledwithaname,dns.lookup() isused),and• anyC++addons thatuseit.
Defaultnumberofthreadsis4,significantlyparallelusersoftheaboveshouldincreasethesize.
Hints:
• ResolveDNSnamesyourself,directly,usingthedirectAPIstoavoiddns.lookup(),andstayoutofthethreadpool.• IncreasethethreadpoolsizewithUV_THREADPOOL_SIZE.
Pollable:signals
Theultimateasync...usestheself-pipepatterntowritethesignalnumbertotheloop.
Notethatlisteningforsignalsdoesn't"ref"theeventloop,whichisconsistentwithsignalusageasa"probablywon'thappen"IPCmechanism.
Pollable:childprocesses
• UnixsignalschildprocessterminationwithSIGCHLD• Pipesbetweentheparentandchildarepollable.
Sometimespollable:C++addons
Addons shouldusetheUV threadpoolorintegratewiththeloop,butcandoanything,includingmakingloop-blockingsystemcalls(perhapsunintentionally).
Hints:
• Reviewaddon code• Trackloopmetrics
Sometimespollable:C++addons
Addons shouldusetheUV threadpoolorintegratewiththeloop,butcandoanything,includingmakingloop-blockingsystemcalls(perhapsunintentionally).
Hints:
• Reviewaddon code• Trackloopmetrics
Youshouldnowbeabletodescribe:
• Whatistheeventloop• Whenisnodemulti-threaded• Whyit"scaleswell"
Thistalk,includingversionsofthepseudo"C"codeheretocompileandplaywith:
• https://goo.gl/N7oLVK orhttps://gist.github.com/sam-github/71d0ebf53a63ab7ca35a1eccc8536e47
BertBelder's talkabouttheNode.js eventloopfromahigherlevel,the"outside in":
• https://goo.gl/EIPclI orhttps://www.youtube.com/watch?v=PNa9OMajw9w&list=PLfMzBWSH11xaxRcsreXF-jB16geIJ8Foc&index=35