netty notes part 3 - channel pipeline and eventloops
TRANSCRIPT
NOTES ON NETTY PART 3
RICK HIGHTOWER’S
CHANNEL PIPELINES, AND EVENT LOOPS
About Rick hightowerAbout Rick
• Implemented Microservices, Vert.x/Netty at massive scale
• Author of QBit, microservices lib and Boon, Json parser and utility lib
• Founder of Mammatus Technology
• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare
Great book on Netty!
https://www.manning.com/books/netty-in-action
Great talk about Netty Best Practices given at Facebook,then Twitter University
https://goo.gl/LbXheq
http://www.infoq.com/presentations/apple-netty
Great talk about Netty at scale
Previous slide deckPrevious SLIDE DECK
• Notes on Netty Basics Slideshare
• Part 1: http://www.slideshare.net/richardhightower/notes-on-netty-baics
• Part 2: http://www.slideshare.net/richardhightower/netty-notes-part-2-transports-and-buffers
• Notes on Netty Basics Google slides
• Part 1: https://goo.gl/aUGm2N
• Part 2: https://goo.gl/xZZhVs
CHANNEL PIPELINES
Chaining channel handlersChannelPipeline
• Channel - Socket
• ByteBuf - container for bytes of message
• ChannelHandler process / transform messages
• ChannelPipeline - forms chain of ChannelHandlers
Lifecycle of a channel Channel Lifecycle states
• Active
• connected to remote peer
• Inactive
• disconnected from remote peer
• Unregistered
• created
• Registered
• channel associated with event loop
methods on ChannelHandler for lifecycle eventChannelHandler Lifecycle Events
• handlerAdded()
• added to ChannelPipeline
• handlerRemoved()
• removed from ChannelPipeline
• exceptionCaught()
• error during ChannelPipeline processing
event notification event methods for ChannelInboundHanderChannelInboundHandler lifecycle methods for
Channel
• channelRegistered() - Channel married to an event loop
• channelUnregistered() - Channel divorced from event loop
• channelActive() - connected
• channelInactive() - disconnected
• channelReadComplete() - read operation completed
• channelRead() - data is read on channel
• channelWritabilityChanged() - outgoing IO buffer/limit is met or not
• userEventTriggered() - someone passed a POJO to the event loop
Let it go. Let it go. Can't hold it back anymoreReleasing messages
• Not a good idea to override read unless you know what you are doing
• you must free up pooled resources calling ReferenceCountUtil.release(message)
• Use SimpleChannelInboundHandler and override channelRead0() instead, netty will free up message resource for you
java -Dio.netty.leakDetectionLevel=ADVANCED
You can leak on write or read
Handler for outbound ChannelOutboundHandler
• Outbound operations
• methods invoked by
• Channel, ChannelPipeline, and ChannelHandlerContext.
• Can defer operation or event
• Powerful level of control
• Many Methods take a ChannelPromise which extends ChannelFuture and provides ability to mark as succeeded or failed (setSuccess(), setFailure())
• You should mark promise failed or succeeded and also release resources
l ifecycle event methodsChannelOutboundHandler
• bind(channelHandlerContext, localSocketAddress, channelPromise)
• listen to address for connections request event
• connect(channelHandlerContext, socketAddress, channelPromise)
• connect to remote peer request event
• disconnect(channelHandlerContext, channelPromise)
• disconnect from remote peer request event
• close(channelHandlerContext, channelPromise)
• close channel request event
l ifecycle event methodsChannelOutboundHandler
• deregister(channelHandlerContext, channelPromise)
• removed from event loop request event
• read(channelHandlerContext)
• read more data from channel request event
• flush(channelHandlerContext)
• flush data to remote peer request event
• write(channelHandlerContext, message:Object, channelPromise)
• write data to channel request event
chain of channel handlersChannelPipeline
• ChannelPipeline is chain of ChannelHandlers
• handlers intercept inbound and outbound events through Channel
• ChannelHandlers process application IO data
• Channel processes IO events
• New Channel assigned to new ChannelPipeline
• relationship is permanent
• no cheating! Channel can’t attach to another ChannelPipeline
• Channel can’t leave ChannelPipeline
• Direction determines if event handled by ChannelInboundHandler or a ChannelOutboundHandler
• Unhanded events go to next Channel in chain
ChannelHandlers can modify ChannelPipelineChannelPipeline can be edited
• ChannelHandler methods to change ChannelPipeline
• addFirst()
• addBefore()
• addAfter()
• addLast()
• remove()
• replace()
Used by Netty to fire events inboundChannelPipeline methods
• fireChannelRegistered
• fireChannelUnregistered
• fireChannelActive
• fireChannelInactive
• fireExceptionCaught
• fireUserEventTriggered - user defined event so you can pass a POJO to the channel
• fireChannelRead
• fireChannelReadComplete
Outbound methodsChannelPipeline methods
• bind - listen to a port, binds channel to port/address
• connect - connect to a remote address
• disconnect - disconnect from a remote address
• close - close the channel after next channel handler is called
• deregister
• flush
• write
• writeAndFlush
• read
Managing passing the events to next handler in chainChannelHandlerContext
• Associates ChannelHandler and ChannelPipeline
• Created when ChannelHandler added to a ChannelPipeline
• Manages interaction of its ChannelHandler to others in ChannelPipeline
• Use context instead of pipeline (most often) as it involves a shorter event flow
• otherwise must go through whole chain from start
MethodsChannelHandlerContext
• channel - returns the Channel
• bind, close, connect, disconnect
• read, write
• deregister
• executor - returns EventExecutor (has thread pool scheduling interface)
• fireChannelActive, fireChannelInactive, fireChannelRead, fireChannelReadComplete
• handler - returns corresponding handler
• isRemoved - has the handler been removed?
• name - name of the instance
• pipeline - returns the associated pipeline
In Bound Exception handling
public class MyInboundExceptionHandler
extends ChannelInboundHandlerAdapter {
…
@Override
public void exceptionCaught(ChannelHandlerContext
channelHandlerContext,
Throwable cause) {
logger.error(cause);
channelHandlerContext.close();
}
}
Outbound Exception handling
public class MyHandler extends ChannelOutboundHandlerAdapter {
@Override
public void write( final ChannelHandlerContext channelHandlerContext,
final Object message,
final ChannelPromise channelPromise) {
channelPromise.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(final ChannelFuture channelFuture) {
if (!channelFuture.isSuccess()) {
logger.error(channelFuture.cause());
channelFuture.channel().close();
}
}
});
}
}
EVENT LOOPS
Threading model specificsEvent Loop And Threading model
• Threading model specifies is key to understanding Netty
• When threads are spawned is very important to your application code
• You have to understand trade-offs and specifics of Netty
• Multi cores are common occurrence
• Netty uses Executor API interfaces to schedule EventLoop tasks
• Netty tries to reduce cost of thread hand off and CPU cache line moving about by limiting the #of threads to handle IO
• Less is more by reducing “context switching"
• Increases CPU Register, L1, L2, cache hits by keeping things in same thread
• Reduce wake up cost of threads and synchronization of shared variables
Tasks can be submitted to EventLoopEventLoop tasks
• EventLoop indirectly extends Java’s ScheduledExecutorService
• Events and Tasks are executed in order received
• You can use ScheduledExecutorService methods to schedule a task for later execution
• Tasks will run in same thread as IO so no sync needed
Use event loop to schedule tasksEventLoop task schedule
ScheduledFuture<?> future =
channel.eventLoop().scheduleAtFixedRate(...);
Netty does task management to make sure only one thread can handle IO
Ensure Channel is handled by ONE Thread
• Any Thread can call methods on a Channel
• No synchronization in is needed
• How?
• If you call a method on a Channel and its not from the IO Thread (EventLoop thread)
• Netty will schedule a task for that method to be run and the task will run in EventLoop thread
• “Netty’s threading model hinges on determining the identity of the currently executing Thread; that is, whether or not it is the one assigned to the current Channel and its EventLoop.” (Netty in Action)
• If method call is from EventLoop Thread, then method is executed right away
• Each EventLoop has a task queue
• task queue is not shared
• Do not put long running tasks in the task queue of an event loop
• If long running use another thread pool (not Netty) to execute and then call channel when done
Relationship for NIOFor NIO
• An EventLoop manages many Channels
• Channels can be sockets/connections/clients
• If you block in one, you block all clients
• Do not block
THANKS AND BUY NORMAN’S BOOK
Netty in ActionIdeas for slides
• Many ideas for slides are directly derived from Netty in Action book by Norman et al and/or his talks on Netty
• BUY THE BOOK Netty in Action!
• The slides are a study aid for myself so I can better learn Netty
• I’ve worked with ByteBuffer, NIO, Vert.x, and have often wanted to use parts of Netty on projects but lacked the knowledge of Netty internals so used NIO or ByteBuffer or OIO when I really wanted to use Netty instead
Previous slide deckPrevious SLIDE DECK
• Notes on Netty Basics Slideshare
• Part 1: http://www.slideshare.net/richardhightower/notes-on-netty-baics
• Part 2: http://www.slideshare.net/richardhightower/netty-notes-part-2-transports-and-buffers
• Notes on Netty Basics Google slides
• Part 1: https://goo.gl/aUGm2N
• Part 2: https://goo.gl/xZZhVs
About Rick hightowerAbout Rick
• Implemented Microservices, Vert.x/Netty at massive scale
• Author of QBit, microservices lib and Boon, Json parser and utility lib
• Founder of Mammatus Technology
• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare
The end… sleepless dev.