high performance rpc with finagle
TRANSCRIPT
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
High Performance RPC with
Sam BESSALAH @samklr
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
RPC Redux
• Control of clients and servers• Need for predictable performance • Communication between different languages• Isolated computations• Fan out, Fan In distributed systems• ...
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Open sourced by Twitter in 2011 to solve this :
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
AsynchronousNon-Blocking
Protocol Agnostic Full Stack RPCbuilt on top of
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scalahttp://monkey.org/~marius/funsrv.pdf
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Load Balancing : Heap BalancerClient Backoff Strategy
BackPressureFailure Detection
Failover/RetryConnection PoolingTracing (via ZipKin)
...
Production ready distributed toolbox
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
• Box• Foursquare• Nest• Uber• Pinterest• NY Times• Rdio• PagerDuty• Soundcloud• Strava• StumbleUpon• ...
Known Users ...
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
@nivdul
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Networking Layer
@nivdul
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Building Blocks
Futures
Services
Filters
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Futures
• Concurrency primitive in Finagle• Different from scala.concurrent.Future and
java.util.concurrent.Future• Less context-switching, easier cancelability• From com.twitter.util.Future• Composables• Error and Exception Handling
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Futures Combinators
• Can be composed with map, flatMap, handle and rescue• Exception handling via:
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Service
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Servicesclass DummyService extends Service[httpx.Request, httpx.Response] {
def apply(req: httpx.Request): Future[httpx.Response] = { val response = req.response response.setStatusCode(200) response.setContentTypeJson() response.write(requestHandler(req.getContentString())) Future.value(response)
}
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
class Filter[-ReqIn, +RepOut, +ReqOut, -RepIn] extends ((ReqIn, Service[ReqOut, RepIn]) => Future[RepOut])
or type Filter[Req, Rep] = (Req, Service[Req, Rep]) => Future[Rep]
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
• Wrap services to modify inputs or outputs (or both) …• Protocol and Network agnostic• Composable
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
val myTimeoutFilter = new TimeoutFilter[String,Int](1.second,DefaultTimer.twitter)
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
val myTimeoutFilter = new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val myService = new DummyService
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
val myTimeoutFilter = new TimeoutFilter[String, Int](1.second,DefaultTimer.twitter)
val myService = new DummyService
val server = myTimeOutFilter andThen myService
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filtersval myTimeoutFilter =
new TimeoutFilter[String,Int]( 1.second, DefaultTimer.twitter)
val myService = new DummyService
val service = myService andThen myTimeOutFilter
val server = Httpx.serve( “:8080”, service)
Await.ready(server)
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Filters
Can be used for:
• Retries• Logging• Serialization• Authentication• Retries• Timeouts• ...
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Clients
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Clients
Finagle client are just full stack services
val myClient: Service[Request, Response] = Httpx.newService("rtb-bidder.com:8081")
val bidder = Httpx.serve(":8080", myClient)
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle in a nutshell
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Protocols
• Thrift• Memcache• Redis• HTTP• SMTP• MySQL• Zookeeper• Mux
Protocol.newService( … )
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle Mux
• RPC Protocol Multiplexing • One network connection per client-server session
• Maximization of available bandwidth without incurring the cost of opening additional sockets
• Elimination of head-of-line blocking
• Explicit queue management
• Pure session layer (OSI )
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle Serial
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle Serial
Use binary format. Scodec, Protobuf, Thrift, Avro, etc.
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Service discovery• Finagle ServerSets via ”Names”• Uses Zookeeper as Name server
val client = Http.newService(
“zk!myzkhost.home.com:2181!/my/zk/path” )
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Twitter-Server
• Template for servers at Twitter built on top of Finagle• Flags for configuration• Logging and Metrics• Admin HTTP UI• Lifecycle management endpoints, handy for Docker and/or Mesos
• Tracing via ZipKin
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle JavaService<Request, Response> service = new Service<Request, Response>() { public Future<Response> apply(Request request) { Response response = Response.apply(); response.setContent(ChannelBuffers.wrappedBuffer("yo".getBytes())); Future<Response> future = Future.value(response); return future; }};
ServerBuilder.safeBuild( service, ServerBuilder.get() .codec(Http.get()) .bindTo(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0)) .name("HttpServer")); }
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finatra• Library to easily build API Services on top of the Twitter Stack (Finagle,
Twitter-Server, Twitter-util, etc.)
• Supports Http and Thrift services
• Brings Sinatra like Style to Finagle for Rest Services
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finatra
class HelloWorldController extends Controller { get("/hi") { request: Request => "Hello " + request.params.getOrElse("name", "unnamed") }
post("/hi") { hiRequest: HiRequest => "Hello " + hiRequest.name + " with id " + hiRequest.id }
}
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finatraobject HelloWorldServerMain extends HelloWorldServer
class HelloWorldServer extends HttpServer { override def modules = Seq(Slf4jBridgeModule)
override def configureHttp(router: HttpRouter) { router .filter[LoggingMDCFilter[Request, Response]] .filter[TraceIdMDCFilter[Request, Response]] .filter[CommonFilters] .add[HelloWorldController] }}
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
● Thin layer of purely functional basic blocks, to build composable REST APIs.
● Built on top of Finagle● Functional wrapper of Finagle-http● Performance close to Bare Metal Finagle
http://github.com/finagle/finch
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Zipkin
• Zipkin is a distributed tracing system that helps us gather timing data for all the disparate services at Twitter. It manages both the collection and lookup of this data through a Collector and a Query service.
• Implementation of Google’s Dapper paper
• Moved to OpenZipkin
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
ZipKin - Open ZipKin
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Finagle : Cons
• Stuck to old versions of Netty. Currently Netty 3 and now moving to 4• API Changes since 6.x. No more Netty API Leaking through Finagle’s• Mismatch between scala.concurrent.Future and com.twitter.util.Future• Slow to update• Documentation not exhaustive. Thanks for gitter.im and the Mailing List• Can be painful to plug to other Metrics and Monitoring systems. Hopefully
it’s changing
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Quick Demo
@YourTwitterHandle#Devoxx #YourTag @samklr#devoxx #finagle #scala
Conclusion