Download - Claire protorpc
Intro to claire-protorpc@fanyu83
What is claire-protorpc
● protobuf-based RPC library● non-blocking● event-driving● multi-core ready● modern C++● for x86-64 Linux● BSD license
Tutorial: echo
● In computer telecommunications, echo is the display or return of sent data at or to the sending end of a transmission. ------ from wikipedia
● Use claire-protorpc develop application and services should do:○ define a .proto file which declared the rpc message
structure and methods○ use generated stub to call echo methods through
RpcChannel○ implement echo services, and register it to RpcServer
Echo Client/Server
Stubs
RpcChannel
HttpClient
Services
RpcServer
HttpServer
HttpConnection
echo.proto
● echo.proto define message and service.
● Use protoc compile the echo.proto○ generate echo.pb.h
and echo.pb.cc
// protoc --plugin=protoc-rpc-gen --rpc_out . echo.proto
package echo;
option cc_generic_services = true;
message EchoRequest {
required string str = 1;
}
message EchoResponse {
required string str = 1;
}
service EchoService {
rpc Echo (EchoRequest) returns (EchoResponse);
}
Echo client
● echo.pb.h define Stub which can call Echo service
● It need RpcChannel to communicate remote server.
int main(int argc, char* argv[])
{
EventLoop loop;
InetAddress server_address(argv[1], 8080);
RpcChannel channel(&loop);
channel.Connect(server_address);
echo::EchoService::Stub stub(&channel);
RpcControllerPtr controller(new RpcController());
echo::EchoRequest request;
request.set_str("0123456789ABCDEF");
stub.Echo(controller, request, boost::bind(&replied, _1, _2));
loop.loop();
}
Echo server
● echo.pb.h define Abstract interface EchoService, server side need implement it.
● EchoService need register to RpcServer.
int main(int argc, char* argv[])
{
::google::ParseCommandLineFlags(&argc, &argv, true);
InitClaireLogging(argv[0]);
EventLoop loop;
InetAddress listen_address(8080);
echo::EchoServiceImpl impl;
RpcServer server(&loop, listen_address);
server.set_num_threads(FLAGS_num_threads);
server.RegisterService(&impl);
server.Start();
loop.loop();
}
Benefits of claire-protorpc
● claire-protorpc provide:○ Define a service only need a function.○ Generate stub to call remote service.○ Automatic message encode/decode.○ Checksum for each message.○ Compress data if user set.○ Very good performance, at least no obvious
bottleneck.
Concept● protobuf provide server declaration but no implementation, it suggest
“provide code generator plugins which generate code specific to the particular RPC implementation.”
● claire-protorpc is one implementation of protobuf-based rpc, and supply itself plugin to generate code for claire-protorpc only.
● It implement 3 important concept of protobuf rpc:○ RpcController○ RpcChannel○ RpcServer
● But we do not use Google defined interface directly, instead claire-version!
Service● Services themselves are abstract interfaces (implemented either by servers
or as stubs), but they subclass this base interface. The methods of this interface can be used to call the methods of the Service without knowing its exact type at compile time (analogous to Reflection).
● claire-protorpc use protoc-rpc-gen to generate the rpc service which inherited from Service class(claire-protorpc’s version)
RpcController
● “An RpcController mediates a single method call. The primary purpose of the controller is to provide a way to manipulate settings specific to the RPC implementation and to find out about RPC-level errors.”
● Now it used to declare caller specific option and get error information.
RpcChannel
● “An RpcChannel represents a communication line to a Service which can be used to call that Service's methods. The Service may be running on another machine. Normally, you should not call an RpcChannel directly, but instead construct a stub Service wrapping it.”
● Now it used to communicate to RpcServer, it do message encode/decode, health detection, compress/uncompress,loadbalance, address resolverm, .etc .
RpcServer
● RpcServer used to store all registered service, and response to all request.○ It also provide monitor, debug, profile, flags
management features, user no need write one line code to use these
Wire Formatm
message RpcMessage {
required MessageType type = 1;
required fixed64 id = 2;
optional string service = 3;
optional string method = 4;
optional bytes request = 5;
optional bytes response = 6;
optional ErrorCode error = 7;
optional string reason = 8;
optional CompressType compress_type = 9;
}
meesage length
checksum
RpcMessage
4 bytes
4 bytes
Architecture
Logging
Symbolizer
EventLoop
Thread
Metrics
Time
String
File
protobuf
tcmalloc
profiler
snappy
ctemplate
rapidjson
boost
gflagsc-ares
RpcController
gtest
RpcChannel
ProtobufIO
RpcServer
System
BuiltInService
HttpConnection
TcpConnection
HttpClient
TcpClient
HttpServer
TcpServer
Resolver
Inspector
protoc-rpc-gen
gen-assets
claire-protorpc
claire-netty
claire-common
external
LoadBalancer
Features
● Health Detection● LoadBalancer(Random, .etc)● Address Resolver(Dns, .etc)● Compress transport● Flags view/modify● Internal statistics view/collection● Online profile● Methods form for test/debug● All built-on Http
/flags
● claire use gflags manage configures.● Through /flags, user can view and modify flags.● Easy for modify flags of a lot of machines through /flags
post method.
/flags
Metrics
● claire-protorpc supply powerful metrics type:○ counter
■ A counter is a value that never decreases. Examples might be "sent bytes" or "receied bytes". You just increment the counter each time a countable event happens, and graphing utilities usually graph the deltas over time.
○ histogram■ A metric is tracked via distribution, and is usually used for
timings.● claire-protorpc support graphing pages & json interface together
/couters
/histograms
/form
● /form page show all register services & method on running server, user can post form to server as post protobuf message
● Easy for test and debug
/form
/pprof
● claire-protorpc integrate with gperftools, so it support online profiling.○ support /pprof/profile, /pprof/heap, /pprof/growth○ also support profile without binary file, through
/pprof/symbol & /pprof cmdlind○ will support /pprof/contension later
/pprof
Question
github.com/robbinfan/claire-protorpc