服务框架: thrift & pastescript
DESCRIPTION
TRANSCRIPT
服务框架:Thrift & PasteScript
BPUG, Sep, 2010
为什么要服务?库不好吗?
云就是服务
• IaaS - Infrastructure as a Service
• PaaS - Platform as a Service
• SaaS - Software as a Service
适合服务的场景
• 分开部署• 独立维护• 错误隔离• 资源节省• 跨语言/跨平台
Service框架让常规的事情简单,让特殊的事情可能
HTTP + web.py
import web
urls = ('/add', 'add')
class add: def GET(self): i = web.input() r = int(i.a) + int(i.b) return str(r)
app = web.application(urls, globals)
if __name__ == '__main__': app.run()
import urllib
def add(a, b): f = urllib.urlopen('http://calculator.services.douban.com/add?a=%s&b=%s' % (a, b)) r = f.read() return int(r)
问题
• 纠结于格式转换• JSON有帮助,但有限
• HTTP和文本协议的overhead过重
Thrifthttp://incubator.apache.org/thrift/
Thrift
• 跨语言• C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, Smalltalk, OCaml
• 可定义数据结构• struct, list, set, map
• 二进制传输协议• 支持oneway calling
• 自带RPC框架实现
Thrift Example
# Make an objectup = UserProfile(uid=1, name="Mark Slee", blurb="I'll find something to put here.")
# Talk to a server via TCP sockets, using a binary protocoltransport = TSocket.TSocket("localhost", 9090)transport.open()protocol = TBinaryProtocol.TBinaryProtocol(transport)
# Use the service we already definedservice = UserStorage.Client(protocol)service.store(up)
# Retrieve something as wellup2 = service.retrieve(2)
class UserStorageHandler : virtual public UserStorageIf { public: UserStorageHandler() { // Your initialization goes here }
void store(const UserProfile& user) { // Your implementation goes here printf("store\n"); }
void retrieve(UserProfile& _return, const int32_t uid) { // Your implementation goes here printf("retrieve\n"); }};
int main(int argc, char **argv) { int port = 9090; shared_ptr<UserStorageHandler> handler(new UserStorageHandler()); shared_ptr<TProcessor> processor(new UserStorageProcessor(handler)); shared_ptr<TServerTransport> serverTransport(new TServerSocket(port)); shared_ptr<TTransportFactory> transportFactory(new TBufferedTransportFactory()); shared_ptr<TProtocolFactory> protocolFactory(new TBinaryProtocolFactory()); TSimpleServer server(processor, serverTransport, transportFactory, protocolFactory); server.serve(); return 0;}
struct UserProfile { 1: i32 uid, 2: string name, 3: string blurb}service UserStorage { void store(1: UserProfile user), UserProfile retrieve(1: i32 uid)}
Thrift通信架构
Transport(socket)
Protocol(binary)
Handler(app code)
Processor(dispatcher)
thrift package generated code
Thrift的问题
• 没有统一的部署方式• 调用代码复杂• 开发调试不方便
我们需要
• 快捷的创建项目方法• 方便的开发调试环境• 统一的部署方式
PasteScripthttp://pythonpaste.org/script/
Paste
• WSGI工具集
• 三部分组成• Paste
• PasteScript
• PasteDeploy
PasteScript
• paster serve
• templates
paster serve
• paster serve --reload --monitor-restart development.ini
[server:main]use = egg:Paste#httphost = 0.0.0.0port = 5000
[app:main]use = egg:MyWSGIApp
# Logging configuration[loggers]keys = root, access_log...
Thrift通信架构
Transport(socket)
Protocol(binary)
Handler(app code)
Processor(dispatcher)
server app
paster serve
• paster serve --reload --monitor-restart development.ini
[server:main]use = egg:DoubanService#thread_poolport = 9090pool_size = 10
[app:main]use = egg:CalculatorServer
# Logging configuration[loggers]keys = root, access_log...
setuptools entry points
entry_points = """[paste.server_runner]thread_pool = doubanservice.server:thread_pool_server_runner"""
DoubanService/setup.py:
entry_points = """[paste.app_factory]main = calculator_server.makeapp:make_app"""
CalculatorServer/setup.py:
app
from .gen.calculator import Iface, Processorfrom .app import Handler
def make_app(global_config, **local_conf): handler = Handler() processor = Processor(handler) return processor
server runnerdef thread_pool_server_runner(app, global_conf, **kwargs): for name in ['port', 'pool_size']: if name in kwargs: kwargs[name] = int(kwargs[name]) pool_size = kwargs.pop('pool_size') host = kwargs.pop('host', '0.0.0.0') transport = TSocket.TServerSocket(**kwargs) transport.host = host tfactory = TTransport.TBufferedTransportFactory() pfactory = TBinaryProtocol.TBinaryProtocolFactory() server = ThreadPoolServer(app, transport, tfactory, pfactory) if pool_size: server.threads = pool_size
server.serve()
我们得到了什么?
• 启动服务的命令 (paster serve)
• 可选的服务器 ([server:main] use)
• 可配置的服务器和应用参数• 代码更改会自动重启服务 (--reload)
• 可配置的日志 ([loggers])
Paste Template
entry_points = """[paste.server_runner]thread_pool = doubanservice.server:thread_pool_server_runner
[paste.paster_create_template]doubanservice = doubanservice.templates:DoubanServiceTemplatedoubanservice_server_py = doubanservice.templates:ServerPyTemplatedoubanservice_client_py = doubanservice.templates:ClientPyTemplate"""
DoubanService/setup.py:
paster create -t doubanservice calculator
doubanservice template
doubanservice/templates/doubanservice/├── +package+.thrift_tmpl├── create-client_tmpl├── create-server_tmpl└── gen_tmpl
vi calculator.thrift./create-server py./create-client py./gen
class DoubanServiceTemplate(paste.script.templates.Template): _template_dir = 'doubanservice' summary = "A DoubanService project"
def post(self, command, output_dir, vars): for filename in ['gen', 'create-server', 'create-client']: os.chmod(os.path.join(output_dir, filename), 0755)
server_py templatedoubanservice/templates/server-py/├── +package+│ ├── __init__.py│ ├── app.py_tmpl│ └── makeapp.py_tmpl├── development.ini_tmpl├── gen_tmpl├── production.ini_tmpl├── remote_tmpl├── scripts│ └── censor-server-py├── serve├── setup.cfg├── setup.py_tmpl└── tests
vi calculator_server/app.py./serve./remote add 1 2
client_py templatedoubanservice/templates/client-py├── +package+│ ├── __init__.py│ └── client.py_tmpl├── gen_tmpl├── setup.py_tmpl└── tests
from calculator_client import calculatorprint calculator.add(1, 2)
class Client(BaseClient): __metaclass__ = ClientMetaClass service_name = '${service}' thrift_module = ${service}_genmod port = 9090 connect_timeout = 1000 read_timeout = 5000
${service} = Client()
Demoa calculate service
广告时间A New OpenSource Project...
OneRingBuild Desktop Applications Using Web Technology
http://code.google.com/p/onering-desktop/
目标
• 用Web技术做桌面应用
• 使用HTML5+CSS3提供用户界面/本地存储
• 使用Javascript编写界面逻辑/业务逻辑
• 用AJAX方式调用本地代码
• 跨操作系统
• Qt (mac, windows, linux)
• Cairo (windows, linux)
• WebKit Framework (mac)
• 跨语言
• C API DLL
• Python - WSGI
#!/usr/bin/env pythonimport jsonimport webimport onering
urls = ( '/init', 'init', '/', 'index',)
class init: def GET(self): web.header('Content-Type', 'application/json') return json.dumps({'width': 400, 'height': 300, 'url': '/'})
class index: def GET(self): web.header('Content-Type', 'text/html') return """<html><head><script type="text/javascript" src="onering://onering/onering.js"></script></head><body><p>Hello, world!</p><button onclick="javascript:ONERING.exit()">Exit</button></body></html>"""
app = web.application(urls, globals())
if __name__ == '__main__': onering.register_wsgi_app("demo", app.wsgifunc()) onering.loop("demo")
Join Us If You Know...• C++ / C / Objective-C / Python
• Qt / win32 / Cocoa / GTK
• WebKit / Gecko
• Javascript / HTML5
• Python C API / py2exe / py2app
• 其他语言的C绑定和发布
• Win/Mac/Linux 下的动态库发布和软件发布
• 软件瘦身
你将得到
• 一个可能会被千万人使用的项目• 推动HTML5应用大潮
• 加入豆瓣的机会
ThanksQ & A