no#callbacks,#no#threads#&ruby#1.9# - …assets.en.oreilly.com/1/event/40/no callbacks, no...

No Callbacks, No Threads & Ruby 1.9 @igrigorik h;p://bit.ly/ruby-‐stack

No Callbacks, No Threads & Ruby 1.9

Ilya Grigorik @igrigorik

async & co-‐opera-ve web-‐servers


The slides… Twi+er My blog


The state of art is not good enough. (we’ve been stuck in the same local minima for several years)


require "active_record”

ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)

threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end

threads.each { |t| t.join }

The “Experiment” vanilla everything…


require "active_record”

ActiveRecord::Base.establish_connection( :adapter => "mysql", :username => "root", :database => "database", :pool => 5)

threads = []10.times do |n| threads << Thread.new { ActiveRecord::Base.connection_pool.with_connection do |conn| res = conn.execute("select sleep(1)") end }end

threads.each { |t| t.join }

# time ruby activerecord-pool.rb## real 0m10.663s# user 0m0.405s# sys 0m0.201s

5 shared connecJons


BHP

WHP

% power loss


> 50% power loss!?


Drivers

Ruby VM

Network

MySQL Mongo

PSQL Couch …

Mongrel Passenger

Unicorn

…

Threads

Fibers GIL …


Drivers

Ruby VM

Network

MySQL Mongo

PSQL Couch …

Mongrel Passenger

Unicorn

…

Threads

Fibers GIL …

We’re as fast as the slowest component

1

2

3

4


Concurrency is a myth in Ruby (with a few caveats, of course)

Global Interpreter Lock is a mutual exclusion lock held by a programming language interpreter thread to avoid sharing code that is not thread-‐safe with other threads.

There is always one GIL for one interpreter process.

hMp://bit.ly/ruby-‐gil


Concurrency is a myth in Ruby s-ll no concurrency in Ruby 1.9

N-‐M thread pool in Ruby 1.9… BeMer but s-ll the same problem!



Concurrency is a myth in Ruby s-ll no concurrency in Ruby 1.9

Nick – tomorrow @ 11:45am



Avoid locking interpreter threads at all costs let’s say you’re wri-ng an extension…

Blocks enJre Ruby VM

Not as bad, but avoid it sJll..


Will fetch_xyz() block the VM? when was the last -me you asked yourself this ques-on?


mysql.gem under the hood

require 'rubygems’require 'sequel'

DB = Sequel.connect('mysql://root@localhost/test')

while true DB['select sleep(1)'].select.first end

22:10:00.218438 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.001100> 22:10:01.241679 mysql_real_query(0x02740000, "select sleep(1)", 15) = 0 <1.000812>

ltrace –MTg -‐x mysql_real_query –p ./example.rb

Blocking 1s call!

hMp://bit.ly/c3Pt3f


gem install mysql what you didn’t know…

1.  Blocking calls to mysql_real_query 2.  mysql_real_query requires an OS thread 3.  Blocking on mysql_real_query blocks the Ruby VM 4.  Aka, “select sleep(1)” blocks the enJre Ruby runJme for 1s

(ouch)


mysqlplus.gem under the hood

static VALUE async_query(int argc, VALUE* argv, VALUE obj) { ... send_query( obj, sql ); ... schedule_query( obj, timeout); ... return get_result(obj); }

static void schedule_query(VALUE obj, VALUE timeout) { ... struct timeval tv = { tv_sec: timeout, tv_usec: 0 };

for(;;){ FD_ZERO(&read); FD_SET(m->net.fd, &read);

ret = rb_thread_select(m->net.fd + 1, &read, NULL, NULL, &tv); ...

if (m->status == MYSQL_STATUS_READY) break; }}

send query and block

Ruby: select() = C: rb_thread_select()


mysqlplus.gem + ruby select

spinning in select


Step 1: Fix the drivers Many of our DB drivers don’t respect the underlying Ruby VM. Don’t blame the VM.


WEBRick

Drivers

Ruby VM

Network

Mongrel made Rails viable s-ll powers a lot of Rails apps today


Rails rediscovers Apache the worker/forker model…


An exclusive Ruby VM for EACH request am I the only one who thinks this is terrible?

…

*nix IPC is fast! Woo!

Full Ruby VM


An exclusive Ruby VM for EACH request am I the only one who thinks this is terrible?

Robustness? That sounds like a bug.

“Does not care if your applica-on is thread-‐safe or not, workers all run within their own isolated address space and only serve one client at a -me for maximum robustness.”


Step 2: consider enJre stack The driver, the web-‐server, and the network must all work together.


1.  Node imposes the full-‐stack requirements 2.  Node imposes async drivers 3.  Node imposes async frameworks

Surprise: Node is “fast”


We can ignore the performance

issues at our own peril or, we can just fix the problem


I’ll take Ruby over JS gem install eventmachine

>


EventMachine Reactor concurrency without thread

EventMachine: The Speed Demon Wednesday @ 11:45am – Aman Gupta

while true do Jmers

network_io other_io

end

p "Starting"

EM.run do p "Running in EM reactor" end

p ”won’t get here"


Non-‐blocking IO requires non-‐blocking drivers:

AMQP h+p://github.com/tmm1/amqp MySQLPlus h;p://github.com/igrigorik/em-‐mysqlplus Memcached h+p://github.com/astro/remcached DNS h+p://github.com/astro/em-‐dns Redis h+p://github.com/madsimian/em-‐redis MongoDB h+p://github.com/tmm1/rmongo HTTPRequest h+p://github.com/igrigorik/em-‐h+p-‐request WebSocket h+p://github.com/igrigorik/em-‐websocket Amazon S3 h+p://github.com/peritor/happening

And many others: h+p://wiki.github.com/eventmachine/eventmachine/protocol-‐implementaJons


em-‐mysqlplus: example async MySQL driver

EventMachine.run do conn = EventMachine::MySQL.new(:host => 'localhost') query = conn.query("select sleep(1)")

query.callback { |res| p res.all_hashes } query.errback { |res| p res.all_hashes }

puts ”executing…”end

# > ruby em-mysql-test.rb## executing…# [{"sleep(1)"=>"0"}]

gem install em-‐mysqlplus

callback fired 1s aner “execuJng”


em-‐mysqlplus: under the hood mysqlplus + reactor loop

non-‐blocking driver require 'mysqlplus'

def connect(opts) conn = connect_socket(opts) EM.watch(conn.socket, EventMachine::MySQLConnection, conn, opts, self)end

def connect_socket(opts) conn = Mysql.init

conn.real_connect(host, user, pass, db, port, socket, ...) conn.reconnect = false connend

EM.watch: reactor will poll & noJfy


em-‐mysqlplus mysqlplus + reactor loop

Features:

-‐  Maintains C-‐based mysql gem API -‐  Deferrables for every query with callback & errback -‐  ConnecJon query queue -‐ pile 'em up! -‐  Auto-‐reconnect on disconnects -‐  Auto-‐retry on deadlocks

h+p://github.com/igrigorik/em-‐mysqlplus


and this callback goes to…


We can do be;er than node.js all the benefits of evented code without the drawbacks


Ruby 1.9 Fibers and coopera-ve scheduling

Ruby 1.9 Fibers are a means of crea-ng code blocks which can be paused and resumed by our applica-on (think lightweight threads, minus the thread scheduler and less overhead).

f = Fiber.new { while true do Fiber.yield "Hi” end}

p f.resume # => Hip f.resume # => Hip f.resume # => Hi

Manual / cooperaJve scheduling!

h+p://bit.ly/d2hYw0


Ruby 1.9 Fibers and coopera-ve scheduling

h+p://bit.ly/aesXy5

Fibers vs Threads: creaJon Jme much lower

Fibers vs Threads: memory usage is much lower


Untangling Evented Code with Fibers h+p://bit.ly/d2hYw0

def query(sql) f = Fiber.current conn = EventMachine::MySQL.new(:host => 'localhost')

q = conn.query(sql)

c.callback { f.resume(conn) } c.errback { f.resume(conn) }

return Fiber.yieldend

EventMachine.run do Fiber.new {

res = query('select sleep(1)') puts "Results: #{res.fetch_row.first}"

}.resumeend

ExcepJon, async!




q = conn.query(sql)





}.resumeend

1. Wrap into a conJnuaJon




q = conn.query(sql)





}.resumeend

2. Pause the conJnuaJon




q = conn.query(sql)





}.resumeend

3. Resume the conJnuaJon


em-‐synchrony: simple evented programming best of both worlds…

Good news, you don’t even have to muck around with Fibers!

gem install em-‐synchrony

-‐  Fiber aware connecJon pool with sync/async query support -‐  MulJ request interface which accepts any callback enabled client -‐  Fibered iterator to allow concurrency control & mixing of sync / async -‐  em-‐h+p-‐request: .get, etc are synchronous, while .aget, etc are async -‐  em-‐mysqlplus: .query is synchronous, while .aquery is async -‐  remcached: .get, etc, and .mulJ_* methods are synchronous

h+p://github.com/igrigorik/em-‐synchrony



Async under the hood

require "em-synchrony/em-mysqlplus"

EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")

res = db.query("select sleep(1)") puts res

EventMachine.stopend



require "em-synchrony/em-mysqlplus"

EventMachine.synchrony do db = EventMachine::MySQL.new(host: "localhost")

res = db.query("select sleep(1)") puts res

EventMachine.stopend


Drivers

Ruby VM

Network

Fibers

EM-‐HTTP, EM-‐MySQL, EM-‐Jack, etc.

Async-‐rack

Thin

One VM, full concurrency, network-‐bound Ruby 1.9, Fibers, Thin: in producJon!

Goliath


Async Rails with EventMachine & MySQL

development: adapter: em_mysqlplus database: widgets pool: 5 timeout: 5000

git clone git://github.com/igrigorik/em-‐mysqlplus.git git checkout ac-verecord rake install

database.yml

require 'em-activerecord’require 'rack/fiber_pool'

# Run each request in a Fiberconfig.middleware.use Rack::FiberPoolconfig.threadsafe!

environment.rb


Async Rails with EventMachine & MySQL

class WidgetsController < ApplicationController def index Widget.find_by_sql("select sleep(1)") render :text => "Oh hai” endend

ab –c 5 –n 10 hMp://127.0.0.1:3000/widgets

Server Soiware: thin Server Hostname: 127.0.0.1 Server Port: 3000

Document Path: /widgets/ Document Length: 6 bytes

Concurrency Level: 5 Time taken for tests: 2.210 seconds Complete requests: 10 Failed requests: 0 Requests per second: 4.53 [#/sec] (mean)

woot! Fiber DB pool at work.


Not only is it doable… it already works.

git clone git://…./igrigorik/mysqlplus git checkout ac-verecord rake install


Ruby 1.9 + Rails 3 + new stack

=

Order of magnitude be;er performance

(aka, enough of a reason to actually switch)


What do you think?

The state of art is not good enough, in fact, it’s terrible!

hMp://bit.ly/aHzdZw

Untangling Evented Code with Ruby Fibers: h+p://www.igvita.com/2010/03/22/untangling-‐evented-‐code-‐with-‐ruby-‐fibers/

Fibers & CooperaJve Scheduling in Ruby: h+p://www.igvita.com/2009/05/13/fibers-‐cooperaJve-‐scheduling-‐in-‐ruby/

EM-‐Synchrony: h+p://github.com/igrigorik/em-‐synchrony

no#callbacks,#no#threads#&ruby#1.9# - …assets.en.oreilly.com/1/event/40/no callbacks, no...

Documents