little big ruby

Code Reading 101

TWO THINGS TO KNOW ABOUT ME

I wrote the TextMate book

My name is Jim Weirich

JAMES EDWARDGRAY II

I wrote two books for the Pragmatic Programmers: Best of Ruby Quiz and TextMate: Power Editing for the Mac

I’ve contributed documentation and patches for some standard libraries, which I now help to maintain

I built FasterCSV (now CSV), HighLine (with Greg), Elif, and a few other libraries people don’t use

I created the Ruby Quiz and ran it for the first three years

HI. I’M JAMES ANDI READ CODE.

HOW MUCH SHOULD YOU READ?

0% 25% 50% 75% 100%Novice

Advanced Beginner

Competent

Proficient

Expert

My opinion based on the Dreyfus Model of Skill Acquisition.

WHY IS CODE READING IMPORTANT?

It can show you common idioms

It’s good to practice breaking down possibly challenging code, because you will always have to work with other’s code

Understanding how something works gives you insight into any limitations it has

Seeing bad code helps you write better code

Knowledge workers always need more ideas

INTRODUCING RESTCLIENT

WHAT IS RESTCLIENT?

Sinatra’s sister library (sometimes called “reverse Sinatra”)

It provides a very clean interface to RESTful web services

Simple well-written code (around 500 lines of clear code for the core functionality)

Plus a couple of exciting features

BASIC GETReading tweets with Twitter’s API

require "rubygems"require "rest_client"require "json"

twitter = RestClient::Resource.new( "http://twitter.com/statuses", :user => "JEG2", :password => "secret" )

json = twitter["friends_timeline.json"].gettweets = JSON.parse(json)tweets.each do |tweet| # ...end

BASIC POSTPosting a tweet with Twitter’s API

require "rubygems"require "rest_client"require "json"

twitter = RestClient::Resource.new( "http://twitter.com/statuses", :user => "JEG2", :password => "secret" )

json = twitter["update.json"].post(:status => "Hello from #mwrc!")tweet = JSON.parse(json)# ...

NETWORKING CODE DONE RIGHT

def process_result(res) if res.code =~ /\A2\d{2}\z/ decode res['content-encoding'], res.body if res.body elsif %w(301 302 303).include? res.code url = res.header['Location']

if url !~ /^http/ uri = URI.parse(@url) uri.path = "/#{url}".squeeze('/') url = uri.to_s end

raise Redirect, url elsif res.code == "304" raise NotModified, res elsif res.code == "401" raise Unauthorized, res elsif res.code == "404" raise ResourceNotFound, res else raise RequestFailed, res endend

def transmit(uri, req, payload) setup_credentials(req)

net = net_http_class.new(uri.host, uri.port) net.use_ssl = uri.is_a?(URI::HTTPS) net.verify_mode = OpenSSL::SSL::VERIFY_NONE net.read_timeout = @timeout if @timeout net.open_timeout = @open_timeout if @open_timeout

display_log request_log

net.start do |http| res = http.request(req, payload) display_log response_log(res) string = process_result(res)

if string or @method == :head Response.new(string, res) else nil end endrescue EOFError raise RestClient::ServerBrokeConnectionrescue Timeout::Error raise RestClient::RequestTimeoutend

def decode(content_encoding, body) if content_encoding == 'gzip' and not body.empty? Zlib::GzipReader.new(StringIO.new(body)).read elsif content_encoding == 'deflate' Zlib::Inflate.new.inflate(body) else body endend

A RESTFUL SHELL (IN 90 LOC)

CURL-ISH REQUESTSFetching the latest tweet !om Twitter’s API

$ restclient \> get http://twitter.com/statuses/friends_timeline.json?count=1 \> JEG2 secret[{"text":"Sent out first round of Twitter client betas…", "user":{"name":"Jeremy McAnally","screen_name":"jeremymcanally",…}, …}]

RESTFUL IRBInteracting with the Twitter API

$ restclient http://twitter.com/statuses JEG2 secret>> post "update.json", :status => "The RestClient shell is fun."=> "{\"text\":\"The RestClient shell is fun.\",…}">> get "friends_timeline.json?count=1"=> "[{\"text\":\"The RestClient shell is fun.\",…}]"

LOGGING IN RUBY

GENERATING RUBYInteractively building a RESTful Ruby script

$ RESTCLIENT_LOG=twitter_fun.rb restclient …>> post "update.json", :status => "The RestClient shell is fun."=> …>> get "friends_timeline.json?count=1"=> …

# twitter_fun.rbRestClient.post "http://twitter.com/statuses/update.json", "status=The%20RestClient%20shell%20is%20fun.", :content_type=>"application/x-www-form-urlencoded"# => 200 OK | application/json 379 bytesRestClient.get "http://twitter.com/statuses/friends_timeline.json?count=1"# => 200 OK | application/json 381 bytes

FASTERCSV ISTHE NEW CSV

THE LESS BORING PARTS OF CSV

Tricky data structures are needed

Rows need ordered access for their columns

Users also like to work with them by header name

Column names often repeat

Now that we have m17n, CSV parses in the encoding of your data (no transcoding is done on your data)

THE ARRAY-HASH-WITH-DUPLICATES

DATA THING

CSV::ROWThe various ways to refer to data

require "csv" # using Ruby 1.9

data = <<END_DATAConsole,Units Sold 2007,Percent,Units Sold 2008,PercentWii,"719,141",49.4%,"1,184,651",49.6%XBox 360,"333,084",22.9%,"743,976",31.1%PlayStation 3,"404,900",27.8%,"459,777",19.3%END_DATAps3 = CSV.parse(data, :headers => true, :header_converters => :symbol)[-1]

ps3[0] # => "PlayStation 3"ps3[:percent] # => "27.8%"ps3[:percent, 3] # => "19.3%"ps3[:percent, ps3.index(:units_sold_2008)] # => "19.3%"

M17N IN ACTION

@io = if data.is_a? String then StringIO.new(data) else data end@encoding = if @io.respond_to? :internal_encoding @io.internal_encoding || @io.external_encoding elsif @io.is_a? StringIO @io.string.encoding end@encoding ||= Encoding.default_internal || Encoding.default_external

sample = read_to_char(1024)sample += read_to_char(1) if sample[-1..-1] == encode_str("\r") and not @io.eof?

if sample =~ encode_re("\r\n?|\n") @row_sep = $& breakend

def encode_re(*chunks) Regexp.new(encode_str(*chunks))end

def encode_str(*chunks) chunks.map { |chunk| chunk.encode(@encoding.name) }.joinend

OTHER POINTSOF INTEREST

The parser

Ruby 1.9’s CSV library uses primarily one ugly regular expression from Master Regular Expressions

The old FasterCSV has switched to a non-regex parser to dodge some regex engine weaknesses

FasterCSV::Table is another interesting data structure that can work in columns or rows

BJ, SLAVE,AND TERMINATOR

WHY THESE LIBRARIES?

They teach how to build multiprocessing Unix software

Covers Thread and fork(), apart and together

Using pipes

Signal handling

And much more

Robust code written by an expert

HOW TO ASK YOUR CHILD TO KILL YOU

def terminate options = {}, &block options = { :seconds => Float(options).to_i } unless Hash === options

seconds = getopt :seconds, options trap = getopt :trap, options, lambda{ eval("raise(::Terminator::Error, '#{ seconds }s')", block) }

handler = Signal.trap(signal, &trap)

plot_to_kill pid, :in => seconds, :with => signal

begin block.call ensure Signal.trap(signal, handler) end end

def plot_to_kill pid, options = {} seconds = getopt :in, options signal = getopt :with, options process.puts [pid, seconds, signal].join(' ') process.flushend

fattr :process do process = IO.popen "#{ ruby } #{ program.inspect }", 'w+' at_exit do begin Process.kill -9, process.pid rescue Object end end process.sync = true processend

fattr :program do code = <<-code while(( line = STDIN.gets )) pid, seconds, signal, *ignored = line.strip.split

pid = Float(pid).to_i seconds = Float(seconds) signal = Float(signal).to_i rescue String(signal)

sleep seconds

begin Process.kill signal, pid rescue Object end end code tmp = Tempfile.new "#{ ppid }-#{ pid }-#{ rand }" tmp.write code tmp.close tmp.path end

OTHER POINTSOF INTEREST

bj – A robust background priority queue for Rails

Noticing changes from the outside world via signals

Managing and monitoring an external job

slave – Trivial multiprocessing with built-in IPC

How to set up a “heartbeat” between processes

How to run DRb over Unix domain sockets

THE ART OFCODE READING

PROCESS TIPS

Take a deep breath and relax

Not all code sucks

Don’t start with Rails

There’s a ton of great stuff in there

But it’s a big and complex beast

Have a goal

GETTING THE CODE

gem unpack GEM_NAME

Use anonymous VCS access to pull a local copy of the code

Open it in your standard environment as you would if you were going to edit it

FINDING THINGS

Try the conventions first

Executables are probably in the bin/ directory

Look for MyModule::MyClass in lib/my_module/my_class.rb

Look for methods in super classes and mixed in modules

But remember Ruby is quite dynamic

Hunt for some “core extensions”

UNDERSTANDINGTHE CODE

Start with the tests/specs if there are any

Check for an “examples/” directory

Try to load and play with certain classes in isolation

irb -r a_class_to_play_with

Remember Ruby’s reflection methods, like methods()

QUESTIONS?About code or other important topics…