polyglot parallelism
TRANSCRIPT
![Page 1: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/1.jpg)
Polyglot ParallelismA Case Study in Using Erlang
and Ruby at Rackspace
![Page 2: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/2.jpg)
The ProblemPart 1
![Page 3: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/3.jpg)
20,000 network devices
![Page 4: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/4.jpg)
9 Datacenters,3 Continents
![Page 5: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/5.jpg)
devices not designed forhigh-throughput
management
![Page 6: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/6.jpg)
we need a high throughput solution
![Page 7: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/7.jpg)
the time spent in I/O is the primary
bottleneck
![Page 8: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/8.jpg)
if you want to speed things up you have to talk to more devices
in parallel
![Page 9: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/9.jpg)
The ProblemPart II
![Page 10: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/10.jpg)
huge blobs of data
![Page 11: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/11.jpg)
lots of backups equals big database
![Page 12: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/12.jpg)
ad-hoc searching is difficult but
important
![Page 13: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/13.jpg)
customer SLA means need to restore from
backup quickly
![Page 14: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/14.jpg)
an event must be generated for each device interaction
![Page 15: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/15.jpg)
migrations are problematic with that much data
![Page 16: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/16.jpg)
rigid schema made adapting to new devices difficult
![Page 17: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/17.jpg)
each device type has different properties
![Page 18: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/18.jpg)
“backup” means different things for each device type
![Page 19: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/19.jpg)
need to grow with the business
![Page 20: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/20.jpg)
Previous Solution
![Page 21: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/21.jpg)
multiple Ruby apps
![Page 22: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/22.jpg)
difficult to scale
![Page 23: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/23.jpg)
vendor device managers
![Page 24: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/24.jpg)
New Solution
![Page 25: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/25.jpg)
![Page 26: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/26.jpg)
the simplest thing that could possibly
work
![Page 27: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/27.jpg)
most db writes come from
scheduled jobs
![Page 28: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/28.jpg)
Rails
MongoDB
Erlang
ReST API
Other Clients
NetworkDevices
![Page 29: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/29.jpg)
![Page 30: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/30.jpg)
Joe Armstrong
![Page 31: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/31.jpg)
AXD301ATM Switch
![Page 32: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/32.jpg)
99.9999999%
![Page 33: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/33.jpg)
Functional
![Page 34: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/34.jpg)
Dynamically Typed
![Page 35: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/35.jpg)
Single Assignment
![Page 36: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/36.jpg)
A = 1. %=> 1A = 2. %=> badmatch
![Page 37: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/37.jpg)
[B, 2, C] = [1, 2, 3].B = 1. %=> 1C = 3. %=> 3
![Page 38: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/38.jpg)
ImmutableData
Structures
![Page 39: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/39.jpg)
D = dict:new().D1 = dict:store(foo, 1, D).D2 = dict:store(bar, 2, D1).
![Page 40: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/40.jpg)
Concurrency Oriented
![Page 41: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/41.jpg)
-module(fact).-export([fac/1]). fac(0) -> 1; fac(N) -> N * fac(N-1).
![Page 42: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/42.jpg)
-module(quicksort).-export([quicksort/1]). quicksort([]) -> [];quicksort([Pivot|Rest]) -> quicksort([Front || Front <- Rest, Front < Pivot]) ++ [Pivot] ++ quicksort([Back || Back <- Rest, Back >= Pivot]).
![Page 43: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/43.jpg)
Details
![Page 44: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/44.jpg)
jobs framework
![Page 45: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/45.jpg)
Runner
CallbackModuleWorkers
![Page 46: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/46.jpg)
Runner Worker CallbackModule
start
ready
process processready
.
.
.stop
item
![Page 47: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/47.jpg)
“behaviour” is interface
![Page 48: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/48.jpg)
behaviour_info(callbacks) -> [ {init, 1}, {process_item, 3}, {worker_died, 5}, {job_stopping, 1}, {job_complete, 2}].
![Page 49: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/49.jpg)
running({worker_ready, WorkerPid, ok}, S) -> case queue:out(S#state.items) of {empty, I2} -> stop_worker(WorkerPid, S), {next_state, complete, S#state{items = I2}};
{{value, Item}, I2} -> job_worker:process(WorkerPid, Item, now(), S#state.job_state),
{next_state, running, S#state{items = I2}} end;
![Page 50: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/50.jpg)
handle_info({'DOWN', _, process, WorkerPid, Info}, StateName, S) -> {Item, StartTime} = clear_worker(WorkerPid, S),
Callback = S#state.callback, spawn(Callback, worker_died, [Item, WorkerPid, StartTime, Info, S#state.job_state]),
%% Start a replacement worker start_workers(1, Callback), {next_state, StateName, S};
![Page 51: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/51.jpg)
handle_cast({process, Item, StartTime, JS}, S) -> Callback = S#state.callback, Continue = try Callback:process_item(Item, StartTime, JS) catch throw: Error -> error_logger:error_report(Error), ok end,
job_runner:worker_ready(S#state.runner, self(), Continue),
{noreply, S}.
![Page 52: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/52.jpg)
story time
![Page 53: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/53.jpg)
ReSTful APIwith Webmachine
The Convention Over Configuration Webserver
![Page 54: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/54.jpg)
HTTP Request Lifecycle Diagram
http://webmachine.basho.com
![Page 55: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/55.jpg)
Webmachine Is Simple As Proven by the “Number of Types of Things”
Measurement of Complexity
If you know HTTP
![Page 56: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/56.jpg)
The 3 Most Important Types of Things In Webmachine
1. Dispatch Rules (pure data--barely a thing!)2. Resources (composed of simple functions!)3. Requests (simple get/set interface!)
![Page 57: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/57.jpg)
Dispatch Rules
GET /devices/12345
Webmachine inspects the device_resource module for defined callbacks, and sets the Request record’s “server”
value to 12345.
{ ["devices", server], device_resource, [] }
![Page 58: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/58.jpg)
Resources
• POEM (Plain Old Erlang Module)• Composed of referentially transparent functions*• Functions are callbacks into the request lifecycle• Approximately 30 possible callback functions, e.g.:
• resource_exists → 404 Not Found• is_authorized → 401 Not Authorized
* mostly
![Page 59: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/59.jpg)
Perma-404resource_exists(Request, Context) -> {false, Request, Context}.
Lucky Authis_authorized(Request, Context) -> S = calendar:time_to_seconds(now()), case S rem 2 of 0 -> {true, Request, Context}; 1 -> {“Basic realm=lucky”, Request, Context} end.
Resource Functions
![Page 60: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/60.jpg)
Requests• The first argument to each resource function • Set and read request & response data
wrq:set_resp_header(“X-Answer”, “42”, Request).
RemoteIP = wrq:peer(Request).
![Page 61: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/61.jpg)
content_types_provided(Request, Context) -> Types = [{"application/json", to_json}], {Types, Request, Context}.
to_json(Request, Context) -> Device = proplists:get_value(device, Context), UserId = get_user_id(Request),
case fe_api_firewall:get_config(Device, UserId) of
{ok, Config} -> success_response(Config, Request, Context);
{error, Reason} -> error_response(502, Reason, Request, Context) end.
Retrieving a JSON Firewall Representation
![Page 62: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/62.jpg)
Gotchas
![Page 63: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/63.jpg)
primitive obsession
![Page 64: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/64.jpg)
string-ish
[<<"easy as ">>, [$a, $b, $c], " ☺\n"].
“hi how are you”
<<“hello there”>>
![Page 65: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/65.jpg)
hashes vs records
![Page 66: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/66.jpg)
to loop is human,to recur divine
![Page 67: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/67.jpg)
Erlang conditionals always return a value
![Page 68: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/68.jpg)
design for testability
![Page 69: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/69.jpg)
don’t spawn,
use OTP
![Page 70: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/70.jpg)
Downsides
![Page 71: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/71.jpg)
Erlang changes very slowly
![Page 72: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/72.jpg)
3rd party libraries
![Page 73: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/73.jpg)
standard librarycan be inconsistent
![Page 74: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/74.jpg)
package management
![Page 75: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/75.jpg)
Questions
![Page 76: Polyglot parallelism](https://reader034.vdocuments.net/reader034/viewer/2022052117/554e9105b4c90573338b4daf/html5/thumbnails/76.jpg)
Phil: @philtolandhttp://github.com/tolandhttp://philtoland.com
Mike: @lifeinzemblahttp://github.com/msassak
http://spkr8.com/t/7806