erlang - concurrent language for concurrent world
DESCRIPTION
Intro to Erlang talk from Erlang Factory Lite Tel-Aviv, October 23 2013TRANSCRIPT
Concurrent Language for Concurrent World
The world is concurrent… I could not drive the car, if I did not understood concurrency …
-- Joe Armstrong, father of Erlang
Object-oriented
Programming
Object-oriented
Programming
object object
object object
object
object
object object
object
object
object
object
object
object
object object
object
object
Concurrency-oriented
Programming
Concurrency-oriented
Programming
process process
process process
process
process
process process
process
process
process
process
process
process
process process
process
process
Imagine a programming language which allows only
1000 objects -- Joe Armstrong, father of Erlang
COP vs OOP Concurrency-oriented vs Object-oriented
• 15 years ago OOP became mainstream
• COP became popular only recently
• Process is active object
– i.e. object with it’s own process of control
– Objects in OOP are “passive objects”
• Processes as cheap as hashtables:
– ~ 300 words (1.2KB) per process by default
– max 134M processes per VM (limited by RAM)
Concurrency-oriented Programming
• Large number of processes (up to millions)
• Complete isolation of processes
• No shared memory
• Copy message data between processes
– instead of just passing pointer to message
• Avoid sequential bottlenecks
• Location transparency
Share Nothing!
No mutable State!
Software Isolation
But, it will not help
if the whole VM crashes
One Process per
Parallel Activity
• try to do only one thing per process
• If you need to do something
– just spawn a new process
• no thread pools!
Let it crash!
• all sequential languages get error handling wrong!
• No defensive programming!
• If you don’t know how to recover from the error – let it crash!
Lenin – Lived Lenin – is Alive Lenin – will Live!
Errors – Existed Errors – Exist Errors – will Exist!
Lenin, read my dissertation: “Making reliable distributed systems in the presence of software errors”
http://www.erlang.org/download/armstrong_thesis_2003.pdf
Erlang Timeline
19841986
1987
1991
1993
OTP
1996
1998
2007
Open Source Distributed Erlang
Early Erlang Prototypes
First fast implementation
CS Lab Experiments
AXD301 switch
Erlang/OTP Stack
Erlang VM – BEAM / ERTS
Functional Sequential Erlang
Concurrent Erlang
OTP – Open Telecom Platform
Distributed Erlang
Erlang P
rogram
min
g Langu
age
Functional Sequential Erlang
• Data types: – Integers, floats, atoms, tuples/records, lists,
binaries, funs
• single assignment • pattern matching & guards • closures (anonymous function data type) • list comprehensions • bit-syntax & binary comprehensions • tail recursion & tail call optimization (TCO)
Single Assignment 1> X.
* 2: variable 'X' is unbound
2> X = 10.
10
3> X = X + 1.
** exception error: no match of right hand side value 11
4> X = X.
10
5> X = 10.
10
6> X = 20.
** exception error: no match of right hand side value 20
No Loops???
• Use instead:
– recursion and tail recursion
– list comprehensions
–binary comprehensions
–Higher-order Functions (HOFs):
• map/foldl/foldr/zip/filter/etc.
• For example:
[ io:format(“~b”,[I]) || I<-lists:seq(1,10) ].
Regular Recursion
Tail Recursion
zvi@host:~$ erl
Erlang R16B02 …
…
1> c(demo).
{ok,demo}
2> demo:fact(0).
1
3> demo:fact(5).
120
4> demo:fact(-10).
** exception error: no function clause matching demo:fact(-10) (demo.erl, line 10)
OOP vs FP
Side-Effects • Erlang is not pure-functional language
• “Controlled Side effects”:
– Messages - Pid ! Msg – send message to process Pid
– Signals - i.e. your linked process is dead
– Exceptions
– I/O
– Process Dictionary - similar to TLS (thread local storage)
– ETS / DETS - fast in-memory / persistent lookup tables
– Mnesia - distributed realtime database (sort of STM)
Erlang Source Code Structure
• OTP Release (self-contained tar.gz archive)
– OTP Application (directory structure)
• Module (file sources in *.erl, compiled in *.beam) – Function
» Clause (syntactic sugar for case statement)
MFA
• Module:Function/Arity
• Module:Function(Arguments)
• M:F(A1,A2,..,An) = apply(M,F,[A1,A2,..,An])
Concurrent Erlang
• Spawn processes
• Send messages to processes
• Selective receive
• Additional data types:
– PIDs, references
Spawn a process
Send Message
Receive Message
Actors / Agents / Active Objects
Process
• Pid – Process ID • optional: registered name • initial call: MFA • message queue (mailbox) • stack • heap • Dictionary (like TLS – thread local storage) • traping exits? (boolean flag) • linked processes • priority (low/normal/high) • Reductions counter
Distributed Erlang
Erlang Run-Time System Erlang Run-Time System
network
A B C
Node name: [email protected] Node name: [email protected]
Distributed Erlang
Erlang Run-Time System Erlang Run-Time System
B ! Msg
network
A B C
Node name: [email protected] Node name: [email protected]
Distributed Erlang
Erlang Run-Time System Erlang Run-Time System
B ! Msg
network
C ! Msg
A B C
Node name: [email protected] Node name: [email protected]
Erlang VM - SMP Schedulers
zvi@host:~$ erl
Erlang R16B02 (erts-5.10.3) [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V5.10.3 (abort with ^G)
1>
Hot Code Reloading
• Module may have 2 version loaded in VM: – old version – current version
• When new version of module is loaded: – it become a “current version” – current version become the “old version”
• All new processes will use current version • The old processes running old version
– In places in the code where module name is explicitly specified - the process will switch to current version: • my_function(Args) - no version switch • my_module:my_function(Args) - switching version
Tools
• IDE: – Emacs, Distel, Plugins for Eclipse and Netbeans
• Testing: – e-unit, meck, proper, QuickCheck (paid), Quick Check mini - free
• Static code analyzer: – Dialyzer
• Debug and trace – Built-in in VM – dbg, redbug
• Build: – emake, rebar, relx, erlang.mk
• Package Managers: – rebar, mix
ERLANG RESOURCES
Docs, Tutorials, Community, Conferences
ErlangCentral.org
Erlan
g b
oo
ks
http://www.erlang.org/download.html
http://erlang.org/mailman/listinfo/erlang-questions
Erlang/OTP is a good fit for:
• Irregular concurrency: – Task-level – Fine-grained parallelism
• Network servers • Distributed systems • Middleware:
– Parallel databases – Message Queue servers
• Soft Realtime / Embedded applications • Monitoring, control and testing tools
Not so good fit for:
• Concurrency for synchronized parallel execution – Data Parallelism
• Floating-point intensive code (HPC) • Text Processing / Unicode
– Unicode support now much better
• Traditional GUI (supports wxWidgets) • Hard Realtime applications • Extensive interop with other languages/VMs
– improvement here with NIFs and Erjang - Erlang on JVM
Thank You! Now Q&A
All images are taken from Google Image search and various other places on the Internet © Copyright of corresponding owners
BACKUP SLIDES
Communicating with External world:
• Part of the OTP: – CORBA/IDL – ODBC/SQL – ASN.1 – SNMP – HTTP/FTP – SSH
• erlcall – execute any Erlang function on remote Node from the
shell or script
• escript – write scripts in Erlang – no need to compile
Communicating with External world In-proc Out-of-proc
Custom Protocol Linked-in Drivers • in-proc (.so, DLLs) • Written in C • custom protocols • Can crash VM
Ports • external executable • like pipes • custom protocols via
fd/stdio/sockets • Dir = os:cmd(“ls –al”).
Generic Protocol NIF • Native Implemented
Functions • Written in C • in-proc (like JNI in Java:
.so, DLLs) • .so/DLLs can be hot code
reloaded at runtime • can block a scheduler • Can crash VM
C-Node • out-of-proc (external executable,
interpreter or VM) • Pretend to be a distributed
Erlang node (but implemented in other lang.)
• JInterface (Java), OTP.NET, PyInterface (Pyton), Ruby-electricity, etc.
• Example: erlcall
LANGUAGES FOR ERLANG VM
ERLANG ON JVM
Erjang – Erlang for JVM
Elixir Books
COMPARISON
Concurrent Programming: Scala+Akka, node.js, Go
Go-lang