exploiting concurrency with dynamic languages
DESCRIPTION
Presentation from JavaOne 2009 on how Dynamic languages can be better suited for writing concurrent applications.TRANSCRIPT
![Page 1: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/1.jpg)
Exploiting Concurrencywith Dynamic Languages
Tobias IvarssonNeo Technology [email protected] Committer @thobe on twitter
![Page 2: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/2.jpg)
$ whoamitobias> MSc in Computer Science
from Linköping University, Sweden> Jython compiler zealot> Hacker at Neo Technology
• Home of the Neo4j graph database• Check out http://neo4j.org
> Follow me on twitter: @thobe• Low traffic, high tech
> Website: http://www.thobe.org (#include blog)2
![Page 3: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/3.jpg)
Exploiting concurrency with dynamic languages
> Background> The languages> The benefits of these languages> Example> Summary
3
![Page 4: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/4.jpg)
Background
4
![Page 5: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/5.jpg)
What Java brought to the table
> Built in, simple support for multi threaded programming
> Synchronized blocks and methods in the language> wait() and notify()> A good, sound memory model for concurrent
situations> Executors, Thread pools, Concurrent collection,
Atomic variables, Semaphores, Mutexes, Barriers, Latches, Exchangers, Locks, fine grained timing...
5
![Page 6: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/6.jpg)
Why use a language other than Java?
> More expressive - put the good stuff to better use> Higher order constructs abstract away tedious
boilerplate> Closures
6
![Page 7: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/7.jpg)
Why use a language other than Java?
> Closures provide clean ways of expressing tasks and thread entries
> Meta-programmability can encapsulate conventions• Operator overloading• Macros and function modifiers
> Capabilities for building DSLs> Built in ways of dealing with modern problems
• Support for concurrency best practices?
7
![Page 8: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/8.jpg)
The Languages
8
![Page 9: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/9.jpg)
Jython
> I’m biased> Python is used a lot in scientific computing
• Could benefit a lot from better concurrency offered by Jython being on the Java platform
> Highly dynamic> Strict, explicit, but compact and readable syntax
9
![Page 10: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/10.jpg)
JRuby
> It is a great language implementation> It is a popular, highly productive language> Highly dynamic> Powerful syntax for closures
10
![Page 11: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/11.jpg)
Scala
> Geared towards scalability in concurrency> Interesting Actor model
• Provide for scalability by restricting data sharing between concurrently executing code
11
![Page 12: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/12.jpg)
Clojure
> Built at core for being good at concurrency> Provides for concurrency by using persistent data
structures• Most objects are immutable
> Objects that are mutable are mutated explicitly • Software transactional memory
12
![Page 13: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/13.jpg)
“Why not Groovy” (or another language)
> Good and popular dynamic language for the JVM> Can do anything that Java, Jython or JRuby can> Not interesting - All examples would be “me too”
> Countless other languages have their specialties, including them all would be too much• Ioke, Kawa, Rhino, Java FX script, Kahlua,
Fortress...
13
![Page 14: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/14.jpg)
What these languages add
14
![Page 15: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/15.jpg)
Automatic resource management - Jython
15
with synchronization_on(shared_resource):data = shared_resource.get_data(key)if data is not None:
data.update_time = current_time()data.value = “The new value”shared_resource.set_data(key, data)
![Page 16: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/16.jpg)
Closures - JRuby
shared_resource.synchronized dodata = shared_resource.get_data(key)if data.nil?
data.update_time = current_time()data.value = “The new value”shared_resource.set_data key, data
endend
16
![Page 17: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/17.jpg)
Closures - JRuby
chunk_size = large_array.size/NUM_THREADSthreads(NUM_THREADS) do |i|
start = chunk_size * i(start..start+chunk_size).each do |j|
large_array[j] += 1end
end
17
![Page 18: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/18.jpg)
Meta functions - Jython (“function decorators”)
@future # start thread, return future refdef expensive_computation(x, y):
z = go_do_a_lot_of_stuff(x)return z + y
# use the function and the futurefuture_value = expensive_computation(4,3)go_do_other_stuff_until_value_needed()value = future_value()
18
![Page 19: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/19.jpg)
Meta functions - Jython
@forkjoin # fork on call, join on iterdef recursive_iteration(root):
if root.is_leaf:yield computation_on(root)
left = recursive_iteration(root.left)right= recursive_iteration(root.right)for node in left: yield nodefor node in right: yield node
for node in recursive_iteration(tree):...
19
![Page 20: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/20.jpg)
Built in Software Transactional Memory - Clojure
(defn transfer [amount from to] (dosync ; isolated, atomic transaction (if (>= (ensure from) amount) (do (alter from - amount) (alter to + amount)) (throw (new java.lang.RuntimeException "Insufficient funds")))))
(transfer 1000 your-account my-account)
20
![Page 21: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/21.jpg)
Transactions as a managed resource - Jython
import neo4jneo = neo4j.NeoService(“/var/neodb”)persons = neo.index(“person”)with neo.transaction:
tobias = neo.node(name=”Tobias”)persons[tobias[‘name’]] = tobiasemil = neo.node(name=”Emil”, CEO=True)persons[emil[‘name’]] = emiltobias.Knows(emil, how=”Buddies”)
21
![Page 22: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/22.jpg)
Actor Execution - Scala
> Concurrent processes communicating through messages
> Receive and act upon messages• Send message• Create new actors• Terminate• Wait for new message
> Message objects are immutable
22
![Page 23: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/23.jpg)
Actor Execution - Scala
case class IncredibleProblem(d:String)class SimpleSolution(d:String)object RichardDeanAnderson extends Actor{
def act = {loop {
react {case IncredibleProblem(d) =>
reply(SimpleSolution(d))}}}}
23
![Page 24: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/24.jpg)
Example
24
![Page 25: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/25.jpg)
Adding two numbers
25
# Jythondef adder(a,b):
return a+b
# JRubydef adder(a,b)
a+bend
![Page 26: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/26.jpg)
Counting the number of additions - Jython
from threading import Lockcount = 0lock = Lock()def adder(a,b):
global countwith lock:
count += 1return a+b
26
![Page 27: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/27.jpg)
Counting the number of additions - JRuby
class Countingdef adder(a,b)
@mutex.synchronize {@count = @count + 1
}a + b
endend
27
![Page 28: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/28.jpg)
Adding two numbers - Clojure
(defn adder [a b](+ a b))
(let [count (ref 0)] (defn counting-adder [a b] (dosync (alter count + 1)) (+ a b)) (defn get-count [] (deref count)))
28
![Page 29: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/29.jpg)
Actors Adding numbers - Scala
class Adder { def add(a:Int,b:Int) = a+b}
29
![Page 30: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/30.jpg)
Actors Adding numbers - Scala
class Cntr(var count:Int) extends Actor { def act = { loop { react { case Inc => count = count + 1 case GetCount => reply(count); exit } } } }
30
![Page 31: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/31.jpg)
Actors Adding numbers - Scala
class CountingAdder(times:Int,adder:Adder,counter:Cntr) extends Actor {
def act = {
loop {
react {
case Add(a,b) => {
val start = nanoTime()
for (i <- 0 until times) {
counter ! Inc
adder.add(a,b)
}
val time = nanoTime() - start
react {
case GetTime => reply(time / 1000000.0)
}
}
case Done => exit
} } } }
31
![Page 32: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/32.jpg)
Performance figures
32
![Page 33: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/33.jpg)
Execution times (ms for 400000 additions)
33
0
175
350
525
700
Without counter
Jython JRuby Clojure
![Page 34: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/34.jpg)
Execution times (ms for 400000 additions)
34
0
12500
25000
37500
50000
With counter
Jython (Lock) JRuby (Mutex) Clojure (STM)Jython (AtomicInteger)
![Page 35: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/35.jpg)
Execution times (ms for 400000 additions)
35
0
2250
4500
6750
9000
Mutex (JRuby) STM (Clojure) Actors (Scala)
![Page 36: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/36.jpg)
Summary
36
![Page 37: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/37.jpg)
Jython
> Nice resource management syntax> Implementation has a few things left to work out> Suffer from high mutability in core datastructures
37
![Page 38: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/38.jpg)
JRuby
> Closures help a lot> Great implementation> Suffer from high mutability in core datastructures
38
![Page 39: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/39.jpg)
Scala
> Actors have good characteristics• Decouple work tasks with no shared state• Asynchronous communication is great
> Syntax is more on the verbosity level of Java
39
![Page 40: Exploiting Concurrency with Dynamic Languages](https://reader034.vdocuments.net/reader034/viewer/2022051611/54b7a4144a795993718b479e/html5/thumbnails/40.jpg)
Clojure
> Immutability frees you from thinking about synchronization
> Transactions for mutation enforces thinking about state
> The combination enforces best practices, but in a way much different from “plain old Java”
40