ph: a parallel dialect of haskell

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science

pH: A Parallel Dialect of Haskell

Jim Cipar & Jacob SorberUniversity of Massachusetts

Amherst* Some material adapted from slides from Arvind(MIT)

and Jan-Willem Maesson(Sun)

UUNIVERSITY OF NIVERSITY OF MMASSACHUSETTSASSACHUSETTS, A, AMHERST • MHERST • Department of Computer Science Department of Computer Science 2

outline Historical Background pH Language Threading in pH Scheduling Where is pH today?


History Dataflow Languages

Describe programs as data flows Inherently parallel Examples: Id and Sisal

Id Introduced I-structures and M-

structures to avoid copying arrays during construction.


History Id worked great on the Monsoon

dataflow architecture.

1x 2x 4x 8x

Mat. Multiply 1.00 1.99 3.90 7.74

Paraffins n=22 1.00 1.99 3.92 7.25

Particles(40K) 1.00 1.95 3.81 7.35

Simple 100 iters

1.00 1.85 3.45 6.27

Speed up

Data flow architecture effectively supports the execution model.


History Haskell

Purely functional programming language

Lazy execution model (expressions only evaluated when needed)

“Sexy” new type system [Jan-Willem Maesson]

People actually use Haskell (?)


pH: parallel Haskell pH = Haskell (syntax, type system)

+ Id (evaluation order, side-effect ops)

Goal: Unite two communities Bring the dataflow and functional

communities under a single language

Facilitate code sharing


Language Structure pH has 3 layers

pH(F) Purely functional Haskell + loops

pH(I) pH(F) + I-structures (Id)

pH(M) pH(I) + M-structures (Id)


pH(F) Adds for and while loops to Haskell

Syntactic sugar (== tail recursion)

for sum=0

in for i <- [1..n] do

next sum = sum + i

finally sum


pH(I) I-structures

I-structure = write-once array Reads are delayed until written All reads return a single consistent

value No data races


State? Programs without

side-effects/state are not usually very useful.

Add state using assignment (ML, Scheme) Results in a sequential evaluation order Goal: evaluate sequentially only when

necessary.


pH(M) == Full pH M-structures (state)

Allows multiple synchronized “takes” and “puts” Take: block until data is written, then remove it Potential for race conditions (use barriers) Useful for classic mark-based graph algorithms Higher performance than pure functional

(sometimes)

Program Version

Executed Instructions

Critical Path

Average Parallelism

Heap used (words)

HA1HA2HA3HA4HA5M-structures

199,75772,847

325,472266,223253,37060,496

1,4527,324

238195

28,833589

13810

13681365

9103

9,2502,100

18,4239

11,3019


M-Structure Syntax Allocate := M_array(1,n) Allocate := M_array((1,n),(1,m)) Put := A![i] = 5 Take := (A![i] + B![i])

def replace a i v =

{ x = a![i];

a![i] = v;

in x};


Barriers (1) Divide a control region into two

subregions that must be evaluated sequentially

Syntax = “---”

def fib n = if n < 2 then n

else {

x = fib(n-1);

---

y = fib(n-2);

in x + y};


Barriers (2) Control parallelism

Reduce exponential resource usage

Sacrifice parallelismdef fib n = if n < 2 then n

else {

x = fib(n-1);

---

y = fib(n-2);

in x + y};


Barriers (3) Force sequential evaluation

def replace a i v =

{ x = a![i];

---

a![i] = v;

in x};


Implicit Parallelism Program execution == expression

reduction Every reduction is evaluated in parallel Exception: lambda expressions and

conditionals

def fib n = if n < 2 then n

else {

x = fib(n-1);

y = fib(n-2);

in x + y};


Implicit Parallelismdef fib n = if n < 2 then n

else {

x = fib(n-1);

y = fib(n-2);

in x + y};

fib()

(-) (-)

fib()

(+)

n 1 2


Parallel Execution Eager evaluation

All expressions are reduced in parallel Limited by data dependences and

barriersf (4*x) (g 25 (5+6))

Implications Not all Haskell programs will

terminate using pH semantics! All pH programs will terminate

using Haskell lazy semantics.


Threading in pH

Tree of Activation

Frames f:

g: h:

loop

Global Heap of Shared Objects


Threading in pH Spawn a new thread only when:

There are multiple dependent blocks One of them actually suspends

Use strictness analysis to determine what needs to be evaluated.


Scheduling Work stealing (Cilk-style)

Follows usual call/return pattern Good temporal locality in practice Low overhead in the common case

I-structures/M-structures? Add yourself to the defer list Run the defer list on a write Messes up temporal locality


Where is pH now?

Hmmm…..


Implementation Status

Limited at best Currently compiles to the Monsoon

dataflow machine Coming soon to the UltraSPARC No documentation

A few papers, mostly about semantics No support

Support for real parallel architectures might encourage involvement from the Haskell community


Where is pH now? Implementation There might be hope

Haskell is actually used Mitre: Speech Recognition System LOLITA: Natural Language Processing

System Monadius: a Haskell shoot ‘em up

Legacy might live on through Fortress

ph: a parallel dialect of haskell

Documents