eden parallel functional programming with haskell

Download Eden  Parallel  Functional Programming with Haskell

Post on 01-Jan-2016

20 views

Category:

Documents

0 download

Embed Size (px)

DESCRIPTION

Eden Parallel Functional Programming with Haskell. Rita Loogen Philipps-Universität Marburg, Germany - PowerPoint PPT Presentation

TRANSCRIPT

Die parallele funktionale Sprache Eden -Entwurf und Implementierung-

Eden Parallel Functional Programming with Haskell

Rita LoogenPhilipps-Universitt Marburg, Germany

Joint work withYolanda Ortega Malln, Ricardo Pea Alberto de la Encina, Mercedes Hildalgo Herrero, Christbal Pareja, Fernando Rubio, Lidia Snchez-Gil, Clara Segura, Pablo Roldan Gomez (Universidad Complutense de Madrid)Jost Berthold, Silvia Breitinger, Mischa Dieterle, Thomas Horstmeyer, Ulrike Klusik, Oleg Lobachev, Bernhard Pickenbrock, Steffen Priebe, Bjrn Struckmeier(Philipps-Universitt Marburg)

CEFP Budapest 201112Rita Loogen: Eden CEFP 2011

Marburg /Lahn 2OverviewLectures I & II (Thursday)MotivationBasic Constructs Case Study: MergesortEden TV The Eden Trace ViewerReducing communication costsParallel map implementations

Explicit Channel ManagementThe Remote Data ConceptAlgorithmic SkeletonsNested WorkpoolsDivide and Conquer

3Lecture III: Lab Session (Friday Morning)Lecture IV: ImplementationLayered StructurePrimitive OperationsThe Eden Module

The Trans classThe PA monadProcess HandlingRemote Data3MaterialsMaterials Lecture NotesSlidesExample Programs (Case studies)Exercisesare provided via the Eden web page www.informatik.uni-marburg.de/~edenNavigate to CEFP!4Rita Loogen: Eden CEFP 201145Rita Loogen: Eden CEFP 2011Motivation56

Our GoalParallel programming at a high level of abstractionfunctional language (e.g. Haskell)=> concise programs=> high programming efficiency

automatic parallelisationor annotationsinherent parallelism67Our ApproachParallel programming at a high level of abstractionparallelism controlexplicit processesimplicit communicationdistributed memory

Eden = Haskell + Parallelismwww.informatik.uni-marburg.de/~eden+functional language (e.g. Haskell)=> concise programs=> high programming efficiency

llll78Rita Loogen: Eden CEFP 2011Basic Constructs89Eden= Haskell + Coordination process definition

process instantiation parallel programmingat a high level ofabstractionprocess :: (Trans a, Trans b) => (a -> b) -> Process a b

gridProcess = process (\ (fromLeft,fromTop) -> let ... in (toRight, toBottom)) ( # ) :: (Trans a, Trans b) => Process a b -> a -> b

(outEast, outSouth) = gridProcess # (inWest,inNorth)process outputs computed byconcurrent threads,lists sent as streams9Derived operators and functionsParallel function application Often, process abstraction and instantiation are used in the following combination

Eager process creation Eager creation of a series of processes

10Rita Loogen: Eden CEFP 2011spawn :: (Trans a, Trans b) => [Process a b] -> [a] -> [b]spawn = zipWith (#) -- ignoring demand control

spawnF ::(Trans a, Trans b) => [a -> b] -> [a] -> [b]spawnF =spawn . (map process)($#) :: (Trans a, Trans b) => (a -> b) -> a -> b

f $# x = process f # x -- ($#) = (#) . process10Evaluating f $# e11Rita Loogen: Eden CEFP 201111graph of process abstractionprocess f #graph of argument expression e will be evaluated by new child processon remote PE will be evaluated in parent processby new concurrent thread and sent to child processmain processchild processcreatesresult of eresult of f $ e1112Defining process nets Example: Computing Hamming numbersimport Control.Parallel.Eden

hamming :: [Int]hamming = 1: sm ((uncurry sm) $#(map (*2) $# hamming, map (*3) $# hamming)) (map (*5) $# hamming)

sm:: [Int] -> [Int] -> [Int]sm [] ys = yssm xs [] = xssm (x:xs) (y:ys) | x < y = x : sm xs (y:ys)| x == y = x : sm xs ys| otherwise= y : sm (x:xs) ys*3*2*5smsm1:hamming13Questions about Semanticssimple denotational semanticsprocess abstraction-> lambda abstractionprocess instantiation-> applicationvalue/result of program, but no information about execution, parallelism degree, speedups /slowdowns

operationalWhen will a process be created? When will a process instantiation be evaluated? To which degree will process in-/outputs be evaluated? Weak head normal form or normal form or ...?When will process in-/outputs be communicated? When will a process be created? When will a process instantiation be evaluated?

To which degree will process in-/outputs be evaluated? Weak head normal form or normal form or ...?

When will process in-/outputs be communicated?

14AnswersEden

only if and when its result is demanded

normal form

eager (push) communication:values are communicated as soon as availableLazy Evaluation (Haskell)

only if and when its result is demanded

WHNF (weak head normal form )

only if demanded:request and answermessages necessary15Lazy evaluation vs. ParallelismProblem: Lazy evaluation==>distributed sequentialityEdens approach:eager process creation with spawneager communication:normal form evaluation of all process outputs (by independent threads)push communication, i.e. values are communicated as soon as availableexplicit demand control by sequential strategies (Module Control.Seq):rnf, rwhnf ... :: Strategy ausing :: a -> Strategy a -> apseq :: a -> b -> b (Module Control.Parallel)16Rita Loogen: Eden CEFP 2011Case Study: Merge Sort16Case Study: Merge SortHaskell Code:mergeSort :: (Ord a, Show a) => [a] -> [a]mergeSort [] = []mergeSort [x] = [x]mergeSort xs = sortMerge (mergeSort xs1) (mergeSort xs2) where [xs1,xs2] = unshuffle 2 xs17UnsortedlistUnsortedsublist 1Unsortedsublist 2Sortedsublist 1sortedSublist 2sortedlistsplitmerge17Example: Merge Sort parallelEden Code (simplest version):parMergeSort :: (Ord a, Show a, Trans a) => [a] -> [a]parMergeSort [] = []parMergeSort [x] = [x]parMergeSort xs = sortMerge (parMergeSort $# xs1) (parMergeSort $# xs2) where [xs1,xs2] = unshuffle 2 xs18UnsortedlistUnsortedsublist 1Unsortedsublist 2Sortedsublist 1sortedSublist 2sortedlistsplitmerge18Eden Code (simplest version):parMergeSort :: (Ord a, Show a, Trans a) => [a] -> [a]parMergeSort [] = []parMergeSort [x] = [x]parMergeSort xs = sortMerge (parMergeSort $# xs1) (parMergeSort $# xs2) where [xs1,xs2] = unshuffle 2 xsExample: Merge Sort Process net19main processchild processchild processchild processchild processchild processchild process1920Rita Loogen: Eden CEFP 2011EdenTV: The Eden Trace Viewer Tool20The Eden-System21Parallel runtime system(Management of processes and communication)parallel systemEdenEdenTVCompiling, Running, Analysing Eden Programs22Rita Loogen: Eden CEFP 2011Set up environment for Eden on Lab computers by callingedenenv

Compile Eden programs withghc parmpi --make O2 eventlog myprogram.hs or ghc parpvm --make O2 eventlog myprogram.hs

If you use pvm, you first have to start it. Provide pvmhosts or mpihosts fileRun compiled programs withmyprogram +RTS ls -N -RTS

View activity profile (trace file) withedentv myprogram_..._-N4_-RTS.parevents 22Eden Threads and ProcessesAn Eden process comprises several threads (one per output channel). Thread State Transition Diagram:23runnableblockedrunningfinishednew threadkill threadkill threadkill threaddeblock threadblock threadrun threadsuspend threadEdenTV

- Diagrams:Machines (PEs)ProcessesThreads

- Message OverlaysMachinesProcesses

- zooming- message streams- additional infos- ...

24EdenTV Demo25Rita Loogen: Eden CEFP 20112526Rita Loogen: Eden CEFP 2011Case Study: Merge Sort continued26Example: Activity profile of parallel mergesort27

Program run, length of input list: 1.000Observation:SLOWDOWNSeq. runtime: 0,0037 sPar. runtime: 0,9472 sReasons: 1999 processes, mostly blocked 31940 messages delayed process creation process placement27How can we improve our parallel mergesort? Here are some rules of thumb.Adapt the total number of processes to the number of available processor elements (PEs), in Eden: noPe :: Int

Use eager process creation functions spawn or spawnF.

By default, Eden places processes round robin on the available PEs. Try to distribute processes evenly over the PEs.

Avoid element-wise streaming if not necessary, e.g. by putting the list into some box or by chunking it into bigger pieces. 28Rita Loogen: Eden CEFP 2011THINK PARALLEL!28Parallel Mergesort revisited29Rita Loogen: Eden CEFP 2011unsortedlistunsortedsublist 2unsortedsublist noPe-1sortedsublist 2sortedsublist noPe-1sortedlistunshuffle (noPe-1)merge many listsunsortedsublist 1mergesortsortedsublist 1unsortedsublist noPesortedsublist noPemergesortmergesortmergesort2914parMap :: (Trans a, Trans b) => (a -> b) -> [a] -> [b]parMap f = spawn (repeat (process f))

A Simple Parallelisation of mapmap :: (a -> b) -> [a] -> [b]map f xs = [ f x | x [a] -> [a]par_ms xs = head $ sms $ parMap mergeSort (unshuffle (noPe-1) xs))

sms :: (NFData a, Ord a) => [[a]] -> [[a]]sms [] = []sms xss@[xs] = xsssms (xs1:xs2:xss) = sms (sortMerge xs1 xs2) (sms xss)

31 Total number of processes = noPe eagerly created processes round robin placement leads to 1 process per PEbut maybe still too many messages31Resulting Activity Profile (Processes/Machine View)32Rita Loogen: Eden CEFP 2011Input size 1.000seq. runtime: 0,0037par. runtime: 0,0427 s

8 Pes, 8 processes, 15 threads2042 messages

Much better, but still

SLOWDOWN

Reason: Indeed too many messages

Previous results for input size 1000Seq. runtime: 0,0037 sPar. runtime: 0,9472 s

3233Rita Loogen: Eden CEFP 2011Reducing Communication Costs33Reducing Number of Messages by Chunking StreamsSplit a list