Charisma: Orchestrating Migratable Parallel Objects

Chao Huang, Laxmikant KaleParallel Programming LabUniversity of Illinois at Urbana-Champaign

HPDC 2007 204/22/23

Motivation Complex structures in modern applications

Large number of components Complicated interactions

Parallel programming productivity Traditional SPMD paradigms break modularity Object-based paradigm obscures global control flow

Goal A language for expressing global view of control

HPDC 2007 304/22/23

Outline Motivation Expressing Flow of Control Language Design and Implementation

Code Examples Results

Performance and Productivity Studies Related Work Future Work

HPDC 2007 404/22/23

Example: MD Structure of a simple MD simulationMPI_Recv(buf,..., MPI_ANY_SOURCE, MPI_ANY_TAG,…);

switch(GET_TYPE(buf)){ case (FOR_ANGLE): /* calculate angle forces */ case (FOR_PAIR_LEFT): /* calculate pairwise forces */ case (FOR_PAIR_RIGHT): /* calculate pairwise forces */}

MPI_Recv(angle_buf,…, ANGLE_SRC, ANGLE_TAG,…);/* calculate angle forces */MPI_Recv(pair_left_buf,…, PAIR_LEFT_SRC, PAIR_LEFT_TAG,…);MPI_Recv(pair_right_buf,…, PAIR_RIGHT_SRC, PAIR_RIGHT_TAG,…);/* calculate pairwise forces */

patch (cell)

compute (cellpair)

HPDC 2007 504/22/23

MainChare::MainChare{ cell.sendCoords();}

MainChare::reduceEnergy(energy){ totalEnerty+= energy; if iter++ ( MAX_ITER cells.sendCoords(); else CkExit();}

Cellpair::recvCoords(coords){ if not coords from both cells received buffer(coords); return; else // all coords ready force = calcForces(); for index in 2 cells cells(index).recvForces(forces);}

Cell::sendCoords(){ for index in 26 neighbor cellpairs cellpairs(index).recvCoords(coords);}

Cell::recvForces(forces){ totalforces += forces; if not all forces from all cellpairs received return; else // neighborhood reduction completed integrate(); mainProxy.reduceEnergy(energy);}

Expressing Flow of Control Charm++: fragmented in object code

HPDC 2007 604/22/23

Charisma

Expressing global view of control Parallel constructs in orchestration code Sequential code separately in user C++ code

Features High level abstraction of control Separation of parallel constructs and sequential code Generating Charm++ code

Automatic load balancing, adaptive overlap, etc

HPDC 2007 704/22/23

Language Design foreach statement

Invokes method on all elements: object-level parallelism Producer-consumer model

Sequential code unaware of source of input values and destination of output values

Data is sent out as soon as it becomes available Various communication patterns Control constructs: loop, if-then-else, overlap

foreach i in workers workers[i].doWork(); end-foreach

foreach i in workers (p[i]) <- workers[i].foo(); workers[i].bar(p[i+1]); end-foreach

HPDC 2007 804/22/23

Code Example: MD with Charisma

Orchestration code: global view of control foreach i,j,k in cells (coords[i,j,k]) <- cells[i,j,k].produceCoords();end-foreachfor iter = 1 to MAX_ITER foreach i1,j1,k1,i2,j2,k2 in cellpairs (+forces[i1,j1,k1],+forces[i2,j2,k2]) <-

cellpairs[i1,j1,k1,i2,j2,k2].calcForces(coords[i1,j1,k1],coords[i2,j2,k2]);

end-foreach foreach i,j,k in cells (coords[i,j,k],+energy) <-

cells[i,j,k].integrate(forces[i,j,k]); end-foreach MDMain.updateEnergy(energy);end-for

HPDC 2007 904/22/23

Code Example: MD with Charisma

Orchestration code: global view of control foreach i,j,k in cells (coords[i,j,k]) <- cells[i,j,k].produceCoords();end-foreachfor iter = 1 to MAX_ITER foreach i1,j1,k1,i2,j2,k2 in cellpairs (+forces[i1,j1,k1],+forces[i2,j2,k2]) <-

cellpairs[i1,j1,k1,i2,j2,k2].calcForces(coords[i1,j1,k1],coords[i2,j2,k2]);

end-foreach foreach i,j,k in cells (coords[i,j,k],+energy) <-

cells[i,j,k].integrate(forces[i,j,k]); end-foreach MDMain.updateEnergy(energy);end-for

HPDC 2007 1004/22/23

Code Example: MD with Charisma

Orchestration code: global view of control foreach i,j,k in cells (coords[i,j,k]) <- cells[i,j,k].produceCoords();end-foreachfor iter = 1 to MAX_ITER foreach i1,j1,k1,i2,j2,k2 in cellpairs (+forces[i1,j1,k1],+forces[i2,j2,k2]) <-

cellpairs[i1,j1,k1,i2,j2,k2].calcForces(coords[i1,j1,k1],coords[i2,j2,k2]);

end-foreach foreach i,j,k in cells (coords[i,j,k],+energy) <-

cells[i,j,k].integrate(forces[i,j,k]); end-foreach MDMain.updateEnergy(energy);end-for

HPDC 2007 1104/22/23

Code Example: MD with Charisma

Sequential codevoid Cell::integrate(Force forces[],

outport coords, outport energy){ for(int i=0;i<mySize;i++){ myAtoms[i].applyForces(forces[i]);

myAtoms[i].update(); } produce(coords,myAtoms,mySize);

double myEnergy = this->calculateEnergy(); reduce(energy, myEnergy, “+”);}

HPDC 2007 1204/22/23

Language Implementation Dependence analysis

Identify inports and outports Organize dependence graph Generate communications

Control transfer Central control vs. Distributed control

Generated code optimization Eliminating unnecessary memory copy Migrating live variables

Library module support

HPDC 2007 1304/22/23

Expressing Flow of Control (Cont.)

Example: Parallel 3D FFT

foreach x in planes1 (pencils[x,*]) <- planes1[x].fft1d(); end-foreach foreach y in planes2 planes2[y].fft2d(pencils[*,y]);end-foreach

HPDC 2007 1404/22/23

Results Scalability Results

2D Jacobi(Size: 163842 on 4096 objects)

3D FFT(Size: 5123 on 256 objects)

HPDC 2007 1504/22/23

Results

2D Jacobi Wator

Productivity Study

HPDC 2007 1604/22/23

Related Work

Producer-consumer modelFortran M

Composing from componentsP-COM2

Visual parallel dataflow languagesHeNCE and CODEVPE

HPDC 2007 1704/22/23

Future Work

Use dependence to optimizeCritical path analysisPrefetch objects (out-of-core execution)Assist communication optimizations

Orchestration for other fieldsStream-based applications

HPDC 2007 1804/22/23

Thank You

Questions?

More details at http://charm.cs.uiuc.edu