mpjdev, the low-level mpj device

mpjdev, the low-level MPJ device

Aamir Shafiaamir.shafi@port.ac.uk

Distributed Systems Group12th March, 2004

Sequence Introduction

History of MPJ Overview of java.nio package ‘mpjdev’ implementation

Introduction Packing/Unpacking at the device level Communication Protocols

Eager Send Rendezvous Protocol

Quick Start Guide to mpjdev What’s next

Trinity: Neo... No one has ever done anything like this.Neo: That's why it's going to work.

Introduction Parallel Programming

Shared Memory Message Passing

Message Passing MPI PVM

Has Java got what it takes to be a parallel programming language ?

Message Passing in Java (MPJ)

Specification Document mpiJava 1.2.5

Uses native MPI

Pure java MPI-like libraries mpiJava 1.2.5

Uses native MPI Best candidate to be developed to MPJ http://www.hpjava.org/mpiJava.html DSG’s project MPJ (http://dsg.port.ac.uk/projects/mpj/) is tightly

coupled to this effort JMPI

Drawback, based on RMI http://euler.ecs.umass.edu/jmpi/

MPJava Based on java.nio Results encouraging No source-code

DOGMA – Dead MPP –

Basic library http://www.mi.uib.no/~bjornoh/mtj/mpp/

Sequence Introduction

History of MPJ Overview of java.nio package ‘mpjdev’ implementation

Introduction Packing/Unpacking at the device level Communication Protocols

Eager Send Rendezvous Protocol

Quick Start Guide to mpjdev What’s next

Oracle: You're cuter than I thought. I can see why she likes you.

New I/O overview Buffer classes,

Conversion among basic data types Direct and Indirect Buffer

Scalable Applications No more “one thread per client” One thread, many clients

Multiplex Synchronous I/O using the selectors

SocketChannel – New Abstraction Pipe – One way communication

Buffer classes in java.nio

‘T’Buffer classes, ‘T’ being all the basic data types

Any basic data type can be copied onto the ByteBuffer, (the basic primitive)

Buffer can be, Direct, allocateDirect(int) Indirect, allocate(int)

a. ByteBuffer buffer = ByteBuffer.allocate(8);b. buffer.putInt(4);c. buffer.flip();

a. ByteBuffer buffer = ByteBuffer.allocate(8);b. ByteBuffer buffer = ByteBuffer.allocateDirect(8);

Selector

Selector provide, connecting, accepting, reading & writing facilities to the SocketChannel OP_WRITE, OP_READ, OP_ACCEPT,

OP_CONNECT Depends on native OS facilities

Selection performs differently on Windows and Linux

Sequence of events for selectors

Taming the NIO circus Taming the NIO circus thread

http://forum.java.sun.com/thread.jsp?forum=4&thread=459338&start=0&range=15&hilite=false&q=

OutOfMemory Exception http://forum.java.sun.com/thread.jsp?thread=433702&f

orum=4&message=2136979 Selectors taking cent percent CPU

http://forum.java.sun.com/thread.jsp?forum=11&thread=494967

http://forum.java.sun.com/thread.jsp?forum=4&thread=494194

J2SE 1.5.0 beta solves many problems

Sequence Introduction

History of MPJ Overview of java.nio package ‘mpjdev’ implementation

Introduction Packing/Unpacking at the device level Communication Protocols

Eager Send Rendezvous Protocol

Quick Start Guide to mpjdev What’s next

Neo: What are you trying to tell me? That I can dodge bullets?Morpheus: No, Neo. I'm trying to tell you that when you're ready, you won't have to.

mpjdev – Introduction

What is device ? Sockets

Similar to ADI in MPICH Meant for library developers (MPJ),

not application programmers jGMA can use mpjdev

mpjdev – Introduction Single JVM Implementation

Processes are threads in the single JVM Native implementation

Uses native MPI LAPI Implementation

Eager-send Rendezous

Buffer packing/unpacking are taken from these implementations

mpjev functions Comm methods

Void send(Buffer, int dest, int tag) Req isend(Buffer, int dest, int tag) Req irecv(Buffer, int src, int tag)

For src, ANY_SRC For tag, ANY_TAG

Status recv(Buffer, int dest, int tag)

Req methods Status Wait() Status Wait(Req[])

‘mpjdev’ – Connectivity

Communication Protocols – Eager Send

Sender Receiver

Sel Sel

1 2 3 4 5 6

Huge Memory at Receiver

Rendezvous Protocol

1

Sender Queue

1

Recv Queue

Sender

Sel Sel

1

Receiver

1

Sender 1:

Sender posts send(), Req stored in Queue

Sender 2:

Sender sends the control message asking if matching recv is posted ?

Sender 3:

Sender receives response confirming there is a matching recv

Sender 4:

Sender sends the actual data

Receiver 1:

Checks if step 2 has already posted recv, if yes, initiates step 3, else posts a req in Que

Receiver 2:

Checks if step 1 has posted a recv request, if yes, initiates step 3, else posts a request.

NOTE :- STEP 1 & 2 needs synchronization

Receiver 3:

Writes back to sender that recv’r is ready to receive

Receiver 4:

Receives the data

Packing/Unpacking of Buffers write(t[] source, int offset,int length) gather(t[] source, int numEls, int offs,

int [] indexes) strGather(t[] source, int srcOff, int

rank, int exts, int strs, int [] indexes) read(t[] dest, int dstOff, int numEls) scatter(t[] dest, int numEls, int offs, int

[] indexes) strScatter(t[] dest, int dstOff, int rank,

int exts, int strs, int [] indexes) Java objects, same concepts

Message Format Primary header (Eight bytes),

First four bytes –nothing Last four byte --size of message

Primary PayLoad, Each primitive data type is written as a section,

SECTION_HEADER (8 bytes) First byte – Type of Message, int, float etc Last four bytes – Number of Elements in this section

SECTION_DATA The actual data itself Should be multiple of 8

Secondary header (Eight Bytes) First Four bytes –nothing Last Four bytes – Size of Java Object

Secondary PayLoad Actual java Object in bytes

Can Java object be written as a section in Primary Payload ?

Message Format, Single Section

int intArray[] = new int[2];

int[0] = 1; int[1] = 2;

WriteBuffer wBuffer = new WriteBuffer(24);

wBuffer.write(intArray, 0,2); wBuffer.pack();

X X X X Size (24) int T X X X 2(NoEls) int[0] int[1]

PRIMARY_HEADER SECTION_HEADER SECTION_HEADER

8 bytes 8 bytes 8 bytes

Message Format, Multiple Section

int intArray[] = new int[2]; long longArray = new long[2];

int[0] = 1; int[1] = 2; longArray[0] = 1L; longArray[1] = 2L;

WriteBuffer wBuffer = new WriteBuffer(48);

wBuffer.write(intArray, 0,2); wBuffer.write(longArray, 0,2);

wBuffer.pack();

X X X X Size (24) int X X X 2(NoEls) int[0] int[1] L X X X 2 long[0] long[1]

PRIMARY_HEADERSECTION_HEADERSECTION_DATA

8 bytes 8 bytes 8 bytes

S_HEADER S_DATA

8 bytes 16 bytes

Control Messages

Used in Rendezvous Protocol Format

Int – Rank of Destination Int – Rank of Source Int – Message Length Int – Message Tag

Should be as small as possible

Sequence Introduction

History of MPJ Overview of java.nio package ‘mpjdev’ implementation

Introduction Packing/Unpacking at the device level Communication Protocols

Eager Send Rendezvous Protocol

Quick Start Guide to mpjdev What’s next

Morpheus: There is a difference between knowing the path and walking thepath.

Directory Structure

Compiling the source files

Dependencies, J2SE1.5.0-beta Apache Ant (1.6.1)

If you want to compile from the source, otherwise

mpjdev.jar is lying in the mpjdev/lib folder

You need to add “mpjdev.jar” in your CLASSPATH

Running Examples - Config

Configuration mpjdev/conf/mpjdev.conf Total Number of Processes For each Process

FULLY_QUALIFIED_NAME@PORT@RANK Sorry – IP address won’t work at present

I never had to to write an IP

Running examples # cd mpjdev/test # javac -classpath ../lib/mpjdev.jar:.

BufferTest3.java # java -classpath ../lib/mpjdev.jar:.

BufferTest3 0 (O is the Rank)

Tracking problems java –version, is it 1.5. ? Are you using IP ? Email me

Writing your own programs As an example,

Two processes Initialize the device

CommImpl.init() Get your ID,

CommImpl.id() One process packs & sends data,

CommImpl.send(Buffer, int dest, int tag) Other process reads data & unpacks

CommImpl.recv(Buffer, int src, int tag) Finalize,

CommImpl.finish();

Sequence Introduction

History of MPJ Overview of java.nio package ‘mpjdev’ implementation

Introduction Packing/Unpacking at the device level Communication Protocols

Eager Send Rendezvous Protocol

Quick Start Guide to mpjdev What’s next

Neo: Why do my eyes hurt?Morpheus: You've never used them before.

A Few experiments No tests, yet Bottlenecks,

Transfer of huge messages seems inefficient Queues, java.util.Vector is not good enough

Would like to see how many processes, can selector handle ?

Potentially thousands Need bench-marks, keeping in view java

issues and bottlenecks, and test them Write our own

Future Badly need a runtime

Shell scripts starting jobs using ‘ssh’ or ‘rsh’ ?

MPJ, Group communications

I thought, I would have completed the whole MPJ, by now

Looking for my PhD project, in the same domain

Morpheus: Unfortunately, no one can be told what the Matrix is. You have tosee it for yourself.

Suggestions

?