a high performance middleware in java with a real application

A High Performance Middleware in Java with a Real Application

Fabrice Huet*, Denis Caromel*, Henri Bal+

* Inria-I3S-CNRS, Sophia-Antipolis, France+ Vrije Universiteit, Amsterdam, Netherlands

Outline• Motivations and Overview

• Jem3D: 3D electromagnetism

• ProActive: High Level Middleware for the Grid

• Ibis: High Performance Communications

• Java experimentations

• Comparison with a Fortran Implementation

• Conclusion

Motivations

• Offer an integrated solution for – Programming – Deploying– Running

applications in an heterogeneous environment

• Investigate the current situation of Java programming for high performance

Overview







• Conclusion

Jem3D• Solves 3D Maxwell equations

• Relies on finite volume approximation method

• Works on unstructured tetrahedral discretization of the computation domain

• In this paper, we compute the propagation of an eigenmode in a cubic metallic cavity (standard test case)

• The complexity of the computation is linked to the number of tetrahedra

• We set the number of points to be used for tetrahedra (Mesh size)

Jem3D (2)

Initialization

Compute mag. field, and update elec. field

Compute elec. field, and update mag. field

Calculation of the discreteelectromagnetic energy

Solution Saving

t < tmax

t = tmax

Communication intensive

Distribution of the ComputationExample on 4 nodes

N

1

4

1 1

2

2

Communication Area: 2*N2Communication Area: 3*N2







• Conclusion

ProActive

• Java library for parallel, distributed and concurrent programming

• Medium grain entities: active object– Has its own thread of execution– Stores and serves requests– Asynchronous communications with transparent future objects

and wait by necessity

• Uses RMI or Soap-like protocol

• High level features: mobility, security, component, group communication, deployment descriptors…

Group Communications

• Idea / Goals– Manipulate groups as ‘simple’ Java objects– Maintain the ‘dot’ notation– Code reusability

• Remote method invocation on a group

• Multi-unicast approach– Based on the ProActive communication mechanism– Replication of n ‘single’ communication– Uses of multiple threads

Deployment Descriptor

• Abstract Away from source code:– Machines names– Creation/Connection Protocols– Lookup and Registry Protocols

• Interfaced with various protocols and Infrastructures:– Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols)– Internet and LAN: rsh, rlogin, ssh– Grid: Globus, Web services, gsissh

• API to query resources from within the application through symbolic names (VirtualNode)

Deployment Descriptor- ExamplesVirtualNode:

jem3dNode

Mapping:

jem3dNode VM1, VM2

Infrastructure:

VM1 Local VM

VM2 SSH host1

then Local VM

VirtualNode:

jem3dNode

Mapping:

jem3dNode VM1

Infrastructure:

VM1 SSH frontend

then pbsProcess

pbsProcess PBS 10 nodes Local VM

VM2VM1

localhost

host1

localhost frontend

VM

VM

VM

VM

VM

VM

VM

VM

VM

VM

App.

jem3dNode

App.

jem3dNode







• Conclusion

Ibis• Grid programming environment with efficient

communications

• Java Implementation for portability

• Native Implementation for performance– TCP– Myrinet– MPI

• Various programming models: Group Method Invocation (GMI), Divide and Conquer (Satin), Remote Method Invocation (RMI)

• We focus here on the Java implementation for portability reasons

Improved RMI implementation• Source compatible with Sun’s RMI (replace

java.rmi.* with ibis.rmi.*)

• Reduced overhead through caching of type information

• Overall, reduce by 10% the amount of data sent over the wire

• Improved serialization– Generation of serialization code for all classes– Specially generated constructor to create empty

objects







• Conclusion

Running Environment

• DAS-2 MultiCluster– 200 Nodes on 5 sites– ≥1GB of memory– We use 72 Nodes on 1 site, other evenly distributed– Intra domain nodes linked by 100Mb/s ethernet– Inter domains linked with 1Gb/s

• OS– RedHat 7.2

• Java– Sun’s 1.5.0 and IBM’s 1.4.1

Jem3D with 51x51x51 mesh

10

30

50

70

90

2 4 6 8 10 12 14 16 18 20Number of Nodes

Exe

cuti

on

Tim

e (s

)

ProActive/RMI IBM ProActive/RMI SunProActive/Ibis IBM ProActive/Ibis Sun







• Conclusion

Comparison with a Fortran Implementation

• Not a Java vs Fortran/MPI Comparison!• We compare:

– The same algorithm– Implemented in very different languages (dynamic

data structures in Java, static in Fortran)– By different persons– With different goals

• We want to know – How much we pay for the Java features– How much we can gain with Ibis

Running Environment• Cluster located in Sophia-Antipolis

– 16 nodes of Pentium 3 at 1Ghz– 512MB of RAM– Nodes linked with 100Mb/s Ethernet

• OS– Linux 2.4.17– Sun’s JDK 1.4.2

• Fortran/MPI– MPICH-1.2.5

Sequential Version

Time Memory Java/Fortran

Mesh Java Fortran Java Fortran Time Memory

21x21x21 45s 18.9s 78M 59M 2.38 1.32

31x31x31 150s 65s 224M 164M 2.30 1.36

41x41x41 387s 156s 483M 366M 2.48 1.31

Speedup, 51x51x51

1

3

5

7

9

11

13

2 4 6 8 10 12 14 16Number of Nodes

Sp

eed

up

Theorical Speedup Fortran/MPI 51x51x51ProActive/Ibis 51x51x51 ProActive/RMI 51x51x51

Speedup, 81x81x81

5

7

9

11

13

15

8 10 12 14 16Number of Nodes

Sp

eed

up

Theorical Speedup Fortran 81x81x81

ProActive/Ibis 81x81x81 ProActive/RMI 81x81x81







• Conclusion

Conclusion

• ProActive + Ibis:– Integrated Solution– Allows a better speedup than plain RMI implementation– Gives to the application a better control on its environment

• Still slower (3x) than a “close” Fortran/MPI implementation but– Easier to compile– Easier to deploy (Jem3D ran on 150 nodes)

• Evaluate the native solutions– The native ibis version is installed/managed by an administrator– We compile against the Java classes only

Resources

• Jem3D: not available yet, but working on it

• ProActive: Available under LGPL– http://www-sop.inria.fr/oasis/ProActive

• Ibis: Available under LGPL– http://www.cs.vu.nl/ibis

• Booth #2345

http://www-sop.inria.fr/oasis/ProActive

http://www.cs.vu.nl/ibis

a high performance middleware in java with a real application

Documents