grid computing framework a java framework for managed modular distributed parallel computing

Grid Computing FrameworkA Java framework for managed modular distributed parallel computing

Presentation Plan

1. Discussion on distributed computing(Concept, examples, insight)

2. What are we doing (new?)(Our concept, innovation, vision)

3. Who is doing what(Technical details, responsibilities, SE Practices)

4. What remains to be done(Or as they say – ‘future scope’)

11/11/2010 Abhishek Ganguly | Apoorv Pandey 2

Grid Computing Framework

Parallel Processing (problem classes)1. SIMD

(Single Instruction Multiple Data)

Used mostly in DSP i.e.Digital Signal Processing applications


Grid Computing Framework – Discussion on distributed computing

Bass Boost

Left | Center | Right



2. MISD(Multiple Instruction Single Data)

Very few actually use or implement this aspect of parallelism

Task replication systems may use this technique for fault tolerance or error masking etc





2. MISD(Multiple Instruction Single Data)

3. MIMD(Multiple Instruction Multiple Data)

Most commonly adopted distributed processing strategy



Parallel Processing (Solution Approaches)






Centralized parallelism

Common Memory (RAM)

Common Die(Multi Core)

Common Board(Multi Processor)




Centralized parallelism

Multi Core

Common Memory (RAM)



• Common Die or Chip. Everything on a single Chip.• Common System bus.• Common L2 cache

• Individual L1 cache• Individual Register file• Individual ALU

• May or may not share a common Clock




Multi Processor

Common Memory (RAM)



• Common Motherboard• Common OS instance• Common RAM

• Individual Clock• Individual System Bus• Individual RAM access

• Higher Throughput than multi core • Higher power consumption than multi core



Distributed parallelismParallel Processing (Solution Approaches)

Reduced Coupling



Distributed parallelism – SUPER COMPUTERParallel Processing (Solution Approaches)

• Highly coupled systems with specialized hardware built for the purpose• May be composed of hundreds of motherboards, each housing multiple CPUs• RAMs spread all over the housing• Interconnected by special high speed bus (Network)• Each board may have its own copy of the OS.

Requires• Special OS• Special software to run atop it• Special programming methodology



Distributed parallelism – CLUSTERSParallel Processing (Solution Approaches)

•These fall between super computers and grid computers•These are networked computers•Computers are connected through high speed LAN•Each computer has its own instance of OS running for it

These are the highest deployed distributed systems.

Uses•Load balancing (as in web servers) •Availability, Robustness, fail-safe (DNS serves)•Throughput increase (database clusters)



Distributed parallelism – GRID COMPUTERSParallel Processing (Solution Approaches)

• Most diffused, heterogeneous and independent form of multi node alliance. • Components of a grid computer can be anything from a commodity PC to clusters of supercomputers.• Components can be located al over the globe.• Internet is the basic networking medium between the components.

Requires• Special programming technique• Specially written software• Extensive GRID framework



Grid Computer Architecture

Grid User(Programmer) Grid Coordinator Server

Workers


Grid Computing Framework – What are we doing (new?)

Project Proposal

To implement such a distributed framework in java

To adapt java multi-threading concepts for scalable distributed computing



Implementing Distributed Framework

Network

MachineMachine MachineMachine MachineMachine

JVMJVM JVMJVM JVMJVM JVMJVM

Grid Frame Work

UserProgram User

Sub-ProgramUser

Sub-Program

Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code Grid CodeGrid Code

Grid APIGrid API Grid APIGrid API Grid APIGrid API



Implementing Distributed Framework

Network

Machine Machine Machine

JVM JVM JVM JVM

Grid Frame Work

UserProgram User

Sub-ProgramUser

Sub-Program

Grid Code Grid Code Grid Code Grid Code

Grid APIGrid API Grid APIGrid API Grid APIGrid API



Adapting Multi Threading Concepts

Common Java Code

Common Java Code



Adapting Multi Threading Concepts

Grid Aware java

Code


Grid Computing Framework – Who is doing what, Technical details etc

Functional Objectives

Research

Design

Coding

Testing by using



Research on existing work

Basic understanding of parallel computingAbhishek & Apoorv

MPI, Java RMI , Directory and Naming servicesApoorv

Object Serialization, CORBA, Dot net RemotingApoorv

Existing grid computing frameworks. BOINC, OPEN-Grid, Abhishek

Distributed file systems, Distributed SynchronizationAbhishek

Software DesignSystem Architecture



Client

Job ServerWorker




Client

Job ServerWorker

Client uses Grid APIorg.accurate.grid

org.accurate.grid.Job

Job

Runnable SerializableThread

void start()void Run()void join()




Client

WorkerJob Server

ServerManages job Queue

&ManagesCompleted jobs’ bin




Client

WorkerJob Server



JobServer.submitJob( Job j );>>




Client

WorkerJob Server


&ManagesCompleted jobs’ bin Job j=JobServer.getJob()>>




Client

WorkerJob Server


&ManagesCompleted jobs’ bin JobServer.doneJob( Job j )>>




Client

WorkerJob Server



Job j =JobServer.getCompleted(this)>>

Software DesignData Flow Diagram 0



Grid FrameworkGrid Framework

Jobs

Completed Results

Software DesignData Flow Diagram 1



1.2Job Queuing

and dispatching

1.2Job Queuing

and dispatching

1.1Job Creation

1.1Job Creation

1.3Job Execution

1.3Job Execution

Code

blockUser Program

User Program

Job Queue Completed Job Store`` ` `

User Program

User Program

Child job modules

All that remains to be done


Grid Computing Framework – Future Scope

Virtual distributed file system.A distributed file system for high volume data interchange

Dedicated application servers.So that long running applications do notrequire their originating PCs to stay on.

Integration with existing grid computing systems.So that jobs can be interchanged between them.

Automatic parallelization.So that code can be parallelized withoutexplicit programmer effort.

All that remains to be done…


Grid Computing Framework – Future Scope

Wrapper to precompiled codeSo that precompiled modules can bereused without reengineering

Worker credit trackingSo that workers can be gifted benefit a/c to work they do

Standardization of APISo that programmers of different platformscan feel comfortable.


Grid Computing Framework – References

90% of the research material has been obtained from the internetAnd rest 10% from text books

Text Books:

Herbert Schildt, Java 2- The complete reference, Osborne.Client Server, Most of practical java

Bruce Eckel, thinking in java, PearsonMost of theoretical java

Galvin, Operating system conceptsScheduling, multitasking, synchronization


Grid Computing Framework – References

90% of the research material has been obtained from the internetAnd rest 10% from text books

Websites :

WikipediaAbout everything has been consulted on Wikipedia

Oracle sun tutorial-trailjava code samples, in-depth class structure

boinc.berkeley.eduBasic understanding of one of the most successful grid.


Grid Computing Framework – Rebuttle

Questions


Grid Computing Framework – Thank you

Thank You

grid computing framework a java framework for managed modular distributed parallel computing

Documents

distributed computing

single chip

common system bus

coupled systems

fault tolerance

specialized hardware

technical details

se practices