SZTAKI Desktop Grid – a Hierarchical Desktop Grid System

P. Kacsuk, A. Marosi, J. Kovacs, Z. Balaton, G. Gombas, G. Vida, A. Kornafeld

MTA SZTAKI

www.lpds.sztaki.hu

Desktop Grid model

Internet/Intranet

Dynamic resource donation

Work package distribution

Company/univ.

server

Donor: Company/

univ. or private PC

Donor: Company/

Univ. or private PC

Donor: Company/

univ. or private PC

Application

Characteristics of the desktop Grid model

• Anybody can donate resources• Heterogeneous resources, that

dynamically join and leave• One or a small number of projects

can use the resources• Asymmetric relationship between donors

and users: U << D

• Advantage: • Donating a PC is extremely easy• Setting up and maintaining a DG server is

much easier than installing the server sw of utility grids

Master/slave parallelismand parameter studies

Internet

Master

Workunit-1

Workunit-2

Workunit-3

Workunit-N

DG Server

Types of Desktop Grids

• Global Desktop Grid• Aim is to collect resources world-wide for grand-

challenge scientific problems

• Examples: • SETI@home• SZTAKI Desktop Grid global version at:

http://szdg.lpds.sztaki.hu/szdg/

• Local Desktop Grid• Aim is to enable the quick and easy creation of

grid for any community (company, univ. city, etc.) to solve their own applications

• Example: • SZTAKI Desktop Grid local version

SZTAKI Desktop Grid: Global version

• Main objective: • Demonstrate the usage of DG systems for

any community

• Supported application: searching 12-dimension binary number systems

• Number of registered donors: ~12500• Number of registered PCs: > 22000• How to register a PC?

• http://www.lpds.sztaki.hu/szdg/

SZTAKI Desktop Grid global version

SZTAKI Desktop Grid performance

TOP 500 entry performance: 1645 GFlops

SZTAKI Desktop Grid local version

• Main objective: • Enable the creation of a local DG for any

community• Demonstrate how to create such a system

• Building production Grids requires huge effort and represents a privilege for those organizations where high Grid expertise is available

• Using the local SZDG package • Any organization can build a local DG in a day with

minimal effort and with minimal cost• The applications of the local community will be executed

by the spare PC cycles of the local community• There is no limitation for the applied PCs, all the PCs of

the organization can be exploited (heterogeneous Grid)

DSP application on a local SZDG in the Univ. of

Westminster• Digital Signal Processing

Appl.: Designing optimal periodic nonuniform sampling sequences

• Currently more than 100 PCs connected from Westminster and planned to extend over 1000 PCs

DSP size Production SZDG

20

22

24

~35min ~1h 44min

~7h 23min

~141h ~46h 46min

The speedup

~5h 4min

Sequential

~3h 33min

~41h 53min

~724h

• If many organizations (universities, companies, towns, etc.) have already set up their own DG, then the next step is to connect these DGs together

● We distinguish 4 levels in the SZDG future vision:● Normal Desktop Grid (the current SZDG, available as

package)● Mixed Desktop Grid (available as package)

● Goal is to include clusters in a local SZDG ● Hierarchical Desktop Grid (available as prototype)

● Goal is to build larger DGs from smaller ones in a hierarchical way● E.g. Enterprise DG can be built exploiting PCs of the dept. DGs● Expected release by September (work in a Jedlik Ányos project)

● Collaborative Desktop Grid (in planning phase)● Goal is to create generic Grid out of local and global DGs

Future vision of SZDG

LocalDEG

LocalDEG LocalDEG

Normal Desktop Grid

University Dept. DG

University Faculty DG

• Each local DG runs the applications of the local community (univ. dept., faculty, enterprise, etc.)

Enterprise DG

LocalDEG

LocalDEG LocalDEG

Mixed Desktop Grid

University Dept. DG

University Faculty DG

Enterprise DG

Local DGs can be extended with local clusters (the current SZDG release includes this feature and used at several universities)

LocalDEG

LocalDEG

LocalDEG

LocalDEG LocalDEG

Hierarchical Desktop Grid

University DG

Enterprise Dept. DG

• Local DGs at the lower level of hierarchy can be used to solve the applications of the higher level DGs.

• E.g. univ. dept. and faculty DGs contribute to the university level DG

University Dept. DG University Faculty DG

Enterprise DG

LocalDEG

LocalDEG

LocalDEG

LocalDEG LocalDEG

GlobalDEG

LocalDEG

Production Grid

Collaborative DG

University DG

Enterprise DG

GlobalDEG

GlobalDEG

Grid service provider

Usage of local SZDG in industry

• Comgenex Ltd.• Drug discovery application• Creating enterprise Grid for prediction of

ADME/Tox parameters• Millions of molecules to test according to potential

drug criteria

• Hungarian Telecom• Creating enterprise Grid for supporting large data

mining applications where single computer performance is not enough

• OMSZ (Hungarian Meteorology Service)• Creating enterprise Grid for climate modeling

Assessment of the desktop Grid model

• Advantages• Easy to create and maintain• Any organization can quickly and cheaply install it• Easy to program and hence no steep learning curve• Robust technology• Industry can use it as enterprise Grid

• Drawbacks• Only limited number of applications can be run on a DG

• Remedy by SZDG• Many DGs can be connected in a hierarchical way – at

every level of the hierarchy several applications can be used

Conclusions

● Desktop Grids are here for any community (universities, companies , etc.). They can

● access and/or ● build Grid systems

● SZTAKI Desktop Grid technology enables ● easy creation and programming of local desktop Grids

● SZTAKI is ready to help any organization ● to set up its local DG(s)● to port applications on local DGs ● to train people how to build and use local DGs

● More information on: http://www.lpds.sztaki.hu/szdg/