The EDGeS project receives Community research funding

1

EDGeS infrastructure for the EGEE user community Peter Kacsuk

kacsuk@sztaki.huMTA SZTAKI

2

What is EDGeS?What is EDGeS?

EDGeS: Enabling Desktop Grids for e-ScienceEDGeS: Enabling Desktop Grids for e-Science1.1. EDGeS was a 2-year FP7 project EDGeS was a 2-year FP7 project

completed on the 31completed on the 31stst of March 2010 of March 20102.2. Its primary goal was to support EGEE user Its primary goal was to support EGEE user

communities in running their applications communities in running their applications on massive volunteer desktop grid (DG) on massive volunteer desktop grid (DG) systemssystems

3.3. To achieve this EDGeS To achieve this EDGeS • created an infrastructure to extend EGEE VOs with

DG systems• provided an application porting and validation service

3

The EDGeS infrastructureThe EDGeS infrastructure

EGEE VO1

EGEE VO2

DG1 DG2 DGn

EGEE VOn

EDGeS bridge services

Jobs

WUs

3G Bridge 3G Bridge 3G Bridge

EDGeSAppl

Repository

EDGeSportal

EDGeS@home (5.000 CPUs)

Extremadura School DG (15.000 CPUs)

Generic Grid-Grid (3G) Bridge to Generic Grid-Grid (3G) Bridge to extend SGs with DGsextend SGs with DGs

Job

Han

dle

r In

terf

ace

Job Database

Queue Manager

Gri

dH

and

ler

Inte

rfac

e

Ou

rGri

dP

lug

in OurGrid

Xtr

emW

eb

Plu

gin Xtrem

Web

DC

-AP

IP

lug

in

BOINC

Job sourceEGEE GRAM

EGEE

Job sourcegUSE submitter

gUSE/WS-

PGRADE portal

3G Bridge

5

3G Bridge3G Bridge as a WS as a WS

3GBridge

JobDB

3GBridge

JobDB

Qu

eu

e M

an

ag

er

Grid

Han

dler

Inte

rfac

e

GridPlugin

2

GridPlugin

3

GridPlugin

n

WS

Sub

mitt

er

WS

Sub

mitt

er

GridPlugin

1

DownloadManager

DownloadManager

HT

TP

DHT

TP

D

Submit jobCheck status

GetOutput

Fetch input files

6

EGEE VO EGEE VO DG system DG system

EGEEWMS

DG-CE

EGEE UIEDGeS

ApplicationRepository

WatchGet output

DG

-LR

MS

EGEE BDII(with DG-CE)

Info

pro

vid

er

CheckEXE

Send job

Submitjob

EGEELB

EGEEVOMS

X509proxy

3GBridge

BOINC1(SZDG)

DC

-AP

I plu

gin

3GBridge

BOINC2(UoW)

DC

-AP

I plu

gin

3GBridge

XWebIN2P3

XW

eb p

lug

in

DG servers

WS

interface

WS

interface

WS

interface

QM

WS

client

EDGeS Services

7

EGEE VO EGEE VO DG system DG system

EGEEWMS

DG-CE

EGEE UIEDGeS

ApplicationRepository

Get EXE

WatchGet output

DG

-LR

MS

EGEE BDII(with DG-CE)

Info

pro

vid

er

Report resourcesand performance

Submit job

CheckEXE

Send job

Submitjob

EGEELB

Log eventsSend output

EGEEVOMS

X509proxy

3GBridge

BOINC1(SZDG)

DC

-AP

I plu

gin

3GBridge

BOINC2(UoW)

DC

-AP

I plu

gin

3GBridge

XWebIN2P3

XW

eb p

lug

in

DG servers

WS

interface

WS

interface

WS

interface

QM

WS

client

EDGeS Services

8

GlobalDGLocalDG

University DG e.g. Univ. of Westminster

Volunteer DG e.g.

EDGeS@home

Service Grid

EGEE

EGEE User access to EGEE User access to various gridsvarious grids

EDGeS

Appl. Repository

9

EDGeS@homeEDGeS@home

10

EGEE EGEE VO extension with a VO extension with a single DGsingle DG

EDGeS VO

UI machine

WMSMachine

Other EGEEservices

EDGeS CE machine

LCG-CE

WN

WN

WNWN

WN

WN

BOINC1 server

DBQM

WS

Inte

rfac

e

DC

-AP

I p

lug

in

QM WS client

EGEE EGEE VO extension with a VO extension with a single DGsingle DG

EDGeS VO

UI machine

WMSMachine

Other EGEEservices

EDGeS CE machine

LCG-CE

WN

WN

WNWN

WN

WN

BOINC1 server

DBQM

WS

Inte

rfac

e

DC

-AP

I p

lug

in

QM WS client

EGEE EGEE VO extension with VO extension with two DGstwo DGs

EDGeS VO

UI machine

WMSMachine

Other EGEEservices

EDGeS CE machine

LCG-CE

WN

WN

WNWN

WN

WN

BOINC1 server

DBQM

WS

Inte

rfac

e

DC

-AP

I p

lug

inQM WS client

BOINC2 server

DBQM

WS

Inte

rfac

e

DC

-AP

I p

lug

in

QM WS client

14

Portal -> SEEGRID ->DG Portal -> SEEGRID ->DG and portal->VOCE and portal->VOCE

infrastructureinfrastructure

SEEGRIDWMS

CE+BridgeClient

WSInterface

Core3G

Bridge

DGServer

DG server machine

AR

EDGeS Bridge Services CE

AR Cache

EDGeSportal

VOCEWMS

VOCECE

Desktop1

Desktop2

DesktopN

15RI-211727 NA3

version: 1.0IntroductionAuthor: Ad Emmen

MoUs with other MoUs with other projectsprojects

Signed MoU with EGEE IIISigned MoU with EGEE IIIJointly select EGEE applications that should be migrated to EDGeSCollaboration in porting these appls to EDGeSEGEE support for the establishment and operation of the EDGeS VOOrganizing joint events (summer school, user forum meetings)

Signed MoU withSigned MoU withEELA-2IberCivis DGSEE-GRID-SCI

15

16

Ported ApplicationsPorted ApplicationsApplication

nameApplicationdescription

User community

ISDEP Integrator of Stochastic Differential Equations in Plasmas, fusion plasma application

BIFI, Spain, Fusion Community of EGEE

X-ray Extraction of X-ray diffraction profiles University of Extramadura, Spain

VisIVO Visualisation Interface to the Virtual Observatory Astrophysical Observatory of Catania, Italy

GRIMP Grid-based tool for high-speed analysis of large scale genome-wide association

Erasmus Medical Centre, The Netherlands

DASP Designing periodic non-uniform sampling sequences for digital alias free signal processing

Centre for System Analysis, University of Westminster, UK

Protein Protein carbohydrate docking simulations using AutoDock

School of Biosciences, University of Westminster, UK

EMMIL E-marketplace model integrated with logistics International Business School, Hungary

CancerGrid Anti-cancer drug design SME participants in the European CancerGrid Project

Laserac Simulate the dynamics of laser devices using a cellular automata based discrete model

University of Seville, Spain

17

Ported ApplicationsPorted ApplicationsApplication

nameApplicationdescription

User community

Signal and Image Processing

Signal and image processing using WSRF-compliant web services to distribute Java and Matlab based routines

Forschungszentrum Karlsruhe, Germany

PLINK Analysis of genotype/phenotype data Atos Origin, Spain

DART Distributed Audio Retrieval Using Triana Cardiff University, UK

Blender 3D Video Rendering using Blender Blender user community

Patient Readmission

Profiling hospitals based on patient readmission statistics

Health and Social Care Modeling Group, University of Westminster, UK

UC-Explorer Understand basic universality classes of non-equilibrium system

KFKI Research Institute for Technical Physics and Materials Sciences, Hungary

InterProScan Protein Functional Analysis CNR, Italy

SLinCa Scaling laws in cluster aggregation to detect defect aggregation in materials

Kurdymov Institute, Ukraine

MVP Multi-scale Video Processing Kurdymov Institute, Ukraine

BinSys Search for binary number systems Eötvös Lóránd University, Hungary

18

Using the EDGeS Using the EDGeS infrastructure outside the infrastructure outside the

EUEU• New FP7 project: DEGISCONew FP7 project: DEGISCO• Objectives: to support non-EU countries to Objectives: to support non-EU countries to

create their national EDGeS infrastructure:create their national EDGeS infrastructure:– Brazil– China– Kazakhstan– Russia– Taiwan– Ukraina

• Representatives of other national grids are Representatives of other national grids are welcome to join DEGISCOwelcome to join DEGISCO

19

Extension of the EDGExtension of the EDGeSeS infrastructure: infrastructure: EDGIEDGI

SG (gLite)

Local DG

Public DG

SG (ARC)

SG (Unicore)

Public DG

Local DG

Local DG

Local DG

Public DG

Public DG

Cloud

Cloud

CloudCloud

Cloud

Cloud

EMI

EDGeS scope only for compute intensive apps for EGEE (glite)

EDGI scope for both compute and data intensive apps for EMI/EGI (gLite, ARC, Unicore)

Academic Cloud

Academic grid/cloud ecosystemAcademic grid/cloud ecosystem

Desktop grids (DGs)(volunteer DGs – home computers,

organizational DGs – institutional desktops)

Cluster based service grids (SGs)(EGEE, OSG, etc.)

Supercomputer based SGs

(DEISA, TeraGrid)OGF PGI

EDGeS/EDGI

EDGI

21

SummarySummary• EDGeS enables to extend gLite VOs with EDGeS enables to extend gLite VOs with

– BOINC or XtremWeb public or local DGs – Current DGs:

• EDGeS is an open infrastructure:EDGeS is an open infrastructure: – Any gLite based VO can be extended with DGs– EDGI helps in EU countries and DEGISCO in non-EU countries

to extend gLite based VOs with DGs– Anyone can support EGEE applications by donating desktop

cycles to EDGeS@home

• Any EGEE user can access EDGeS:Any EGEE user can access EDGeS:– EDGI and DEGISCO helps any EGEE user community in porting

their applications into EDGeS– EDGeS Application Repository stores applications already

ported to EDGeS and usable by any EDGeS users

22

For more information please visit the EDGeS Website:

Join the EDGeS User & Industry Forum here!

http://www.edges-grid.eu/

Please contact us if you need support in porting your application to EDGeS

Email: kacsuk@sztaki.hu

Thank you for your Thank you for your attention …attention …

Anyquestions?