EUROPEAN UNION

PL-Grid:PL-Grid:National Grid Initiative in PolandNational Grid Initiative in Poland

for supporting Computational Science for supporting Computational Science in the European Research Spacein the European Research Space

Jacek KitowskiJacek KitowskiInstitute of Computer Science AGH-USTInstitute of Computer Science AGH-UST

ACK CYFRONET AGH, Cracow, PolandACK CYFRONET AGH, Cracow, Poland

in collaboration within collaboration withPL-Grid RepresentativesPL-Grid Representatives

Michał Turała, Kazimierz Wiatr, Marian Bubak, Tomasz Szepieniec,Michał Turała, Kazimierz Wiatr, Marian Bubak, Tomasz Szepieniec,Marcin Radecki, Jacek Niwicki, Alex Kusznir, Zofia Mosurska, Mariusz SterzelMarcin Radecki, Jacek Niwicki, Alex Kusznir, Zofia Mosurska, Mariusz Sterzel

Piotr Bała, Wojciech Wiślicki, Norbert Meyer, Krzysztof Kurowski, Piotr Bała, Wojciech Wiślicki, Norbert Meyer, Krzysztof Kurowski, Józef Janyszek, Bartłomiej Balcerek, Mścisław Nakonieczny, Rafał TylmanJózef Janyszek, Bartłomiej Balcerek, Mścisław Nakonieczny, Rafał Tylman

....NEC 2009: XXII International Symposium on Nuclear Electronics and Computing

Varna, September 7-14, 2009

National Grid Initiative in Poland in a nutshell Motivation Systematic approach

to establishing the Consortium and NGI Rationales and Foundations International Collaboration Status

PL-Grid Project Infrastructure Workpackages Activities Grid Examples

Summary

OutlineOutline

E-Science approach to research Integration activities ongoing in the world

Need for e-Science ApproachNeed for e-Science Approachin spite of complexity: Computing, Storage, Infrastructurein spite of complexity: Computing, Storage, Infrastructure

E-Science: collaborative research supported by advanced distributed computations Multi-disciplinary, Multi-Site and Multi-National Building with and demanding advances in Computing/Computer Sciences

Goal: to enable better research in all disciplines System-level Science: beyond individual phenomena, components interact

and interrelate to generate, interpret

and analyse rich data resources• From experiments, observations

and simulations

• Quality management, preservation and reliable evidence

to develop and explore models and simulations• Computation and data at all scales

• Trustworthy, economic, timely and relevant results to enable dynamic distributed collaboration

• Facilitating collaboration with information and resource sharing

• Security, trust, reliability, accountability, manageability and agility

M. Atkinson, e-Science (...), Grid2006 & 2-nd Int.Conf.e-Social Science 2006, National e-Science Centre UKI. Foster, System Level Science and System Level Models, Snowmass, August 1-2, 2007

Thanks to CYFRONET activity

five High Performance Computing Polish Centres analysed according to: Participation in International and National Projects and Collaboration Needs by Polish Scientific Communities Computational resources to date European/Worldwide integration Activities National Network Infrastructure ready (thanks to Pionier National Project)

Rationales behind PL-Grid ConsortiumRationales behind PL-Grid Consortium

GEANT2GEANT2

~35 international projects FP5, FP6, FP7 on Grids (50% common) Grid Projects: Crossgrid, Gridstart, GridLAB, Unicore, EGEE I,II,III, K-WfGrid,

Coregrid, Virolab, Gredia, int.eu.grid, Eurogrid, Grip, Unigrids, Balticgrid I,II, Gridlab, Intelligrid, Ringrid, Brein, Beingrid, Qoscosgrid, Chemomentum, EUChinagrid, PRACE,…

~15 Polish projects (50% common) Pionier, Progress, SGIgrid, Clusterix, NDS, Platon, IT-SOA, Powiew, NewMAN...

Close collaboration with Institute of Computer Science AGH-UST

Participation in Projects and CollaborationParticipation in Projects and Collaboration

Need by Polish Scientific Communities Need by Polish Scientific Communities

Scientific Community /Representative

% Polish publications

Warsaw / ICM 29,0 %

Cracow / Cyfronet 16,4 %

Wrocław / WCSS 11,1 %

Poznań / PCSS 10,1 %

Gdańsk / TASK 6,8 %

SUM 73,4 %

Poland 100 %

Distribution of Polish publications in period 01.2004 – 04.2008 according to

Science Citation Index Expanded + Social Science Citation Index + Arts & Humanities Citation Index

Partners’ Resources to DatePartners’ Resources to Date

TOP500 – Nov.2008 Polish SitesTOP500 – Nov.2008 Polish Sites

Partners’ Resources to DatePartners’ Resources to Date

TOP500 – June 2009 Polish SitesTOP500 – June 2009 Polish Sites

PL-Grid Consortium Founders PL-Grid Consortium Founders

• Academic Computer Center Cyfronet AGH (ACK CYFRONET AGH) Coordinator • Poznań Supercomputing and Networking Center (PCSS)• Wrocław Centre for Networking and Supercomputing (WCSS)• Academic Computer Center in Gdańsk (TASK)• Interdisciplinary Center for Math. and Computat. Modelling, • Warsaw University (ICM)

GEANT2GEANT2

PL-Grid Foundations – SummaryPL-Grid Foundations – Summary

Motivation E-Science approach to research EGI initiative ongoing in collaboration with NGIs

Milestones: Creation of Polish Grid (PL-Grid) Consortium: http://plgrid.pl

Consortium Agreement signed in January 2007 PL-Grid Project (2009-2012)

Application in Operational Programme Innovative Economy, Activity 2.3 (in Sept. 2008) Get funded March 2, 2009 (via European Structural Funds)

Consortium made up of five largest Polish supercomputing and networking centres (founders) ACK CYFRONET AGH (Cracow) – Coordinator

Polish Infrastructure for Supporting Computational Science in the European Research Space

Response to the needs of Polish scientists and ongoing Grid activities in Poland, other European countries and all over the world

Grid infrastructure (Grid services) PL-Grid

App

licat

ion

App

licat

ion

App

licat

ion

App

licat

ion

Clusters High Performance Computers Data repositories

National Computer Network PIONIER

DomainGrid

Advanced Service Platforms

DomainGrid

DomainGrid

DomainGrid

Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in

other countries. Specialized, domain Grid systems – including services and tools focused on specific types of

applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of

separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World

Grids thanks to cooperation with teams involved in the development of European Grid systems (EGI, EGEE, DEISA, OMII, C-OMEGA, ESFRI).

Plans for HPC and Scalability Computing enabled.

Tighly-coupled activities

PL-Grid Base PointsPL-Grid Base Points

Elements and FunctionalityElements and Functionality

PL-Grid software will comprise: user tools (portals, systems for

applications management and monitoring, result visualization and other purposes, compatible with the lower-layer software used in PL-Grid);

software libraries; virtual organization systems: certificates,

accounting, security, dynamic ; data management systems: metadata

catalogues, replica management, file transfer;

resource management systems: job management, applications, grid services and infrastructure monitoring, license management, local resource management, monitoring.

Users

Nationalcomputernetwork

Grid Application

Programming Interface

Virtual organizations andsecurity systems

Basic Grid services

Gridservices

LCG/gLite(EGEE)

UNICORE(DEISA)

OtherGrids

systems

Gridresources

Distributedcomputational

resources

Grid portals, development tools

Distributed data

repositories

Three Grid structures will be maintained: production, reseach, development/testing.

ChallengesChallenges

Short term To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To select computational/storage resources for PL-Grid infrastructure

Long term – continously To provide the necessary infrastructure (!!)

• Computer rooms, electrical power, many organizational issues Be prepared / work on approaching paradigms and integration development

• HPC and Scalable Computing (Capability and Capacity Computing)

• Clouds (internal-external, computing clouds, data clouds)

• SOA paradigm, knowledge usage …

• „Future Internet” as defined by EC in Workprogramme

The PL-Grid Project is split into several workpackages

Planned Realization of AimsPlanned Realization of Aims

PLANNING AND DEVELOPMENT

OF INFRASTRUCTURE

P2

CoordinationStructure

Operation RulesDissemination

PROJECT MANAGEMENTP1

SECURITY CENTER

P6

Training

SUPPORT FOR VARIOUSDOMAIN GRIDS

P5P4 GRID SOFTWAREAND USERS

TOOLS DEVELOPMENTEGEE DEISA … .

OPERATIONS CENTERP3

Main Project Indicators: • Peak Perf.: 215 Tflops • Disk Storage: 2500 TB

WP1: Management and OrganizationsWP1: Management and Organizations

WP2: Planning of Infrastructure DevelopmentWP2: Planning of Infrastructure Development

Hardware

• Analysis of users’ requirement

• Continuous analysis of worldwide activity in infrastructure development and vendors’ offers

• Expecting results: Current activity: organization of pilot PL-Grid VO using EGEE infrastructure Current and long-term activity: Analysis of hardware trends and offers

Recommendation from analysis: technical details of clusters with x86 processors

WP3: Operations Center’s tasksWP3: Operations Center’s tasks

EGIGroup

EGI Testing

Middleware

EGI ProductionMiddleware

Coordination of Operation Management and accounting EGI and DEISA collaboration

(Scalability and HPC Scalability Computing)

Users’ requirements analysis for operational issues

Running infrastructure for: Production Developers Research

to consider:

Computational Cloud Data Cloud Internal and External

Clouds Virtualization aspects

WP4: Grid Software and Users Tools WP4: Grid Software and Users Tools

Workpackage on Development/Research

• Analysis of users’ requirements

• Software repository (different tools and components developed by Partners)

• Reengineering of tools and applications (legacy codes, APIs, SOA applications)

• Workflows, applications compositions

• Virtual Laboratory (component applications, workflows, applications monitoring) – from EU FP6 Virolab Project

• High-level virtual organizations construction using knowledge (semi-automatic creation using contract, semantic description of resources, monitoring of SLA due to QoS), data access, output contracts in OWL for security configuration – from EU FP6 Gredia Project

• Tools for management, proactive monitoring and security (Bazaar, Permis)

• In general: software services, virtualization on various levels (computing, storage, network, operating system…) knowledge, semantics, QoS and SLA in various aspects… XaaS (SaaS, HPCaaS, Scalable Computing aaS)

WP5: Users’ Support and TrainingWP5: Users’ Support and Training

„User Interface”

• Running Help-desk Identification of software and problems with licences

• ATLAS, ALICE, CMS, LHCb,

• Gussian Turbolome, NAMD, CPMD, Gamess, Dalton, NWChem. ADF

• Compass, MOPAC, Blast, ClustalW2, OpenBabel, Macromodel, Autodock

• Charmm, Compchem,

• Making commercial software available for the users (license activity) Training, education

WP6: SecurityWP6: Security

Users’ support

• Authorization, authentication, ...

• Certificates ...

Some examplesSome examples

Concerning applications Concerning operational activities

21

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688 t.szepieniec@cyfronet.pl

• GAUSSIAN in Grid• Gaussian VO created supported by several Partners (EGEE

activity)

• Accepted by Vendor

• Registration: https://voms.cyf-kr.edu.pl:8443/vo/gaussian/vomrs

Computational Chemistry

• VO Manager: Mariusz Sterzel (CYFRONET), EGEE II Comput. Chemistry coordinator m.sterzel@cyfronet.pl

ACK: Tomasz Szepieniec, Mariusz Sterzel

Enabling Grids for E-sciencE

EGEE-II INFSO-RI-031688 t.szepieniec@cyfronet.pl

Biotechnology in Grid

ComputingElement

Euchina Virtual Organization (EGEE)

StorageElement

2. Transfer

application

User InterfacePortal

AABTDDSAD

1.Submit

sequence

3. Store protein

4. Visualize

PDB1.32 3.23 3.442.77 4.33 5.661.32 3.23 3.44

ACK: Irena Roterman, Jagiellonian University, Tomasz Szepieniec

Never Born Protein Folding

Other Activities – Collaboration and DisseminationOther Activities – Collaboration and Dissemination EGI_DS – CYFRONET activity

Policy Board (deputy: M. Turala), NGI Observers -- „Polish Experts” (WP3, WP5) participation in Workshops…

EGI Steering Group of Policy Board - proposal preparation (Alex Kusznir from Cyfronet and Ivan Maric from Croatia) Contribution to proposals EGI.org, SSC (from PL-Grid: T. Szepieniec, M. Sterzel, P. Bala)

EGEE I-III From Cyfronet: chairing Resource Allocation Group in EGEE-III – trying to influence EGI

• Operate CE ROC

• More: https://twiki.cern.ch/twiki/bin/view/EGEE/RAG E-IRG – CYFRONET activity

Workshops Discussions on EGEE – EGI – NGI future

EU Unit F3 „Research Infrastructure” Experts in Program Committee, nominated by the Polish Ministry of Science and High Education (since 2007) from CYFRONET (M. Bubak, J. Kitowski)

ENPG seminar in Cracow (2008) Dissemination, promotion, conferences:

International ENPG Seminar in Cracow (2008) International Cracow Grid Workshop 2007, 2008, 2009 Cyfronet Users’ Annual Conference (2008 and 2009) Cyfronet Open Day (2007, 2008) International . Conf. Computational Science 2008 EGEE 2009, Barcelona event Inter. Users Conf. HP-Cast 2009, Madrid

Polish NGI (PL-Grid) Expression of Interest Polish NGI (PL-Grid) Expression of Interest for Hosting Selected EGI Global Task for Hosting Selected EGI Global Task (EGI.org and SSC Proposals)(EGI.org and SSC Proposals)

O-E-10: Coordination of resource allocation and of brokering support for VOs from NGIs (Cyfronet – T. Szepieniec)

O-E-5: Grid operation and oversight of the e-Infrastructure (Cyfronet – M. Radecki) O-E-11: Coordination of interoperations between NGIs and with other Grids (Cyfronet – T. Szepieniec) O-E-13: Coordination of definition of best practices, operations procedures, operations requirements

(Cyfronet – T. Szepieniec) O-E-14: Operation of production Grid core services, catch-all services for international VOs

(Cyfronet – T. Szepieniec) O-E-3: Operation of the grid repositories storing monitoring and performance data, and other related information. (PCSS

Poznan – M. Lichwała) O-E-9. Coordination of middleware roll-out and deployment, middleware pilot and certification testbeds

(Cyfronet – M. Radecki)

Chemical and Material Science and Technology Specialised Support Centre (CMST SSC)based on Computational Chemistry Cluster of Excellence in EGEE

Coordinator : Cyfronet – M. Sterzel Deputy: Uni.of Perugia – A. Lagana Participants:

• University of Perugia, Dept of Chemistry Italy

• ACC Cyfronet AGH Poland

• National Center for Biomolecular Research Czech Republic

• IT Center for Science ltd Finland

• Ente nazionale energie alterantive Italy

• Consorzio Interuniversitario CINECA Italy

• Theoretical chemistry and computational grid applications Switzerland

• Foundation for Research and Technology Hellas, Inst. Electronic structure and lasers Greece

• Democritos, ICTP, Trieste Italy

• University of Barcelona Spain

EGEE-IIIEGEE-III / CE ROC / CE ROC

• CE ROC – coordinator: ACK CYFRONET-AGH

• Consortium consists of institutes from 7 countries:Austria, Bielarus, Croatia, Czech, Hungary, Poland, Slovakia, Slovenia

• Operate, maintain and support EGEE Grid Infrastructure in CE region

• Virtual Organizations supported:a) LHC: alice, atlas, lhcb, cmsb) Biomed: medical imaging, bioinformatics, drug

discoveryc) vo.cta.in2p3.fr: Monte Carlo and Data Production for

Cherenkov Telescope Discoveryd) imon, imain: VO from Interactive Grid for managemente) auger: Pierre Auger Cosmic Ray Observatory

Poznan-PSNC

Cracow-CYFRONET

Warsaw-ICM

cores storage

Published by sites

EGEE-III / CE ROCEGEE-III / CE ROC

27

Summary – ActivitiesSummary – Activities

Short term To start – establishing PL-Grid VO using Partners’ local and EGEE resources To provide resources for covering operational costs To provide resources for keeping international collaboration ongoing To select and install computational/storage resources for PL-Grid infrastructure

Plans for 2009: 100Tflops, 700TB Long term – continously

To provide, keep and extend the necessary infrastructure (!!)

• Computer rooms, electrical power, many organizational issues

• Towards Cloud Computing (internal, external; computational, data)

• Towards HPCaaS and Scalable computing

http:plgrid.plhttp:plgrid.pl

29

Back-up slidesBack-up slides

30

National and International Integration InitiativesNational and International Integration Initiatives

Ongoing European and Worldwide activities and consolidation EGEE EGI_DS, EGI DEISA PRACE ......

European e-

Infrastructure

2000 Testbeds 2010 Utility ServiceRoutine Usage

National

International

SGI Grid, Progress(Clusterix, National Data Store ...)

Chemomomentum, Virolab, CoreGrid, Gredia, int.eu.grid, Baltic Grid, GridLab,

Porta Optica, RINGRid, Phosphorus, QoSCoSGrid, Intelligrid, K-WfGrid, Unicore...

Other ActivitiesOther Activities

Computational Chemistry (VO) in EGEE Third CPU power consumer

• Nearly 3 million of jobs executed during 2007 Lead by Cyfronet since 2006

• Recently management of Computational Chemistry Cluster of Excellence In parallel current effort include:

• Grid ports of commercial and non commercial software packages with particular focus on their parallel version

• Development of “experiment centric” grid web portal for chemists

EGEE III / CE ROC

Grid infrastructure (Grid services)PL-Grid

App

licat

ion

App

licat

ion

App

licat

ion

App

licat

ion

Clusters High Performance Computers Data repositories

National Computer NetworkPIONIER

DomainGrid

Advanced Service Platforms

DomainGrid

DomainGrid

DomainGrid

Assumptions Polish Grid is going to have a common base infrastructure – similar to solutions adopted in

other countries. Specialized, domain Grid systems – including services and tools focused on specific types of

applications – will be built upon this infrastructure. These domain Grid systems can be further developed and maintained in the framework of

separate projects. Such an approach should enable efficient use of available financial resources. Creation of a Grid infrastructure fully compatible and interoperable with European and World

Grids thanks to cooperation with teams involved in the development of European Grid systems (EGEE, DEISA, OMII, C-OMEGA, ESFRI).

Plans for HPC and Scalability Computing enabled.

PL-Grid InfrastructurePL-Grid Infrastructure

Users

Nationalcomputernetwork

Grid Application

Programming Interface

Virtual organizations and

security systems

Basic Grid services

Gridservices

LCG/gLite(EGEE)

UNICORE(DEISA)

OtherGrids

systems

Gridresource

s

Distributedcomputational resources

Grid portals, development tools

Distributed data

repositories