introduction to the course - ut€¦ · introduction to the course week #1 basics of grid and cloud...

Introduction to the course

Week #1Basics of Grid and Cloud computing

Hardi [email protected]

Tartu ÜlikoolFebruary 13 2013

Satish [email protected]

Basics of Grid and Cloud computing 2/33

Overview

● BGCC administrative information

● Introduction to Grid Computing

● Authentication procedures


BGCC course● 2005 – 2009 Basics of Grid computing

– by prof. Eero Vainikko

– gLite middleware

● 2010 – Basics of Grid and Cloud computing– Dr Satish Narayana Srirama joined with Cloud part

– gLite and ARC middlewares

● 2013 – same but– Eero Vainikko is at the University of Bath, UK

– almost no gLite

● 2014 – Course will not be delivered in the current form– Cloud Computing becomes an independent course (6 ECTS)


Web and discussion

● Course web page – http://courses.cs.ut.ee/2013/cloud

– Contacts

– Times and rooms

– Lectures slides

– Lab exercises and deadlines

● Mailing list– [email protected]


Course Structure

● Basics of Grid Computing● 7 Lectures ● 7 Labs ● Structure will be introduced in the coming slides

● Basics of Cloud Computing● 7 Lectures● 7 Labs● No course projects for this year :-)

● Structure will be discussed on 3rd April 2013● http://courses.cs.ut.ee/2013/cloud


Grades and points

● Grid part 50% and Cloud part 50%

● You can earn 50 Grid points:– 25p Grid part of the exam

– 20p Grid Labs

– 5p Active participation at lectures● Devising questions for on-line study-questionary in 24h after each

lecture

● Two different courses – you should pass both– You have to score at least 50% in both courses


Grid part given by

● Hardi Teder– Lectures and Labs on Wednesday

● Pelle Jakovits– PhD student at Mobile Cloud Lab

– Labs on Tuesday and Thursday

● Ilja Kromonov– Student at Mobile Cloud Lab

– Labs on Thursday


About me*

● Related with Grids and HPC since 2003

● at EENet since 2003● Grid projects:

– BalticGrid I-II, EGEE/EGI, Nordic Grid Neighbourhood

● Computing and storage resources

● X509 certificates and PKI● AAI and Federations

● Head of Distributed computing department at EENet

● EGI council member● EGI security policy group

member● Baltic Grid CA manager

– EUGridPMA, IGTF, OGF CAOPS-WG

● TERENA – TCS and REFEDS

* “me” is Hardi


7 Grid lectures

● Introduction to Grid computing

● Grid middlewares and infrastructures

● Grid information services

● Data management on Grid

● Grid security

● Grid job management

● About Grid experiments


7 Grid labs● Requesting the Grid certificate and VO registration and Python

warm-up. (2p)

● Hello Grid! Hello ARC! (2p)

● Grid information services (2p)

● Data Management on Grid (3p)

● Grid security and breaking RSA (4p)

● Grid experiment – Job management (4p)

● Grid experiment - GPUs on Grid (3p)

● Usually the deadline of exercises is 2 weeks but not always ● 50% of points after deadline in Grid exercises


Introduction to Grid ● Who needs it?

● What is Grid?

● What is Grid useful for?


Driving forces of computational science

● Environment simulation● Climate changes● Prediction of amount of fish in

Norwegian fjords● Ice glacier flow simulation

● Solving fluid dynamic problems

● Weather predictions● Design of hypersonic airplanes● Design of more efficient cars● Extremely quiet submarines● Design of efficient and safe

nuclear power stations

● Simulation of nuclear explosions

● Satellite data analysis

● Data analysis of DNA-sequences

● Simulation of 3D proteine molecules

● Simulation of global economical processes

● etc. in more and more fields


Driving forces of computational science

● Common to all examples - need for larger than usual set of resources

● CPU cycles● Data volumes● Special devices producing

data● Parallel processing

● Common problems● how to store data?● how to move data?● which algorithms can be

used?


Why the Grid?

● Science is becoming increasingly digital and needs to deal with increasing amounts of data.– Large Hadron Collider (LHC), Radio

telescopes, gene research

● More complicated simulations grew bigger than HPC centres could provide resource

● Collaboration– Grid provides infrastructure for sharing

resources.

Concorde(15 Km)

Balloon(30 Km)

CD stack with1 year LHC data!(~ 20 Km)

Mt. Blanc(4.8 Km)


Grid

Grid is securely share distributed resources (computation, storage, etc) so that users can collaborate within Virtual Organisations (VO):


GRID

MIDDLEWARE

Visualization

Supercomputer, PC-Cluster

Data-storage, Sensors, Experiments

Internet, networks

Desktop

Mobile Access


Middleware

● Tools and packages for building Grid:– Globus toolkit

– Nordugrid ARC

– gLite

– Unicore

● EMI

European

Middleware

Initiative


Grid foundations


Resource management

● Computing resources

● Storage resources

● Other specific resources


Information services

● Maintains information about hardware, software, services and people participating in a Virtual Organization– Should scale with the Grid´s growth

– Sharing jobs

– Logging and accounting

– Monitoring


Data management

● Data access and transfer– Simple, automatic multi-protocol file transfer tools: Integrated

with Resource Management service● Move data from local machine to remote machine, where the job is

executed (input file staging)● Move the output files from the remote computer to the local

machines (output file staging)● Pull executable from a remote location

– To have a secure, high-performance, reliable file transfer over modern WANs: GridFTP

● Data replication and management


Grid security

● Basic security:– Authentication: Who we are on the Grid?

– Authorization: Do we have access to a resource/service?

– Protection: Data integrity and confidentiality

● Grid Security Infrastructure (GSI):– Grid credentials: digital certificate and private key

● International trust– IGTF

– EUGridPMA


AuthN and AuthZ

Grid resources (A)

Grid resources (B)

Certification Authority (CA)BobCert request

User Interface (UI)

Bob´s Grid certificate

VO Database

VO ServiceVO

Manager

VO membership request

VO

VO Account

Pool

VO Account

Pool

Automatic mappingfor Bob

Automatic mappingfor Bob

voms-proxy-init


Virtual Organizations

● Distributed people and resources

R

R

RR

R

R

R

R

R

RR

R


Virtual Organizations

● People and resources

● Network connections

● Sharing resources

● Dynamic

● Fault tolerant

RR

R

R

R

R

R

RR

R

R

R

VO-BVO-A


Grid User Interfaces

● Linux command line UI

● GUIs

● Web portals


Estonian Grid

● 2004 started with Grid developments in Estonia

● Started with NorduGrid middleware

● BalticGrid and BalticGrid-II projects for developing Grid in Baltic States 2005-2010

● Estonian NGI at European Grid Infrastructure (EGI)


ETAIS● Estonian Scientific

Computing infrastructure– To provide computing and

storage resources for science and education

– 2011-2012

● Partners– Tartu Ülikool

– Tallinna Tehinkaülikool

– Keemilise ja Bioloogilise Füüsika Instituut

– EENet


European Grid Infrastructure

● Central body for coordinating international Grid collaboration and standardization and interoperability

● 34 members

● EENet represents Estonian NGI


Cloud computing

● Gartner: “Cloud computing is a style of computing where massively scalable IT-related capabilities are provided ‘as a service’ across the Internet to multiple external customers”

● Why it is getting cloudy? – Development

– Business model

– Management model

– Virtualization level


Grid vs Cloud

● Rather – Science

– Real machines

– Similar resources (Linux clusters, mostly SLC5)

– Many resource providers in the same group

– Collaboration in big Vos

– PKI

● Rather– Business

– Virtual machines

– Different resources (Linux, Windows, etc)

– One (few) resource providers per cloud

– Services for special groups

– AAI


What is Grid useful for?

● Resource sharing● simplified access to remote resources: computing, databases,

software● multiple geographically apart resource aggregation● flexibility: in case of sudden need for large amount of

computing resources● reliability: network cuts, resource downtimes ● collaboration: remote working groups and developing teams


Thank you● More information from:

● Hardi Teder [email protected]● Pelle Jakovits [email protected]● Ilja Kromonov [email protected]

● http://courses.cs.ut.ee/2013/cloud

● [email protected]

introduction to the course - ut€¦ · introduction to the course week #1 basics of grid and cloud...

Documents