a csel presentation based on i. foster, z. yong, i. raicu, and s. lu, "cloud computing and grid...

King Mongkut’s University of Technology North BangkokFaculty of Information Technology

1

ON THE TECHNOLOGICAL FRONTIER WITH ANALYTICAL MIND AND PRACTICE

Grid and Cloud Overview and Comparison

A CSEL presentation based onI. Foster, Z. Yong, I. Raicu, and S. Lu, "Cloud Computing and Grid Computing 360-Degree Compared," in Grid Computing Environments Workshop, 2008. GCE '08, 2008, pp. 1-10.

2


OverviewGrid and Cloud Comparison

Business ModelArchitectureResource ManagementProgramming ModelApplication ModelSecurity Model

Content

3


To meet computational needsSingle supercomputer is too expensiveA distributed systemCoordination of existing resources (e.g. computer

clusters) to form virtual supercomputerMostly used in universities and government laboratories1 hard-working person VS many average personParallelizing jobs

Overview: Grid

4


Do you know what exactly is Cloud computing?Born as a relative to Grid computing and

Utility computingCluster computingDistributed systems

Moving computation on PC to centrally managed resourcesBy oneself - private CloudBy third-party – public Cloud (e.g. Amazon)Hybrid

Overview: Cloud

5


Grid Computing“In the mid 1990s, the term Grid was coined to describe technologies that would allow consumers to obtain computing power on demand. Ian Foster and others posited that by standardizing the protocols used to request computing power, we could spur the creation of a Computing Grid, analogous in form and utility to the electric power grid.”

I. Foster, Z. Yong, I. Raicu, and S. Lu, "Cloud Computing and Grid Computing 360-Degree Compared," in Grid Computing Environments Workshop, 2008. GCE '08, 2008, pp. 1-10.

6


Resources: compute resource, file (data) exchange, software and others.

Virtual Organization (VO): organisations usually with common interest contribute to (and thus form) the Grid

Grid Computing“…coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations.”

I. Foster, "The Anatomy of the Grid: Enabling Scalable Virtual Organizations," in Proceedings of the 7th International Euro-Par Conference Manchester on Parallel Processing, 2001, pp. 1-4.

7


The Grid provides high computation capacity and data storage through the coordination of shared distributed resources.

Economies of scaleDriven by the need for higher computation capacity and

storage while reducing costs, organizations share their resources - thus forming the Grid environment - in exchange for access to the resources owned by other organizations inside the Grid

Grid Computing

S. Smanchat, "Scheduling Parameter Sweep Workflow in the Grid," PhD, Caulfield School of Information Technology, Monash University, Australia, 2012.

8


Enabling ways to solve problem that were not possibleEnables organisations to perform the tasks that require

computation power exceeding their own by delegating tasks to be executed by others.

Increasing resource utilizationOther parties can make use of any idle resources in the Grid

according to resource sharing agreement

Grid Computing

S. Smanchat, "Scheduling Parameter Sweep Workflow in the Grid," PhD, Caulfield School of Information Technology, Monash University, Australia, 2012.

9


Ground concept remains similar: reducing cost, increasing reliability and flexibility by using federated resources

Context changes:Big DataVirtualization

Clusters are expensive Virtualization of commodity servers (or even clusters) is a better

choicesProblems in the resource management also remains similar.

UsageDiscoveryProgramming

Is Cloud a Grid?

10


A definition of Cloud Computing

“A large-scale distributed computing paradigm that is driven by economies of scale, in which a pool of abstracted, virtualized, dynamically-scalable, managed computing power, storage, platforms, and services are delivered on demand to external customers over the Internet.”


11


Massively scalableAbstract entity

No complicated setup is required (unlike Grid)Different levels of services (e.g. packages) for end users:

Economies of scaleServices can be re-configured and delivered on demand.

Cloud Computing

12


Cloud is a distributed systemCloud evolves from Grid computing

And may use Grid computing as supporting infrastructureUtility computing

Computing resources are metered (e.g. utility Grid)Service-Oriented Architecture (SOA)

Cloud and its predecessors

13



Grids and Clouds Overview

14


G. Mateescu, W. Gentzsch and C. J. Ribbens, "Hybrid Computing—Where HPC meets grid and Cloud Computing," Future Generation Computer Systems, vol. 27, pp. 440-453, 2011.

15


“Grid is a system that1) Coordinates resources that are not subject to centralized

control

2) Using standard, open, general-purpose protocols and interfaces

3) To deliver nontrivial qualities of services”

Grid: a three point checklist

I. Foster, "What is the Grid? - a three point checklist," GRIDtoday, vol. 1, 2002.

However, central administration control may be present in some Grids

Globus Toolkit Version 4 (de facto standard)

16


CloudUsage-based payment (e.g. EC2 and S3)VS traditional one-time payment for softwareAccess 100,000 cores without full investment

GridResource sharing (e.g. CPU cycle) within VOProject-orientedUsually used in academic institutionsGive and take

Business Model

17


Grid Architecture

I. Foster, "The Anatomy of the Grid: Enabling Scalable Virtual Organizations," in Proceedings of the 7th International Euro-Par Conference Manchester on Parallel Processing, 2001, pp. 1-4.

18


Fabric layer – access to resourcesE.g. Condor, GARA (General architecture for advanced

reservation)Connectivity layer – communication and authentication

GSI (Grid Security Infrastructure)Resource layer – protocols related to resources

GridFTP, GRAM (Grid Resource Access and Management)Collective layer – monitor and discover VO resources

E.g. Condor-G, Nimrod-GApplication layer – whatever you want

E.g. Grid workflow, Grid portal

The 5 Layers of Grid

Globus Toolkit covers the first four layers

19


Cloud Architecture


IaaS

SaaS

PaaS

20


Fabric layer – hardware resourcesUnified resource layer – abstract resources through

virtualizationPlatform layer – specialized services providing

development platformE.g. web hosting

Application layer – applications on Cloud

The 4 Layers of Cloud

21


IaaS – Infrastructure as a ServiceHardware (virtual) and software for application

environmentScalable (and reconfigurable) dynamicallyE.g. EC2 and S3

PaaS – Platform as a ServiceHigher level environment for deployment of applicationE.g. Google’s App Engine

SaaS – Software as a ServiceSpecialized software accessed remotely through the

Internet e.g. Office Web Apps

The Three Service Levels

22


Cloud Operating Systems

G. Sakellari and G. Loukas, "A survey of mathematical models, simulation approaches and testbeds used for research in cloud computing," Simulation Modelling Practice and Theory, 2013.

23


Amazon AWS Console

AWS Console as of August 2014

24


IBM Bluemix

25


Grid compute modelBatch-scheduled modelUsers submit job > request resource > wait for resource >

runUsually long wait queue and data stagingHardware-bound queuing system

Cloud compute modelResources are shared by all users at the same time in the

Cloud via virtualizationNo hardware-bound queuing

Resource Management

26


Cloud data modelEverything on the Cloud

VS Ian Foster’s vision

Can we trust Cloud security? Do we want to do things offline?

Listening to music on the Cloud? Desktop supercomputer

Resource Management


27


Grid data modelData Grid

E.g. management of data replicasGridFTP - analogous to BitTorrentVirtual data – data abstraction

Location transparency Distributed metadata catalog Privacy and access control Materialization transparency (transfer VS recompute)

Resource Management

28


Other concernsData locality

File system and data transferMerging compute and data management

Moving data around for computationVirtualizationMonitoringProvenance

Log history of execution Implemented in workflow management systems

Resource Management

29


MPI (Message Passing Interface) – most commonly usedFor multi-core architecture / cluster

MPICH-G2 - Grid-enabled MPI with Globus Toolkit integration

Grid programming focuses on management of large numbers of datasets and tasks.The motivation for Grid workflow system

MapReduce – distributes program to dataIntegration and interoperability are the challenges in the

Cloud

Programming Model

30


GridHPC / HTC applications

MPIWorkflow applications

Loosely coupled applicationsHomogeneous / heterogeneous tasks of small / large sizes

CloudNot well-defined yetTransaction-orientedInteractive

Application Model

31


GridHardware are shared by different institutionsLikely to be heterogeneous resourcesEach site has its own security measureSingle sign-onGSI (Grid Security Infrastructure) based on PKI

CloudHardware usually belong to single organizationLikely to be homogeneous resourcesSimpler and less secure (please check for update)Most important for Cloud’s success

Security Model

32


Data only accessed by the authorized usersSecurity certificate of Cloud providerData (physical) location and privacyData segregation – separate data of each userRecoverySupport for investigationData viability, even though the provider is taken over

Cloud Security Issues

33


Random interesting scenariohttp://mashable.com/2013/01/28/eve-online-asakai/

Question?

a csel presentation based on i. foster, z. yong, i. raicu, and s. lu, "cloud computing and grid...

Documents