c-dac cloud computing: initiatives & offerings meet 2013-pdfs/session5/ta… · cdac cloud...

Dr. B.B Prahlada Rao Joint Director-SSDH

email: [email protected]

Centre for Development of Advanced Computing

C-DAC Bangalore, India

GARUDA-NKN Partners Meet

26th July 2013

C-DAC Cloud Computing: Initiatives & Offerings

26 July 2013 CDAC CLoud Offerings 1

Presentation Outline • Introduction to Cloud computing.

o Tech Trends o Definitions, Benefits & Bariers o Virtulization o Characteristics of Cloud Computing

• Cloud computing Initiatives in India o C-DAC Cloud Initiatives o Govt Clouds o Medical Clouds o Cloud Innovation Council

• Cloud Trends o Medical Clouds o Sensor Clouds

• Conclusion


Rise & Fall of Computing Technologies

Mainframes Minis PCs Clusters Grids

Minis PCs Network Comp Grid Comp Cloud Comp

1970 1980 1995 2000 2007


2012 Gartner “IT Hype Cycle” for Emerging Technologies

CDAC CLoud Offerings 26 July 2013

4

Technologies influencing a new era in computing…

• Microprocessor Technology

• Optical Networking Technology

• Storage Technology

• Wireless Technology

• Sensor Technology

• Global Internet Infrastructure

• WWW and Web Services (SOA)

• Open Source Movements

CDAC CLoud Offerings 26 July 2013 5

Definitions- Cloud Computing

• Cloud Comp is a style of computing, where massively scalable IT related capabilities are provided ‘as a service’ across the Internet to multiple external customers. ….. Gartner – It means Cloud should deliver a service on a subscription basis,

allow access from anywhere and provide economies of scale. – Cloud will go cross vendor and feature shared resources.

Q. Is Cloud a metaphor for the Internet? Cloud computing is an evolution of the Internet computing or usage of the

Internet as a computing infrastructure & resource. Cloud computing, is used to provide services such as Data Storage,

Software Applications, and email and file exchanges • Cloud Computing is Ubiquitous network access

• Cloud Computing promised infinite elastic computing resources.


Science with Cloud Computing

Ease of Access

Application Scaling

Easy Automatic Deployment

Virtual Ownership of resources

26 July 2013

CDAC CLoud Offerings 7

Revolution in IT?

Classical Computing

– Buy & Own

Hardware, SystemSW,

Applications: to meet peak needs.

– Install, Configure, Test,

Verify, Evaluate

– Manage

– ..

– Finally, use it

– $$$$....$(High CapEx)

Cloud Computing

– Subscribe

– Use It

– $ - Pay for what you use,

based on QoS

– No CapEx ,Only OpEx

Ev

ery

18

mo

nth

s?


New IT Trends

• Ownership -> Leasing Model

– Infrastructure, platform, applications

• PC -> Internet Data Center

– Building applications for a single user large

number of concurrent consumers

• Web 2.0 & Virtualisation

– Building 3rd party IT services by composing

multiple independent services


Grid Computing

Utility Computing

SaaS Computing Cloud Computing

Solving large

problems with

Parallel Comp.

Made

Main Stream

by Global

Alliance

Offer Comput.

resources as

Metered

Service

Introduced in

late 1990s

Network-based

subscriptions

to applications

Gained

momentum

in 2001

Next-Generation

Internet

computing

Next-Generation

Data Centers

Evolution of Cloud Computing

Ref No: 6 CDAC CLoud Offerings

Characteristics of Clouds

Autonomic

Elastic

Market

Oriented

(Pay As You Go)

Virtualized

Service

Oriented

Dynamic &

(Distributed)

Shared

(Economy of

Scale)

Cloud Computing


CDAC CLoud

Offerings 26 July 2013

Promised Benefits of Public Clouds

• No upfront infrastructure investment – No procuring hardware, setup, hosting, power, etc

• On demand access – Lease what you need and when you need

• Efficient Resource Allocation – Globally shared infrastructure, can always be kept busy by serving users

from different time zones/regions.

• Nice Pricing – Based on Usage, QoS, Supply and Demand, Loyalty

• Application Acceleration – Parallelism for large-scale data analysis, what-if scenarios studies

• Highly Availability, Scalable, and Energy Efficient

• Supports Creation of 3rd Party Services & Seamless offering – Builds on infrastructure and follows similar Business model as Cloud

12

Benefits of Clouds

Cloud Computing: New model of how IT operates as a

business!

Agility

Service Management

Do More- Virtualization

Flexibility

Cost Efficiency

Automation

User Metering

Accessibility

Ref No: 6, 11 CDAC CLoud Offerings

Cloud Benefits And Barriers

CDAC CLoud Offerings

Cloud Deployment Models

Private/Enterprise Clouds

Cloud computing model run

within a company’s own Data Center / infrastructure for internal and/or partners use.

Public/Internet

Clouds

3rd party, multi-tenant Cloud

infrastructure & services:

* available on

subscription basis (pay as you go)

Hybrid/Mixed Clouds

Mixed usage of private and public

Clouds: Leasing public cloud services

when private cloud capacity is insufficient


Cloud Architecture

Virtual Machine (VM), VM Management and Deployment

QoS Negotiation, Admission Control, Pricing, SLA Management,

Monitoring, Execution Management, Metering, Accounting, Billing

Cloud programming: environments and tools

Web 2.0 Interfaces, Mashups, Concurrent and Distributed

Programming, Workflows, Libraries, Scripting

Cloud Applications:

Social computing, Enterprise, ISV, Scientific, CDNs, ...

Ad

ap

tive M

an

ag

em

en

t

Core Middleware

User-Level Middleware

System level

User level

Au

ton

om

ic / Clo

ud

Eco

nom

y

Apps Hosting Platforms

Cloud resources

16 26 July 2013 CDAC CLoud Offerings

Inside Cloud

Storage Provisioning

VM Provisioning

Network Provisioning

Pricing

Billing

Pricing

Accounting

Monitoring

Cloud Middleware

Interface

Virtual Resources

Physical Resources

User Corporate


Cloud Taxonomy



Cloud Anatomy

Application Services (Services on Demand)

Gmail, GoogleCalender

Payroll, HR, CRM etc

Sugarm CRM, IBM Lotus Live

Platform Services (Resources on Demand)

Middleware, Integration, Messaging, Information, connectivity etc

AWS, IBM Virtual images, Boomi, CastIron, Google App engine

Infrastructure as services (Physical Assets)

IBM BlueHouse, VMWare, Amazon EC2, Microsoft Azure, Sun Parascale …

Three models based on the type of service offered to the user

26 July 2013 19

• ‘That which is not real ‘ but may display the salient qualities of Real.

• Virtualization is the creation of a virtual (rather than actual) version of something, such as an Server, a Storage device or Network resources.

• Dis-associating the tight bond between S/W and Hardware.

• Process of Abstracting computer resources.

Virtualization

Motivation for Virtualization: • Multitasking Computer systems.

• Do More with Less.


Traditional Arch. Vs Virtualization Arch.

Hardware

Operating System

App App App

Traditional Stack Hardware

App

OS

App App

Hypervisor

Virtualized Stack

OS OS

Virtual Architecture

• Many OS to one server

• Separation between software and Hardware

• Many applications on one server

• Typical load on server is 80 –95%

• Dynamically optimized resources

Traditional Architecture • One OS one server one application

• Tightly coupled hardware and software

• Typical load on server is 5 –15 %


Indian Cloud Computing-Initiatives

• Pan C-DAC Cloud Computing-Initiative: – Meghdooth: Open Source Cloud Stack

– Scientific Cloud Computing: Scientific Cloud Stack, IaaS, StaaS, PaaS

– E-Sikshak: C-DAC’s E-Learning on Cloud

– MANTRA: MAchiNe Assisted TRAnslation System over CLOUD

– Disaster Recovery: (DR) solution on Cloud

– eSanjeevani : Web based Telemedicine Appn, CDAC Mohali.

– Megh Shushrut: C-DAC Hospital Mgmt Information System

• MEGHA: A Cloud interface for GARUDA

• Govt Cloud Initiatives


Meghdoot Stack


• One stop solution for implementing Cloud environment.

• One single middleware bundle that incorporates all pre-requisites to establish a complete functional cloud

• Provision to offer IaaS, PaaS, SaaS

• Free and Open source

• Based on Open standards

Meghdoot


C-DAC Scientific Cloud


Scientific Cloud

Scientific Cloud is a model for on-demand access to a shared pool of HPC resources (e.g.,networks, servers, storage, applications) that can be easily provisioned as and when needed by the researchers and scientists.

Benefits of Scientific Cloud

• Easy access to large infrastructures

• On demand access to HPC resources

• Virtual ownership of cloud resources

• Ease of deployment

• Scientific Cloud provide an alternative for HPC outreach


Scientific Cloud: HPC as a Service

• Cluster as an Infrastructure Service (IaaS) • Provisioning of on demand HPC clusters • Provisioning of Map reduce clusters to process large data sets • Support for MPI as a tightly-coupled applications category. • Design methodology for providing/supporting installation of MPI

libraries in multiple machines simultaneously • Storage as a Service (StaaS): CloudVault • SaaS:

– Job Submission Portal (C_JSP) , Cloud Access Portals – Monitoring tools

• PaaS: Program development and analysis Tools • Applications as Service (AaaS) PSEs

– PSE for Climate Modelling – PSE’s for Bio Informatics

• Types of applications supported – Sequential & Concurrent Applications – Data Parallel Applications - Map Reduce

– Parallel Programs: OpenMP ,Pthreads, MPI


SciCloud Service Offerings

26 July 2013 CDAC CLoud Offerings

GUI

IaaS PaaS

SaaS

Cloud Portal ( Selects the Resource)

Virtual Machine (OS, Time, Type)

Virtual Cluster

(No. of nodes, type, Time)

Hadoop Cluster

(No. of nodes, Time)

MPI Cluster

(No. of nodes, Time)

Cloud Vault (StaaS)

Web interface

Small

Medium

Large

PSEs for

Bioinformatics Climate Modelling

Job Submission

Portal

28

Cloud Vault

End-User

Desktop Application

Web Application

Mobile Application

Swift

Storage and Data Center

Physical Storage

OpenStack Object Storage

Cloud Vault is your personal hard drive in the

cloud. Store any kind of file like Documents, Music, Videos, Photos, and Virtual Machine

Images on CDAC Cloud Vault secure servers.


Cloud Vault: Features • Reliable Storage 24x7 • Storage Access through Internet :

• Requested MB/GB Storage • Required duration

• Interfaces GUI - Access cloud vault files

• File /Directory Operations : – Create Directory – Files /Directories Listing – File/ Directories Upload/ Download – File Editing – Files Synchronization

Java APIs • Authentication/Login

• File Operations – Listing – File upload – File Download

Functionality: • Virtual image storage (used for provisioning

of IaaS)

• Storage of any type of file including mpeg,

jpeg, videos

• Unstructured Data Storage (NO-SQL

database)

• Backup, Archive/ Retention

• Scientific Databases in the cloud

Technologies & Softwares:

– Openstack Swift Storage

– GlusterFS

– Cyberduck

– Rackspace Storage APIs


MEGHA: A Cloud interface for GARUDA

• Provides virtual Infrastructure on Garuda resource

– Supports IaaS and SaaS – Supports various OS flavors RHEL, Ubuntu and Fedora.

• Provides Virtual Clusters Inbuilt with – clustering tools – Parallel paradigm like Mpi & OpenMP. – Torque as scheduler

• Provides on demand Scientific Application like – SFM :Seasonal Forecasting Model, – FastA :Protein Sequence Analysis – Scilab :Open Source Equivalent of Matlab – VRGeo : Collaborative mapping platform for Geospatial information

• Supports Open-Id for user login • Present focus includes

– Providing Scientific tools in the area of CFD, Bioinformatics, Molecular Biology, Statistical Computing etc

– VO enabled On demand Grid


MEGHA: A Cloud interface for GARUDA

• Introduction to C-DAC Initiatives on Cloud? • Various Offerings possible: • Option 1: Cloud Offering1 -Basic Users *1 node+ Storage * 2 node+ Storage * 3 node + Storage • Option 2: Storage on Cloud • Option 3: Cloud Offering – Advance Users VC with 1-8 node + Storage + MPI applications • Option 4: Cloud Offering – Advance Users VC with 16 node + Storage • MPI applications -Framework Hadoop Applications-Framework • Option 5: Cloud based PSE for SFM • Option 6: Cloud based PSE for NG Each offering The Benefits of the usage, Targeted Use Cases, C-DAC Servise for the Above Offerings: Free Softwares Help in Installations Trainings Support & Advise .. etc


Indian Government: Heading To Cloud

• DietY declared E-governance on national cloud with all data centers connected.

• National Network Cloud - all states will have their private cloud

• National Cloud Benefits

– With Reduced IT management load agencies can concentrate on core activities

– Elasticity

– Agility


SuMegha Cloud Kit

SuMegha Cloud Kit

• Packaged software kit to enable the automated deployment of Private Scientific Clouds on the hardware.

• Suitable for Academic institutes & Research organizations who want to setup cloud labs.

• Enables the users to setup cloud lab without having expertise of system administration.

Outcomes

1. CD

Release Notes

Terms & Conditions, Usage policy

SuMegha software stack

Installation and configuration scripts

User Manuals

Tutorials

2. SuMegha website : www.SuMegha.in

Release Notes

Terms & Conditions, Usage policy

SuMegha software stack

Installation and configuration scripts

User Manuals Tutorials

3. Training & Support

Workshops (CD,1-day training)

Online Tutorials on website

Institute / organization level trainings

for mass

Telephonic/ Mail Support

4. Costing model

Academics

Organization

Individual Research students

SuMegha Software Stack & Supported Images Components :

Hypervisor Xen

Cloud Middleware Nimbus, Openstack Swift

Storage Glusterfs

Portal SuMegha portal Job Submission Portal Cloud Vault Portal

PSE PSE for SFM PSE for NGS Pipeline

Supported Images: Virtual Machines

CentOS (small, large, medium)

Job Submission Portal

Virtual Clusters

MPICH enabled with job submission portal

HADOOP enabled

PSE enabled images

SFM

NGS Pipeline

SuMegha Cloud Stack

Cloud Product Offerings

Various Cloud Offerings Possible:

1. Cloud Offering- Basic Users – 1 node+ Storage

– 2 node+ Storage

– 3 node + Storage

2. Storage on Cloud

3. Cloud Based PSE – Advance Users – Cloud based PSE for SFM

– Cloud based PSE for NGS

Cloud Product Offerings

1. Cloud Offering – Advance Users – VC with 1-8 node + Storage + MPI applications

1. Cloud Offering – Advance Users

– VC with 1-16 node + Storage + – MPI Applications Framework – Hadoop Applications Framework

• Cloud Based PSE Offering – Advance Users – Cloud based PSE for SFM – Cloud based PSE for NGS

• Cloud Servises to the Above Offerings:

– Free Softwares – Installations & Trainings – Support & Advise

Sensing As A Service & BigData

• Modern world is full of devices comprising sensors, data

processor.

• Such resources enable sensing, capturing, collection, and

processing of real time data from billions of connected

devices.

• Fact:

• 2010-Total data on earth exceeded one ZB (zettabyte).

• End of 2011- Total data number grew up to 1.8 ZB.

• 2020- Expected that the number will reach 35 ZB in 2020


Dissemination: Talks / Publications @ Conferences / Workshops

• Prahlada Rao B.B., “Resent Trends in Cloud Computing” Technology day Talk C-DAC Bangalore 10th May 2013.

• Prahlada Rao B.B., “CDAC Cloud Initiatives” Workshop on National Cloud Innovation Council, C-DAC Bangalore 29th April 2013.

• Prahlada Rao B.B., “HPC for BigData” International Workshop on Data Analytics and Applications (IWDAA) 2013, BITS, Goa Campus, 1st Mar2013.

• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Tutorial “Cloud Computing for HPC” C-DAC’s National Conference on Parallel Computing Technologies (ParCompTech2013), NIAS, Bangalore, 21-23rd Feb 2013.

• Prahlada Rao B.B, Payal Saluja, Neetu Sharma, Ankit Mittal, Shivay Veer Sharma, “Cloud Computing for IOT & Sensing Based Applicationss”, in IEEE International Conference on Sensing Technologies, ICST 2012, Kollkata, India, Dec 2012.

• Prahlada Rao B.B., Payal Saluja, Vineeth Simon Arackle, Half-Day Tutorial “Open Source Platforms for Scientific Clouds” CSI’s Annual Conference on Advances in Cloud Computing (ACC2012), 26th July 2012, Bangalore.(60+ participants from Industry, Academia)

• Payal Saluja, Prahlada Rao, Ankit Mittal, Rameez Ahmad, “CDAC Scientific Cloud: On Demand Provisioning of Resources for Scientific Applications”, in Proceedings of 18th International Conference PDPTA 2012, Las vegas, Nevada, USA, 16-19 July 2012.

• Prahlada Rao B.B., Tutorial On “Cloud Computing”, International Conference on Trends in Knowledge and Information Dynamics, ICTK 2012, organized by ISI, NIMHANS Conv Centre, Bangalore, India, 10 July 2012.(150 participants from Industry, Academia)

• Dr. Prahlada Rao BB, Cloud Computing for Telemedicine, National Workshop on Telemedicine Today & Tomorrow, C-DAC Mohali, An Invited Talk, 6th July 2012.

• Prahlada Rao B. B. , National Workshop on Cloud Computing, conducted at CDAC Knowledge Park, Bangalore, 21st May 2011 .(80+ participants from Industry, Academia)

• Vineeth Simon Arackal, Arunachalam B, Payal Saluja, Prahlada Rao B B, Co-existence of Cloud and Grid: A Case Study in Service Oriented Grid GARUDA. The International Conference on Information Technology- New Generations (ITNG 2011), Las Vegas, USA. 11-13April 2011.

• Vineeth Simon Arackel, “ Coexistence of Grid and Cloud” at "Grid To Cloud (G2C)- Challenges and Applications" Workshop at VIT, Vellore, 2nd April 2011.

42 26 July 2013 CDAC CLoud Offerings

Conclusion

• Next Generation Complex Systems are becoming a Reality – Peta/ExaScale Computing Systems – Combines Big Compute & Big Data to deliver – Exascale Performance in Supercomputers

• National Missions are Emerging

– National Grid Computing:GARUDA – National Knowledge NetWork:NKN – National Supercompution Mission – National BioDiversit Organization – National Cloud Innovation Council

• HPC Grid and Cloud infrastructures are converging:

– Compute Resources on demand to Grids – Cloud Bursting Research: Grids Vs Clouds – On Demand Grid Services on Cloud , Cloud Services on Grid

• Newer Services Models “Sensing as a Service” are Evolving

• C-DAC is Engaged in: HPC, Grid, Cloud and BigData


26 July 2013 CDAC CLoud Offerings

Thank You

44