bringing hyperscale computing to the enterprise hyperscale computing to the enterprise the need for...

41
Bringing HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems

Upload: voquynh

Post on 09-May-2018

219 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Bringing HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems

Page 2: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Global Presence

Atlanta Pune

Bangalore Chennai

MSys: Corporate Overview

Established In: 2007

Self-funded, profitable

Over 350 employees

Global Presence

Product Engineering Services Technology Storage Services

Product Engineering / Development

Test Engineering

Rapid Prototyping

Maintenance & Support

UI Engineering

Cloud Computing

Big Data Analytics

Storage

DevOps

Infrastructure Tech

Filesystem Development

Kernel Development

Backup & DR

Cloud Storage

Storage QA

Storage Certifications

MSys Advantages

• End-to-end Storage Expertise

• 75% revenue from storage companies

• Dedicated team with focused skills

Page 3: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

About Me:

• VP Engineering – Msys Technologies

• Co-founder – Clogeny Technologies (acquired

by Msys)

• Built cutting edge products in Storage, Cloud

and Big data space including:

• Cloud Mobility platform

• Cloud Backup Solution

• Recommendation Engines

Page 4: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Agenda

• Understand the changes taking place in

the Enterprise – Application and Data

• Need to re-think approach to

Infrastructure

• Understand what is Hyperscale

• Different Approaches in Hardware

• Different Approaches in Software

• The Full Picture

Page 5: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT • Real-time Data

streams • Near real-time

recommendations

• Massive amounts of data

• Cost effective

solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 6: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT • Real-time Data

streams • Near real-time

recommendations

• Massive amounts of data

• Cost effective

solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 7: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT

– Available as-a-

Service

– Scale on demand

– Always on

operation

– Automated

Provisioning

– Agility, Flexibility

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 8: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT • Real-time Data

streams • Near real-time

recommendations

• Massive amounts of data

• Cost effective

solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 9: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Real-time Data streams

– customers

– partners

– supply chains

– applications

– universe of

things

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 10: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT

• Real-time Data streams

• Near real-time recommendations

• Massive amounts of data

• Cost effective solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 11: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Near real-time recommendations – real-time

transactional and analytical systems

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 12: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT • Real-time Data

streams • Near real-time

recommendations

• Massive amounts of data

• Cost effective

solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 13: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Massive amounts of data

– structured,

unstructured

– archive

– readily available

for analytics,

compliance,

outages

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 14: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Web-Scale IT • Real-time Data

streams • Near real-time

recommendations

• Massive amounts of data

• Cost effective

solutions

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 15: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Next-gen Application and Data

• Cost effective solutions – cost-effect

performance

– Minimize cost of

Human resources

– Minimize costs of

infrastructure

Ref: http://dionhinchcliffe.com/2012/10/23/imagining-the-future-of-the-enterprise/

Page 16: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Current Constrains- Storage

• Multiple Storage Array Platforms and

Arrays

• Slow Access Speed and throughput of

Disk Drives

• Difficulty of migrating current

management of data on DAS and SAN

to more flexible topologies

• Difficulty in managing the volume,

growth and complexity of

unstructured data

• A fragmented and often fragile Data

backup process

Ref: http://wikibon.org/w/images/f/f3/Softwareledstorage.png

Page 17: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Exabyte Questions (Million Dollar Question)

How are exa-bytes of data going to be:

–backed up?

–restored?

–accessed?

Page 18: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

How did “they” do it?

Page 19: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Micro-service Micro-

service Micro-services

Internet Scale Applications

Real-time Analytics and Batch processing

Real Time streams of data Containers

Internet of things

Supply Chains

Click streams Security

Page 20: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs

Page 21: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs SSDs

• Flash Storage and Solid State Drives

Page 22: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs

Controller VM/Software

Servers

All Flash/Hybrid arrays

SSDs HDDs

• Flash Storage and Solid State Drives

• All Flash or Hybrid Flash Arrays

Page 23: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

SSDs HDDs

• Flash Storage and Solid State Drives

• All Flash or Hybrid Flash Arrays

• Software Defined Software

Page 24: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

SSDs HDDs

• Flash Storage and Solid State Drives

• All Flash or Hybrid Flash Arrays

• Software Defined Software

• Hyper converged Infrastructure

Page 25: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs

• Flash Storage and Solid State Drives

• All Flash or Hybrid Flash Arrays

• Software Defined Software

• Hyper converged Infrastructure

• IT-as-a- Service

Page 26: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs

Page 27: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Page 28: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

• Object Storage

• No RAID – but RAIN !

• Erasure Coding

Page 29: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

What about the Software to use ?

Page 30: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Page 31: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Storage– Ceph, Gluster, Lustre, HDFS

Page 32: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Storage– Ceph, Gluster, Lustre, HDFS

• Distributed Parallel Filesystems

• Non of complexities of RAID

• Replication, Zero Downtime and

Self-healing

• Commodity hardware with lowest

cost

Page 33: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Systems – Mesos, Zookeeper

Distributed Storage– Ceph, Gluster, Lustre, HDFS

Page 34: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Systems – Mesos, Zookeeper

Distributed Storage– Ceph, Gluster, Lustre, HDFS

• Apache Mesos - Operating system for the Datacenter

• Apache Zookeeper – Distributed Co-ordination

Service

• Consul, Eureka – Service Discovery

Page 35: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Systems – Mesos, Zookeeper

Distributed Storage– Ceph, Gluster, Lustre, HDFS

In-memory Computing Burst Buffer Solutions

Page 36: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM VM VM

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

Servers

Erasure Coding

RAIN Architecture

High Performance

Distributed Systems – Mesos, Zookeeper

Distributed Storage– Ceph, Gluster, Lustre, HDFS

In-memory Computing Burst Buffer Solutions

• Tachyon – Reliable data sharing at memory-speed across clusters

• Burst Buffers - Latency reduction, greater bandwidth and high IOPS

performance

Page 37: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM

Distributed Systems – Mesos, Zookeeper

Micro-service Micro-

service Micro-services

Internet Scale Applications

Distributed Storage– Ceph, Gluster, Lustre, HDFS

VM VM

Real-time Analytics and Batch processing

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

In-memory Computing Burst Buffer Solutions

Servers

Erasure Coding

RAIN Architecture

High Performance

Real Time streams of data Containers

Universe of things

Supply Chains

Click streams Security

Page 38: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM

Distributed Systems – Mesos, Zookeeper

Micro-service Micro-

service Micro-services

Internet Scale Applications

Distributed Storage– Ceph, Gluster, Lustre, HDFS

VM VM

Real-time Analytics and Batch processing

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

In-memory Computing Burst Buffer Solutions

Servers

Erasure Coding

RAIN Architecture

High Performance

Real Time streams of data Containers

Universe of things

Supply Chains

Click streams Security

Page 39: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM

Distributed Systems – Mesos, Zookeeper

Micro-service Micro-

service Micro-services

Internet Scale Applications

Distributed Storage– Ceph, Gluster, Lustre, HDFS

VM VM

Real-time Analytics and Batch processing

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

In-memory Computing Burst Buffer Solutions

Servers

Erasure Coding

RAIN Architecture

High Performance

Real Time streams of data Containers

Universe of things

Supply Chains

Click streams Security

Page 40: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

SSDs HDDs

Hypervisor

VM

Controller VM/Software

Controller VM/Software

Servers

Servers

Hyper-converged / Software defined

All Flash/Hybrid arrays

VM VM

Distributed Systems – Mesos, Zookeeper

Micro-service Micro-

service Micro-services

Internet Scale Applications

Distributed Storage– Ceph, Gluster, Lustre, HDFS

VM VM

Real-time Analytics and Batch processing

SSDs HDDs SSDs HDDs

Scalable Exascale Distributed Object

Storage

In-memory Computing Burst Buffer Solutions

Servers

Erasure Coding

RAIN Architecture

High Performance

Real Time streams of data Containers

Universe of things

Supply Chains

Click streams Security

Page 41: Bringing HyperScale Computing to the Enterprise HyperScale Computing to the Enterprise The need for Enterprises to overhaul their IT systems Global Presence Atlant a Pune Bangalore

Thank You!

MSys Chennai Bristol IT Park, 4th Floor, Plot No. 10, South Phase, Thiru Vi Ka Industrial Estate, Guindy, Chennai 600032 Ph. +91-44-39167015

MSys Bangalore No: 56/3, Ground Floor, Vakil Square, Bannerghatta Road, Bangalore -560 029 Ph. +91-80-41158363

MSys Georgia 4385 Kimball Bridge Rd, Suite 203, Johns Creek, Georgia – 30022 Ph. +1 770-809-3217

E: [email protected] W: www.msys-tech.com

/msystech @msys_tech

Clogeny Pune Plot no. 34/2, Rajiv Gandhi Infotech Park – Phase 1, Hinjewadi, Pune - 411 057 Ph. +91 20 661 43 482 US Ph. +1 408 556 9645