cloud computing gets put to the test

Enterprise Strategy Group | Getting to the bigger truth.™

© 2015 by The Enterprise Strategy Group, Inc.

Scott Sinclair, Storage Analyst Bernie Behn, Principal Engineer, Avere Systems

Cloud Computing Gets Put to the Test

PREPARED BY ESG FOR

Today’s Speakers

Scott SinclairStorage Analyst

ESG

Bernie BehnPrincipal Engineer

Avere Systems

Housekeeping

• Recording• Available on-demand approximately 5 minutes after

today’s presentation

• Resources• Complete Lab Report• Links

• Questions• Please rate this webinar


Agenda

Review ESG research on growth factors driving cloud

Review ESG’s Lab Findings on Avere vFXT

Open Up to Questions

About ESG• ESG is an IT analyst, research, and strategy company.• Our firm was founded in 1999 with headquarters in Milford, MA / an analyst and client

relations presence in Silicon Valley, CA.• ESG conducts research with and for IT vendors, IT professionals, business professionals,

and channel partners.• We maintain ongoing analyst coverage in cloud computing, networking, storage, data

protection, cybersecurity, data management and analytics, application development and deployment, enterprise mobility, and channels.

• Capabilities include: Analyst services, market research, technical performance testing, economic validation, consulting, and custom content.


OverviewWith cloud computing being a top technology area of increased spending, many enterprises are validating cloud compute resources for the promise of cost savings, efficiency, and dependability.

Organizations that rely on compute-intensive operations such as media, financial analysis, and scientific research, find the ability to quickly scale compute resources up and down as well as create off-site archives invaluable to operational advancement.


ESG Research Overview2014 Next Generation Storage Trends

• 25 in-depth interviews with IT professionals responsible for their organization’s long-term IT infrastructure planning and strategy, especially storage

2015 IT Spending Intentions Survey • Quantitative study of 601 IT decision makers• 65% senior IT management (C-, VP-, and director-level)

and 35% IT management

2015 Storage Trends Study• Quantitative study of 373 IT professionals responsible for

evaluating, purchasing, and managing data storage technology – including external disk-based storage systems – for their organization


Overview of Cloud Research Findings

Cloud computing services lead IT spending growth expectations

Use cases for cloud led by secondary storage workloads

Cloud storage dominates initiatives that will impact storage spend


Server infrastructure (N=355)

Information management software (N=183)

Business applications, middleware, etc. (N=262)

Databases (N=282)

Network infrastructure (N=323)

Information security (N=311)

Business intelligence, analytics, and/or big data (N=175)

Storage infrastructure (N=320)

Virtualization/private cloud infrastructure software (N=195)

Enterprise mobility (N=152)

Cloud computing services (N=308)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

47%

48%

50%

50%

53%

59%

60%

61%

61%

61%

66%

41%

43%

42%

43%

39%

32%

33%

31%

29%

34%

24%

12%

9%

8%

7%

8%

9%

7%

9%

10%

5%

10%

To the best of your knowledge, to what extent will your organization’s 2015 IT spending for each technology listed below change relative to 2014? (Percent of respondents)

2015 spending will increase 2015 spending will stay flat 2015 spending will decrease

2014-2015 Spending Change, by Technology

Data Insights

For the fourth year in a row, more organizations will increase spending on cloud than on any other area of technology.


Additional resource to accommodate spikes in workload

Application bursting

Run internal production applications

Use as temporary compute resources for time-limited projects

Business intelligence/analytics

Use for high performance and/or scientific computing applications

Run internally/externally-facing Web servers

Primary storage for files

Disaster recovery

Test and development

Data backup and archive

0% 10% 20% 30% 40% 50% 60%

25%

25%

27%

27%

30%

31%

32%

34%

35%

38%

49%

For which of the following purposes does/did your organization use cloud infrastructure services? (Percent of respondents, N=327, multiple responses accepted)

Cloud Infrastructure Use Cases

Data Insights38% of companies that are less than 10 years old say they use cloud infrastructure for BI/analytics, making it the third-most-common use case for these companies.


IT Initiatives That Will Impact Storage Spending Over the Next 12 Months

Increasing applications delivered via software-as-a-service (SaaS) model

Deploying application- or workload-specific systems/appliances

Deploying converged/hyper-converged infrastructure solution

File sharing/transfer with remote offices

Collaboration tools and initiatives

“Green” initiatives to reduce data center power and cooling requirements

Regulatory compliance initiatives

Business continuity/disaster recovery programs

Major application deployments or upgrades

Internet of Things (IoT)

Supporting mobile and remote users

Database implementation/upgrade/migration

Business intelligence / data analytics

Desktop virtualization/thin client initiative (e.g., VDI)

Supporting server virtualization/private cloud implementation(s)

Data center consolidation

Using “cloud storage” service as way to source storage capacity without buying new on-premises infrastruc-ture

14%14%

15%15%

16%17%17%

19%20%20%20%

21%21%

23%24%

25%37%


Insights from IT Leader Discussions on Cloud1

Vendor lock-in and data security were commonly mentioned concerns

Lack of insight into file storage added complexities to which workloads could be migrated to the cloud

Cloud pricing complexity created challenges to anticipate which workloads would benefit from cloud storage

Despite the high interest in cloud, IT leaders still have concerns

1 From 2014 Next Generation Storage Architectures research study


"Cloud allows the community to access top of the line technology in a fast manner”

“Certain data types will never go to the cloud” "As a bank, we are waiting for the first few banks to move to the cloud and see how the regulators react…after that there could be a tidal wave to the cloud”

“We don’t feel its mature enough, the last thing we would want is for the company to end up in the news due to a hack or a data leak”

“Once they’ve got you, they’ve got you"

Cloud Storage: Quotes


"All our employees have to sign an NDA, and I’m not sure if they could require the cloud company to do the same”

"When you walk through our doors here, you fall under our controls. We need to know who is using what, where and when. As soon as we lose track of that, everyone asks, can we withstand an audit?”

"On premises is always cheaper however we have modeled it”

"To be successful in any cloud undertaking you need to bring in a trusted partner to help you pick the provider”

Cloud Storage: Quotes Continued


ESG Lab: Hands-On Validation Testing of Avere vFXT

Validate whether a product works as advertised and explain both the technical and business benefits.

Validate performance, scalability, usability, agility, efficiency, etc…

Why This Matters

More than 600 ESG Lab Reports


ESG Lab Hands-On Validation of Avere vFXT Solution

Simple Cloud Deployment.

Flexibility to access data between cloud and on-premises without disruption to existing workflows.

Agility to automate movement of data to the right place, at the right time.

Cost Effectiveness of Avere vFXT in the Cloud.


About the ESG Lab Test• To substantiate the time and cost savings as well as degree of

operational impact of deploying Avere Virtual FXT (vFXT) Edge filers into cloud computing workflows.

• The Avere vFXT Edge Filer enables NAS file-services in the Cloud, connecting on-premises storage to cloud computing resources.

• Google Cloud Platform: Google Compute Engine and Google Cloud Storage selected for testing environment


ESG Lab Test: Why Are Cloud File-Services Needed?• Moving application stacks from on-premises into the compute cloud is generally

straightforward – keeping the applications saturated with I/O is challenging

• Applications that have historically run on-premises against file-based storage will require the same interface when running at-scale in the cloud

• Running applications in the cloud directly over dedicated private links will affect application performance due to network latency and bandwidth limitations

• Forklifting entire datasets into cloud-storage-based file services may not be feasible for “short bursts of compute activity”


Cloud Compute and Storage Resource Lifecycles• Cloud bursting applications utilize two components:

– Permanent Infrastructure: On-Premises Storage• User content remains stored on-premises customer-owned NAS• Computational processing takes place in Google Compute Engine

– Temporary/Transient Infrastructure: Google Cloud Project• VPN between Google Cloud Project and on-premises network• Google Compute Engine

– n1-highmem-8 compute instances with persistent SSD disks for vFXT caching– Linux instances running applications with mount points to vFXT

• Born-in-cloud Applications– Permanent Infrastructure: Google Cloud Project

• Google Compute Engine– Application and file-services are active cloud compute instances– Can be powered on/off on-demand in response to application needs

• Google Cloud Storage– Object-based cloud storage bucket where at-rest encrypted data resides


Overall Cloud Bursting Architecture

• Permanent on-premises and transient in-cloud infrastructure• Compute project can be spun-up and torn-down on-demand• Key benefits:

– Data resides permanently on-premises– Cloud infrastructure is completely elastic/on-demand


Straightforward Deployment via API Commands

• gcloud API– gcloud compute instances create <vFXT deployment orchestration>

• Avere API:– corefiler.add <IP address of on-premises filer>– vserver.createJunction <add on-premises export for in-cloud access>

• Client Compute Instances:– gcloud compute instances create <compute node deployment>– NFS-Mount client compute instances to vFXT file-services IP addresses

• GO!– Further scale the Avere vFXT cluster and application compute farm via API calls


Cloud Bursting Application Performance

• Google Compute Engine clients run the workload against application data from on-premises• Clustered file system cache of on-premises filer is running inside of Google Compute Engine• Cloud-scale compute demands scalable file-services performance: Avere vFXT Cluster• On-Premises to Cloud network connectivity bottleneck is relieved by file caching• Workload results are cached and acknowledged in the cloud and streamed back on-premises


Cloud bursting Cost Analysis

• Supercomputer capabilities built on-demand, pay-as-you-go• Run applications that were historically run on-premises without modifications• Quick spin-up and spin-down of application instances• Cloud architects can focus on running large-scale apps, not data management


Map/Reduce Analytics

• Cloud-based workflows can deposit file data directly on vFXT Edge Filer• Source data generation and gathering can happen without HDFS• Map/Reduce jobs can access data directly with shared storage NFS mount points• Hadoop grid accrues compute charges only when running analytic workloads


Hadoop Environment Setup

• Google “bdutil” deployment script builds Hadoop cluster on-demand• Set up NFS mount points to vFXT on each Hadoop worker• Update the “mapred_site.xml” and “core-site.xml” files to use mount points• Source data is already collected, in-place, and available on the Avere vFXT• Dispatch map-reduce jobs in typical fashion against paths served by Avere

hadoop jar <jar file> <job routine> /avere/data/input/ /avere/data/output

• Output data is placed directly on shared storage for post-consumption• Removed the need for copy-in and copy-out of map-reduce data


Running Distributed Map-Reduce Jobs

• Online archive of file data stored in Google Cloud Storage• Recently written data is handled by vFXT, in-cache• More efficient map-reduce runs against in-cache data• Multiple jobs in parallel maximized data throughput• Hadoop cluster can be powered-off or torn down when jobs

are not actively being dispatched• Data is always available when vFXT cluster is powered on


Hadoop Analytics Cloud Costs

• Scale map-reduce worker-node independently from storage performance• Less cost overhead when compared to HDFS backed by persistent disk• Minimize application compatibility by writing to file storage instead of object• Skip data ingest and export for streamlined hadoop job execution


Conclusions

• Simplify cloud computing usage with file-based access solutions• Overcome “data inertia” without moving data• Increased agility to minimize costs of cloud overhead• Trivialize network bottlenecks between on-premises and cloud infrastructures• Agile permanent or transient usage of cloud computing/storage resources• Minimize full-copy data movement with Edge filer caching• Move entire workflows into the cloud without re-writing applications


Thank You

Enterprise Strategy Group | Getting to the bigger truth.™

http://www.twitter.com/esg-global

http://www.facebook.com/ESGglobal

https://www.linkedin.com/groups?gid=1295607&trk=myg_ugrp_ovr

http://www.youtube.com/user/ESGglobal

FOLLOW ESGScott Sinclair, Storage Analyst [email protected]@Scott_Sin

Bernie Behn, Principal [email protected] SystemsVisit: AvereSystems.com@Averesystems

mailto:[email protected]

mailto:[email protected]