data fabric enabled solutions for hybrid cloud - netapp hci · data fabric enabled solutions for...
TRANSCRIPT
White Paper
Data Fabric Enabled Solutions for Hybrid Cloud
Joe CaraDonna, Jim Holl, Arthur Lent, NetApp
November 2017 | WP-7259
Abstract
NetApp® Data Fabric enabled solutions empower organizations to use data to make intelligent
decisions about how to optimize their business and get the most out of their IT infrastructure.
They provide essential data services for data visibility and insight, data access and control,
and data protection and security.
The purpose of this paper is to describe how a NetApp Data Fabric can solve the challenges
of data services in today’s distributed, ever-changing IT landscape. All solutions described
here are designed to manage a data-driven approach across a broad array of use cases.
Send your questions and comments to [email protected].
ii Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
TABLE OF CONTENTS
1 Introduction ........................................................................................................................................... 1
1.1 About This White Paper ..................................................................................................................................2
2 Creating a True Data Fabric ................................................................................................................. 3
2.1 A Data-Centric View of IT Infrastructure .........................................................................................................3
2.2 Design Principles of a True Data Fabric .........................................................................................................3
2.3 Data Fabric Use Cases and Solutions Overview ............................................................................................4
2.4 Data Fabric Capabilities ..................................................................................................................................5
Data Visibility and Insight .............................................................................................................................5
Data Access and Control ..............................................................................................................................5
Data Protection and Security ........................................................................................................................6
2.5 Data Fabric Deployment Models .....................................................................................................................6
Private Clouds and Service Provider Clouds in the Data Center ..................................................................6
Public Clouds................................................................................................................................................7
Colocation.....................................................................................................................................................7
Hybrid Clouds and Multiclouds .....................................................................................................................7
2.6 Data Fabric Architecture Overview .................................................................................................................8
3 Data Fabric Enabled Solutions ............................................................................................................ 9
3.1 Data Analytics in the Hybrid Cloud ................................................................................................................ 10
NFS Deployments for Managed Hadoop Systems ..................................................................................... 11
Combining NFS and Object Deployments for AWS Elastic Map Reduce ................................................... 11
Hybrid Cloud Data Pipelines for Object Storage ......................................................................................... 12
3.2 Enabling Infrastructure as a Service ............................................................................................................. 14
NetApp Approach to IaaS Security ............................................................................................................. 14
Two Architectural Approaches Provide Choice........................................................................................... 15
3.3 Automatic Data Tiering in the Hybrid Cloud .................................................................................................. 16
StorageGRID for Data Tiering .................................................................................................................... 17
Fabric Pool for Data Tiering ........................................................................................................................ 17
3.4 DevOps in the Hybrid Cloud.......................................................................................................................... 18
Data Protection for Continuous Integration, Continuous Deployment Hybrid Cloud DevOps ..................... 19
3.5 Workload Migration to, from, and Among Public Clouds ............................................................................... 21
Transfer Data Efficiently with SnapMirror Data Transport .......................................................................... 21
Workload Migration for Mission-Critical Data .............................................................................................. 22
3.6 Disaster Recovery and Business Continuity ................................................................................................. 23
Mirror Data to ONTAP Cloud for Disaster Recovery Without Investing in a Secondary Site ...................... 24
NetApp Private Storage for Disaster Recovery........................................................................................... 24
3.7 Data Protection ............................................................................................................................................. 25
iii Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Protecting SaaS Data with Cloud Control ................................................................................................... 25
Protecting Your Data Center Data with AltaVault in the Cloud ................................................................... 27
Protecting NFS Data with AltaVault and SnapCenter ................................................................................. 28
3.8 Controlling Specialized Industry Applications in the Cloud ........................................................................... 29
3.9 Hybrid Cloud Resource Monitoring and Managing ........................................................................................ 30
Maintaining Service Levels in the Hybrid Cloud.......................................................................................... 30
Controlling Cost in the Hybrid Cloud ........................................................................................................... 31
Proactively Monitor the Data Fabric ............................................................................................................ 31
Resources .................................................................................................................................................. 33
LIST OF FIGURES
Figure 1) Data Fabric use cases. ...................................................................................................................................4
Figure 2) Data Fabric deployment models......................................................................................................................6
Figure 3) Standard NetApp Private Storage deployment architecture. ...........................................................................7
Figure 4) NetApp Data Fabric architecture. ....................................................................................................................8
Figure 5) NFS data center deployments for managed Hadoop systems. ..................................................................... 10
Figure 6) NetApp Data Fabric Cloud Sync service rapidly moves data to and from AWS, converts the file-based NFS dataset to the S3 object format, and can kick off a cloud service. ................................... 12
Figure 7) With StorageGRID you can integrate cloud data services, optimize unstructured data, and automate troubleshooting and analysis of critical applications across a distributed environment. ......................... 13
Figure 8) Three deployment options for Fabric Pool: in the data center, in the cloud, and hybrid. ............................... 18
Figure 9) Continuous integration workflow with JFrog Artifactory. ................................................................................ 20
Figure 10) NetApp SnapMirror is the primary transport for the Data Fabric. ................................................................ 22
Figure 11) With a NetApp Private Storage solution, you get all the benefits of workload migration from the cloud of your choosing without sacrificing control over your data. .................................................................. 23
Figure 12) With ONTAP Cloud, a disaster recovery environment can be easily deployed in the cloud. ....................... 24
Figure 13) AltaVault can back up any array to a large selection of clouds, with support for a wide variety of backup software options. .............................................................................................................................. 27
Figure 14) SnapCenter can orchestrate backups of NFS data to the cloud using SnapMirror and AltaVault technologies. ................................................................................................................................................. 28
Figure 15) NetApp Private Storage enables specialized industry applications to benefit from cloud resource access while your data remains secure within the colocation facility. ............................................................ 29
Figure 16) The visibility delivered by OnCommand Insight can help manage service levels and control costs in hybrid cloud environments. .................................................................................................................. 31
LIST OF TABLES
Table 1) The Data Fabric enabled solutions support the nine most common customer use cases. ...............................5
1 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
1 Introduction
“Data is becoming as critical to life as electricity. We at IDC estimate that in only eight years, nearly 20% of the data in the world will be critical to our daily lives, and nearly 10% of that will be hypercritical—of life-or-death importance.”
Laura DuBois Group Vice President for Computing, Platform, Storage, and Infrastructure Software, IDC
Successful businesses recognize the importance of data and digital transformation. Their IT leaders must
meet the challenges of building a data-centric organization with limited time, skills, and budget while
managing data that is increasingly distributed, dynamic, and diverse.
As the data authority for hybrid cloud, NetApp provides a full range of hybrid cloud data services that
accelerate digital transformation. Together with our partners, we are realizing our vision for the future of
data services. With the NetApp Data Fabric, data is accessible where it is needed most, organizations
can realize the full potential of their data, and IT can make the best IT strategy decisions for the business.
For many organizations, the rush to the cloud has resulted in data silos that make it impossible to fully
leverage and control data. The Data Fabric addresses this challenge and others, including:
• Protecting data and addressing security issues. Wherever an organization’s data resides, IT is still responsible for data security, protection, and governance to make sure of regulatory compliance.
• Inability to move data. After an organization’s data moves into a particular cloud, it can be difficult or impossible to move that data into a different cloud.
• Difficulty managing data consistently. Each environment has a different set of tools, APIs, and management software that make it difficult to apply consistent policies to data. IT staff must learn how to use these new tools and applications effectively.
• Limited flexibility. New technologies and services that do not integrate with existing environments are difficult to adopt. Limited technology choices affect IT’s ability to exploit the capabilities of new and existing environments.
• Lack of control. IT must be the stewards of data no matter where it is. Storing data in a cloud where there is little visibility into how it is protected and governed can put businesses at risk.
2 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
1.1 About This White Paper
The purpose of this paper is to describe how a NetApp Data Fabric can solve the challenges of data
services in today’s distributed, ever-changing IT landscape. It is organized into four sections:
• Section 1: Introduction
• Section 2: Creating a True Data Fabric
• Section 3: Data Fabric Solutions
Data Analytics in the Hybrid Cloud
Enabling Infrastructure as a Service
Automatic Data Tiering in the Hybrid Cloud
DevOps in the Hybrid Cloud
Workload Migration to, from, and Among Public Clouds
Disaster Recovery and Business Continuity
Data Protection
Controlling Specialized Industry Applications in the Cloud
Hybrid Cloud Resource Monitoring and Managing
• Resources
3 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
2 Creating a True Data Fabric
Data Services and Data Fabric Architecture
2.1 A Data-Centric View of IT Infrastructure
The NetApp Data Fabric is built for data-driven enterprises in a data-centric world. In “Creating a Data-
Driven Organization,” Carl Anderson starts off by saying, “Data-drivenness is about building tools,
abilities, and, most crucially, a culture that acts on data.” An organization must adopt a specific IT strategy
focused on data in order to become data centric. And that’s where the Data Fabric comes in. The NetApp
Data Fabric enables organizations to put data at the heart of their businesses.
Data is created and used in multiple locations, and it often needs to be leveraged and shared with other
locations, applications, and infrastructures. Customers want a way to manage data that is consistent and
integrated. The NetApp Data Fabric architecture weaves together data across sites, beyond physical
boundaries, and across applications. It provides a way to manage data that puts IT in control and
simplifies ever-increasing IT complexity.
The Data Fabric architecture helps catalyze digital transformation by providing these capabilities:
• Data visibility and insights. Insights into performance, capacity, and cost for resources and application planning, along with analytics application integrations
• Data access and control. Migration to and from on-premises and cloud environments, replication, business continuity, data tiering for efficiency, and access across locations and applications
• Data protection and security. Data backup and recovery, archive and retention, copy data management, encryption, and application integration
2.2 Design Principles of a True Data Fabric
NetApp is delivering a Data Fabric built for a data-centric world. The NetApp Data Fabric simplifies and
integrates data management across cloud and on-premises environments to accelerate digital
transformation. The NetApp Data Fabric delivers consistent and integrated hybrid cloud data services for
data visibility and insights, data access and control, and data protection and security.
A true Data Fabric architecture delivers on five major design principles:
• Control. Securely retain control and governance of data regardless of its location: on the premises, near the cloud, or in the cloud.
• Freedom. Choose cloud, application ecosystem, delivery methods, storage systems, and deployment models, with freedom to change.
• Integration. Enable the components in every layer of the architectural stack to operate as one, while extracting the full value of each component.
• Access. Easily get data to where applications need it, when they need it, in a way they can use it.
• Consistency. Manage data across multiple environments using common tools and processes, regardless of where it resides.
The NetApp Data Fabric architecture is designed to support these principles.
4 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
2.3 Data Fabric Use Cases and Solutions Overview
NetApp Data Fabric enabled solutions help manage data across customers’ diverse use cases. They
offer a choice of endpoints and application ecosystems and connect data between on-premises and cloud
environments.
Figure 1) Data Fabric use cases.
5 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
2.4 Data Fabric Capabilities
As the leader in data services, NetApp is evolving Data Fabric capabilities in three key domains to
facilitate the creation, deployment, and use of Data Fabric solutions, applications, and services.
Data Visibility and Insight Data visibility and insight go hand in hand. Without visibility into the organization’s data, IT lacks the ability to gain the insights it needs to make sure service levels are met and environments are operating at peak efficiency. In a hybrid cloud environment, getting visibility into huge volumes of data using disparate toolsets for different environments and clouds is particularly challenging. Data Fabric capabilities in this domain enable services that traverse all environments to provide critical information about who created the data, when was it last used, and who modified the data.
Data Access and Control
Access is a key element of control over an enterprise’s hybrid cloud data. To maintain the organization’s
security profile and to facilitate adherence to geo and industry regulations, it is important that IT always
controls who can access the data and where that data is located. Data services in this domain rely on
auditing and logging, role-based access and control, digital rights management, and secure multitenancy
capabilities.
Table 1) The Data Fabric enabled solutions support the nine most common customer use cases.
6 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Data Protection and Security
Data-driven organizations thrive on data. Data loss or breaches in data security can cost an organization
more than financial loss; they can cost the organization its reputation. Data protection and security are
essential to success. With the proliferation in the volume of data, diversity of data types, and the
interdependencies in virtual environments, many organizations struggle to meet recovery time and
recovery point objectives. Traditional backup and restore methods are not designed for hybrid cloud and
the stringent service-level objectives (SLOs) that today’s businesses demand from IT.
In the Data Fabric architecture, data protection and security are provided as services by the
infrastructure. Capabilities in this domain relate to replication, backup and restore, archiving, failover,
encryption, and key management.
2.5 Data Fabric Deployment Models
Hybrid cloud deployment models offer the choice, efficiency, and flexibility today’s organizations need.
When clouds are connected by the Data Fabric, IT can draw from the resources of each, move data and
applications to new cloud services, and put every workload on the most appropriate platform. The Data
Fabric transforms how organizations manage, protect, and move their data, regardless of where it lives.
Because enterprises will consume IT from all three types of cloud resources for the foreseeable future,
data needs to be mobile. Data will move between providers, between hyperscale clouds, and between
private clouds.
A Data Fabric transforms how organizations manage, protect, and move their data, regardless of where it
lives. It provides the control and choice organizations need to make the hybrid cloud real as they take
advantage of the new infrastructure options that work best for them.
Private Clouds and Service Provider Clouds in the Data Center
Private clouds can reside in an organization’s own data center or be hosted in a remote facility. In either
case, the hardware and software architecture choices are the same, and there are plenty from which to
choose.
Figure 2) Data Fabric deployment models.
7 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Public Clouds
Public clouds are made available by service providers that own and operate their own data centers and
infrastructure. Although the largest cloud providers operate at a scale at which they can design
proprietary architectures, service providers typically choose from the same options used by enterprises
architecting private clouds. Doing so enables the service providers to focus less on infrastructure and
more on their core business of service innovation.
Service providers utilizing NetApp infrastructure can enable customers to expand their Data Fabrics to the
cloud by offering NetApp SnapMirror® or NetApp SnapVault® services. This approach allows the service
provider’s customer base to efficiently onboard data into the cloud for use with the service provider’s
services, paving the way for hybrid cloud architectures.
In addition, NetApp ONTAP® Cloud software can be used to quickly create an ONTAP endpoint in the
public cloud, bringing the value of ONTAP data management to cloud storage.
Colocation
A hybrid cloud might also connect colocation managed and/or dedicated services with cloud resources.
For example, with NetApp Private Storage (NPS), organizations can deploy a private FAS cluster in a
colocation facility and use a network exchange to connect it to public cloud compute resources. This
deployment model allows for a low-latency hybrid architecture, combining the security of private storage
with the elasticity of public cloud compute resources. IT departments retain control of their data, from its
physical location to retention policies and SLOs, and gain the benefit of data mobility across
environments through Data Fabric connectivity.
Hybrid Clouds and Multiclouds
Hybrid cloud is a mix of on-premises traditional or private cloud IT resources with public cloud resources
within or across workloads.
In its simplest form, a hybrid cloud deployment might consist of a FAS array in the corporate data center
and ONTAP Cloud in a public cloud, connected using the SnapMirror transport to replicate data from one
Figure 3) Standard NetApp Private Storage deployment architecture.
8 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
location to the other. This simple architecture establishes the Data Fabric, enabling application data to be
served and managed the same way in both locations.
2.6 Data Fabric Architecture Overview
Creating a Data Fabric is about taking a data-centric view of IT infrastructure across the environment—
endpoints, connections, ecosystems, applications, and services—while maintaining an organization’s
security posture and framework. The overall architecture is composed of products and solutions that
unbind data from underlying systems so that data can be accessed across the fabric. With NetApp, IT
architects have many building blocks from which to choose at each layer that have been designed with
the principles of a true Data Fabric in mind.
We organize our discussion of the Data Fabric architecture around the layers that create the foundation
for the Data Fabric and the capabilities that enable the data services that run in each layer of the
architecture. Data Fabric capabilities are organized into three crucial domains: data visibility and insight,
data access and control, and data protection and security.
There are four conceptual layers to the infrastructure. At the foundation are the endpoints. These are the
engineered and software-defined storage systems. The diversity of Data Fabric endpoints provides the
flexibility organizations require to make a decision today and change it tomorrow.
In the Data Fabric, data is free to move across endpoints by virtue of the connections layer. Connections
can be enabling technologies such as protocols or fully featured applications such as NetApp
SnapCenter®. Connections make it easy to get data to the applications that need it, when they need it, in
a way that they can use it.
The ecosystems layer enables the components in every layer of the architectural stack to operate as one.
The Data Fabric integrates storage and data management functionality with popular application software
frameworks, tools, and processes no matter where data resides.
The applications and services layer is the top layer of the architecture. Data Fabric–integrated
applications and services from NetApp and its partners deliver high-value, secure solutions that leverage
the foundational capabilities of the underlying fabric.
Figure 4) NetApp Data Fabric architecture.
9 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
3 Data Fabric Enabled Solutions
In today’s data-driven world, digital transformation tops the strategic agenda in many organizations. Data
can make the difference between surviving and thriving. Thriving in this environment requires creating a
data-centric organization and adopting a holistic approach to managing data that is secure, efficient, and
future-proof and provides freedom of choice.
While IT leaders everywhere are under pressure to unleash the power of data, organizations that
embrace the Data Fabric are using their data to learn more about their customers, reduce costs, and
boost efficiency. They are also turning their data into new lines of business that produce additional
revenue streams.
NetApp Data Fabric enabled solutions help organizations use data to optimize their operations and get
maximum value from their IT infrastructure. NetApp Data Fabric enabled solutions deliver consistent and
integrated hybrid cloud data services for data visibility and insights, data access and control, and data
protection and security.
Data Fabric Enabled Solutions
3.1 Data Analytics in the Hybrid Cloud
3.2 Enabling Infrastructure as a Services
3.3 Automatic Data Tiering in the Hybrid Cloud
3.4 DevOps in the Hybrid Cloud
3.5 Workload Migration to, from, and Among Public Clouds
3.6 Disaster Recovery and Business Continuity
3.7 Data Protection
3.8 Controlling Specialized Industry Applications in the Cloud
3.9 Hybrid Cloud Resource Monitoring and Managing
10 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
3.1 Data Analytics in the Hybrid Cloud
Leverage Cloud Computing to Get the Most Value from Data
Data Fabric Use Cases: Accelerate Workloads and Analytics | Integrate Cloud Data Services
Data centers are generating a huge amount of unstructured data that contains valuable information. Data
visionaries use this data to inform their business strategy and achieve operational excellence. No longer
is data locked away on devices hidden behind firewalls. Instead, it is becoming distributed, dynamic, and
diverse, making it incredibly difficult to manage and mine for insights. The intense compute requirements
required for data analytics are making cloud the standard for bursty compute workloads due to its on-
demand ability to scale infrastructure as datasets increase in size. Rather than build a fixed environment
in the data center, cloud services make nearly endless compute capacity available on a pay-for-only-the-
resources-you-require, when-you-require-them basis.
IT can manage its own analytics platform in the data center, or it can leverage a public cloud service such
as AWS Elastic Map Reduce (EMR). NetApp offers two deployment options for hybrid cloud data
analytics:
• The NFS deployment uses SnapMirror to connect on-premises datasets with NetApp Private Storage near the cloud and data analytics engines and compute in the cloud.
• The object deployment uses Cloud Sync, a NetApp Data Fabric application, to move datasets to and from AWS; convert the file-based NFS datasets to the Amazon Simple Storage Services (Amazon S3) object format; automatically launch cloud services such as RedShift, EMR, or RDS; and return results to the original location of the data.
The task of getting large datasets from the data center to the cloud and back again can be daunting.
Organizations must consider: How long is it going to take? Can we keep the data up to date? Is it in the
right format? How much will it cost to get our data back?
Figure 5) NFS data center deployments for managed Hadoop systems.
11 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
NFS Deployments for Managed Hadoop Systems
Apache Hadoop software is a popular big data framework that enables the distributed processing of large
and varied datasets across clusters of computers. Historically, Hadoop has been used primarily for
incoming external data. However, many organizations need to use Hadoop for existing data stored in
network-attached storage (NAS). Typically, this approach requires setting up a storage silo to host the
Hadoop Distributed File System (HDFS) and then running the Hadoop analytics on that storage. This
siloed approach complicates data management, contributes to inefficiency, and increases costs as a
result of having to move the data between NAS and HDFS.
The NetApp In-Place Analytics Module for Hadoop allows Hadoop to run natively on NFS without needing
to move the data or create a separate data silo on HDFS. IT can swap out HDFS for NFS or run NFS
alongside HDFS. The NetApp In-Place Analytics Module works with MapReduce for compute or
processing and supports other Apache projects, including HBase (columnar database) and Spark
(processing engine compatible with Hadoop).
The NetApp In-Place Analytics Module can support many types of workloads: batch, in memory,
streaming, and more. The module is applicable to multiple deployment models, whether they be ONTAP
on the premises, near the cloud with NetApp Private Storage, or in the cloud with ONTAP Cloud.
Combining NFS and Object Deployments for AWS Elastic Map Reduce
AWS offers an EMR service for organizations that do not want to install and manage their own analytics
engine. EMR loads data primarily from S3 buckets.
However, this configuration is not without its own challenges in hybrid cloud environments. Tools such as
the rsync utility and the homegrown scripts used with it to perform these tasks are often inefficient and
cumbersome to work with. Organizations need a way to efficiently and securely transfer unstructured data
residing in local (on-premises) file shares to the public cloud, run the analytics, and then retrieve the
results. Additionally, they still need a solution to handle file/object format transformation and keep files or
objects in sync as they change.
Automatically Analyze Most Recent Data with Cloud Sync
The NetApp Data Fabric Cloud Sync service rapidly moves a dataset to and from AWS, converts the file-
based NFS dataset to the S3 object format, and can kick off a cloud service (such as EMR, RedShift, or
RDS).
Cloud Sync supports any NFS v3 file share (NetApp or third party). It effectively manages a large number
of files by rapidly traversing directory structures and transferring them to AWS in parallel. After the
baseline is copied, Cloud Sync’s continuous sync capability synchronizes the dataset incrementally as
changes occur, minimizing the time it takes to transport new data and get results.
Envision the Future: Extending Support for Hadoop
Today the NetApp In-Place Analytics Module supports Hortonworks
Hadoop clusters deployed on the premises or in the cloud. We are
extending the NetApp In-Place Analytics Module support to native cloud
analytics platforms, including HD insight, EMR, and DataBricks.
12 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
While data is being analyzed in S3, users can still use the dataset in the active-active mode. After data is
transferred, Cloud Sync can trigger the AWS cloud analytics service of your choice and return the results
back to the original location of the data, whether on the premises or in the cloud.
Hybrid Cloud Data Pipelines for Object Storage
Reengineering process bottlenecks or bolstering process efficiency with statistical analysis, exception
handling, and predictive monitoring is a key part of building a responsive, data-driven architecture. To
reduce process handling, organizations must define the way they measure processes to assure accurate
evaluation. They must also assemble data from a variety of digital resources to evaluate their products
and services as well as monitor customers’ responses.
To achieve this data-driven architecture, organizations need a data pipeline across multiple services,
some located on the premises and others residing in the public cloud. They must be able to access
resources (that is, other services and data) located on either side of the on-premises and cloud divide.
NetApp StorageGRID® creates data pipelines that transparently use the best resource for the task at
hand, regardless of where these resources live. By deploying a StorageGRID architecture, organizations
can integrate cloud data services, optimize unstructured data, and automate troubleshooting and analysis
of critical applications across a distributed environment.
StorageGRID is an object storage solution and distributed content repository with metadata-controlled
data management executing on servers in virtual machines and block storage attached to the physical
servers. It is an ideal solution for a hybrid deployment because of its ability to scale across geographies
and integrate with Amazon S3.
Envision the Future: Expanding Cloud Sync Protocol Support
Today Cloud Sync can sync between NFS and S3 buckets. We are
expanding support to include CIFS and data synchronization across all
combinations of supported protocols: CIFS, NFS, and S3.
Figure 6) NetApp Data Fabric Cloud Sync service rapidly moves data to and from AWS, converts the file-based NFS dataset to the S3 object format, and can kick off a cloud service.
13 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Object storage is uniquely suited for the cloud because it is HTTP based, so it can be directly accessed
from a cloud. Applications sitting in the cloud, with the right networking configurations, can read data
directly from an on-premises object store. And, with CloudMirror, organizations can replicate data into the
cloud, allowing customers to use their existing cloud investments to house on-premises data.
StorageGRID Webscale supports features that enable deep integration with the cloud to create data
pipelines that span multiple services, regardless of their location. This integration allows organizations to
use the most efficient resources for the task at hand, such as flexible cloud compute, fast and low-latency
on-premises storage, or specialized cloud services.
In addition, StorageGRID supports two key features that make creating automated, event-driven
workflows seamless:
• CloudMirror replicates objects in a native format to the cloud and automates event notifications to create pipelines as required. For instance, when firms need operational analysis to address a significant or meaningful change in the state of a customer-facing application or website, pairing StorageGRID with AWS Elasticsearch can help.
• StorageGRID with AWS notification integration publishes notifications directly to AWS Simple Notification Service (SNS). In this case only the notification is published, while the actual object is still on the premises. That notification then triggers an AWS Lambda function that can either push or pull objects as wanted to other AWS services. For example, metadata can be pulled into AWS Elasticsearch, where the data is indexed for analysis in near real time (less than one second). Using the Kibana plug-in, administrators can visualize the data and perform operational analyses such as finding outages and problems. Troubleshooting teams can then search the index and perform statistical aggregations to find the underlying root cause and resolve the issues.
Figure 7) With StorageGRID you can integrate cloud data services, optimize unstructured data, and automate troubleshooting and analysis of critical applications across a distributed environment.
14 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Resources: Data Analytics in the Hybrid Cloud
NetApp Cloud Analytics Solutions www.netapp.com/us/solutions/cloud/analytics.aspx
3.2 Enabling Infrastructure as a Service
Simplify Performance Management and Confidently Deliver Per-Tenant Service-Level Objectives
Data Fabric Use Cases: Build Cloud Hosting Infrastructure | Gain Data Center Efficiencies |
Simplify and Automate
Managing data that is increasingly dynamic and diverse in the face of rapidly changing business
requirements can be a costly, ongoing struggle. Increasingly, organizations are taking advantage of the
flexibility provided by infrastructure-as-a-service (IaaS) cloud offerings to meet the challenge. IaaS allows
organizations to provision compute, storage, network, and additional resources on which IT can deploy
and run operating systems and applications. IT also has the option of creating private IaaS offerings to
provide these advantages to their business users.
NetApp Approach to IaaS Security
Security is a primary concern for anyone considering an IaaS solution. The hypervisor of a virtual
environment is a key component of an IaaS solution and is the focal point of NetApp’s IaaS security.
NetApp’s approach to IaaS security focuses on three main areas:
• Hypervisors and virtual machines (VMs). The number of virtual solutions across the Data Fabric increases exposure to VM-based attacks. The impact of these attacks can be exponential. A breach of one VM can affect many other VMs in the same hypervisor or physical server. Virtual machine–based rootkits, which directly affect the hypervisor, can propagate attacks to subsequent VMs, applications, and software in the solution. It is critical to protect virtual machines by protecting the hypervisor from traditional attacks and vectors such as DDoS and DNS-based attacks.
• Virtual network and infrastructure. Attacks against the virtual network infrastructure are prevalent. These attacks usually target the virtual switch or router that controls the flow of traffic across the fabric. Attacks such as VLAN hopping and manipulation or modification of ARP tables are examples of virtual network vulnerabilities.
Envision the Future: Enterprise-Class Cloud Storage
NetApp and Microsoft have partnered to deliver enterprise-class cloud
storage as a native Microsoft Azure service. For the first time, the NetApp
Data Fabric enables customers to extend directly into a hyperscaler
environment. Azure users can provision and manage NFS v3 or v4 shares,
snapshots, and clones directly from the Azure portal. The service is built
upon the capabilities of ONTAP, delivering the same high availability, high
performance, and high durability our customers expect from our engineered
systems. Data on NFS shares can be synchronized from the data center to
Azure cloud and back using Cloud Sync or SnapMirror. With the NFS service,
customers can reliably run POSIX workloads in Azure, as well as new
workloads, such as analytics of unstructured data with HDInsight integration.
To learn more, visit www.netapp.com/azure.
15 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
• Management functions. Colocation, tenancy, and segmentation are the key management functions that pertain to IaaS:
Colocation refers to sharing of physical resources, where multiple VMs share compute, storage, memory, and other resources. The sharing of such resources inherently increases the attack surface and therefore increases risk.
Tenancy or multitenancy makes sure that when different organizations and users share the same applications and hardware in the cloud environment, the information from their respective logical environments is isolated and never shared.
Network segmentation creates logical separation and therefore isolation and reduction of the attack surface while providing key points of visibility.
Two Architectural Approaches Provide Choice
NetApp offers two architectural approaches to support IaaS delivery. One features ONTAP software
running on FAS system endpoints, and a second features the Element® OS running on SolidFire® and/or
NetApp HCI systems. Both choices offer all-flash performance with quality of service functionalities that
simplify performance management and enable IT to deliver per-tenant service-level objectives and charge
appropriately for service levels.
NetApp IaaS solutions provide logical network segmentation and secure multitenancy to reduce the threat
landscape for security in a highly virtualized environment. They provide the flexibility IT requires to
increase capacity as needs increase and to introduce new services to users more quickly.
Through intensive integration, NetApp’s IaaS solutions provide consistent data management across
multiple protocols, multiple hypervisors, and new applications without significant reinvestment for
reengineering.
ONTAP for IaaS
With ONTAP, IT can scale FAS systems both vertically and horizontally without service disruption. Flash
capacity can be expanded to increase performance. High-density drives can be added to increase raw
capacity. Systems can be scaled up to create higher-end arrays. To distribute workloads, systems can be
scaled out by adding storage nodes.
SnapMirror and SnapVault connection services allow easy onboarding of infrastructure users. After being
onboarded, users can easily access data through a portal using NetApp APIs for automation and
management integration.
16 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Element OS for IaaS
The elastic Element OS scale-out architecture allows for expansion or contraction of storage pools and
predictable performance gains as nodes are nondisruptively added to (or removed from) the HCI or
SolidFire cluster. Data is automatically redistributed in the background across all nodes in the cluster,
maintaining balance as the system grows. With this level of flexibility, IT can quickly and easily reallocate
storage (and in the case of HCI, compute) resources where they are needed as business demands
change.
Native QoS capabilities allow for delivery of firm performance SLAs for applications, workloads, and
tenants across the entire infrastructure: an important capability for cloud service providers and enterprises
building clouds.
Resources: Enabling Infrastructure as a Service
Build Cloud Hosting Infrastructure www.netapp.com/us/solutions/build-cloud-infrastructure
3.3 Automatic Data Tiering in the Hybrid Cloud
Simplify Management and Optimize Cost Benefits with Automatic Data Placement
Data Fabric Use Cases: Gain Data Center Efficiencies | Integrate Cloud Data Services | Optimize
Unstructured Data | Simplify and Automate
Data is dynamic. Access patterns constantly vary, making it difficult to balance the delivery of frequently
requested data while delivering cost optimization for rarely accessed or inactive data. Data is also diverse
and, in a data-driven organization, valuable throughout its lifetime. What is inactive today might unlock a
major business breakthrough tomorrow.
Envision the Future: Expanding SnapMirror Data Transport Endpoints
Today, SnapMirror data transport is available for ONTAP and AltaVault™
endpoints. It is being expanded to include SolidFire and HCI.
SnapMirror data transport allows data to be accessed and consumed in a
form native to each endpoint. For example, SolidFire serves a given
dataset to clients using its iSCSI protocol, but after the data is moved to
FAS using SnapMirror, the FAS may serve the dataset to clients using
iSCSI or Fibre Channel.
Extending SnapMirror to include SolidFire and HCI opens up a host of use
cases between the platforms, from backup and disaster recovery to
workload migration and acceleration.
Envision the Future: ONTAP Select Integration with SolidFire and HCI
Tighter integration with ONTAP Select allows for integrated file services on
the SolidFire and HCI platforms.
17 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Governance, regulation, privacy, and sovereignty policies apply to data, restricting what type of data can
reside where. When taking advantage of cloud resources, IT must make sure that the organization
complies with company policies and all levels of governmental regulation. Through automated data
tiering, policies automatically and transparently move data between different classes of storage with
different cost and performance structures. The data is always accessible, but users experience different
performance and latency depending on the definition of each storage tier.
StorageGRID for Data Tiering
StorageGRID object storage organizes data into flexible-sized data containers, called objects. Each
object has both data (an un-interpreted sequence of bytes) and metadata (an extensible set of attributes
describing the object). Object-based storage uses unique IDs to identify files and packages these along
with extensible metadata about the object. This allows data to be referenced and queried based on
anything about the file. The types of identifier tags used allow for the indexing of files in quantities several
orders of magnitude higher than a file system, making object storage ideal for enterprise storage
distributed between on-premises data centers and public clouds.
StorageGRID’s dynamic policy engine places newly ingested data into tiers automatically across on-site
locations such as storage arrays and tape as well as public cloud locations such as AWS Simple Storage
Service (S3). As data characteristics change over time the administrator can create (or change) policies
after data ingestion and existing objects will flow to follow the new policy.
Fabric Pool for Data Tiering
The NetApp Data Fabric seamlessly connects file and object repositories on the premises and in the
cloud. With policy-based data tiering, IT can automatically make sure the right data is in the right place.
Volume- and object-level metadata is used to define region placement policies to help make sure of data
privacy compliance. Hot data resides on the fastest (most expensive) media, and cold data resides on the
slowest (least expensive) media. Cold data automatically moves to object stores on the premises or in the
cloud, and hot data is loaded into SSD storage on demand or predictively based on access patterns.
NetApp ONTAP Fabric Pool technology enables automated data tiering within aggregates that consist of
a set of SSDs and HDDs. The HDDs are the slower, less costly, high-capacity tier. The SSDs are the
faster but more expensive tier. With such a hybrid configuration, the hot data automatically gravitates to
the SSD tier, allowing for the highest performance data access.
ONTAP intelligence automatically moves data to the tier that best suits the observed access patterns.
The customer benefits by getting the best of all worlds: SSD-level performance for hot data, HDD prices
for capacity, and automatic data placement to exploit both.
18 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Fabric Pool takes the concept of data tiering and expands it to object stores and hybrid clouds. In the
case of NetApp All Flash FAS, consider an aggregate consisting of a set of SSDs and an object store
bucket. As with Flash Pool™ aggregates, the hot data gravitates to the SSDs, while the object store
operates as the inexpensive, deeper, slower capacity tier.
With All Flash FAS systems, the Fabric Pool object store can take many forms, including StorageGRID on
the premises, Amazon S3, and Azure Blob Storage. On-premises StorageGRID is available in a hardware
appliance form factor or as a software-defined offering running on third-party storage. Paired with ONTAP
9, StorageGRID becomes the capacity tier for infrequently accessed or “cold” data. What’s more a
FabricPool license is not required when using StorageGRID as the capacity tier.
ONTAP Cloud uses the same technology to automatically tier data between AWS EBS block storage and
AWS S3 object stores.
With ONTAP supporting object storage as a native tier, object stores become tightly woven into the fabric.
This level of system integration gives customers flexibility in choosing storage architectures, SLOs, and
price points, while unifying data management.
Resources: Automatic Data Tiering in the Hybrid Cloud
Data Infrastructure Management www.netapp.com/us/products/data-infrastructure-management
3.4 DevOps in the Hybrid Cloud
Confidently Leverage Hybrid Cloud to Accelerate DevOps
Data Fabric Use Cases: Accelerate DevOps Transformation | Simplify and Automate
Application development is now happening at scale on cloud-based platforms. The immediate and
frictionless fulfillment of developer requests makes the public cloud compelling for build, development,
and test workloads.
In traditional development workflows, checking out code, checking in code, and executing builds are time
consuming and can lead to larger build windows and unnecessary delays in time to market. Developers
and their operations counterparts can’t move their projects forward while they’re waiting for requests to be
fulfilled and tests to be run. The longer the build times, the more features and fixes that get packed into a
single check-in. The results can be higher error rates, slower error resolution times, and sacrifices to
quality and the customer experience.
Figure 8) Three deployment options for Fabric Pool: in the data center, in the cloud, and hybrid.
19 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
By adopting DevOps processes and technologies that increase collaboration, flexibility, and automation,
organizations can improve the speed with which they respond to customer and business requirements
without sacrificing quality or the user experience. With the power of an agile DevOps infrastructure,
developers can quickly adjust their behavior. Creating new workspaces is nearly free, so developers
create more and keep builds longer for test and error resolution purposes.
With a NetApp Data Fabric, organizations can make development tool, service, and data management
choices without worrying about where data resides or where the work is done. They can adopt a hybrid
cloud DevOps model to achieve fast, reliable, efficient application development at scale anywhere
developers choose to develop and deploy applications. Development and deployment choices are no
longer driven by what’s best for the technology. DevOps on the Data Fabric is about what’s best for the
business.
Data Protection for Continuous Integration, Continuous Deployment Hybrid Cloud DevOps
Jenkins is one of the most common continuous integration (CI) tools used in software development.
Continuous integration is the process of testing small code changes iteratively and identifying bugs at an
early stage in the development process. This early intervention can accelerate time to market and
improve code quality.
Jenkins integrated with NetApp provides seamless integration of Git, Jenkins, and JFrog Artifactory on
Docker containers. The Docker containers mount persistent datastores from NetApp ONTAP using
RESTful APIs and the NetApp Docker Volume Plug-In. The framework accelerates the development
process by provisioning preconfigured (source code prebuild artifacts and binary) workspaces instantly for
onboarding hundreds to thousands of developers at scale. This process also reduces compute and
network resource consumption to improve infrastructure efficiency on the premises and in cloud
environments.
ONTAP provides a high degree of efficiency with its data management capabilities, which allows IT to
provision more workloads for less data space.
JFrog Artifactory is the most popular universal binary repository manager, supporting different software
packages created in any language to accelerate the software delivery process in continuous deployment
(CD) workflows. Developers write and test code using artifacts—libraries, compilers, and tools—and
deliver a successful final build. One of JFrog Artifactory’s key features is consistent version control for all
the binary packages on which development teams work prior to production release.
Together with JFrog Artifactory, NetApp software manages all the build data that is created. In addition to
providing a robust and scalable data management platform, NetApp ONTAP helps developers meet
requirements such as data protection and data availability. By configuring JFrog Artifactory data volumes
on NetApp systems, developers can use SnapCenter to protect against data loss by taking application-
consistent Snapshot™ copies of the database and the repository.
20 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
This integration gives developers more control over the data generated during the CI process. These
Snapshot copies are taken on the data volumes that provide persistent storage to Artifactory running on
Docker containers. They are instantaneous, thin provisioned, and highly efficient.
When SnapCenter creates the thin-provisioned Snapshot copies, developers can clone the data instantly
to test the applications during user acceptance validation and staging. Data can be replicated for disaster
recovery to a remote site or to public hypervisor cloud environments such as Amazon Web Services
(AWS) or Microsoft Azure.
During development, developers can use Artifactory as the binary repository manager on the premises.
During deployment, they can use Artifactory in hyperscaler clouds such as AWS. Because SnapCenter
and Artifactory integration enables a Data Fabric environment, application and asset owners can move
data to public hyperscaler clouds without vendor lock-in.
Resources: DevOps in the Hybrid Cloud
Accelerate DevOps www.netapp.com/devops
Figure 9) Continuous integration workflow with JFrog Artifactory.
21 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
3.5 Workload Migration to, from, and Among Public Clouds
Accelerate Application Workloads and Analytics
Data Fabric Use Cases: Accelerate Workloads and Analytics | Gain Data Center Efficiencies | Integrate
Cloud Data Services | Protect and Secure Data | Simplify and Automate
Public clouds are often compared to utilities, such as an electricity grid. However, data is more like a
water system, with reservoirs and pipes. And data, like water, isn’t easy to move.
Enterprises want to leverage the agility of public cloud resources to run existing workloads without having
to rearchitect them and their processes. In many cases, applications are moved individually over an
extended period. Over time, some applications might move back on the premises.
With ONTAP Cloud and NetApp OnCommand® Cloud Manager, enterprises can create a storage platform
in the cloud for which their applications are already optimized and that their staff and processes already
understand. ONTAP Cloud is a cloud-based data management solution that provides protection, visibility,
and control of cloud-based workloads. SnapMirror technology allows organizations to migrate their data to
the cloud efficiently and securely. Applications managed by SnapCenter can continue to operate without
modification, and existing manageability tools, such as OnCommand Insight, continue to provide a
complete monitoring solution.
For data that needs to be more tightly controlled, NPS allows enterprises to keep their data in a secure
colocation facility while the applications connect from the cloud or multiple clouds.
Transfer Data Efficiently with SnapMirror Data Transport
The ability to move data across a Data Fabric requires a transport mechanism to establish lines of
communication between the endpoints. When endpoints in the fabric have their own native data formats,
they require a transport technology to interoperate and move data efficiently between them. A Data Fabric
transport allows the secure movement of data from one endpoint to another and enables the data to be
accessed and consumed in a form native to each endpoint. The more connectors the fabric has, the
stronger it becomes, and the more solutions IT has at hand to address customer problems.
NetApp SnapMirror is the primary transport for the NetApp Data Fabric. The SnapMirror protocol
connects endpoints in the Data Fabric to transfer data efficiently and minimize the cutover period. It
enables applications to move data nondisruptively where it is needed most: to a cloud, across storage
tiers, or across clusters. Applications are unaware of any data movement.
The ONTAP family of products (FAS, All Flash FAS, ONTAP Cloud, and ONTAP Select) share a common
WAFL® file system format. SnapMirror interoperability, when used between ONTAP endpoints, also
OnCommand Cloud Manager: Centralized Management Across
Your Hybrid Cloud
From a simple, single interface, NetApp OnCommand Cloud Manager
enables IT to manage your ONTAP data across the hybrid cloud, from
physical FAS and All Flash FAS systems to ONTAP Cloud
deployments in AWS and Azure. Cloud Manager is also the
deployment environment for ONTAP Cloud software, offering drag-
and-drop installation and automatic selection and assignment of
resources to each instance.
Cloud Manager then creates the appropriate infrastructure behind the
volume to operate as you defined. You don’t need to know the ins and
outs of storage; you just need to know what volumes your application
requires.
22 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
preserves deduplication and compression efficiencies, meaning data does not rehydrate when moving
from one endpoint to another. This transport is the fastest way to move data in bulk around the fabric.
This fact is especially important when the fabric spans WANs for hybrid cloud architectures.
Cloud Connectivity
When endpoints plug into the fabric, they not only get the benefit of data exchange with FAS and each
other, which is important for managing SLOs and total cost of ownership, but also become cloud enabled.
With the SnapMirror transport, data can be moved efficiently to and from these endpoints to cloud
endpoints: ONTAP Cloud or FAS systems deployed as NetApp Private Storage.
Workload Migration for Mission-Critical Data
Some data is too sensitive for enterprises to comfortably or legally migrate to the cloud. In instances
where maintaining complete control over data is of the utmost importance, IT can still leverage the agility
of the cloud to accelerate existing workloads using NPS.
With NPS, you keep your data on NetApp storage “next to” rather than “in” multiple clouds. You get the freedom to connect to the clouds you want while keeping your data on your own dedicated, private NetApp storage systems. Workloads can run in the cloud without the data ever leaving your systems.
Figure 10) NetApp SnapMirror is the primary transport for the Data Fabric.
23 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
With access to multiple clouds, you can match your clouds to your workloads and choose the right mix of clouds to meet your needs. For example, you might decide to run an analytics application in Amazon Web Services, but prefer to use Microsoft Azure to cloud-burst from an existing on-premises Microsoft infrastructure.
Another possibility is using cloud agility for development and test. After these are completed, you could choose to run the application in the cloud of your choosing or deploy it on the premises.
Resources: Workload Migration to, from, and Among Public Clouds
OnCommand Cloud Manager www.netapp.com/us/products/data-infrastructure-management/cloud-manager.aspx
Data Infrastructure Management www.netapp.com/us/products/data-infrastructure-management
3.6 Disaster Recovery and Business Continuity
Pay Only for the Resources You Use When You Use Them
Data Fabric Use Cases: Gain Data Center Efficiencies | Integrate Cloud Data Services | Protect and
Secure Data | Simplify and Automate
Data-driven organizations are especially vulnerable to data loss and unavailability. Business stops when
data stops flowing. Natural and human-made disasters are a constant reminder that disaster recovery
cannot be considered an optional IT capability.
It wasn’t too long ago that disaster recovery solutions were out of reach for many organizations because
they required a duplicate infrastructure to be on standby in the event of an emergency.
Figure 11) With a NetApp Private Storage solution, you get all the benefits of workload migration from the cloud of your choosing without sacrificing control over your data.
24 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Today, organizations can leverage the cloud to implement a fully functional secondary site without having
to duplicate infrastructure. The usage-based model of cloud computing is well suited for disaster
recovery, and it offers a viable alternative to on-premises or remote site solutions. Moving disaster
recovery operations to the cloud reduces the required data center space and storage infrastructure,
resulting in cost savings and flexibility. The elasticity of cloud resources produces considerable savings
when you only pay for what you use when you use it.
Organizations wanting to deploy a disaster recovery solution without investing in a secondary data center
also have the option of using cloud-connected data centers such as Equinix colocation facilities. These
facilities offer customers a secure space with high-speed connections to leading hyperscaler cloud
providers. Deploying storage into a colocation facility saves organizations the costs of building a
secondary site while also allowing them to leverage pay-as-you-go cloud resources.
Mirror Data to ONTAP Cloud for Disaster Recovery Without Investing in a Secondary Site
ONTAP Cloud is a cloud-based data management solution that provides protection, visibility, and control
of cloud-based workloads. Organizations can leverage ONTAP Cloud as a disaster recovery solution to
take advantage of cloud cost and efficiency benefits while meeting critical disaster recovery requirements.
A disaster recovery environment can easily be deployed in the cloud and targeted from on-premises
ONTAP environments, including FAS and All Flash FAS arrays.
SnapMirror is the vehicle that transports data between onsite data centers and the cloud. NetApp
Snapshot copies provide near-instantaneous point-in-time backups, while efficiency features such as
deduplication and compression help to minimize the storage footprint. IT can set recovery point objectives
(RPOs) ranging from minutes to hours with the ability to fail over to a specific point in time.
Organizations can also retain the same security and control standards in the cloud that are required on
the premises. Data can be encrypted at rest and in flight with externally managed keys, protecting it from
unauthorized access.
ONTAP Cloud for disaster recovery is available in both AWS and Azure cloud environments.
NetApp Private Storage for Disaster Recovery
NetApp storage or converged infrastructure solutions can be deployed at cloud-connected data centers
such as Equinix colocation facilities as a secondary site, with efficient data replication to and from the
Figure 12) With ONTAP Cloud, a disaster recovery environment can be easily deployed in the cloud.
25 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
central data center. In the event of a primary site outage, the secondary storage can be brought online
quickly with compute served locally or on demand from the cloud.
NetApp SnapMirror is a feature built into ONTAP and leverages NetApp Snapshot technology to enable
efficient block-level data replication between endpoints. Data is replicated asynchronously to protect
volumes, LUNs, and storage virtual machines (SVMs), across any distance, for SAN and NAS data.
SnapMirror preserves any data reduction implemented on the primary storage to reduce the amount of
data transported over the network. The unified replication technology combines data protection for both
backup and DR scenarios whether on the premises, at remote data centers, or in the cloud.
For even higher availability, NetApp MetroCluster™ delivers zero RPO and near-zero RTO to protect your
most critical applications. MetroCluster supports flexible configurations with “set it and forget it” simplicity,
making it ideal for the most demanding service levels.
Resources: Disaster Recovery and Business Continuity
Protect and Secure Data www.netapp.com/us/solutions/data-protection
3.7 Data Protection
Protect Data Across Data Silos
Data Fabric Use Cases: Integrate Cloud Data Services | Protect and Secure Data | Simplify and
Automate
Legacy approaches to backup, such as tape libraries, can be costly and time-consuming to operate,
resulting in unnecessary operational delays for IT organizations with requirements to keep data online 24
hours a day, 7 days a week. More and more, IT professionals are adding cloud-integrated solutions to
help protect their data as an alternative to on-premises tape backup and off-premises tape storage.
According to research by ESG,1 backup and recovery is the number-one use case for cloud deployments.
The cloud is an ideal location for backup archive of tier 2 and tier 3 data, such as file shares and home
directories.
Software-as-a-service (SaaS) applications relieve IT from having to manage applications and IT
infrastructure, but they do not free them from being responsible for protecting the data that these
applications contain or produce. Cloud service agreements often contain clauses that the customer is
responsible for securing and protecting its data in the cloud. IT is responsible for:
• Retaining control of sensitive data as it moves users, folders, and mailboxes to Office 365
• Enabling business continuity with fault-tolerant data protection
• Storing backup data securely
• Making sure that the organization’s security posture and regulatory compliance programs are in effect
Protecting SaaS Data with Cloud Control
Cloud providers do not offer guarantees or SLAs regarding data recovery or the corruption of data that is
stored in their services. If application/user access or natural disaster results in data corruption or deletion,
the cloud provider is not liable.
Therefore, to mitigate risk to business continuity, it is just as important for IT administrators to maintain
backup copies of their SaaS resident data as they do with primary data in their data centers.
1
26 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
NetApp Cloud Control, a data protection service for your Data Fabric, enables backup of Microsoft Office
365 data. Cloud Control secures data by performing automatic daily backups of Exchange, SharePoint,
and OneDrive for Business. Administrators can control the archive period and backup destinations to
meet regulatory compliance guidelines.
Because Cloud Control is an integrated solution, customers can be confident that their business-critical
data is fully protected. They can eliminate the stress and costs associated with monitoring, managing,
and acquiring additional storage.
With Cloud Control, customers have a consistent, flexible, and reliable way to protect critical data across
multiple storage destinations. All content in Office 365 instances is protected by restoring Exchange and
SharePoint components—including individual items—without native Office 365 rollback burdens. Backups
are stored either in the data center or in the public cloud using ONTAP destination endpoints or object
stores supported by AltaVault (for example, StorageGRID, OpenStack Swift, and public cloud). The “bring
your own storage” nature of Cloud Control allows customers to securely retain control and governance of
data, whether it resides in cloud-based targets or on-premises endpoints.
Cloud Control offers granular restore options. Administrators have the option to restore data into the
original user’s mailbox/folders/sites or another user’s mailbox/folder/sites as needed.
Cloud Service Agreements Often Contain Clauses That the Customer
Is Responsible for Securing and Protecting Its Data in the Cloud
“Although Salesforce does maintain back up data and can recover it, it’s
important to regularly back up your data locally so that you have the ability to
restore it to avoid relying on Salesforce backups to recover your data.
“The recovery process requires [sic] is time and resource consuming and typically
involves an additional fee.”
Salesforce Knowledge Article 000004037
Retrieved September 2016
“Account Compromised”
“If you can't find your messages in All Mail, Spam, or Trash, or by performing a
search, then they've been permanently removed from Gmail, possibly deleted by
someone else. We regret that we are unable to recover messages that have
been permanently deleted. If you're concerned that someone may have gained
access to your address, we suggest that you take the following measures to
protect your account.”
https://support.google.com/a/answer/112445
Google Apps Administrator Help
Retrieved September 2016
Envision the Future: Cloud Control for SalesForce
Cloud Control for SalesForce offers automated backups, on-demand backups, and
granular restore of organization, object, record, and field data. Customers have control
of and flexibility with their backup deployment in the cloud or on the premises to meet
their regulatory compliance needs.
27 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Protecting Your Data Center Data with AltaVault in the Cloud
NetApp AltaVault cloud-integrated storage addresses the data protection challenges of NAS file services
by enabling secure backup of data to any cloud. Inline deduplication and compression reduce I/O traffic
and improve replication performance by as much as 60%. Only changed blocks are replicated, further
reducing I/O traffic.
Backing Up to Cloud Object Stores: Secure, Efficient, and Fast Data Transfers
An AltaVault appliance is a modern-day replacement for tape solutions. AltaVault provides the economic
and geolocation advantages of cloud implementations, while maintaining security levels needed for
sensitive and private data. Data is encrypted as it is ingested by the appliance and at rest with customer-
controlled keys.
Beyond its deployment flexibility, AltaVault offers ingest rates of more than 13TB per hour. Local caching
helps to minimize the recovery time objective (RTO) of recent data, and integration with SnapVault allows
for even faster and more efficient data transfers.
With ONTAP as a source, incremental NetApp Snapshot data can be pushed in its native format to
AltaVault. This streamlined payload minimizes both source and destination system load, which, in turn,
allows for less application impact, a greater number of concurrent backup and restoration sessions, and
improved RPOs.
In addition, backing up to AltaVault using the SnapMirror data transport mechanism allows for an
“incremental forever” backup strategy, eliminating the need for periodic full backups. This implementation
minimizes backup windows, provides faster data recovery, and reduces storage capacity requirements for
the destination.
For additional cost efficiency, AltaVault supports multiple cloud storage tiers (for example, AWS S3, IA,
and Glacier).
Figure 13) AltaVault can back up any array to a large selection of clouds, with support for a wide variety of backup software options.
28 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Protecting NFS Data with AltaVault and SnapCenter
Organizations can back up NFS and SMB NFS file service and home directories to the cloud using
SnapMirror technology, an AltaVault cloud provider connection, and the SnapCenter graphical interface.
Using this solution, administrators can protect their NAS file services data on FlexVol® volumes in public
or private clouds.
SnapCenter is first used to initiate a baseline backup from ONTAP to AltaVault. After that initial backup,
all other backups of that volume from ONTAP are incremental. These incremental backups are initiated
by the SnapCenter policy-driven scheduler on demand.
It is also possible to manage SnapMirror shares directly in AltaVault through its command line interface
(CLI). For example, if IT needs to remove NAS file services data protection from a volume, it can
manually delete the Snapshot copies from shares in AltaVault.
Resources: Data Protection
Protect and Secure Data www.netapp.com/us/solutions/data-protection
A Case for Multicloud Endpoints
CodeSpaces.com was a startup that offered code repository services in the cloud.
It committed exclusively to the AWS cloud platform, using EC2 for compute, EBS
and S3 for primary storage, and Glacier for backups.
Hackers acquired the CodeSpaces AWS account credentials and systematically
deleted all of the company’s cloud resources. Because CodeSpaces did not own
or control any of its data, the security breach instantly put a multimillion-dollar
startup out of business.
Multicloud endpoints could have helped CodeSpaces protect its data, its business,
and its customers. For example, AltaVault could have backed up CodeSpaces
data to a secondary cloud of its choice and then easily recovered the hacked data.
Alternatively, ONTAP Cloud could have been used with SnapVault to copy the
data to a service provider offering NetApp ingest backup as a service.
Figure 14) SnapCenter can orchestrate backups of NFS data to the cloud using SnapMirror and AltaVault technologies.
29 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
3.8 Controlling Specialized Industry Applications in the Cloud
Retain Control Over Sensitive Data While Harnessing the Power and Flexibility of the Cloud
Data Fabric Use Cases: Accelerate Workloads and Analytics | Integrate Cloud Data Services |
Protect and Secure Data | Simplify and Automate
Specialized industry applications are critical applications specific to an industry vertical. Examples include
the financial industry, genomics, electronic design automation (EDA), oil and gas, and manufacturing. In
some verticals, one or more specific applications are critical. Examples include Synopsys and Cadence in
the EDA industry and Landmark in the oil and gas industry.
Enterprises are under pressure to migrate their specialized industry applications to the cloud to take
advantage of scalability and dynamic pricing. However, these applications operate on the enterprise’s
most valuable data. Data must remain under the control of the enterprise to protect intellectual property,
comply with regulations, and meet governance requirements. Control is further complicated when an
enterprise licenses access to its data to another enterprise. Also, some of these applications have
performance requirements beyond what can be delivered by a public cloud.
By simplifying and integrating data management across cloud and on-premises resources, the NetApp
Data Fabric enables specialized engineering applications to thrive without sacrificing control,
performance, or advanced data management capabilities.
Specialized applications can run on a public cloud, with the data stored in NPS. NPS resides in a
colocation facility, such as those run by Equinix, but it’s accessible by applications over a high-speed
connection. Optionally, data can be moved between an on-premises data center and the colocation
facility with SnapMirror technology.
With NPS, the enterprise keeps control of its data. It owns the key manager and encryption at rest and
over the wire and can tell regulators exactly where the data is. The data’s owner can even grant
controlled access to the data to another enterprise without opening access to the on-premises data
center.
Figure 15) NetApp Private Storage enables specialized industry applications to benefit from cloud resource access while your data remains secure within the colocation facility.
30 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
The enterprise-grade performance of NPS meets the needs of the most demanding applications. For
many enterprises, more of the annual budget for specialized industry applications goes to software
licensing than to infrastructure. Because many of these applications are licensed in a consumption or
pay-as-you-go model, a reduction in the time it takes to complete a project translates directly into return
on investment.
In addition, NPS provides advanced data management capabilities. High availability makes sure that data
is available when it’s needed most. NPS can be combined with manageability products such as NetApp
Service Level Manager, OnCommand Unified Manager, and OnCommand Insight to automate the
provisioning of storage by service level and monitor and report storage utilization.
Resources: Controlling Specialized Industry Applications in the Cloud
Accelerate Application Workloads and Analytics
www.netapp.com/us/solutions/applications
3.9 Hybrid Cloud Resource Monitoring and Managing
Simplified Monitoring and Management of All Endpoints, Network Traffic, and Compute Resources
Across Your Hybrid Environment
Data Fabric Use Cases: Deliver Data Insights and Control | Gain Data Center Efficiencies | Integrate
Cloud Data Services | Simplify and Automate
IT leaders are challenged to maintain their service levels while controlling costs in the hybrid cloud due to
the complexity and growth that occur as the environments evolve. With the adoption of the hybrid cloud,
the number of disparate toolsets, the volume of resources, and the complexities of the interconnections
can be overwhelming. IT teams need a way to monitor the health of their endpoints, assess network
traffic, track compute resources, and discover issues before problems arise, all while making sure that the
costs are kept under control and the service levels are met.
Maintaining Service Levels in the Hybrid Cloud
OnCommand Insight addresses these key challenges. With OnCommand Insight, users can see
aggregated rollups of their hybrid infrastructures and drill down into specific resources from there.
OnCommand Insight has configurable, policy-driven alerting that triggers alerts on threshold breaches for
all performance metrics. Alerts can be customized to only alert specific users about certain types of
events, so the right people are notified when there is an issue. In addition, IT can utilize machine learning
to proactively identify potential issues with OnCommand Insight’s anomaly detection.
31 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Controlling Cost in the Hybrid Cloud
NetApp OnCommand Insight management software delivers a consistent view across the hybrid cloud
environment, making it easier to monitor and manage hybrid, multivendor storage, compute, and
switching infrastructures.
With OnCommand Insight, IT leaders can provide rolled-up bills to each line of business with automated
reports that aggregate costs by business unit, project, or application. It’s also possible to drill down into
these reports to show detailed spend by individual users and services used.
OnCommand Insight can also help optimize spend in the public cloud. It provides simple reporting into
hourly spend and the information IT needs to understand instance lifecycles and predict growth.
Customers can identify waste across their hybrid environment with OnCommand Insight’s reports and
determine whether services are aligned and optimized or where waste might be removed.
Proactively Monitor the Data Fabric
NetApp Active IQ® combines predictive analytics and proactive support to help organizations optimize
their data infrastructure across the hybrid cloud. NetApp AutoSupport® telemetry built into NetApp Data
Fabric endpoints sends more than 36 billion data points to Active IQ every day. This telemetry data, which
includes configuration information, performance data, and logs, feeds into a multipetabyte data lake.
Using advanced machine learning, Active IQ draws on this data to generate community wisdom and peer
comparisons from across NetApp’s installed base.
Active IQ unique community wisdom provides actionable intelligence and predictive, self-healing care
across your hybrid cloud. Active IQ can recommend optimal system configurations and proactively identify
misconfigurations. You can see at a glance how much storage you are using and forecast when you
might require additional storage. Continuous risk assessments, predictive alerts, and proactive anomaly
detection prevent problems before they affect your business.
Figure 16) The visibility delivered by OnCommand Insight can help manage service levels and control costs in hybrid cloud environments.
32 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Resources: Hybrid Cloud Resource Monitoring and Managing
Deliver Data Insights and Control www.netapp.com/us/solutions/data-insights
Envision the Future: Using the Data Fabric to Leverage the Public Cloud for
Machine Learning on AutoSupport Data
Building a machine learning model requires access to a lot of computational power for
short bursts of time. The public cloud is ideal for this.
AutoSupport data cannot be migrated to the public cloud for security, compliance, and
regulatory reasons. However, NetApp can leverage the Data Fabric to transparently
move AutoSupport data from on-premises ONTAP endpoints to NetApp Private Storage
in a colocation facility. Using NPS, NetApp can then take advantage of the vast
computational resources available in Amazon AWS or Microsoft Azure.
NetApp continues to work with leading analytics providers in the cloud to make it easy to
use their software on data stored in NPS.
33 Data Fabric Enabled Solutions for Hybrid Cloud © 2017 NetApp, Inc. All rights reserved.
Resources
Data Fabric Enabled Solutions for Hybrid Cloud Resources
General
NetApp Data Fabric cloud.netapp.com/datafabric
NetApp www.netapp.com
Data Analytics in the Hybrid Cloud
NetApp Cloud Analytics Solutions www.netapp.com/us/solutions/cloud/analytics.aspx
Enabling Infrastructure as a Service
Build Cloud Hosting Infrastructure www.netapp.com/us/solutions/build-cloud-infrastructure
Automatic Data Tiering in the Hybrid Cloud
Data Infrastructure Management www.netapp.com/us/products/data-infrastructure-management
DevOps in the Hybrid Cloud
Accelerate DevOps www.netapp.com/devops
Workload Migration to, from, and Among Public Clouds
OnCommand Cloud Manager www.netapp.com/us/products/data-infrastructure-management/cloud-manager.aspx
Data Infrastructure Management www.netapp.com/us/products/data-infrastructure-management
Disaster Recovery and Business Continuity
Protect and Secure Data www.netapp.com/us/solutions/data-protection
Data Protection
Protect and Secure Data www.netapp.com/us/solutions/data-protection
Controlling Specialized Industry Applications in the Cloud
Accelerate Application Workloads and Analytics
www.netapp.com/us/solutions/applications
Hybrid Cloud Resource Monitoring and Managing
Deliver Data Insights and Control www.netapp.com/us/solutions/data-insights
Refer to the Interoperability Matrix Tool (IMT) on the NetApp Support site to validate that the exact product and feature versions described in this document are supported for your specific environment. The NetApp IMT defines the product components and versions that can be used to construct configurations that are supported by NetApp. Specific results depend on each customer’s installation in accordance with published specifications.
Copyright Information
Copyright © 2017 NetApp, Inc. All rights reserved. Printed in the U.S. No part of this document covered by copyright may be reproduced in any form or by any means—graphic, electronic, or mechanical, including photocopying, recording, taping, or storage in an electronic retrieval system—without prior written permission of the copyright owner.
Software derived from copyrighted NetApp material is subject to the following license and disclaimer:
THIS SOFTWARE IS PROVIDED BY NETAPP “AS IS” AND WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE, WHICH ARE HEREBY DISCLAIMED. IN NO EVENT SHALL NETAPP BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
NetApp reserves the right to change any products described herein at any time, and without notice. NetApp assumes no responsibility or liability arising from the use of products described herein, except as expressly agreed to in writing by NetApp. The use or purchase of this product does not convey a license under any patent rights, trademark rights, or any other intellectual property rights of NetApp.
The product described in this manual may be protected by one or more U.S. patents, foreign patents, or pending applications.
RESTRICTED RIGHTS LEGEND: Use, duplication, or disclosure by the government is subject to restrictions as set forth in subparagraph (c)(1)(ii) of the Rights in Technical Data and Computer Software clause at DFARS 252.277-7103 (October 1988) and FAR 52-227-19 (June 1987).
Trademark Information
NETAPP, the NETAPP logo, and the marks listed at http://www.netapp.com/TM are trademarks of NetApp, Inc. Other company and product names may be trademarks of their respective owners.