BACKUP & RECOVERY FOR VMWARE

ENVIRONMENTS WITH AVAMAR 7.1 A Detailed Review

ABSTRACT With the ever-increasing pace of virtual environments deployed in the enterprise

cloud, the requirements for protecting these environments are quickly being identified

as the bottleneck to deployment expansion. This white paper highlights several new

features within EMC® Avamar® 7.1 that specifically target and greatly advance the

capabilities, performance, and management of VMware® data protection. With these

features, EMC Avamar is enabling the rapid continued expansion of cloud

environments, required by the demand for IT-as-a-Service (ITaaS).

April, 2014

Copyright © 2014 EMC Corporation. All Rights Reserved.

EMC believes the information in this publication is accurate as of its publication date. The information is subject to change without

notice.

The information in this publication is provided “as is.” EMC Corporation makes no representations or warranties of any kind with

respect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitness for a

particular purpose.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license.

For the most up-to-date listing of EMC product names, see EMC Corporation Trademarks on EMC.com.

VMware, vCenter, vSphere and vCloud Director are registered trademarks or trademarks of VMware, Inc. in the United States and/or

other jurisdictions. All other trademarks used herein are the property of their respective owners.

Part Number H8945.3

TABLE OF CONTENTS

EXECUTIVE SUMMARY 4

AUDIENCE 4

INTRODUCTION 5

AVAMAR: EFFICIENT DATA PROTECTION FOR VMWARE 5

Guest-level Backup & Restore 5

Image-Level Backup & Restore 6

VM Recovery Flexibility: Avamar VM Instant Access and

Changed-Block Tracking for Restore 8

VM Data Protection Management: Avamar vCenter Integration 10

VM Data Protection Management: EMC Avamar Plug-In

for vSphere Web Client 11

Cloud-Ready Data Protection: Avamar Plug-In for vCloud Director 13

CONCLUSION 14

4

EXECUTIVE SUMMARY Backup administrators face many challenges as they witness virtual cloud environments growing exponentially, and no aspect can

pose greater difficulty to this growth than in the area of backup and recovery. While virtualization technology optimizes the use of

the physical resources on a physical server basis, ultimately the underlying resources and applications available to efficiently protect

these environments can limit planned expansion from a data protection manageability perspective.

Today’s virtual environments typically contain hundreds if not thousands of VMs. With the lower amount of VMs as we have seen in

the past, the data protection scenario can be thought of as somewhat manageable, however given the more recent higher VM counts

we have witnessed today, data protection of the environment is no trivial task.

Balancing limits on VM growth due to data protection requirements and the expansion requirements requested from business units to

cloud administrators is a daunting challenge. Oftentimes the environment continues to grow but portions are left unprotected or

protected with less than ideal processes in place.

EMC® Avamar® and its integration with the VMware®’s Storage, APIs - Data Protection (VADP), addresses and removes these

constraints for VMware environments. Avamar’s data deduplication technology seamlessly integrates into virtual environments,

providing rapid backup capabilities utilizing the Changed Block Tracking (CBT) feature within the API.

As we all know, backup is performed for one reason and that is recovery. The ability to restore data from backup as fast as possible

is paramount to your business. Introduced since the Avamar 6.0 release, the CBT process is now leveraged to speed the restore

process as well assuring you recover your VMs to production in minutes. Couple this with our VM Instant Access capability in Avamar

7.0 or later, for those customers leveraging the flexibility of EMC Data Domain System integration, the ability to boot and run a VM

from the backup image in seconds is also an option.

Another important factor to consider in a virtualized backup solution is a robust, flexible, yet easy-to-use backup and recovery

management interface. Avamar continues to provide robust integration of live vSphere client data within the main Avamar

Administrator UI. To better reduce operational expenses and enhance cross-functional operations in typical data center, Avamar has

introduced management functionality embedded within the native vSphere Client interface making backup and recovery

management possible from the VMware administrator perspective while still retaining backup administrator control over schedule and

retention policies. Couple this with the ability for providing individual VM owners with the ability to complete self service restores

without making a restore request, you can accelerate your reduction in overall backup and recovery operational overhead.

For those taking the next step in the cloud journey, and have deployed robust ITaaS infrastructures, Avamar in the newly released

7.1 version includes integration with VMware’s vCloud Director®. With the EMC Avamar Plug-In for vCloud Director organizations

providing on-demand application environments to internal Lines-of-Business (LOBs) or those providing robust pay as you go

application hosting portal to external clients, now have access to all of the foundational technologies discussed here as well as

flexible management and orchestration features and APIs allowing data protection services to be included with any on-demand

application portal offering (i.e. BaaS).

Avamar’s proven deduplication process along with these latest enhancements result in vastly less data traversing the virtual and

physical network, and greatly reduces the amount of data being physically stored – translating into realized capital savings and

operational benefits for protecting and recovering virtualized environments and their data.

AUDIENCE

The white paper assumes the reader has a general understanding of the basic concepts and technologies behind EMC Avamar and

VMware, and is intended for systems engineers, technical architects, implementation specialists, technical consultants, and

individuals interested in leveraging the integration of currently shipping products and emerging technologies.

5

INTRODUCTION This whitepaper provides an overview of the features implemented in Avamar for fast, efficient, and secure backups of VMware

virtual environments.

There are essentially two methods of employing Avamar for virtual machine backup:

Guest-level backup

Image-level backup via VMware’s vSphere Storage APIs -Data Protection (VADP)

While Guest-level backup may provide the necessary solutions for some virtual cloud environments, the Image-level backup feature

leveraging the VADP feature set from VMware is where the maximum benefit can be achieved when leveraging Avamar. This is

especially true in larger virtual cloud environments where scale and backup windows are the ultimate concern. Avamar allows for

both guest-level and image-level backups to run simultaneously, allowing the administrator greater flexibility and efficiency of

deployment in their particular environment’s use case.

For the scope of this paper we will concentrate mostly on the foundational features implemented via VADP to enable Image-level VM

backups and restores.

The reader should familiarize himself with the following terms used throughout this document in the context of backup and recovery.

A virtual machine (VM) houses the production data in its virtual environment.

This solution was tested in an enterprise environment, so ESXi® Server was used as the VMware platform. Virtualization allows

customers to reduce IT costs while improving efficiency, availability, flexibility, and manageability of their environment.

Datastore is a VMware term for the storage that is added to house VMware File Systems (VMFS). The VMFS holds all of the

files needed to leverage virtual machines.

A VMDK file is a VMware image file that has the data that will be accessed by the application that runs in this VM. It is where

the operating system and application are installed along with the data for the respective application.

Changed Block Tracking (CBT) is a new VMkernel feature that keeps track of the storage blocks of virtual machines as they

change over time. The VMkernel keeps track of block changes on virtual machines, which enhances the backup process for

applications that have been developed to take advantage of VMware’s new vSphere Storage APIs.

VMware vSphere Storage APIs - Data Protection (VADP) enable backup software to perform centralized virtual machine

backups without the disruption and overhead of running backup tasks from inside each virtual machine.

EMC Avamar is a deduplication backup and recovery software and system solution. It embodies client deduplication that

eliminates data from backup streams before it crosses the network. Savings are realized on bandwidth, storage, and backup

windows.

EMC Avamar Data Store is a bundled EMC solution that combines Avamar software and EMC hardware in a preconfigured,

turnkey solution for simplified ordering, installation, and service. The Avamar Data Store scales from a single node to a multi-

node solution in order to meet the needs of remote offices to data centers.

AVAMAR: EFFICIENT DATA PROTECTION FOR VMWARE

GUEST-LEVEL BACKUP & RESTORE

With the guest backup option client agents are deployed in the same manner as an Avamar solution is implemented for physical

server backup. The Avamar client package is installed on each individual virtual machine and backups are sent to the Avamar server

via the VMs virtual network interface. Just as with physical servers, specific configuration may need to be done via application

specific plugins to support various application consistent backups supported via the agents installed on each client.

As shown in Figure 1 below, the same efficiencies apply as is observed with physical server backup using Avamar, however naturally

in this use case, the savings are in virtual resources allocated to the VM, not physical client hardware resources.

6

Some of the highlighted advantages to the Guest-level feature set:

Highest level of data deduplication

Support for backup of applications inside the virtual machines

Application consistent backups

Support for partial or file-level restores

Identical backup methods for physical and virtual machines

No requirement for advanced scripting or VMware software knowledge

Unchanged day-to-day procedures for backing up

IMAGE-LEVEL BACKUP & RESTORE

With the image based backup option EMC Avamar integrates with VMware’s vStorage API for Data Protection, a feature set within the

vSphere server to offload the backup-processing overhead from the client to a backup proxy server running as a VM. This proxy

server can communicate with the vCenter server to mount a snapshot of a particular VM’s vmdk to perform image level backup of

that virtual machine. With this method deduplication is provided on the file level as well as on the .vmdk level. Performing an image-

level backup of a VMDK, the ability to execute a bare metal restore replacing a VM instance or creating a separate instance is also

possible and supported. VMs protected via Image-level in Avamar can reside on NFS, iSCSI, or SAN storage.

Given the scalability and sheer explosion in the size of some VM environments the workload burden placed on one proxy server can

quickly build up. In this scenario, the recommendation is to provision multiple proxies to handle the combined workload and increase

the amount of parallelism occurring when performing VADP processes. With the provisioning of two proxy servers, we can now

perform the VADP process on two VMs simultaneously for backup by Avamar. Previously, VMs were assigned manually by the

administrator to specific proxy servers to assure proper load balancing. With the latest enhancements introduced within Avamar, this

load balancing occurs natively. Avamar will now utilize any idle proxy server to automatically load balance workload and maximize

parallelism through the backup process.

Figure 1: Client resource usage for traditional vs. Avamar backups

7

While offloading the backup-processing overhead from the client under backup to a proxy server provides a lot of benefit, the usage

of a proxy alone does not introduce efficiencies and intelligence needed to reduce the amount of data blocks that are scanned and

processed by that proxy server.

To introduce further efficiency into the VMware image backup process, Avamar fully utilizes the power of VADPs “Changed Block

Tracking” (CBT) feature. As depicted in Figure 2 below, leveraging CBT allows the VM Image Proxy to query the vSphere

environment and obtain an intelligent list of which blocks have changed on a specific VM since the last backup.

Therefore, Avamar will only scan and process blocks, which are included in the “changed-block” list provided by the CBT log

mechanism within vSphere. This greatly reduces the backup time of a given VM image and is the cornerstone of Avamar’s ability to

process large numbers of VMs within a particular given backup window.

Prior to Avamar 6.0, Restore operations were an “all-or-nothing” implementation as depicted in Figure 3. When restoring a particular

VM image from a backup within Avamar, the entire VMDK would need to be sent over the network and restored. Recovery of only the

individual changed blocks since the last backup within that VM image was not possible. With the introduction of Avamar 6.0 and later

versions, CBT is now leveraged by Avamar software to introduce more efficient and lightning-fast restores.

Some of the highlighted advantages to this VMware Image backup feature set:

Provides full image backups of running virtual machines

Utilizes efficient transport (SCSI hotadd), which avoids copying the entire vmdk image over the network

Provides file-level restores from image-level backups for Windows

Deduplicates within and across all .vmdk files protected by Avamar

Uses changed block tracking for faster backups & restores.

Minimizes network traffic by deduplicating and compressing data

Eliminates the need to manage backup agents in each virtual machine for most scenarios

Leverages proxy server load balancing to achieve parallelism for superior backup throughput

Figure 2: Avamar VM image backup utilizing Changed-Block Tracking (CBT)

Figure 3: Avamar VM image restore classic scenario (non-CBT)

8

VM RECOVERY FLEXIBILITY: AVAMAR VM INSTANT ACCESS AND CHANGED-BLOCK TRACKING

FOR RESTORE

As mentioned briefly in the previous section of this document, Avamar provides for the fastest recovery possible for virtual machines

in the industry. We will now take a look at how Avamar makes this possible in more detail.

When a VM Recovery is executed on a particular vmdk, as depicted in Figure 4, Avamar will query the vCenter via VADP to determine

which blocks have changed since the recovery point chosen, and then only recover or replace those blocks within the VM image file.

This reduces data transfer traffic within the vSphere environment traversing the wire during a restore operation, and more

importantly reducing the restore time objective (RTO) of a Virtual Machine Image.

To further add to this intelligence, the Avamar software will automatically evaluate the workload between both restore methods (Full

Image Restore, or a recovery leveraging CBT) and perform which method will result in the fastest restore times. This is particularly

useful in the scenario where the change rate since the last backup on a VM being restored is very high and the overhead of a CBT

analysis operation would be more costly than a direct full image recover. Avamar will intelligently decide which deployment method

will result in the fastest VM Image recovery times for your particular scenario or enviornment.

There are some use cases however, where the typical SLA that a CBT based recovery provides is not sufficient enough. Certain VMs

running mission critical applications need to be brought online immediately with no time to spare waiting for a full Avamar Recovery

leveraging VMware CBT.

Avamar’s integration with vCenter database structure provides the backup administrator with the flexibility for different VM recovery

options. A VM backup can be restored to the same VM instance from which the backup was taken or, we can perform a re-directed

restore and create an entirely new VM instance within vCenter. All of the steps required to create the VM entry within vCenter have

been done for you through this integration with Avamar. Once the recovery is complete, the only remaining step is to power on the

VM that was created from the recovery process.

Figure 4: Avamar VM image CBT for restore scenario

9

As shown in Figure 5, Avamar even gives you the flexibility to deploy a re-directed restore of a VM to a different VM data store

entirely.

Recovery flexibility is further extended through our integration with EMC Data Domain systems to allow the administrator to boot and

run a VM in production directly from the Backup image. This allows for critical time access for the most important of mission critical

applications. Let’s now take a closer look at how this is implemented.

We will use the illustration in Figure 6 as a reference for walking through a VM Instant Access recovery workflow. In this example, we

have 3 VMs, all 3 run mission critical Java applications. Unfortunately, “VM3” experienced some partial corruption and is currently

unavailable. At this point, the backup administrator can exercise two options in the combined Avamar/Data Domain solution,

leverage a full recovery back to the production VMFS datastore as described above, or orchestrate a VM Instant Access operation.

For this illustration, we will go ahead and bring the VM back online to the end users immediately and boot it directly from last-night’s

backup image. As you can see, our VM backup image is currently protected on the Data Domain system enabling us to leverage this

feature.

Figure 6: Running a VM from a backup with VM Instant Access

Figure 5: Ability to recover .vmdk files to different storage

10

From the Avamar Administrator UI as shown in Figure 6 below, an Instant Access operation is requested for last night’s backup.

The following steps will automatically be kicked into motion:

1. Avamar orchestrates the creation of an NFS datastore via an NFS Export of the VM Backup Image stored on Data Domain

2. Avamar interfaces with the vCenter environment automatically to create a new VM entry in the appropriate VM database, along

with all VM system state and configuration files associated with the VM being recovered.

3. VM is now ready for power-on and boot-up.

Since Avamar automatically integrated with vCenter to create the virtual machine entry as described earlier in this section, the

administrator simply just executes a power on of the newly added VM. End-users of the Java application running inside this VM can

now resume operations as normal.

While the VM is now running from the Backup, Avamar

does not overwrite the original backup image. Any new

I/O to the VM running from the backup goes into a

separately tracked VM snapshot file. In order to

complete the recovery of the VM to production storage,

the administrator can leverage the normal vSphere

interface to easily execute a Storage vMotion from the

temporary NFS data store export on Data Domain to the

production VMFS data store location.

The key to this use case is that rather than waiting for a

full restore to production storage that takes only

minutes, the administrator can literally bring back a

downed application from backup storage in a much

shorter timeframe. Also, with the tight integration with

VMware APIs this entire process is completed in seconds

and is very easily managed and orchestrated, just takes

a couple of clicks.

VM DATA PROTECTION MANAGEMENT:

AVAMAR VCENTER INTEGRATION

Avamar provides several unique integration capabilities

with VMware’s vCenter management utility. The first is

the integration through vCenter APIs that allows the Avamar Administrator UI to routinely query a single instance or group of

vCenter instances, and provide key data protection information that will simplify management of backup and recovery activities.

One of the important features possible with this management integration with vCenter as shown in Figure 7 below, is the ability to

quickly see from the backup software user-interface the current protection

status of each VM on a vCenter server, which is imported into the Avamar

environment. This saves the backup administrator the trouble of having to

correlate backup policies with newly created VMs that the application

administrators are deploying. Most likely, this would be done by cross

correlating the backup policy management UI with VMware’s vCenter client

UI. Now, the backup administrator, or even the application administrator

(with the correct Avamar management permissions assigned) can be given

access to the Avamar administrator UI and quickly assure that newly

deployed VMs are indeed protected under policy and exactly which policy

is protecting them.

Figure 6: Avamar Administrator UI Instant Access option

Figure 7: Determine a data protection approach by virtual machine

11

By implementing these features, the Avamar vCenter integration provides all administrators with the easiest and most efficient

visibility into the current state of VM data protection.

Key benefits and features of Avamar vCenter integration include:

Discovery of VMs and their associated groups in the Avamar UI

Ability to add individual VMs or groups and define “ image” and/or “guest” backup policies

Ability to define multiple VMware Image Proxies

Ability to initiate VM image or guest backup/restore operations

Ability to monitor backup/restore operations in the Activity Monitor

Ability to view VM protection status (guest, image, or none)

Simple views of whether VMs have been backed up or not; and the simple ability to remediate by quickly adding unprotected

VMs to a backup policy group.

Automatically adding a backup policy to virtual machines as they are added

VM DATA PROTECTION MANAGEMENT: EMC AVAMAR PLUG-IN FOR VSPHERE WEB CLIENT

The second major integration Avamar provides within VMware’s vCenter ecosystem is the ability to install the Avamar Plug-In for

vSphere Web Client. This is a plug-in that enables a VM Administrator with access to the vSphere Web UI, to take direct control of,

and monitor Avamar backup and recovery for VMs within that specific vCenter environment.

As shown here in Figure 9, the

plug-in brings Avamar

orchestration natively within the

vSphere UI as an added sub-

menu option. From this main

interface, the administrator can

go ahead and create backup

policies, and orchestrate VM

restores quickly. The biggest

value add this Avamar plug-in

for vSphere brings for IT staff is

significant savings and

efficiencies with no requirement

to train VM administrators in

new backup and recovery tools

and interfaces. Avamar

functionality is incorporated

directly in the user interface

style VM administrators are

already familiar and comfortable

with.

More critical to an IT staff is the

ability to empower application administrators to perform data and system recovery with minimal intervention and dependency on the

backup team. In a VMware scenario, the ability to empower the VM Administrator to perform his or her own VM image recoveries is a

foundation design point in bringing this management flexibility to Avamar customers. As shown below in Figure 9, it is easy to

leverage the plug-in to do just that. This plug-in provides all of Avamar’s efficient recovery features at the VM Administrator’s

fingertips.

Figure 8: EMC Avamar Plug-In for vSphere Web Client home screen

12

While being able to orchestrate backups and recoveries efficiently is critically important to IT organizations, it is also equally

important that all members of the IT team have a unified view of the operational status. Within the context of the same plug-in

Avamar provides in the vSphere interface for executing VM backup and recovery operations, the same administrator can take the

initiative independent of the backup administrator to gain insight into reporting metrics of the Avamar solution.

As you can see in Figure 10 below, from the “Reports” tab the VM administrator can also quickly look into system space utilization,

along with the status of any scheduled or on-demand VM backup and restore jobs. All of this information is displayed within the

management constructs and terminology of the vSphere workflow, so as all information is clear and concise in a language and style

the VM Administrator understands right away. With this dashboard style display, an Administrator can quickly identify if any regular

backups have failed or, if there are any VMs that have not been backed up entirely.

Figure 9: EMC Avamar plug-in for vSphere Web Client restore tab

Figure 10: EMC Avamar Plug-In for vSphere Web Client reports tab

13

As you can see using either the Avamar Administrator UI or the vSphere Web UI, both administrators are able to view consistent

data metrics and status across the entire VM backup and recovery policy set. Consistency across these various management

interfaces which are tailored for different administration styles bring the added benefit of clear and concise information sharing

between IT roles that otherwise would be time consuming to and involve human interaction to maintain. Clearly in today’s globally

distributed IT environments providing tools to enhance or eliminate the need for verbal or written communication for coordination

clearly provides cost savings, reduced risk, and operational efficiencies.

CLOUD-READY DATA PROTECTION: AVAMAR PLUG-IN FOR VCLOUD DIRECTOR

Building upon the foundational features we have outlined in this paper, new to Avamar’s 7.1 release is support for backup and

recovery at the cloud object layer. Rather than managing backup and recovery policies on individual VMs or VM folders, you can

apply Avamar protection workflows to vCloud Director vApps, and VDCs (Virtual Data Centers). Here are some of the key capabilities

that have been added to provide for robust data protection of cloud environments leveraging VMware vCloud Director:

Awareness of vApp constructs, including VM to vApp and vApp to VDC relationship understanding.

Creation of backup policies at the vCloud Director object level, including adding entire vApps or VDCs to a particular backup

policy group with defined schedule and data retention specifications.

Plug-In interface (as illustrated in Figure 12 below) enables the ability to orchestrate backups across multiple Avamar systems,

therefore allowing the ability to direct the data to the appropriate system based on policy.

Capability for Service Providers and Enterprises to incorporate EMC Avamar backup and recovery services into their already existing

or newly deployed ITaaS portals when leveraging our newly provided REST APIs. This provides for a robust cloud offering that

includes data protection.

It is clear that with most IT organizations today running their operations on an entirely virtualized infrastructure foundation, the

natural progression is to build upon this foundation and bring agile on-demand self-service application portals online. With these new

portals the power and agility that is delivered to clients or other internal LOBs will truly accelerate the speed of business and data

computation essential to that business. These new virtual application workflows are managed at an abstraction layer higher than the

Figure 11: EMC Avamar Plug-In for vCloud Director UI

14

virtual machine level. This fact alone calls for a redefined backup and recovery workflow, which speaks in the native object language

of these agile application cloud deployment mechanisms. The key to remember is that all of the foundational features also discussed

in this paper are included in these new workflow constructs such as our industry leading VM Image Level Backup and Recovery with

CBT and Instant Access can still apply. With an efficient management and API layer implemented that is Cloud-Ready, together with

our industry-leading backup and recovery for virtualization technologies, Avamar continues to lead the way in protecting your cloud

environment.

For further information on leveraging Avamar in a vCloud Director implementation, please refer to the following white paper on

emc.com or support.emc.com: EMC Avamar For vCloud Director Environments

CONCLUSION With the ever-increasing demands placed on virtual cloud infrastructures via new IT-as-a-service deployments, the need to protect

the environments with some sort of data protection scheme still remains. Employing traditional backup methods to the virtual

environment may be sufficient at the onset, however as the environment quickly grows in size, backup windows begin to be largely

unmanageable.

Avamar has implemented the VMware features described in this paper and is already providing customers today with a more efficient

VMware backup and recovery solution.

Avamar 7 extends the improvement in backup process efficiency for VM backups, and continues to build upon its industry leading VM

backup feature set with tighter integration and backup process management options for both the Backup and VM administrator. With

features detailed in this whitepaper, EMC Avamar is continues to lead the industry in innovation making backup and restores for

virtual environments more efficient and more importantly faster, allowing for worry-free and cost-effective expansion.