automating backup & archiving with aws and commvault
TRANSCRIPT
Automating Backup & Archiving with AWS and CommVault
Chris GondekPrincipal Architect, CommVault Australia and New Zealand
Physical Virtual Public Cloud
ApplicationsServersNetworkStorage
Private Cloud
CommVault Simpana Software
Recovery Tier Snapshot / Replica
Protection / Archive Tier Disk / Appliance Based
Long Term Retention Tier Object / Cloud / Tape Based
Software/IP IntelliSnap Data Replicator Hardware/Array
Datacore SANsymphony-V Dell Compellent & EqualLogic EMC Isilon, VMAX / Symmetrix, VPLEX, VNX / Celerra & VNX / CLARiiON Fujitsu ETERNUS DX HDS HP 3PAR & EVA Huawei IBM SVC & XIV LSI NetApp Nimble Storage Oracle ZFS
SAN/DAS FC FoE iSCSI SCSI NAS CIFS NFS Appliance Data Domain Exagrid VTL
Object / Cloud Amazon S3 (Native) Amazon S3 (Based) Amazon Glacier AT&T Synaptic Storage Caringo DDN WOS EMC Atmos (Native) EMC Atmos (Based) Hitachi Content Platform Microsoft Azure Storage OpenStack Object Storage (Native) OpenStack Object Storage (Based) Rackspace Cloud Files Tape
For full support details refer to: http://documentation.commvault.com
Recovery Tier Snapshot / Replica
Protection / Archive Tier Disk / Appliance Based
Long Term Retention Tier Object / Cloud / Tape Based
Real World Example: Global Investment Bank
Data Age
Res
tore
Job
s
Recovery Tier Snapshot / Replica
Protection / Archive Tier Disk / Appliance Based
Long Term Retention Tier Object / Cloud / Tape Based
00
5,000
10,000
15,000
20,000
Less Than 7 Days 2 - 3 Weeks 1 - 2 Months 3 - 6 Months 9 - 12 Months Greater than 2 Years
2,396 1,384
466 562 992 1,319 2,838
1,588 1,716 3,547
19,447
Real World Example: Global Investment Bank
Data Age
Res
tore
Job
s
Number of Restore Jobs in a 12 Month Period Based on Data Age
Deployment Models
Public Cloud Private Cloud
Hybrid Cloud
Deployment Models
Deployment Models
TO AWS Cloud
Data Management Use Cases
DATA
Public Cloud Private Cloud
Hybrid Cloud
Deployment Models
Deployment Models
TO AWS Cloud
Data Management Use Cases
DATA
IN AWS Cloud
DATA
Public Cloud Private Cloud
Hybrid Cloud
Deployment Models
Deployment Models
TO AWS Cloud
Data Management Use Cases
DATA
IN AWS Cloud
DATA
FROM AWS Cloud
DATA
Public Cloud Private Cloud
Hybrid Cloud
Deployment Models
AMI
Amazon EBS
Provisioning
Lifecycle Management
AMI
Amazon EBS
Provisioning
Snapshot Managem
ent
PIT Snapshot(s)
Lifecycle Management
AMI
Amazon EBS
Amazon S3 Bucket
Provisioning
Snapshot Managem
entBackup & Archive
PIT Snapshot(s)
Lifecycle Management
AMI
Amazon EBS
Amazon S3 Bucket
Bucket Amazon Glacier
Provisioning
Snapshot Managem
entBackup & Archive Vaulting
PIT Snapshot(s)
Lifecycle Management
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)Client
Amazon S3 Support
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)Client
Amazon S3 Support
Natively Supported = No Dedicated Appliances
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)Client
Amazon S3 Support
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Client
Amazon S3 Support
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Client
Amazon S3 Support
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Client
Amazon S3 Support
More Streams = Greater Performance!
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)Client
Amazon S3 Support
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Cloud Connector
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Client
Amazon S3 Support
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Cloud Connector
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Client
Amazon S3 Support
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Traditional ServerClient
Amazon EC2Client
/
Disk Cache
Cloud Connector
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Sfile (Deduplication)
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux / Unix) - Deduplication Node - Media / Library Manager
Memory CacheAmazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Client
Amazon S3 Support
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Tunable Parameters for Optimised Read / Write
Data Mover
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
Cloud Storage Target(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move objects from S3 bucket into
Glacier archives
Amazon Glacier Support
Data Mover
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
S3 Bucket#2
Cloud Storage Target(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move objects from S3 bucket into
Glacier archives
Amazon Glacier Support
Data Mover
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
S3 Bucket#2
Cloud Storage Target(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move objects from S3 bucket into
Glacier archives
Amazon Glacier Support
Data Mover
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
S3 Bucket#2
Cloud Storage Target(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move objects from S3 bucket into
Glacier archives
Amazon Glacier Support
Create a new S3 Bucket with Glacier Lifecycle Policies Enabled
Data Mover
Disk Cache
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Policy Based Copy Management
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud Connector
Cloud Connector
Chunk File (Backup to Disk)Re-Hydration
CommVault Archive File ContainerData Offset / Index
CommVault Media Agent(Window / Linux ) - Deduplication Node - Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket #1
Cloud Storage Target(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon GlacierAmazon Lifecycle Policies move
objects from S3 bucket into Glacier archives
Auxiliary Copy
Amazon Glacier Support
Glacier VaultS3 Bucket
#2
Archive
Use Cases
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Endpoint Protection
Amazon S3 S3 Bucket
user
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Endpoint Protection
Amazon S3 S3 Bucket
user
On-Demand Self Service
CommVault infrastructure running in the AWS Cloud
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Endpoint Protection
Amazon S3 S3 Bucket
user
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Endpoint Protection
Amazon S3 S3 Bucket
user
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Recovery to Amazon WorkSpaces
Amazon S3 S3 Bucket
user
Amazon WorkSpaces
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Recovery to Amazon WorkSpaces
Amazon S3 S3 Bucket
user
Amazon WorkSpaces
Remote Lock & Remote Wipe
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Recovery to Amazon WorkSpaces
Amazon S3 S3 Bucket
user
Amazon WorkSpaces
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Internet
CommVault CommServe(Master Server)
Mobile Client
Recovery to Amazon WorkSpaces
Amazon S3 S3 Bucket
user
Amazon WorkSpaces
Maintain Productivity
InternetCommVault Media Agent(s)
(Window / Linux) - Deduplication Node - Media / Library Manager
CommVault CommServe(Master Server)
Mobile Client
File Sync
Amazon S3 S3 Bucket
user
InternetCommVault Media Agent(s)
(Window / Linux) - Deduplication Node - Media / Library Manager
CommVault CommServe(Master Server)
Mobile Client
File Sync
Amazon S3 S3 Bucket
user
Mobile Client
InternetCommVault Media Agent(s)
(Window / Linux) - Deduplication Node - Media / Library Manager
CommVault CommServe(Master Server)
Mobile Client
File Sync
Amazon S3 S3 Bucket
user
Mobile Client
Synchronise Files / Folders
InternetCommVault Media Agent(s)
(Window / Linux) - Deduplication Node - Media / Library Manager
CommVault CommServe(Master Server)
Mobile Client
File Share
Amazon S3 S3 Bucket
user
InternetCommVault Media Agent(s)
(Window / Linux) - Deduplication Node - Media / Library Manager
CommVault CommServe(Master Server)
Mobile Client
File Share
Amazon S3 S3 Bucket
user
users
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Amazon S3 Bucket
Backup in the AWS Cloud
CommVault CommServe(Master Server)
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Amazon S3 Bucket
Backup in the AWS Cloud
CommVault CommServe(Master Server)
CommVault infrastructure running in the AWS Cloud
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical ServerAmazon S3 Bucket
Backup to the AWS Cloud
CommVault CommServe(Master Server)
Direct Connect
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical ServerAmazon S3 Bucket
Backup to the AWS Cloud
CommVault CommServe(Master Server)
Onsite / Local Copy (Cache)
Direct Connect
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
Backup to the AWS Cloud
CommVault CommServe(Master Server)
Amazon S3 BucketDirect
Connect
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
Backup to the AWS Cloud
CommVault CommServe(Master Server)
Offsite / DR Copy
Amazon S3 BucketDirect
Connect
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
Vaulting to Glacier
CommVault CommServe(Master Server)
Amazon S3 Bucket
Bucket Amazon Glacier
Direct Connect
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
Vaulting to Glacier
CommVault CommServe(Master Server)
Amazon S3 Bucket
Bucket Amazon Glacier
Policy Based Copy Management
Direct Connect
From retaining this…
Reduce LTR Costs with Reference Copy
From retaining this…Relevant Data
to this…
Reduce LTR Costs with Reference Copy
Reduce up to 70% of retention costs
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
CommVault CommServe(Master Server)
Amazon S3 Bucket
Automated Provisioning and Recovery
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
CommVault CommServe(Master Server)
Amazon S3 Bucket
Automated Provisioning and Recovery
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
CommVault CommServe(Master Server)
Amazon S3 Bucket
Policy Based Provisioning
Automated Provisioning and Recovery
AMI
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
CommVault CommServe(Master Server)
Amazon S3 Bucket
Automated Provisioning and Recovery
CommVault Media Agent(s)(Window / Linux)
- Deduplication Node - Media / Library Manager
Corporate / Provider Data Center
Physical Server
CommVault CommServe(Master Server)
Amazon S3 Bucket
Increase Productivity, Reduce Costs & Downtime
Automated Provisioning and Recovery
Demo
“2 Clicks to the Cloud with Commvault and AWS”
Data Protection
Summary
End Point Protection
Data Protection
Summary
Disaster Recovery
End Point Protection
Data Protection
Summary
Archiving
Disaster Recovery
End Point Protection
Data Protection
Summary
Archiving
eDiscovery
Disaster Recovery
End Point Protection
Data Protection
Summary
Physical Virtual Public Cloud
ApplicationsServersNetworkStorage
Private Cloud
Summary
