(bac309) automating backup and archiving with aws and commvault | aws re:invent 2014
DESCRIPTION
Are you looking to automate backup and archive of your business-critical data workloads? Attend this session to better understand key use cases, best practices, and considerations to help protect your data with AWS and CommVault. This session will feature lessons learned from CommVault customers that have: migrated onsite backup data into Amazon S3 to reduce hardware footprint and improve recoverability; implemented data tiering and archived data in Amazon Glacier for long term retention and compliance; performed snapshot-based protection and recovery for applications running in Amazon EC2; and provisioned and managed VMs in Amazon EC2. Sponsored by CommVault.TRANSCRIPT
November 12, 2014 | Las Vegas, NV
Paul McClure, CommVault
Physical Virtual Cloud
Applications
ServersNetworkStorage
Hosted
Recovery Tier
Snapshot / Replica
Protection / Archive Tier
Disk / Appliance Based
Long Term Retention Tier
Object / Cloud / Tape Based
Software/IP
IntelliSnap Data Replicator
Hardware/Array
Datacore SANsymphony-V
Dell Compellent & EqualLogic
EMC Isilon, VMAX /
Symmetrix, VPLEX, VNX /
Celerra & VNX / CLARiiON
Fujitsu ETERNUS DX
HDS
HP 3PAR & EVA
Huawei
IBM SVC & XIV
LSI
NetApp
Nimble Storage
Oracle ZFS
SAN/DAS
FC
FoE
iSCSI
SCSI
NAS
CIFS
NFS
Appliance
Data Domain
Exagrid
VTL
Object / Cloud
Amazon S3 (Native)
Amazon S3 (Based)
Amazon Glacier
AT&T Synaptic Storage
Caringo
DDN WOS
EMC Atmos (Native)
EMC Atmos (Based)
Hitachi Content Platform
Microsoft Azure Storage
OpenStack Object Storage
(Native)
OpenStack Object Storage
(Based)
Rackspace Cloud Files
Tape
For full support details refer to:
http://documentation.commvault.com
Recovery Tier
Snapshot / Replica
Protection / Archive Tier
Disk / Appliance Based
Long Term Retention Tier
Object / Cloud / Tape Based
19,447
3,547 1,716 1,588
2,838 1,319 992 562 466
1,384 2,396
-
5,000
10,000
15,000
20,000
25,000
Less Than 7Days
1 - 2 Weeks 2 - 3 Weeks 3 - 4 Weeks 1 - 2Months
2 - 3Months
3 - 6Months
6 - 9Months
9 - 12Months
1 - 2 Years Greaterthan 2 Years
Real World Example:
Global Investment Bank
Data Age
Res
tore
Jo
bs
Number of Restore Jobs in a 12
Month Period Based on Data Age
Deployment
models
TO
AWS
Data
management
use cases
DATA
IN
AWSD
AT
A
FROM
AWS
DATA
Cloud Hosted Hybrid
Architecture
AMI
Amazon EBS
Amazon S3 Bucket
Bucket Amazon Glacier
ProvisioningSnapshot
ManagementBackup & Archive Vaulting
PIT Snapshot(s)
Traditional Server
Client
Amazon EC2
Client
/
Disk Cache
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux / Unix)
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target
(REST / BLOB Storage)Client Natively Supported =
No Dedicated Appliances
Traditional Server
Client
Amazon EC2
Client
/
Disk Cache
Chunk File (Backup to Disk)
Sfile (Deduplication)
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux / Unix)
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target
(REST / BLOB Storage)
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Client
Traditional Server
Client
Amazon EC2
Client
/
Disk Cache
Chunk File (Backup to Disk)
Sfile (Deduplication)
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux / Unix)
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target
(REST / BLOB Storage)
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Client
More Streams =
Greater Performance!
Traditional Server
Client
Amazon EC2
Client
/
Disk Cache
Cloud
Connector
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Sfile (Deduplication)
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux / Unix)
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Client
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Traditional Server
Client
Amazon EC2
Client
/
Disk Cache
Cloud
Connector
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Sfile (Deduplication)
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux / Unix)
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
S3 Bucket
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Client
Data Pipe
Stream / Job)
Stream #1/2 / Job)
Stream #2/2 / Job)
Tunable Parameters for
Optimized Read / Write
Data Mover
Disk Cache
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target
(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Data Mover
Disk Cache
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
S3 Bucket
#2
Cloud Storage Target
(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Data Mover
Disk Cache
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
S3 Bucket
#2
Cloud Storage Target
(REST / BLOB Storage)
Amazon Glacier
Glacier Vault
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Create a new S3 Bucket with
Glacier Lifecycle Policies Enabled
Data Mover
Disk Cache
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Re-Hydration
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3
Bucket
#1
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Amazon Glacier
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Auxiliary Copy
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Re-Hydration
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3
Bucket
#1
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon Glacier
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Auxiliary Copy
Glacier VaultS3 Bucket
#2
Data Mover
Disk Cache
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Re-Hydration
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3
Bucket
#1
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon Glacier
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Auxiliary Copy
Glacier VaultS3 Bucket
#2
Policy Based
Copy Management
Data Mover
Disk Cache
Cloud
Connector
Cloud
Connector
Chunk File (Backup to Disk)
Re-Hydration
CommVault Archive File Container
Data Offset / Index
CommVault Media Agent
(Window / Linux )
- Deduplication Node
- Media / Library Manager
Memory Cache
Amazon S3
S3 Bucket
#1
Cloud Storage Target
(REST / BLOB Storage)
Sub-File / BLOB (16MB Default)
Upload Threads (2 Default)
Amazon Glacier
Amazon Lifecycle Policies move
objects from S3 bucket into
Glacier archives
Auxiliary Copy
Glacier VaultS3 Bucket
#2
Archive
Backup and archiving use cases
with AWS
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Internet
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
On-Demand
Self Service
CommVault infrastructure
running in the AWS Cloud
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Internet
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Internet
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
Amazon
WorkSpaces
Remote Lock &
Remote Wipe
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Internet
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
Amazon
WorkSpaces
Maintain
Productivity
InternetCommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
Mobile
Client
Synchronize
Files / Folders
InternetCommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
CommVault CommServe
(Master Server)Mobile
Client
Amazon S3 S3 Bucket
user
users
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Amazon S3 Bucket
CommVault CommServe
(Master Server)
CommVault infrastructure
running in AWS
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Corporate / Provider
Data Center
Physical ServerAmazon S3 Bucket
CommVault CommServe
(Master Server)
Onsite / Local
Copy (Cache)
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Corporate / Provider
Data Center
Physical Server
CommVault CommServe
(Master Server)
Offsite / DR
Copy
Amazon S3 Bucket
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Corporate / Provider
Data Center
Physical Server
CommVault CommServe
(Master Server)
Amazon S3 Bucket
Bucket Amazon Glacier
Policy Based
Copy Management
From retaining this…
Relevant Data
to this…
Reduce up to 70% of retention costs
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Corporate / Provider
Data Center
Physical Server
CommVault CommServe
(Master Server)
Amazon S3 Bucket
Policy Based
ProvisioningAMI
CommVault Media Agent(s)
(Window / Linux)
- Deduplication Node
- Media / Library Manager
Corporate / Provider
Data Center
Physical Server
CommVault CommServe
(Master Server)
Amazon S3 Bucket
Increase Productivity,
Reduce Costs & Downtime
Archiving
eDiscovery
Disaster recovery
Endpoint protection
Data protection
Physical Virtual Cloud
Applications
ServersNetworkStorage
Hosted
