aws re:invent 2016: reinventing disaster recovery leveraging aws cloud infrastructure, with special...
TRANSCRIPT
![Page 1: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/1.jpg)
© 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
John Fox, Principal Technical Product Manager
Commvault Cloud Business Unit
Shaown Nandi, VP, Head of Infrastructure and Cloud, Dow Jones News Corp.
STG305
Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure
![Page 2: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/2.jpg)
Agenda
•What are you looking for in the cloud?
•Key Considerations for DR to the Cloud
• Initial Protection — Data to the Cloud
•Seeding Cloud Instances
•Disaster! – DR Scenario with AWS
•DR Case Study using Commvault and AWS
•Protection in the Cloud – AWS Focused
•Key Takeaways
•Q & A2
![Page 3: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/3.jpg)
What are you looking for?
![Page 4: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/4.jpg)
Key Considerations for DR to the Cloud
![Page 5: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/5.jpg)
Key Considerations for DR to the Cloud
5
• Multiple Sources that are Interrelated. Physical, Virtual, different OSs and Applications
• Production is Dynamic – The DR Plan for today doesn't work Tomorrow
• Is this a full or Partial DR
• Prioritization – Core, Tier 1, Tier 2 and then Tier 3
• Should Work Versus Knowing it will Work
• Test, Test, Test
• Spot test key Applications – Virtual Application Testing
![Page 6: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/6.jpg)
Getting Data to the Cloud: Step 1
![Page 7: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/7.jpg)
Getting Data to the Cloud
Storage / offsite tape replacement
• AWS and 40+ other storage targets natively supported
• Combined local backup, archive with cloud offsite
• Short-term retention for fast local recoveries
• Dedupe, encryption pre-cloud transfer
• Distributed indexes — data movers are commodity resources, restore from anywhere
• Incremental forever — efficient WAN utilization
• Multiple accounts / account segmentation supported
• Data portability7
![Page 8: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/8.jpg)
Not All Data Can Be Sent Over the Wire
• Large-scale datasets (petabytes)
• De-duplication baseline, or incompressible datasets(video, audio)
• Bandwidth $$
• Physical limitations and reliability of WAN links
8
Dataset Size
Link Size
1 GB 10 GB100 GB
1 TB 10 TB 100 TB 1 PB 10 PB
10 Mbit 14m 2.2 hrs 22.2 hrs 9.2 days 92.6 days – – –
100 Mbit 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days – –
1 Gbit 8s 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days –
10 Gbit 0.8s 8s 1m 20s 13m 20s 2.2 hrs 22.2 hrs 9.2 days 92.6 days
![Page 9: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/9.jpg)
Achieving Petabyte Scale Data Transport
CLOUD LIBRARY SEEDING SUPPORT
• Not all datasets can be sent over WAN –physical transportation required for large scale transfers
• Parallelize as needed
• AWS transfer achieved = Transfer more than 1 petabyte per week
• Simplified Logistics
• End-to-end Custody chain
• De-dupe via Commvault
![Page 10: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/10.jpg)
Amazon Storage Class Selection
Ensuring “fit-for-purpose”
• Amazon S3 - Hot
• S3-IA - Warm
• Amazon Glacier - Cold
• Cost models
• Ingress, egress
• Recall thresholds
• Special conditions
• Use cases
• Short- vs. long-term retention
• Backup vs. archive vs. compliance
• Data retrieval policies (Amazon Glacier)
10
![Page 11: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/11.jpg)
Backup to the Cloud
Table stakes – S3, S3 IA, Amazon Glacier
11
![Page 12: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/12.jpg)
DR Time: Step 2
![Page 13: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/13.jpg)
How would you recover?
”My DR plan is a pile of tapes and hoping for the best.”
![Page 14: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/14.jpg)
Workload Transformation
VMware Recovery to AWS
14
![Page 15: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/15.jpg)
Replicate and Recover to the Cloud
15
Data Protection Operation is run against VMs to be replicated
After Protection Operation is Complete Changed Blocks are Shipped to Standby VMs
2
OR DASH Copy is Performed to Secondary Site
Then Changed Blocks are Overlaid on Replicated VMs
4
Primary Datacenter
Secondary Datacenter
From VMware to AWS
31
43
21
4
![Page 16: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/16.jpg)
DR to the Cloud
VMware recovery to AWS — Recovery and Replication
Let’s See it in Action!
16
![Page 17: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/17.jpg)
“Now that I’m in the cloud, I don’t need backup … right?”
![Page 18: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/18.jpg)
Protection in Cloud: Step 3
![Page 19: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/19.jpg)
Reasons to Back Up in the Cloud
Expect components to fail, design for it – don’t wait for it to happen
Data loss can happen
• Instances retire
• Applications can become corrupted
19
![Page 20: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/20.jpg)
Commvault Cloud Protection
20
![Page 21: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/21.jpg)
What Should I Be Protecting?
Data classification matters — identifying valued data
Stateful machines
•Protect with VSA, BLB, FS, App agents
Stateless machines — “worker” nodes
•Where’s the valued data written out to? Protect that instead.
Data silos
•Object storage, Amazon EFS, Amazon EBS
21
![Page 22: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/22.jpg)
Virtual Server Agent for AWS
Expanding Capabilities
22
Agent-less backup of an Amazon EC2 instance’s EBS volumes
Use agent-in-guest for Application Consistent protection, and as part of a comprehensive data protection strategy
![Page 23: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/23.jpg)
What’s New? - EBS Snapshot Management (VSA for AWS context)
• Snapshots auto-rotated based on # of Snaps, or Days (Retention)
• Extraction to S3 optimized snapshot storage model to reduce storage size
• Copy Management features (Multiple Copies, cross-account/providers)
• Known snapshot names (SP_2_10242_2266_1468318241) and Tags to differentiate between Commvault-created and other snapshot
25
![Page 24: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/24.jpg)
Amazon RDS Support
Protecting Database-as-a-Service
26
Protect & Manage Databases -- whether they live in Amazon RDS, on EC2-instances, or both!
• Data protection for Databases running as a service within Amazon RDS
• Customer can protect Databases whether they reside in an EC2 instance, or in Amazon RDS
Supported mechanisms
• Application-consistent Snapshot backup Snap Restore, Snap Replicate
![Page 25: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/25.jpg)
Keeping It Simple
Use the right tool for the jobVSA for Cloud Image: Agentless instance protection
Agent-in-Guest: App-consistent protection (File, SQL, Oracle, many more!), granular app-specific restore features
DBaaS: Snapshot orchestration (coordinate with other systems)
Cloud Snapshot Management: (Oracle on Linux, Linux FS)
27
![Page 26: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/26.jpg)
Protection in Cloud
You Need to Protect once you Fail Over
Protection of AWS workloads
Let’s See it in Action!
28
![Page 27: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/27.jpg)
Dow Jones / News Corp NAShaown Nandi – VP Head of Infrastructure and Cloud
![Page 28: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/28.jpg)
![Page 29: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/29.jpg)
CLOUD TRANSFORMATION JOURNEY
• Save $100 Million annually
• Reduce from 50 data centers globally to 6
• 75% of computational power in the Cloud
AUDACIOUS GOALS SET IN FY14 BY NEWS CORP
Increase in AWS instances
400%
Dow Jones c loud compute
58%
![Page 30: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/30.jpg)
32
Dow Jones journey to the cloud with Commvault and AWS
1. Challenges
2. Preparation
3. Migration
4. Benefits
5. Cloud of the Future
Provision Cloud Resources Automated Recovery
Cloud Orchestration Workflow Automation
![Page 31: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/31.jpg)
Dow Jones Legacy DR Operation and Challenges
Traditional DR Strategy Included:
• Management of over 30,000 tapes in cycle
• Poor RoI
• Offsite Storage Costs
• Unacceptable recovery SLA’s for mission critical applications
• Limited resiliency
33
Traditional DR Operations
Lowest CostSlowest Recovery
User Data Sets
Files, eMail, Mobile, Laptop, etc
![Page 32: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/32.jpg)
Preparation & Planning
Leveraged Commvault and AWS Extensive Planning Resources
34
• Jointly Validated Reference Architectures for Commvault and AWS• Cloud Transformation Services with Commvault• Commvault Cloud Architecture Guide for AWS• AWS High Availability Best Practices
Commvault and AWS Jointly Validated Reference Architectures
![Page 33: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/33.jpg)
Dow Jones Cloud Based Protection Solution – Real World ResultsResilient, Automated In Cloud Protection with Commvault
35
1. Started sending on-premisebackups to S3 and S3-IA –exploring Glacier
2. Ultimately protect over 4,000 instances in the cloud with Commvault
3. Investigating protected VM’s with Commvault directly into EC2
4. Building out full business continuity plan to include automated disaster recovery testing and failover.
![Page 34: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/34.jpg)
Commvault Vision
![Page 35: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/35.jpg)
Commvault Cloud Vision
•Common Management Platform
•Platforms
•On-premises
•Cloud subscriptions
•Policies
•APIs
•Applications
37
![Page 36: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/36.jpg)
Commvault Ties together on-prem and cloud data strategies
Commvault Orchestrates the Enterprise
• Back up in the Cloud
• Back up to the Cloud
• Disaster Recovery to the Cloud
• Workload Portability
38
AWS and Commvault together combine to minimize networking, storage and
infrastructure costs, while providing the business a sound data protection and disaster
recovery strategy.
![Page 37: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/37.jpg)
Takeaways
![Page 38: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/38.jpg)
Key Takeaways
40
•Consider the Multiple Sources
•Keep the Plan Current
•Know what is going Where
•Bring up the key bits First!
•Test and know it Works
•DR doesn't need to be an Annual Event. Test Applications all year long.
![Page 39: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/39.jpg)
Questions?
Don’t forget to visit us in Booth 1015
![Page 40: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/40.jpg)
Thank you!
![Page 41: AWS re:Invent 2016: Reinventing Disaster Recovery Leveraging AWS Cloud Infrastructure, with special guest, Dow Jones (STG305)](https://reader031.vdocuments.net/reader031/viewer/2022030317/586f7b081a28ab10258b7611/html5/thumbnails/41.jpg)
Remember to complete your evaluations!