shift your tape backups to aws
TRANSCRIPT
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Shift your tape backups to AWS to save time and money
Bhavin Patel
Product Manager
AWS
S T G 2 1 7
Paul Walsh
Systems Administrator
Ryanair
Rob Czarnecki
Product Manager
AWS
… you have come to the right session
Are you dealing with these backup challenges?
Physical tapes & infrastructure complexity
Expensive offsite tape storage
Shrinking capex budget
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Agenda
Tape Gateway overview and benefits
Why Ryanair switched to AWS for archives
Tape Gateway deep dive
Demo
Storage Gateway overview
Summary
Amazon S3 Glacier and Amazon S3 Glacier Deep Archive for archives
AWS CloudCustomer premise
Files
(NFS/SMB)
Volumes
(iSCSI)
Tapes
(iSCSI VTL)
AWS Storage Gateway
Integrated with AWS Identity and Access Management
(IAM), AWS Key Management Service (AWS KMS),
AWS CloudTrail, Amazon CloudWatch services
Amazon S3
Glacier
AWS Storage Gateway
Configuration: VMware ESXi, Microsoft Hyper-V,
Amazon Elastic Compute Cloud (Amazon EC2),
Hardware appliance
Amazon S3
Glacier Deep
Archive
Storage Gateway serviceStorage Gateway
HTTPS Amazon S3
Amazon Elastic
Block Store
(Amazon EBS)
AWS Backup
Learn more about Storage Gateway in STG305 session on Wednesday, 9:15-10:15 AM
Storage Gateway family
FILE
TAPE
VOLUME
Store and access objects in Amazon S3 from
SMB/NFS file-based applications with local caching
Drop-in replacement for physical tape infrastructure
backed by cloud storage with local caching
On-premises iSCSI block storage backed by cloud with
local caching, int. with AWS Backup & Amazon EBS
You can use all gateway types to backup on-premises workloads to cloud
Oracle/
Databases/VMs
Apps needing block storage target
File Gateway
Tape Gateway
Volume Gateway
SMB /NFS
iSCSI BLOCK
Amazon S3
objects
AWS Storage
Gateway virtual tapes
Amazon EBS
snapshots
YO
UR
O
N-
PR
EM
IS
ES
A
PP
S
IN
-C
LO
UD
A
CC
ES
S
TO
DA
TA
iSCSI VTL
SQL Server DBs
APP APP
Tape Gateway overview and use casesPresent cloud-backed virtual tapes to on-premises backup applications
Customer premise
iSCSI VTL
Tape Gateway
HTTPS
Backup
applicationStorage Gateway service
(S3 Glacier Deep Archive)
OR (S3 Glacier)
Tape library(Amazon S3)
Tape shelf
• Backup to cloudUse cases
AWS Cloud
• Archive to cloud
Tape Gateway benefits
Tape Gateway benefits: 1 of 4
Drop-in replacement to your physical tape infrastructure
Lower TCO than physical tape backups
Pay-as-you-go pricing
Tape Gateway benefits: 2 of 4
No changes to backup process
Only manage tapes and gateway
Gateway managed and updated by AWS
Multiple deployment options
Tape Gateway benefits: 3 of 4
Read-only tapes upon archive
Data encrypted on wire
Server-side or KMS encryption
* SOC (1, 2, 3) **ISO (9001, 27001, 27017, 27018)
HIPAA eligible
PCI, SOC*, & ISO** compliant
Tape Gateway benefits: 4 of 4
Local caching for faster backup and recovery times
Compresses data prior to upload
Easy and online retrieval
A current Tape Gateway customer testimonial
“Using Tape Gateway, we are now in a position to power off two tape libraries, two physical servers, stop filing a manifest with our offsite
vendor every week, stop loading and reloading the physical tape libraries and stop paying support on all of it.”
Tape gateway supported backup software
What’s new with Tape Gateway since re:Invent 2018?
Mar.2019
May2019
Sep.2019
Jul.2019
Aug.2019
Jun.2019
Nov.2019
NEW
Jan.2019
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
About Ryanair
What I do at Ryanair?
Part of IT team under CTO
Challenges in
my roleResponsibility
500 people in IT
71 people in support
All ESX/Hyper-V hosts
All storage systems (Dell EMC Compellent)
Backups/archiving of all Ryanair back office systems
Growing data volumes
Need to manage all this within a reasonable budget
Our backup strategy prior to using AWS
• We run our main apps in Head Office Datacenter
• Need to backup critical flight systems and back office systems
• We backup systems on premises for quick recovery, replicate backups to DR site, and then use physical tapes for archiving offsite
Location Strategy
HQ Backup to disk 28 copies on disk
DR site Backup copy to disk Backup copies replicated daily from HQ
Offsite
archiveBackup to physical tapes Monthly tape archives sent offsite
Challenges with our backup environment using physical tapes
• CRC errors / Tape damage
• Monthly purchase of tapes
• Unloading and loading of tape libraries
• Cleaning drives
• Library maintenance
• Shipping tapes offsite
• Retrieving tapes from offsite
Ryanair Head Office Datacenter
Veeam and Micro
Focus Backup
Applications
Local
2
1
Ryanair DR site
Local
Veeam and Micro
Focus Backup Apps
3
Our physical tape replacement alternatives
Keep using LTO tapes (LTO8)
Tape Library as a service (LTO6 and lower)
Why it was not suitable?
Same challenges as what we have today
Deduplication storage appliancesDidn’t fit budget, slow, capex, needs a hosting location
Backup to disk for long-term storageCapacity planning, forecasting, capex, needs a hosting location
Option
Then we turned to AWS
Moving to Tape Gateway
• Immediate support with Veeam
• Quick and easy deployment guide
• Immediate support with Micro Focus Data Protector
• Quick and easy deployment guide
• No more monthly purchase of tapes
• No more tape library maintenance
• No more shipping or requesting tapes to/from offsite archive
• No more failed restores due to CRC errors / damaged tapes
• Pay as you go model instead of upfront purchase
• Cost and time savings
Our backup strategy now
• We run our main apps in Head Office Datacenter
• Need to backup critical flight systems and back office systems
• We backup systems on premises for quick recovery, replicate backups to DR site, and then use Tape Gateway to archive to the cloud
Location Strategy
HQ Backup to disk 28 copies on disk
DR site Backup copy to disk Backup copies replicated daily from HQ
AWS Backup to AWS Monthly archives to virtual tapes in AWS
Our backup environment after migration to AWS
Ryanair Head Office Datacenter
Tape Gateway
Veeam and Micro
Focus Backup
Applications
AWS Cloud
Ryanair DR site
Local
Tape Gateway
Local Tape Gateway Tape Gateway
Storage
Gateway serviceAmazon S3 S3 Glacier or
Glacier Deep Archive
3
2
HTTPS
1
HTTPS
Tape Gateway meets our backup time requirements
Ph
ysi
cal
tap
e
AW
S v
irtu
al
tap
e
28 mins
run time
25 mins
run time
Backup times observed in Veeam
How Tape Gateway solved Ryanair challenges
Challenge Physical tapes AWS
CRC errors / Tape damage On-going risk No risk once job is completed
Monthly purchase of tapesRaising purchase orders, waiting for
delivery, etc.
Virtual tapes created almost instantly
in the AWS Storage Gateway console
Unloading and loading of
tape libraries
Time consuming, engineers could be
doing better things with timeAutomatic at end of job
Cleaning drivesCleaning required monthly; cleaning
tapes requiredNot required
Library maintenanceManual firmware updates, replacing
faulty parts, etc.
Automatic software update during
your selected maintenance window
Shipping tapes offsite or
onsite
Required to keep backups safe, or to
restore, minimum 24 hour wait
Tapes are already offsite once the job
completes, ready to restore quickly
Cost
Ongoing monthly purchases, ongoing
yearly support, large capex on hardware
upgrades
Monthly bill, pay per use
65% cheaper per month
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Archive poolVirtual tape library
Customer premise
• Virtual tapes, tape drives, and media changer are presented via iSCSI to backup app
• Backup application writes data to virtual tapes which are stored in Amazon S3
• When you eject tapes from backup app, gateway marks tapes as read-only and archives them to S3
Glacier or S3 Glacier Deep Archive
How does backup work with Tape Gateway?
Media changer
Tape gateway
Tape drives
Application
servers
Backup
server
Storage
Gateway serviceAmazon S3
Amazon
S3 Glacier or
S3 Glacier Deep
Archive
Virtual tapes
AWS Cloud
HTTPS
How does restore work with Tape Gateway?
• Retrieve archived tapes from S3 Glacier or S3 Glacier Deep Archive to S3 using AWS console or API
• When tapes are in S3 (Status=Retrieved), you can present them to backup app
• Use existing gateway or create a new gateway on premises or in cloud for restore
Customer premise
Backup
server
Application
servers
Amazon S3 Tape
GatewayStorage
Gateway service
S3 Glacier or
S3 Glacier
Deep Archive
Tape
Gateway
AWS Cloud
HTTPS
Tape Gateway configuration options and thresholds
Cache size 150 GiB – 16 TiB
Upload buffer size 150 GiB – 2 TiB
Virtual tape size 100 GiB – 5 TiB
Ga
tew
ay
Ta
pe
sp
eci
fic
Number of slots in virtual tape library 1,500
Number of tape drives 10
Total size of all tapes in virtual tape library 1 PiB
Maximum number of virtual tapes in archive No limit
Total size of all tapes in archive No limit
Multiple options to deploy Tape Gateway
Virtual machines Amazon EC2 Hardware appliance
Wh
ere
Wh
en
Limited time incentive for Storage Gateway Hardware Appliance MONDAYCYBER
Customers asked to: Feature we delivered:
CloudWatch logging and metrics in gateway console
New maintenance options for granular control over software updates
High availability for VMware-based gateways
Latest features for Tape GatewayNEW
3x cloud restore performance
Increased gateway visibility and insightsFor gateways running on virtual or hardware appliances
• Real-time visibility into
throughput, cache
utilization, and gateway
access patterns through
CloudWatch metrics
• Administrators can tune
gateway resources to
optimize performance
based on application
needs
Visualize CloudWatch metrics within Monitoring tab of Storage Gateway console
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Simple Storage Service (S3)
Decreasing prices and more storage options
2006 2019
Decreasing storage prices
S3 Standard
(2006)
S3 Glacier
(2012)
S3 Standard-IA
(2015)
S3 One Zone-IA
(H1-2018)
S3 Intelligent-
Tiering
(H2-2018)
Accelerating innovation
2006 2019
S3 Glacier Deep
Archive
(2019)
> 80% savings
Scale globally with resilience in every region
Region & number of
Availability Zones
Announced Regions
Cape Town,
Jakarta, and Milan
Typical AWS AZAWS Region
AZs separated by miles
Redundant, dedicated
network interconnect
Transit
Transit
AZ
AZ
AZ
AZ
Data center Data center
Data center
Each AZ has
independent power
infrastructure
S3 Glacier Deep Archive: lowest cost cloud storage
No physical
tapes to
manage
Designed for
99.999999999%
durability
Recover data in
hours vs.
days/weeks
$0.00099 per GB-
monthLess than 1/4 the cost of
S3 Glacier
S3 Glacier and S3 Glacier Deep Archive
Priced from $0.00099/GB-Month
Standard restore tier typically within 12 hours
Priced from $0.004/GB-Month
Standard restore tier typically within 3-5 hours
Tape Gateway costs less than physical tapesCustomer example: Every month,100 TB written and 1 tape retrieval; 1-year retention
$56.8K/year
Admin time
Tape library maintenance
Tape media costs
Offsite archive service
$-
$10K
$20K
$30K
$40K
$50K
$60K
On-premises Tape
Robotic tape library
*based on U.S. N. Virginia region
$16.7K*/year
Tape Gateway
Data written via Gateway
Data transfer out and tape retrieval
Virtual tapes archived in
S3 Glacier Deep Archive
Admin time
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Storage Gateway: Creating a Tape Gateway
Tape Gateway: Creating virtual tapes
Tape Gateway: Viewing the backed up tape
Windows Client: Backing up to the Tape Gateway
Windows Client: Backing up to the Tape Gateway
Windows Client: Backing up to the Tape Gateway
Tape Gateway: Viewing the backed up tape
Windows Client: Ejecting the tape from the backup software
Tape Gateway: Viewing the archived tape
Tape Gateway: Assigning the tape to Deep Archive Pool
Tape Gateway: Assigning the tape to Deep Archive Pool
Tape Gateway: Retrieving the tape
Tape Gateway: Retrieving the tape
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Tape Gateway key takeaways
minimal learning curve
better durability, security,
and restore
Reduceinfrastructure
costs
How to get started?
Product information
Product documentation
Pricing
Cost calculator
Learn Execute Get help
aws.amazon.com/storagegateway/vtl
aws.amazon.com/console forums.aws.amazon.com
Create AWS account
PoC a Tape Gateway
Community help
Your AWS account team
AWS Pro Services
AWS Support
aws.training
Free training classes
Learn more about hybrid cloud storage in these sessions
• STG231 Lift and shift your tape-based backup workflows to AWS
• STG226 Hands-on with hybrid block storage using a Volume Gateway
• STG30 Build hybrid cloud storage architectures with AWS Storage Gateway
• STG213— —Storage for hybrid cloud and edge computing: Bring AWS to you
• STG313 Hybrid architectures for database backups & file migrations
• STG336— Using hybrid cloud storage to close a data center and migrate
Thank you!
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Bhavin Patel
© 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.