jp morgan remote to core concepts
DESCRIPTION
JP Morgan Remote to Core Concepts for North America Presented by John Napier, PMP, CSMTRANSCRIPT
January 2007
Remote to Core – Part 1 - Design Concepts
S T
R I
C T
L Y
P R
I V A
T E
A N
D
C O
N F
I D E
N T
I A L
This deck is for Remote to Core presentation purposes and process understanding. It is not intended to replace detailed installation, deployment guidelines or
standards documentation maintained by those groups primarily responsible for the associated tasks described.
Author: J.Napier
2
Contents
Purpose
Solution
Component Architecture
Data Flow Diagram
Snapvault Functionality
Volume/Qtree Convention
Snapvault Requirements
High Level Data RPO
3
Purpose
Eliminate Tape Creation in non-CORE Datacenters
Provide Standard Storage Platform for Strategic MS/AD File/Print Services
Drive Risk and Cost out of the Branch Backup environment by eliminating tape handling/media expenses, and reallocating\decommissioning branch level backup infrastructure
Develop efficiencies and increase speed related to data recovery
4
Solution: Network Appliance Snapvault Management
Network Appliance Snapvault utilizes snapshot and replication functionality to perform backups on NFS-mounted and/or CIFS-mapped file systems residing on NetApp filers. A NetApp Snapshot copy is created on the primary storage system and is then replicated to a NetApp secondary storage system.
Based on the quick creation of a snapshot and a transfer of only changed blocks via NetApp Qtree Snapmirror replication, fast backups and restores can be easily accomplished from local to remote locations.
Backups in the form of Snapshot copies on the primary(Remote) and secondary(Core) NetApp storage systems are managed (creation, retention, deletion) with Snapvault scheduling.
All backup data stored is in file format, including all incremental backups, and each incremental backup can be viewed as a full backup image by any administrator or end-user with associated access control and security rights. Data to be restored can be quickly located with a full view into what the environment looked like at the time the backup was created, and as far back as data retention will permit.
The “file system” view of backups can be accessed by end users as well, and empowers the user to perform self-restores, if the administrator permits.
Standard Solution for Remote to Core provides for 30days local backup protection and 60 days remote backup retention – all backups on disk and online.
5
Remote to Core Component Architecture
[reproduced from M.Weitman’s - Component Architecture.ppt]
Core NAS Device
WAFS or WAN-Accel Engine [Future]
WAFS or WAN-Accel Engine [Future]
Single Instance/ Data Reduction Engine or
Data Movement [Future]
72F
13 12 11 10 09 08 07 06 05 04 03 02 01 00
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
FAS270C
P ower
Status B
Shelf ID
Activity B
Status A
Activity A
V6070
NetApp
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
V6070
NetApp
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14
M K2
FC
NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
WAN
Remote Site
Core Site
d
Existing File Servers, Clients Requiring Backup
a
NAS Deviceb
Ethernet Etherchannel,
Multi-homed
c
e
f
Single Instance/ Data Reduction Engine or
Data Movement [Future]
g
i
Other Storage Infrastructure
Robocopy ServerNeeded for sites that do not use Filer as primary storage- Copies data between F/P Servers and Filer
6
Remote to Core Data Flow Diagram
72F
13 12 11 10 09 08 07 06 05 04 03 02 01 00
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
FAS270C
Power
Status B
Shelf ID
Activity B
Status A
Activity A
V6070
NetApp
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
V6070
NetApp
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
NetworkAppliance
Power
System
Shelf ID
Loop B
Fault
Loop A
72F
D S 14M K2
FCNetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
WAN
Remote Site
Core Site
Initial Data Seeding From Existing Servers to Filer Sharesa
Volumes/Qtrees/Shares Created to match Existing infrastructureStandard R2C Snapvault ScheduleStandard R2C Snapshot Schedule
d b
Snapvault Secondary Volumes created for
R2C Standard. c
Other Storage Infrastructure
F&P File Transports, incl
Full Volume Copy and Nightly
Incremental (if required)
Snapvault schedule triggers Snapshot and Qtree replication transfers Image to Core Volume
e
e
e
e
e
Subsequent Snapvault only transfer changed blocks. Standard Snapshot schedule retains 30 Daily Snapshots in rotation
f
Snapvault Standard schedule retention
(60days) of daily images in rotation
@Core
g
f
f
f
Daily Incrementals follow same path (f). Curve drawn for diagram simplicity
Transfer is IP-based Storage Replication
(Qtree SnapMirror)
7
Snapvault Functionality Process
Remote Site
Core Site
Snapvault Secondary Volumes
Existing F&P Servers
User Volumes Replicated to Filer Volumes
a
WAN
V6070
NetApp
c
c
d
d
Bocada
NetApp Snapvault Primary (remote)c. Perform Local Snapshot d. Source to Target Relationship [Qtree to Qtree]d. Source for Qtree SnapMirror Replication
NetApp Snapvault Secondary (core)c. Snapshot (Retention)d. Target for Snapvault Replication
Operations Manager Functions:Centralized Scheduling, Monitoring, Alerting and Reporting mechanism for Filer configuration and Snapvault processing.
Snapvault Functions:R2C Standard Schedules, Frequencies and Retentions for Snapshots and Snapvault within OnTapNetApp Snapvault executes on local snapshots and filer-based replication per schedule trigger from Op Mgr.Perform Primary and Secondary volume snapshots per schedulerSubsequently, perform Qtree-based Snapmirror replication between Source data and Target DSU
72F
13 12 11 10 09 08 07 06 05 04 03 02 01 00
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
FAS270C
Power
Status B
Shelf ID
Activity B
Status A
Activity A
Tape
Bunker
g
g
Netbackup Functions:If the requirement to have LT Retention (ie “Monthly Full backups @2yr retention”) exists, then NBU Master/Media
at Core has Policy and schedule for that requirement. Source volume for NBU will be on Core filer (Snapvault Secondary) – a mirror of the remote site volume, and target for Media server can be Tape or Bunker via NDMP –
Target is TBD (Other disk, Bunker, C2B, Tape)
Operations Manager Cluster (core)b. Schedulere. Monitoring/Alertingf. Reporting(+feed)
e
Monitoring Infrastructure
f Bocada,Other
TBD
Snapvault Secondary Snapshots(Retention)
8
Remote to Core Volume and Qtree Definition
\vol\vol1\filer1_qtree1
\vol\vol1\filer1_qtree2
\vol\vol9\filer1_qtree98
\vol\vol9\filer1_qtree99
... ...
vol1filer1_vault1
\vol\vol1\filer1_qtree1
\vol\vol1\filer1_qtree2
\vol\vol9\filer1_qtree98
\vol\vol9\filer1_qtree99
\vol\vol1\filer2_qtree1
\vol\vol1\filer2_qtree2
\vol\vol9\filer2_qtree98
\vol\vol9\filer2_qtree99
... ...
vol1
filer2_vault1
\vol\vol1\filer2_qtree1
\vol\vol1\filer2_qtree2
\vol\vol9\filer2_qtree98
\vol\vol9\filer2_qtree99
Rem
ote
Site
AR
emot
e S
ite B
Core Site
72F
13 12 11 10 09 08 07 06 05 04 03 02 01 00
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
FAS270
P ower
Status B
Shelf ID
Activity B
Status A
Activity A
72F
13 12 11 10 09 08 07 06 05 04 03 02 01 00
72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F 72F
FAS270
Power
Status B
Shelf ID
Activity B
Status A
Activity A
V6070
NetApp
WAN
......
vol9
vol9
All qtrees sharing Standard Retention (60 day) will be located on a common vault volume per Remote site
9
SnapVault Management Requirements.
NetApp Hardware Any platform combination that SnapVault supports
Data ONTAP Primary: Data ONTAP 6.5.2 or later (7.1.1 initial deployment)Secondary: Data ONTAP 7.1 or later (7.2 initial deployment)
NetApp Software Snapshot, SnapRestore[1]
Primary: sv_ontap_priSecondary: sv_ontap_sec
Protocols NFS, CIFS
Monitoring Operations Manager (Backup module)
Scheduling Operations Manager (Data Protection Manager) – StrategicFiler-based scheduling - Tactical
10
High-Level Process – Data Recovery Options:
Data Recovery Point and Retention
Remote Site (Local) Core Site Daily Backup Images will reside (Local) for 30 days Daily Backup Images will exist 60 Days Snapshots (Local) will provide intraday file recovery - Backup Images expire per schedule - space permitting - Retention > 60 days – Bunker or Tape
8 AM
6 PM
EODay
Week -2 Last Week Current WeekWeek-3
Mon Tue Wed Thu Fri Sat Sun Mon Tue Wed ThuSunSunSun
Week-4
Time
x
Example Recovery Point Options: Failure Mid-day on Wednesday
Rem
ote
Sit
eC
ore
EODay
Week-8
. . . .
. . . .
M T
W T
F S
M T
W T
F S
M T
W T
F S
End of 30 Day Remote Site Retention
= Snapshot= Snapvault(L)
= Snapvault(R)
Remote Site (Local) Core Site Daily Backup Images will reside (Local) for 30 days Daily Backup Images will exist 60 Days Snapshots (Local) will provide intraday file recovery - Backup Images expire per schedule - space permitting - Retention > 60 days – Bunker or Tape