safe and secure data migration
DESCRIPTION
Safe and Secure Data MigrationTRANSCRIPT
Safe and Secure
Data Migration
“It’s Not That Tough”
David Clark
Director of Product Marketing
QLogic, Storage Solutions Group
Agenda
• Drivers for Data Migration
• Why Data Migration is complex
• Seven Gremlins to a Safe DM
• Methods / Tools used for Data Migration
• A Better Data Migration tool
Data Migration
“It’s Not That Tough”
Data Migration
“Data Migration is one of those tasks that never
seems to be done. The data center ebbs and
flows through periods of storage and server
expansion & consolidation. The need to
migrate data never goes away, it just takes a
break”.
Leading Drivers
• End of Life/lease on storage systems
• Technology transitions
– Speed changes on FC 2Gb – 4Gb – 8 Gb
– iSCSI, FCoE
• Operating Systems and Application Migration
• M&A activities
• Regulatory Compliance
• VMware / Virtualization driven server consolidation
• Storage Consolidation
• Tiered Storage Strategies
• Information Life Cycle Management
Why Data Migration is complex
Data Migration cuts across:
1. Multiple OS and applications
2. Multiple SAN islands & Storage protocols
3. Multi-Vendor Storage
FC SAN
Servers
iSCSI Storage
FC
SAN
iSCSI
SAN
FC Storage
Vendor A
Vendor B
O/S “A” O/S “B” O/S “C” O/S “D”
IT Organizations within Data Centers
• Application team
– Manages applications
• Network team
– Manages Networks
• Storage team
– Manages storage
Data Migration involves interactions
amongst many IT teams
Storage
Network
Applications
Different Types of Data Migration
• ONLINE / OFFLINE– Depends on how much down time application can tolerate
• Local– Within Data Center
• Remote– Migration of Data Centers
• Across Storage Protocols– FC, iSCSI, FCoE
• Many-to-one– Consolidation of volume grown over multiple LUNs to a
single large volume
Customer Concerns re: DM
6%
25%
32%
36%
37%
48%
58%
0% 10% 20% 30% 40% 50% 60% 70%
Other
No Problems
Lost Data
Applications Performance
Data Corruption
Compatibility Issues
Extend Downtime
Data Source - ESG Survey
1. Downtime
2. Human error
3. Data corruption
4. Infrastructure incompatibilities
5. Departmental conflict
6. Costs and resources
7. Poor planning (too many details)
Gremlins of Data Migration
Data Migration Methods
• Manual Methods
• Software based
• Network based appliance
• Array based
• New paradigm
Manual Methods
• Backup & Restore– Simple to use
– Protects Meta Data
– Requires Backup device
– Extended down time
– Expensive
– Requires scheduling with backup windows
• Manual copies– If small amount of file-based data set, most inexpensive method
– For repetitive work, develop scripts
– Tedious & prone to human error
– Must ensure the copy of application metadata
– Extended down time
FC
SAN
Source Array
Server
Target Array
SAN
Source Array
Server
Target Array
VTL
Software only Migration
tools• Typically installed on the
server
• Manages Migrations at Volume level
• Supports Migration– From DAS to SAN
– SAN to SAN
• Independent of Storage and SAN
• OS Specific
• Need a different tool or version of SW per OS type
• Compatibility with many different versions of OS and software is required
• Server Reboot Required after install
FC
SAN
Source Array
Servers
Target Array
Migration is OS
vendor
dependent
O/S “A” O/S “B” O/S “C”
Software Migration Steps
1. Schedule application down time
2. Storage Manager kills Application Manager (or whatever
is needed to load DM agents/apps)
3. Ensure that Agent/App supports proper OS release(s)
4. Create volume-to-LUN map
5. Take application offline
6. Run migration for selected volumes/LUNs
7. Rezone new storage to application
8. Reboot server/application
Network Based DM Tools
• Deployed in the Network
• Independent of Operating
System & Storage Array
• Block based migration tool
• ONLine / OFFline Migration
• Today’s DM tools work only in a vendor specific Network
• May require a specific Fabric infrastructure already installed
• Cumbersome to setup
• Downtime during install
• Expensive
• May require new multi-path SW
FC
SAN
Source Array
Servers
Target Array
Fabric Vendor
Dependent
O/S “A” O/S “B” O/S “C”
Network DM Steps
1. Determine fabric-based or direct-attach
2. Network Manager kills Storage Manager
a) Attach and zone new device
b) Disconnect storage and rezone
FC SAN
Source Array
Servers
Target Array
Fabric Vendor
Dependent
O/S “A” O/S “B” O/S “C”
Network DM Steps
3. Ensure fabric compatibility
4. Determine which Apps/LUNs to be migrated
5. Schedule downtime for some Apps (SAN-
attached) or all Apps (direct-attached)
6. Reboot server/Apps to new storage
FC SAN
Source Array
Servers
Target Array
Fabric Vendor
Dependent
O/S “A” O/S “B” O/S “C”
Storage Array based
Deployment
• Software License on an array
• New class of array
Independent of Operating System and Fabric vendors
Cost effective if used in very narrow use case
Incompatible across heterogeneous storage
• SW often not compatible within different family of arrays even from same vendor
• Software License tied to a specific array, allows migration in and out of that specific array
• New class of storage requires a fork lift upgrade
Array-based Steps
1. Check array family compatibility
2. Acquire and load license for EACH array
3. Determine Volume-to-LUN map
4. Take application(s) offline
5. Perform block-level transfers
6. Rezone and reboot
FC SAN
Source Array
Servers
Target Array
O/S “A” O/S “B” O/S “C”
A Better Data Migration Solution
“It can’t be that tough?”
Need for DM tools
Block based offline migration Service
Must be Independent of:
OS, Fabric, Storage Vendors
LUN 3
LUN 2
LUN 1
FC
SAN
Old StorageNew Storage
LUN 3
LUN 2
LUN 1
Migrated LUNs
Server “A” w/ LUN 1
Server “B” w/ LUN 2
Server “C” w/ LUN 3
DATA
PATH
Data Mover
OFFLINE
OFFLINE
OFFLINE
Low cost:
Small Capital investment
Pay as you go model
Simple to transport
Minimize downtime
Fast install and configuration
Simple to use
Associate Server volumes to physical
LUNs
Use only allocated array bandwidth
DM Tool requirement for Enterprise
• Independent of :
– Operating System
– SAN Vendor
– Storage protocols (FC, iSCSI, FCoE)
– Storage Array
• Need to support :
– Online / Offline
– Local / Remote
• Many to One
– Volume consolidation from many small LUNs to a single LUN
DM Tool requirement for Enterprise
• Least intrusive– Should not require any changes to Host Software or any other infrastructure
already in place
– Should be able to insert ONLINE
• Management– Manage Remotely
– Simple to install, configure and deploy
– Should provide detailed intelligent migration reports that are simple to understand
– Flexible scheduling of migration Jobs
– Should facilitate communication amongst multiple groups with Data Center
– Association of Application Volume to Physical Luns
– Accountability
DM Tool requirement for Enterprise
• Provide Protection against Human errors – Accidently migrating Destination LUN to Source LUN
– Selection of one of the other Data LUN as Destination LUN
• Security– Provide the access to Job configuration only for authorized
users
– Alerts on unauthorized attempt
• Address cost– Low capital cost
– Should not require fork lift upgrades
– Pay as you go model
Enterprise Online DM
1. Install appliance and new storage (non-
disruptive)
2. Create Volume/LUN map
3. Migrate and verify
4. Rezone and reboot
LUN 3
LUN 2
LUN 1
FC
SAN
Old StorageNew Storage
LUN 3
LUN 2
LUN 1
Migrated LUNs
Server “A” w/ LUN 1
Server “B” w/ LUN 2
Server “C” w/ LUN 3
DATA
PATH
Data Mover
Online DM Scenario
• Standard zoning before DM
– Dual SAN, split path
SAN A
Old StorageNew Storage
SAN B
Online DM Scenario
• Rezone dual-path with DM appliance as target for server
• Rezone dual-path with DM appliance as initiator for storage
• Maintain one server path while configuring the other path
SAN A
Old StorageNew Storage
SAN B
Online DM Scenario
• Simplified topology for illustration
SAN
Old Storage New Storage
Source LUN
Sector 0 Sector N
Destination LUN
Sector 0 Sector N
Online DM Scenario
• Simplified topology for illustration
SAN
Old Storage New Storage
Source LUN
Sector 0 Sector N
Destination LUN
Sector 0 Sector N
New
write
Sync Log
SRC sector 3
Online DM Scenario
• Simplified topology for illustration
SAN
Old Storage New Storage
Source LUN
Sector 0 Sector N
Destination LUN
Sector 0 Sector N
Sync Log
SRC sector 3
Summary
• Data Migration should be:
1. Managed by one group
2. Independent of OS, Fabric and Storage
environments
3. Minimally intrusive
4. Highly automated – less prone to error
5. Cost-effective
Safe and Secure Data Migration is now possible,
“it’s not that tough”!
Questions?
THANK YOU