customer experiences with tsm 6,pctyppt - ibm · pdf filecustomer experiences ... – tsm...

TSM 6.1 Development Process and Customer Experiences

Shawn Jaques

Manager, Storage Software Product Management

© 2009 IBM Corporation

1

TSM Development Process and Beta Program

• Abstract:

– During this session, you will learn about the TSM 6 Development Process and the Beta Program. In addition, we will discuss experiences from three customers who participated in the beta program.

• Learning objectives:

– Deeper understanding of the extensive beta testing of Tivoli Storage Manager 6.1.

– Indirect customer feedback about key features tested in the beta program.

– Better appreciation for how Tivoli Storage Manager can help solve key challenges.

2

Tivoli Storage Manager 6 Development Process

Dec, 2005 Dec, 2007 Dec, 2008

Nov, 2005

TSM 6 Concept

approved

Dec, 2006

Planning and Architecture

Aug, 2007

TSM 6 Plan

approved

Database

Development

Beta

Program

Other

development and testing

Feb, 2009

Quality Certification

approved

Mar 27,

2009General

Availability

3

TSM 6.1 Beta Program

• Initial kickoff meeting March 2008 and is wrapping up.

• 216 total participants enrolled

• Longest and largest beta in TSM history!

4

TSM 6.1 Beta Statistics

• 133 defects written and incorporated into code and publications

• 16 code drops delivered in several program iterations

• Over 2700 forum appends written

5

TSM 6.1 Beta Program Participants

• 207 total participants participating on 3 levels– Advocate (24) – work closely with development advocate providing

weekly feedback

– Enablement (107) – forum access for defect/questions providing feedback via surveys as requested

– Solo (85) – forum access for defect/questions, provides no survey feedback

• Breakout by participant type:– 93 customers

– 80 business partners

– 43 internal IBM participants

6

TSM 6.1 Beta Program Significant Feedback

• Verification of base server code as stable

• Reporting function delivered early in beta – install feedback indicated that code was “cumbersome” and “difficult”

– Resulted in complete rework of Reporting install

• DB Upgrade Testing – target to have 50 customer databases upgraded

– 88 databases upgraded by beta participants with a first pass success rate of 93%

– Verification of upgrade process with variety of databases

• Installation process exercised thoroughly by beta participants resulting in resolution of numerous installation defects

• Overall average rating for Admin Center to date is very satisfied (testing still underway)

7

TSM 6.1 Beta Program

• Following DB upgrade performance fix: “I have just finished doing the database extract with the latest AIX server version. The previous server version took over 23 hours and this latest version took 3 hours. Quite an improvement!”

• On Reporting function: “Excellent direction for product to be going”

• On Admin Center: “TSM always has been a feature rich and powerful product and over the years it has become much more userfriendly especially when the ISC/AC and TSM reporter became available. It's a solid product.”

• On the Configuration Wizard: “I noticed it configures the db2 instance for backups, great! (saves a lot of annoying work).”

• Beta Experience: “The working together with the TDP exchange team is a great example of why being a beta tester can really benefit both IBM and the customer.”

TSM 6.1 Beta Testing

9

● Summary

– TSM 6.1 Reporting – Nice new addition to TSM 6.1.

– Admin Center - Improved on performance and more intuitive.

– Common install interface for 6.1 Server, Reporting, ISC, Admin Center.

– Upgrades to 6.1 is an entirely different process than previous upgrades.

– Our custom SQL scripts running against DB2 are noticeably faster than in our 5.5.2 DB.

– We are taking a closer look at our backup data to determine goodcandidates for DEDUP.

– Our Exchange administrators like the Exchange mailbox level recovery.

10

● Benefits of being part of TSM 6.1 beta team.

– Establishing contacts within the development teams.

– Hear how other TSM environments are setup.

– Able to resolve problems and put in fixes much quicker now than if it was already GA.

– Appreciation for how hard all the IBM developers, project managers and beta advocates all worked on this new version of TSM.

– Other benefits:

�Test new features of TSM.

�Plan for how 6.1 new features will fit in our environment.

�Worked through problems and fixes because of how our environmentis setup.

11

● TSM Production hardware Environment

– 5 pSeries , AIX 5.3, 27 TB of TSM disk buffer

– 4 IBM 3584 tape libraries, 29 LTO-4 drives

● TSM Production software Environment

– 6 TSM servers running TSM 5.5.2

– Over 1500 TSM clients � this number made up of:

�AIX, Linux, and Windows physical and virtual clients.

�Oracle and Exchange TDP agents.

● TSM Weekly Backup stats.

– NDMP average about 22TB

– Oracle/SQL/Exchange average about 38TB

– Servers, user shares, and department shares average about 11TB

�Total Weekly backups average about 71TB.

12

● TSM 6.1 Server Test Environment

– pSeries server running AIX 5.3

– 1.5 TB of disk storage for DB, Logs, disk buffers and sequential disk pools.

– TSM server version TSM 6.1 and TSM 5.5.2.

– Test exchange 2007 server.

– Test Windows 2003 server for TSM 6.1 Reporting and Admin Center.

● Chesapeake helped Beta test.

– TSM 6.1 Reporting.

– TSM 6.1 Server install.

– TSM 6.1 Server data DEDUP.

– TSM 6.1 TDP for Exchange.

– TSM 5.5.2 to 6.1 Upgrade to same system using network.

– TSM 5.5.2 to 6.1 Upgrade to different system using media.

– Currently testing backups off Netapp filers to TSM using SNAPDIFF.

13

● TSM 6.1 Reporting

– Components installed were ISC, Admin Center, DB2, and reporting code.

– Installed and testing reporting on Windows 2003 server.

– Help in tracking and showing growth trends on our TSM servers.

– Tested against our 5.5.2 and 6.1 TSM servers.

– Next steps for us – Creating custom reports.

14

Tivoli Enterprise Portal sample display for Client Activity

15

● TSM 6.1 Server Install

– Components installed TSM Server and DB2.

– DB2 install and setup is automated – No DB2 experience was required.

– Disk layout and setup is important for performance.

● TSM 6.1 Server data DEDUP

– Looking for backups that have short retention and can be kept on disk for the primary copy.

– Backups Full or Incremental?

– How often does the data change?

– Monitor TSM server CPU impact.

– Looking at using DEDUP against some NDMP full backups, Oracle, and SQL full backups that are kept for two weeks,

16

● TSM 6.1 TDP for Exchange

– Exchange system administrators are doing the mailbox level restores.

– Worked together with IBM developers to resolve issues because of how our environment was setup.

– Exchange administrators can do mailbox level restores from the 5.5 or 6.1 TDP agent.

– Backups from the 6.1 agent can go to a 6.1 or 5.5 TSM server.

17

● TSM 5.5.2 to 6.1 Upgrades.

– Tested upgrades to same system using network and media.

– Tested export node from 5.5.2 to 6.1.

– Our TSM 5.5.2 databases inserted into DB2 were about the same size after migration.

– Insert times throughout the beta testing have continued to improve.

– Test the upgrade options to determine what works best in your environment.

2/9/2009 Kirsten Glöer

University of Heidelberg

2/9/2009 Kirsten Glöer 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15

Tivoli Storage Manager 6.1Customer Panel Discussion

University of Heidelberg

Computing Center

IBM Pulse 2009


Heidelberg

2/9/2009 Kirsten Glöer 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15

Heidelberg is located in

southern Germany

It is known for its castle of the 14th century

and for one of the oldest Universities in

Germany with 35.000 students


Agenda

2/9/2009 Kirsten Glöer 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15

• TSM Cooperation

• TSM Environment

• TSM V6.1 Test Environment

• New Features in TSM V6.1

• Installation of TSM V6.1

• MS Exchange Item Level Recovery

• Data Deduplication Option

• NetApp SnapMirror Backup

• Upgrade to TSM V6.1 on a new System

• Time Measurements

• Key Benefits of TSM V6.1

• Summary

2/9/2009 Kirsten Glöer2/9/2009 Kirsten Glöer 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15

Network

51 TB Backup

8 TB Archive

138 TB Backup

113 TB Archive

75 TB Backup

7 TB Archive

11 TB Archive

159 TB Backup

Network

TSM data:

Archive: backup stg of Archive stg

Backup: backup stg of Backup stg

and database backups

52 TB Backup

7 TB Backup

31 TB Archive

2 TB NDMP

TSM cooperation

between the

Universities

in south-west

Germany

TSM Cooperation


2x IBM pSeries p5-550Q

Modell 9133-55A

14 GB RAM, 8 Proz.

AIX 5.3 ML 7, 64-bit

TSM V5.5.2

7,6 TB disk storage

7 Backup/Archive-Server

⇒ 745 TB data

2x IBM pSeries 660

Modell 7026-6H1

2 GB RAM, 4 Proz.

AIX 5.3 ML 7, 64-bit

TSM V5.5.2

1 Tape Library Server

3 Backup/Archive-Server

3,5 TB disk storage

⇒ 177 TB data

IBM 3584 Library

• Tape drives:

• 16 Jaguar2

• Tapes:

• 2300 Jaguar2 a 500 GB

=> 922 TB stored data, 40 TB NDMP

Gigabit Ethernet BackboneGigabit Ethernet Backbone

ServerClient Server Client

TSM Environment

Cisco MDS9509

Cisco MDS9513

HDS Lightning 9980V

FC disks for TSM server db

HDS TagmaStore AMS500

SATA disks for TSM serverdisk storagepools

2110 TSM Clients:

• TSM Nodes

• TDP for Oracle

• TDP for MS SQL Server

• TDP for MS Exchange


IBM pSeries RS/6000

Modell 7043-260

4 GB RAM, 2 Proz.

AIX 5.3 ML7, 64-bit

TSM V5.5.1.1

700 GB disk storage fordb, log, stgs

IBM pSeries RS/6000

Modell 7043-260

4 GB RAM, 2 Proz.

AIX 5.3 ML7, 64-bit

TSM V6.1

700 GB disk storage for

db, log, stgs

Tivoli Storage

Manager V5.5.1.1

and V5.5.2.0

Tivoli Storage

Manager V6.1

TSM V6.1 Test Environment

Furthermore:

• Test Exchange server 2003

• NetApp filer N5200


New Features in TSM V6.1

TSM Server Windows 2008 support

MS-SQL 2008 Support

New Reporting and monitoring feature

Updated TSM V6.1 Administration Center

Item Level Recovery for Active Directory Objects

MS Exchange Item Level Recovery

UTF-8 support for Linux-Clients

NetApp SnapMirror to tape integration

Expire Inventory enhancements with additional parameters:

NODE, DOMAIN, TYPE, DURATION, RESOURCE

Data Duplication option for sequential-access disk stgs

DB2 database

TestedNew in Tivoli Storage Manager V6.1


Installation of Tivoli Storage Manager V6.1:

• TSM V6.1 uses DB2 database. When installing TSM V6.1,

DB2 is installed during the installation process

• DB2 knowledge is not necessary

• At the beginning of the betatest in July last year, a lot of errors arises

during installation like:

• FatalInstallException: Install Exception

• ANR2678E Server database format failed

• Installation directories are missing:

/opt/tivoli/tsm/db2 was installed

/opt/tivoli/tsm/license was installed

/opt/tivoli/tsm/server is missing

• Some of these error messages arises because of out of space condition

• Disk space requirements decreased for the directory /opt

from 8 GB to 2 GB

• TSM V6.1 needs a lot of processor and I/O resources

Now, TSM V6.1 installation is easy without any problems

and TSM V6.1 is very stable

Installation of TSM V6.1


MS Exchange Item Level Recovery

MS Exchange Item Level Recovery:

• Really good search function for item-level

recovery operations in MS Exchange Server

2003 or 2007

• In Exchange 2003 a Recovery Storage Group

needs to be created manually, this is not

necessary in Exchange Server 2007


Data Deduplication Option

Data Deduplication:

Data deduplication is a method of eliminating redundant data in

sequential-access disk storage pools.

• At the beginning of the test a backup from a deduplicated storagepool

to a non deduplicated storagepool completed with failures

This is already solved

We will not use Data Deduplication because:

• Data Deduplication needs a lot of processor and I/O resources

• Our TSM Clients backup the data to a disk staging pool before the whole

backup data is migrated to tape. On disk the data will be deduplicated,

when migrating to tape, the data will be de-deduplicated.

• There is a very small likehood of identifying false friends instead of

duplicates


NetApp SnapMirror backup:

NetApp SnapMirror backup is a fast block-level backup and restore

method in order to copy large NetApp volumes to respectively from

tape. SnapMirror backups are created with the command:

backup node <node_name> type=SnapMirror

At the beginning of the betatest the NDMP SnapMirror backup process

terminates always with the message -> internal server error detected

This is already solved

NetApp SnapMirror Backup


1. Installation of TSM V5.5.1.1

3. Database of production

server restored

4. Upgrade utility installed

5. TSM V5 database preparedRun DSMUPGRD PREPAREDB

9. Extraction process startedRun DSMUPGRD EXTRACTDB

2. Installation of TSM V6.1

6. Login as the TSM server

instance owner userid

7. Empty database formated

to receive dataRun DSMSERV LOADFORMAT

8. Insertion process for the

new server startedRun DSMSERV INSERTDB

Upgrade to TSM V6.1 on a new system (network

method)

For best performance it is better to use multiple LUNs for:

• Active log

• Archive log

• Database directories


Time Measurements

16 hr 10 mins16 hr 43 mins48 hr 26 mins61 hr 22 minsTotal Time

Network Method

5 hr 19 mins6 hr 44 mins15 hr 10 mins16 hr 55 minsExtract Step

Driver 0344Driver 0344Driver 0324Driver 0295Driver Level

47,8 GB47,8 GB84,7 GB84,7 GB

Database size

3 LUNs used for the database directories

1 LUN used for the active log

1 LUN used for the archive log

• The database upgrade time does not only depend on the database size,

it does furthermore depend on consistency checks.

• It is not possible to predict the time due to the database size.

• Therefore, it is better to upgrade a production TSM server in a test

environment.

2/9/2009 Kirsten Glöer2/9/2009 Kirsten Glöer 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13| 14 | 15

Key Benefits of TSM V6.1

Key Benefits of Tivoli Storage Manager V6.1:

• Use of DB2 database leads to a higher scalability and to

increased system stability because of the increased recovery log size

• Item Level Recovery capability of MS Exchange

• UTF-8 support for Linux clients

• NetApp SnapMirror to tape integration

The Principal Financial GroupPulse 2009

Butch Umble, IT System Analyst – Senior

TSM 6.1 Customer Panel Discussion

The Principal Financial Group

• One of the nation's 401(k) leaders and total retirement solutions

innovators offering businesses, individuals and institutional

clients a wide range of financial products and services.

• A member of the Fortune 500, serving 19.0 million customers

worldwide from offices in Asia, Australia, Europe, Latin

America and the United States.

• Ranked #1 for IT innovation by InformationWeek (2006)

• Seventh straight year voted by FORTUNE 100 as one of best

companies to work for (Current rank - 17th)

TSM environment - 1998

• MVS Environment/Configuration:

– One TSM started task backing up CFS/UNIX/EXCHANGE:

• 80GB disk buffer/1,600 Media1 data cartridges

• 36 clients (8 Exchange/14 Novell/14 Unix)

• 2.11GB/Hour average throughput

TSM environment - 2009

• AIX Environment/Configuration:

• 5 AIX P550/P520 clusters managing 23 TSM instances:

– 40TB disk buffer/9000 Media5 data cartridges

– 2,306 clients (22 Exchange/1,823 Windows/461 Unix/44 UDB)

• 3.9TB/Hour Average throughput

• 20 AIX P550/P520 clusters managing 20 TSM/CM instances:

• 480 TB of CAS/Centera storage managed

Growth Challenges

0

500

1000

1500

2000

2500Feb-0

7

Mar-07

Apr-07

May-0

7

Jun-0

7

Jul-07

Aug-0

7

Sep-0

7

Oct-07

Nov-0

7

Dec-0

7

Jan-0

8

Feb-0

8

Mar-08

Apr-08

May-0

8

Jun-0

8

Jul-08

Aug-0

8

Sep-0

8

Oct-08

Nov-0

8

Dec-0

8

Jan-0

9

Feb-0

9

Growth of TSM Nodes from 2007 to 2009

0

5

10

15

20

25

Feb-

07

Mar-

07

Apr-

07

May-

07

Jun-

07

Jul-

07

Aug-

07

Sep-

07

Oct-

07

Nov-

07

Dec-

07

Jan-

08

Feb-

08

Mar-

08

Apr-

08

May-

08

Jun-

08

Jul-

08

Aug-

08

Sep-

08

Oct-

08

Nov-

08

Dec-

08

Jan-

09

Feb-

09

Daily Total Backup from 2007 to 2009 in TB

0

0.5

1

1.5

2

2.5

3

Feb-0

7

Mar-07

Apr-07

May-0

7

Jun-0

7

Jul-07

Aug-0

7

Sep-0

7

Oct-07

Nov-0

7

Dec-0

7

Jan-0

8

Feb-0

8

Mar-08

Apr-08

May-0

8

Jun-0

8

Jul-08

Aug-0

8

Sep-0

8

Oct-08

Nov-0

8

Dec-0

8

Jan-0

9

Feb-0

9

Daily Data Stored from 2007 to 2009 in PB

TCR test environment

• Configured one VM guest (1 processor/2GB Ram) for TCR

• Templates available for all aspects of TEMS/TCR

• Configured TEMS database for trending with TCR

• Defined 3 TSM images for reporting analysis collection

• Dynamic data presentation within TEMS

Viewing TEMS reports

Node Activity

Tape volume utilization

Client node Occupancy

Operational Improvements with TSM6.1/TCR

• Improved reporting with TCR provides real time

monitoring (TEMS) and long term trending benefits

(TCR/BIRT)

• Reporting tool packaged with twelve predefined

reports to assist with your daily administration tasks

• Improved performance and scalability with DB

backend provides larger footprints/databases

• Introduction of data dedup functionality provides a

method for reducing storage costs

TSM6 Beta Experience

• Communication from IBM developers, advocates, and

test participants was very open and timely.

• Very challenging and well planned project, information

about all events/releases were very organized.

• The timing of TSM version 6 was perfect and meets our

requirements for the challenges within our TSM

environment

Thank you!

45

Trademarks and disclaimers

Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries./ Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. IT Infrastructure Library is a registered trademark of the Central Computer and Telecommunications Agency which is now part of the Office of Government Commerce. ITIL is a registered trademark, and a registered community trademark of the Office of Government Commerce, and is registered in the U.S. Patent and Trademark Office. UNIX is a registered trademark of The Open Group in the United States and other countries. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others. Information is provided "AS IS" without warranty of any kind.

The customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer.

Information concerning non-IBM products was obtained from a supplier of these products, published announcement material, or other publicly available sources and does not constitute an endorsement of such products by IBM. Sources for non-IBM list prices and performance numbers are taken from publicly available information, including vendor announcements and vendor worldwide homepages. IBM has not tested these products and cannot confirm the accuracy of performance, capability, or any other claims related to non-IBM products. Questions on the capability of non-IBM products should be addressed to the supplier of those products.

All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.

Some information addresses anticipated future capabilities. Such information is not intended as a definitive statement of a commitment to specific levels of performance, function or delivery schedules with respect to any future products. Such commitments are only made in IBM product announcements. The information is presented here to communicate IBM's current investment and development activities as a good faith effort to help with our customers' future planning.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput or performance improvements equivalent to the ratios stated here.

Prices are suggested U.S. list prices and are subject to change without notice. Starting price may not include a hard drive, operating system or other features. Contact your IBM representative or Business Partner for the most current pricing in your geography.

Photographs shown may be engineering prototypes. Changes may be incorporated in production models.

© IBM Corporation 1994-2009. All rights reserved.References in this document to IBM products or services do not imply that IBM intends to make them available in every country.

Trademarks of International Business Machines Corporation in the United States, other countries, or both can be found on the World Wide Web at http://www.ibm.com/legal/copytrade.shtml.

customer experiences with tsm 6,pctyppt - ibm · pdf filecustomer experiences ... – tsm...

Documents