scheduling operations in networker - dell · so if the requirement was to perform a backup every 2...

EMC Proven Professional Knowledge Sharing 2010

Scheduling Operations in NetWorker

Aaron Kleinsmith

Aaron KleinsmithP&E Consultant, EMC EducationEMC² [email protected]

2010 EMC Proven Professional Knowledge Sharing 2

Table of Contents Scheduling in NetWorker .................................................................................................... 3 Different ways to start backups .......................................................................................... 3

Group resource ............................................................................................................... 3 Scheduling the Group backups ......................................................................................... 4 On-demand Group backup ................................................................................................ 5 Restarting a Group backup ............................................................................................... 5 Savegrp ............................................................................................................................. 6

Savefs and save ............................................................................................................. 8 Windows Task Scheduler or Unix/Linux cron ................................................................. 9 External scheduling applications .................................................................................... 9

Using Schedules effectively ................................................................................................ 9 Re-occurring events in the Schedule ........................................................................... 10

Week or Month ................................................................................................................ 10 Schedule Overrides ......................................................................................................... 10 Backing up at intervals longer than a single day ............................................................. 12 Planning for dependent save sets ................................................................................... 14 Retention Policy ................................................................................................................ 15 How to use Directives effectively ...................................................................................... 15

Excluding files from a backup ....................................................................................... 16 Use Null or use Skip? ................................................................................................... 16 Using Groups, Schedules and Directives for Application Backups ............................. 17

Cloning and staging .......................................................................................................... 18 Overview of Cloning ...................................................................................................... 18 Backing up and Cloning with Disk ................................................................................ 19 Scheduling cloning jobs ................................................................................................ 20

Cloning as part of a backup Group ................................................................................. 21 Cloning using nsrclone .................................................................................................... 21 Examples using nsrclone ................................................................................................ 23 Retentions on Cloned Save Sets .................................................................................... 24 Using nsrclone with mminfo ............................................................................................ 25 Cloning by Volume .......................................................................................................... 26 Recovering from a Cloned Save Set ............................................................................... 26

Overview of Staging ...................................................................................................... 26 Staging Resource............................................................................................................ 27 Using nsrstage command ............................................................................................... 27 Other ways to schedule scripts in NetWorker .................................................................. 28 Conclusion ........................................................................................................................ 28

Disclaimer: The views, processes, or methodologies published in this compilation are those of the authors. They do not necessarily reflect EMC Corporation’s views, processes, or methodologies.


Scheduling in NetWorker

Many activities need to be scheduled in NetWorker™ but performing a backup is the

most important. There are many different requirements depending on what type of

system, what type of data, or who owns the data. These requirements help you to decide

when to start the backup, how long you have to complete the backup, how long the data

needs to be recoverable, and whether or not the data will need to be transported offsite.

There are many technologies available to change the way data is backed up and stored.

This great amount of flexibility complicates the way we manage backups. This article

describes strategies that you can use to schedule backup and cloning tasks in

NetWorker.

Different ways to start backups

Group resource

Configuring a Group resource to “Autostart” at a specific time is the most common

method to start a backup in NetWorker. It is straight-forward; just create a Group

resource with a configured start time listed in 24-hour time (for example, a setting of

19:00 for a start time of 7:00 pm), enable the Autostart attribute, and ensure one or more

clients are in the group. This type of backup is a server-initiated backup since the

NetWorker server requests all the clients in a group to send data for backup.

Which save sets that client will be sending for backup are configured In each client

resource. When performing file system backups for clients, the NetWorker server will

initiate separate save commands on the clients to start for each save set that needs to

be backed up. Prior to starting the save commands, start the savefs program on each

client to first verify save set paths or discover the save sets paths if a save set keyword

of “All” was configured for a client. The savefs command running on each client helps

build a worklist of save sets that will be sent for backup. The NetWorker server will

collect the worklists of these save sets from each client, and organize the data to the

needed storage nodes.


Creating multiple groups to begin at staggered start times, and dividing clients between

these groups is a good strategy when scheduling the backups. The size of the

environment, the number of clients, start time requirements, and backup windows will

help you to decide how many groups and when they should be configured to start. When

you stagger the groups, the NetWorker server will not be left with too many clients and

save sets to queue and manage at one time. This allows higher-priority clients to be

backed up first by placing them in the groups that are scheduled to start first.

Scheduling the Group backups

Once a day backup

The NetWorker Group resource can be left with the default “Interval:” attribute of “24:00”

that will run the group once every 24 hours if the group backup is required to be

performed once every day at the same time. Even though a once a day backup is

common, there are some other strategies for scheduling the groups more or less

frequently than once a day. Backing up more frequently than once a day is covered in

the next section. The Group will need to be combined with an appropriate Schedule if

there is a need to back up less frequently than once a day.

Backing up at an interval time

If you must do a specific backup more frequently than every 24 hours, the Interval

attribute can be shortened to the necessary requirement such as 30 minutes (00:30) or

every 2 hours (02:00). This scheduling works well when you need to back up data that

changes frequently such as database logs or other types of transactional logging data

from important applications.

Backing up at an interval during certain hours

Starting a backup at an interval can be very useful, but in certain circumstances there

may be a window of time you don’t want the backups to occur – a black-out window. A

group can start at a specific interval, but NetWorker does not allow you to stop that

interval for a period of time. For example, if you need to back up transaction logs of a

database during business hours and not at all during its nightly, level full, 8-hour backup


window, the interval does not help to prevent any overlap between these two backups.

In this case, you can configure multiple groups with needed start times and select them

in each client resource so the client can start backing up at these required times.

So if the requirement was to perform a backup every 2 hours during normal business

hours (9am-5pm), five Group resources could be created each with an interval of 24

hours and different start times: 09:00, 11:00, 13:00, 15:00, 17:00. A client can be made

part of multiple groups so the client resource that needs the frequent backup can be

made part of each of these groups. A nightly, once a day backup for an overnight

backup would also be needed, so include one more group for that.

Create more groups if you need to perform the backup more frequently than every 2

hours. If you need to perform backups every hour, create groups at 1 hour intervals

during the needed window. For example, there may need to be 9 or 10 groups such as

8:00, 9:00, 10:00, 11:00, 12:00, 13:00, 14:00, 15:00, 16:00, 17:00.

Use a name that describes the purpose and time the group runs; avoid using spaces or

special characters when deciding on group names. When you use spaces in group

names, and it needs to be called from the command line, surround it with double-quotes.

Consider whether the savepnpc (save Pre aNd Post Command) will ever be needed

when choosing a group name. This feature uses the group names to create and name a

configuration file that will be created on the client. NetWorker will have no trouble

creating files for this feature if there are no spaces or special characters in the names.

On-demand Group backup

Once you create a Group resource, start it from the Monitor section of the NetWorker

management console Graphical User Interface (GUI) window under the Groups tab.

Right-click the Group and choose ‘Start’ to begin the group backup. You can set the

Group resource to either ‘enabled’ or ‘disabled’ and still be started.

Restarting a Group backup

You can also initiate a restart from the same right-click menu as the on-demand group

backup. If a group is restarted, any save sets that had not completed in the last run of

the group will be started and any save sets that had completed from beginning to end


will not be started again. The inclusion or exclusion of data from the restarted group is

based on the individual save sets. You could include an earlier failed save set of a client

in the restart, but different successful save sets of that same client will not need to be

started again. For a restart to behave in this manner, it will need to be started within 12

hours of the last time the group finished in order for it to include only failed save sets.

Every save set in the group will be started again if a group is restarted out of this 12 hour

restart window. This “Restart window:” is an attribute setting in the Group resource that

you can customize and change from the default “12:00” hour window to any value that is

less than the interval setting of the group.

Savegrp

You can also start a Group from the command line using the command “savegrp” on the

NetWorker server. This command runs only on the NetWorker server and can be called

directly from the command line as savegrp with appropriate options, or included in a .bat

or script file run from the command line. To automate the start of the backup using the

command, the savegrp command can be put into a .bat or script file and called on the

NetWorker server through an external scheduler such as the Windows Task Scheduler

or UNIX/Linux cron.

Different switches can be applied to the end of the savegrp command to allow for

different ways of starting the backup. If the switches overlap or conflict with options

already set in the GUI, the options used at the end of the command will over-ride or take

precedence over the GUI options. For example, there is a lower-case “L” switch (-l) that

specifies the level of backup. If this is included as “-l Full” then regardless of what is

scheduled in the GUI this backup will be of level Full. There are also switches to perform

a preview backup with verbose options (-pvvv) that is useful to verify configuration or

troubleshooting, back up specific clients (-c client_name), or to force a particular level of

backup.


Here are some examples of the savegrp command:

savegrp command Description of backup started

Savegrp Start the Default group

savegrp NightlyBackup Start group named “NightlyBackup”

savegrp –pvvv NightlyBackup Start “NightlyBackup” group with preview

and verbose settings

savegrp –c client1 NightlyBackup Start only client with hostname “client1” in

the “NightlyBackup” group

savegrp –l Full NightlyBackup Start NightlyBackup group and back up all

savesets at level full regardless of client

schedules

Another useful form of the savegrp command is the upper-case “O” option that will back

up Only the client file indexes of the clients in the group being started. It will ignore the

save set attribute in the client resources. If the backup server is part of the group, the

NetWorker server client file index and bootstrap will also be backed up. This is useful

when you need to back up all the indexes and bootstrap at one time, such as just prior to

performing an upgrade of the NetWorker server.

For daily operations, include the command in a .bat or script file and scheduled to run

with an external scheduler to have all the indexes back up at the same time instead of

only when the groups end at different times throughout the evening. Start a Group

resources with all clients belonging to it to ensure that all client file indexes and the

bootstrap are captured in the backup. You can create a special group with all clients

belonging to it, but leaving the Autostart: attribute set to Disabled. This group can be

used strictly for performing client file index and bootstrap backups with the savegrp –O

option. All the clients will belong to more than one group – the normal scheduled group

and the one that is created for the savegrp –O command.


Savefs and save

You can also start a backup from the command line on the client system; this is a client-

initiated or manual backup. When a group backup starts, the NetWorker server contacts

the clients that are part of the group and requests that a savefs command be started on

each of the clients to:

• collect information

• build a work list of what needs to be backed up

• call the save command(s) that will be needed for copying and sending the data

to an appropriate storage node.

You can also call the savefs command or the save command directly on the NetWorker

clients from a command line, or put in a .bat or script file and call through an external

scheduler. Include a process to back up the client file indexes of these clients when

choosing to start backups only from the client. Normally, NetWorker will automatically

back up the client file indexes at the end of a group backup of the client, but will not

when an administrator initiates the backup from the client itself.

You do not need to run the save commands separately if using the savefs command

since part of the savefs’ job is to call necessary save commands. Save commands can

be used directly to back up specific save sets as necessary if you are not using savefs.

When you run save from the client, the backup level will be recorded as ‘manual’. You

can force a level using a lower-case “L” option. For example, using the following

command syntax to record the backup as level Full “save –s NetWorker-server-

hostname –l full /etc” where “NetWorker-server-hostname” is the hostname of the

NetWorker server and “/etc” is the directory being backed up.

Use the Level attribute or a pool if you need to sort the client-initiated backups to specific

media pools in NetWorker. Sorting data to media pools using Group names is a common

practice when configuring media pools in NetWorker. They can’t be sorted by Group

name for the client-initiated backups since they don’t start as part of a group. Rather,

sort them based on other criteria. “Manual” is one of the Level attribute values that you

can use for a pool configuration to sort the client-initiated backups away from the Default

pool and other backup data.


Windows Task Scheduler or UNIX/Linux cron

The Windows operating system has a scheduler referred to as the Windows Task

Scheduler (@) or Scheduled Tasks option in the Start menu. This scheduler allows an

administrator to schedule .bat or script files to run at different times. There is some

flexibility in regard to weekly, monthly, or other re-occurring events that are not available

with NetWorker Groups. Commands like “savegrp” with appropriate options can be

included in a .bat or script and called from the Windows Task Scheduler.

Likewise, on UNIX and Linux, the included cron scheduler can be used to call scripts

that include the savegrp command. This may allow for more flexibility than relying strictly

on the Group options in the NetWorker GUI.

One disadvantage of scheduling jobs outside of NetWorker is that they need to be

monitored and updated separately. The flexibility achieved by using an outside

scheduler can be configured with an appropriate combination of Groups, Schedules, and

Directives in NetWorker to have everything in one interface. This is described later in this

article.

External scheduling applications

There are other job scheduling (or batch scheduling) software products that you can use

with NetWorker by calling commands directly or through .bat or script files. They may be

a good option if there is already job scheduling software being used in the environment.

Examples of job scheduling software products are BMC Control-M, CA Autosys, IBM

Tivoli, NetIQ, etc.

Using Schedules effectively As mentioned earlier, you sometimes schedule events once a week, once a month, or

even once a year but can’t accomplish that with the NetWorker Group resource alone.

NetWorker Groups can be configured in conjunction with Schedules to handle re-

occurring events that are longer than 24 hours without the help of an external job

scheduling tool. A NetWorker Schedule resource is used to indicate what level of backup

should take place on a given day such as level Full, Incremental, or Differential (1-9)

backups. You can include a ‘skip’ that schedules NetWorker to not back up on a


particular day. With a ‘skip’ scheduled, a Group resource may be set to “Autostart:” and

still run at a given time of day, but once the ‘skip’ level is encountered in the Schedule

the clients this schedule applies to will be skipped and no data will be backed up for

them. A Schedule resource that has a Full backup on Sunday but set to skip for the

other six days of the week will only back up once a week. A schedule could similarly be

set up with a skip on all days of the month except one to do a once a month backup and

skip all days but one in a year to accomplish a once a year backup.

Re-occurring events in the Schedule

Week or Month

Many EMC customers need to perform a NetWorker backup on a once a month

schedule, but it needs to happen on a specific day of the week – such as a Friday or a

weekend. Typically, in the Schedule resource of NetWorker, the interval for a backup

can only be set to a Week or Month. When using “Week,” a change to the schedule is

made only for a specific day of the week (for example, an incremental backup on

Monday) and all similar days of the week (incremental on every Monday of every week)

will be updated with that same level. With the Month period of time, a specific numerical

day of the Month (for example, the 1st of the month) will be updated to the selected level

for every month throughout every year. That numerical day of the month will usually fall

on different days of the week. So even though it may be necessary to get a particular

backup done at the beginning of a month, if the 1st is on a Wednesday that may not be

convenient since Wednesday may be a day in the middle of the week and there may not

be a large enough backup window. It might be better to perform the backup on the first

Saturday of the month when there is an appropriate backup window.

Schedule Overrides

NetWorker Schedules have ‘overrides’ to help set levels outside of the normal week and

month. You can easily set the overrides in the NetWorker GUI one day at a time

wherever needed. When an override is used, it normally will change a very specific day


(to set a level skip on December 25th, 2010 for example) displaying an asterisk beside

the set level and not having it replicate anywhere else in the schedule.

If there are many overrides to set – such as the first Saturday of every month – it is not

convenient to select a single day in each month of each year. Override using text key

words. Right-click on any one of the Schedules resources in the GUI to unselect the

option called “Show Schedules as Calendars.” This allows the schedules to be shown in

text mode – or similar to the way they are stored in NetWorker’s Resource database.

Opening the properties of a schedule allows an administrator to directly update the

Override: attribute with one or more custom text overrides. This could mean entering in a

level and specific date in a MM/DD/YYY format such as “skip 12/25/2010.” Likewise, a

specific day of a month for every year could be entered by leaving off the year value and

including only MM/DD, with the syntax such as “skip 12/25.” Leaving off the year will

cause the level to repeat on the same day every year. The general format for the text

overrides include the level first and then the date or frequency that needs to be applied.

There can be multiple overrides on the same schedule and they will be applied in the

order listed in the attribute.

Some examples of acceptable syntax for the text overrides:

Description of needed policy Attribute setting

Perform Full backup on first day of the year full 1/1

Perform a Full backup last Friday every month full last Friday every month

Perform a level 5 on second Tuesday every month 5 second Tuesday every month

Perform Full backup first Saturday every quarter full first Saturday every quarter

Perform full once a year on last Friday full last Friday every year

You can use the nsradmin command to update these overrides in Override: attribute.

Use the nsradmin command to create or modify resources in the NetWorker server

resource database.


Backing up at intervals longer than a single day

Here are three examples of when a backup interval of longer than one day is useful:

• when there is a need to perform an archive type backup such as a full once a

week for onsite backups

• full once a month to be sent off site

• full once a year to be archived for longer period of time

Typically, a backup is a secondary copy of primary data that is kept in case of failed

equipment, human error, or a site disaster that causes the primary copy of data to be

lost as opposed to being archived. Saving a “once a month backup” for an extended

period of time may seem like a good strategy to archive data, but there could be many

files created and deleted within that month that may not be captured in the backup.

Consider an archive option with NetWorker, or other archive software such as

DiskXtender, if it is important to keep certain data for an extended period of time off the

production servers for archive purposes.

There is a difference between archiving data and archiving a backup of data.

Sometimes, you want to keep a certain backup for a longer period of time than the

normal rotation. For example, a common business requirement might be to keep the first

full backup of the month for 7 years, while keeping regular day to day backups for a

shorter period of time (i.e., 3 months). You can implement this strategy with NetWorker,

but be careful since there may be gaps in continuity between the archived backups.

Do not use backup to keep an archive of a particular file that has had its primary copy

intentionally removed from a system. Backup data is subject to retentions; the copy will

eventually expire from the backup environment and be overwritten. Archive is used to

copy or move data off primary storage to a secondary location and provide a method for

easily accessing that file if it is needed – without subjecting the copy to expiration. There

may be certain files or data that are not captured in an archived backup when a full

backup is archived for a longer period of time than the normal backups.

For example, if you are considering a strategy for a file server for a busy office where

thousands of users are creating, updating, and removing documents on a regular basis,


it is unlikely we could predict whether a critical file will exist on the system during a

monthly full backup. For example, if we have a file created on Tuesday of the 2nd week

of the month, and it is deleted the following week, it may not exist on the day of one of

the monthly backups.

However, there are occasions where we must keep or store data for different lengths of

time depending on when the backups took place or their frequency. A common

requirement is to perform a Daily backup and keep one retention period, perform a

Weekly backup and keep for a second retention, perform a Monthly backup and keep for

a different retention, and sometimes perform a Yearly backup for a completely different

retention. One strategy that you can use within NetWorker to accomplish this backup

configuration is to create four parallel settings of Groups, Schedules, Retentions, and

Pools for Daily, Weekly, Monthly, and Yearly requirements. Here is what a generic setup

may look like in NetWorker to accomplish this kind of requirement:

Daily Group => Daily Schedule => Daily pool => Daily Clone pool

Weekly Group => Weekly Schedule => Weekly pool => Weekly Clone pool

Monthly Group => Monthly Schedule => Monthly pool => Monthly Clone pool

Yearly Group => Yearly Schedule => Yearly pool => Yearly Clone pool

All the groups are set to run every day at a specified time, but the schedules are used to

determine if an actual backup will take place. For example, the Daily groups will have a

schedule that shows regular incremental backups during the weekdays with a ‘skip’

configured for one weekend day, and the days Monthly and Yearly backups are to run.

The Weekly backups would be configured as the weekend Full backups with ‘skip’

overrides to be configured the days the Daily, Monthly, Yearly backups will run. The

Monthly Schedule will have a ‘skip’ scheduled for all the days any Daily or Yearly

backups are set to run. The Yearly schedule will be set with a skip 364 days of the year

except for the one day the Yearly backup is set to run.

The idea is to have a complete picture but uses four different schedules. The chart

displays a 31 day month calendar with a Monthly Full level backup on the first Saturday

of the month and a Yearly level Full on the last Saturday of the month that could be sent


offsite. Following the Monthly and Yearly backups are Weekly Full backups the next day

that can be kept onsite for immediate recovery requests.

SUNDAY MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY SATURDAY 29 – Daily 30 – Daily 31 - Daily 1 - Daily 2 – Daily 3 - Daily 4 – Monthly

5 – Weekly 6 - Daily 7 - Daily 8 - Daily 9 – Daily 10 - Daily 11 - Weekly

12 - Daily 13 - Daily 14 - Daily 15 - Daily 16 – Daily 17 - Daily 18 - Weekly

19 - Daily 20 - Daily 21 - Daily 22 - Daily 23 – Daily 24 - Daily 25 – Yearly

26 - Weekly 27 - Daily 28 - Daily 29 - Daily 30 - Daily 31 - Daily

An advantage of configuring the schedules in this manner is that everything is configured

and managed directly within the NetWorker GUI. Fewer configured resources may be

needed in NetWorker if an external scheduler was used to accomplish the same

schedule. However, once these resources are initially configured, it will be easier to

meet ongoing changes in requirements for the backup environment and much easier to

hand the environment off from one administrator to another since everything is

configured in one place.

Planning for dependent save sets

Consider dependent save sets when a full backup is archived or stored out of sequence

from the other backup data. When performing a Level Full backup once at the beginning

of a week, the remaining incremental backups are dependent on the full. The

incremental backups are only changes that occurred on the save sets since the previous

backup was performed. You need to restore the full backup first, and restore the

following incremental backups afterwards to bring the data back as it looked on a

particular day of the week to restore an entire save set. If the full backup is not available,

it will be impossible to restore the entire save set back to a point later in the week since

any of the unchanged files were only kept in the full level backup.

If a full backup has been flagged as the monthly full offsite copy to maintain once it is

sent offsite, you would need to recall files from that day and also recover from any of the

following dependent backups as well. That is why in the previous chart example the

Monthly and Yearly full backups show a Weekly full on the following day.


Retention Policy

Organizations often need to keep different types of backup data for different lengths of

time. In NetWorker, a save set expires and is marked as recyclable when the Retention

policy expires. The expiration date is assigned to the save set during the backup and

comes from the Retention setting in the client resource. The expiration date is recorded

in the media database as the ssretent value.

It is a good practice to keep data with different retentions sorted to different pools. This

makes sense especially when backing up to tape or virtual tape because it is more likely

a number of save sets on the same tape will expire around the same time. The entire

tape can be expired and re-used by NetWorker once all the save sets expire. When

backing up to different media types, such as disk backups, virtual tape, or physical tape,

the pools can also be used to help direct a backup to the correct media type. The Device

attribute in the Pool resource can be used to restrict a pool to using certain devices or

you can select a chosen volume preference type.

How to use Directives effectively

Directives are used to direct the NetWorker save command that is used for file system

backups to perform special functions such as encrypting data, compressing data,

excluding files during a backup, etc. A common reason to use a directive is to exclude

files from a file system backup.

Directives can be applied in two ways, either in a file on the client or through a directive

resource in the NetWorker GUI. If applying a directive on the client, you will need to

place the directive statements in a .nsr file for UNIX/Linux clients and in a nsr.dir file for

Windows clients. However, creating a directive resource and applying it to a client

resource is a better way to manage the directives since they can all be viewed and

managed in one place – the NetWorker Management Console GUI. With the local

directive, it is easy to lose track of which systems have them and where they were

placed. With the directive resource, they are listed in one place in the GUI so they are

much easier to track and manage.


Excluding files from a backup

You may need to exclude one or more file types from a backup. Simply list only the file

systems, directories or files that you want to back up in the client resource save set

attribute. However, use the “All” keyword for the save set attribute when you want to

perform a backup of most everything from a client. Using “All” as the save set attribute

lets NetWorker discover the save sets on the client at the time of the backup. This is

helpful when you are performing backups of servers managed by other administrators

since any new file systems that are added to a client will be captured during the backup

without any additional configuration. However, there are situations when it is not

necessary to back up everything on a client. Directives help bridge the gap between

these two methods by using “skip” or “null” statements to exclude data from the backup.

Use a save set of “All” to back up all save sets and a directive applied to the client to

keep from backing up what is not wanted.

Use Null or use Skip?

Whether you use Null or Skip in a directive to exclude data from backup can make a

difference, even though they both will effectively exclude your data. The primary

difference has to do with the way the directories and files are recorded in the client file

indexes. When you decide to use skip in the directive it effectively records a kind of

‘blank’ or whitespace in the client file index in place of the directory or file being skipped

for that point in time. This means you will not see any record where that file or directory

would have existed.

When you use null it will not record anything in that place in the client file index. If a

particular file or directory was backed up previously, you would still see those previous

record entries, not from the current backup but from previous backups.

Perform backup and recovery testing to be sure you are getting the result you need from

either skip or null. Testing will help to understand the difference so that you can make an

informed decision about which is needed.


Using Groups, Schedules, and Directives for Application Backups

Creating client configurations for backing up application data is one very useful situation;

we can use many of these configuration strategies with Groups, Schedules, Directives,

and Pools. When performing a file system backup with NetWorker, the server will

request a client to run the savefs and save commands to perform the backup. The save

command is used to capture a copy of data and send it to a storage node. However, the

save command does not understand how to successfully capture data from a file system

that has data on it that may be locked or is changing during a backup – such as

application or database data. This is where the base NetWorker product will need help

from special commands that are installed and included with the NetWorker Modules.

The NetWorker Modules are installed on the clients and used to back up the applications

using special commands, shared application library files, and scripts. This will ensure

that NetWorker backs up the applications in a consistent state that can be successfully

restored. The NetWorker Modules are designed to ensure that each application or

database vendor recommendations are followed and any needed backup utilities or APIs

provided by the vendors are used. Although the NetWorker Modules do allow for level

Full and other granular levels of backups (for example, transaction logs, archived redo

logs, block-level incremental, etc.) they do not capture all the data from the systems.

There is data such as the operating system, configuration files, and other file data that is

not part of the application that may need to be backed up on these application servers.

You must create at least two client resources in NetWorker using the client hostname to

make sure all data is captured from a client. One client is used to back up the NetWorker

Module specific save sets and the other to perform file system backups of the client.

There may need to be even more client resources created if some backups need to

include special commands or schedules for incremental or transaction log type backups.

Needing multiple retentions for different types of data may be another reason to have

more than two client resources created for the same client since the retention policy is

configured in each client resource.


When creating the client to handle file system backups, use a save set value of “All” to

capture all local (and SAN) file systems for a client. However, using “All” alone will mean

that NetWorker will also capture any application data on the client. This may include

getting errors when NetWorker tries to open locked files on Windows servers or capture

application data that is active and changing on UNIX and Linux servers. To avoid this,

create and apply a Directive resource for the file system client that is used to exclude the

known locations of the application files that will be captured by the NetWorker Module

backup. Additionally, if backing up a Windows-based server, use a “save operations”

setting to exclude Microsoft Volume Shadow Copy Services (VSS) from backing up

applications such as MS-SQL and MS-Exchange if they are being backed up with a

NetWorker Module instead. The save operations attribute in the client resource can be

used to tell NetWorker to turn off the capability to request backups using VSS on

Windows 2003, 2008, and Vista systems. Only one method should be used for backing

up the applications when backing up MS-SQL and MS-Exchange so that all transaction

logs are found in the same place when performing a restore.

Cloning and staging

Overview of Cloning

Cloning makes additional copies of data that has already been previously backed up.

Cloning is a valuable function in NetWorker to help ensure data is protected on multiple

pieces of media. It is gaining even more importance to help with integrating backup to

disk and backup to tape technologies. Cloning helps to implement different tiers of disk

storage, virtual tape libraries, or physical tape and keeps backup copies on each of

these different media for different lengths of time. The introduction of specific clone

retention policies (clretent in media database) and the ability to set or extend the browse

policy of clones save sets has helped with managing tiered backup storage.


Backing up and Cloning with Disk

Disks are part of a backup strategy and in some instances they are the only media used

for storing backup data. There are different ways that disk storage can be configured for

NetWorker. For example, if using a SAN storage array, the disk is presented to the

NetWorker server or storage node as one or more file systems and then configured as

an advanced file type device (adv_file). Disk can also be presented as a CIFS or NFS

share to the NetWorker server or storage node and used as an advanced file type

device. This will allow network attached storage (NAS) devices or disk libraries with NAS

features (such as the Data Domain® deduplication appliances) to be a destination for

storing backup data. Another method for using disk with NetWorker is to use a virtual

tape library such as an EMC Disk Library (EDL) to be presented as if it were a tape

library. The Data Domain deduplication appliances can be presented as a tape library to

NetWorker with a virtual tape library option as well.

With all of these ways to use disk, the backup speeds will usually be quicker than what

can be achieved through traditional tape. Seek and find for recovery with disk storage is

definitely quicker than traditional linear tape due to the random-access nature of disk

storage. It is beneficial to have the original backup data reside on the disk for some

period of time in case a recovery is required. Typically, recovery requests are needed

shortly after the backup so having the backup data on disk allows a quick recovery.

However, you may have to keep some or all of the backup data recoverable for a longer

period of time than is practical or cost-efficient for keeping it on backup disk. Copying or

moving the data from the disk media to another media allows you to extend the length of

time the data is available for recovery. Cloning and staging operations in NetWorker can

be used to copy or move the primary backup data from disk to another media type like

physical tape.

Another reason for copying the data off of the original disk media is to have or maintain a

copy in case the original back copy is lost. The original copy could be lost due to

mechanical failure, data corruption, human error, backup software mis-configuration, site

disaster, etc. It is important to have the backup data stored on more than one piece of


media. NetWorker cloning is ideal for helping to protect against these possible failures

soon after the backup, while still allowing the original backup copy to be available in

case of any recovery requests.

Copy the original backup data to physical tape and then transport it if sending the data

offsite. If a method other than physically transporting tape is needed, NetWorker can

also clone the data across a LAN-WAN connection to a destination NetWorker storage

node at a remote location. Cloning is more ideal than staging in these situations since it

will allow the original data to be available onsite in the event a recovery is needed.

To free up space and make room for new backups, different copies of the save sets can

have different retentions. The backup copy on disk can have a retention (ssretent) to

expire the data sooner than the clone copies (clretent) that can have a longer retention.

For example, the original backup copies on disk can have a shorter retention, such as 7

days, while the clone copies on tape can have a longer retention, such as 1 year.

Scheduling cloning jobs

A cloning operation can be started in three ways:

1. through a setting in the Group resource

2. by running nsrclone from the command line

3. by manually selecting volumes or save sets in the NetWorker Management

Console GUI

A cloning operation reads an original backup of a save set and then write out the data

that is read to a destination device. Two devices are used for every clone operation that

is started; one for reading the original backup and one for creating the clone copy. The

number of devices available will affect your decision about when to begin cloning. For

example, if there are many devices available for backup and even more devices can be

added and dedicated for cloning, it might be alright to begin cloning data while a backup

is occurring. This scenario is possible when using Advanced File type devices or virtual

tape drives in a virtual tape library.


Cloning as part of a backup Group There is an attribute setting in the Group resource to enable clones and choose a

destination clone pool to begin a clone job automatically through the NetWorker

Administration GUI. When this option is enabled (set to “Yes”), after a Group completes,

all save sets that were backed up in that group will be cloned to the destination clone

pool. If there are Groups that are scheduled to start afterwards, they will be fighting for

device resources with the clone operation for the group. If we have many Groups

starting and ending in the same backup window, it can be difficult for NetWorker to find

enough devices to complete all operations in a reasonable time with Groups and Cloning

running at the same time. When a clone operation starts, it will use two devices that

could be used for backups and further delay the time the overall backup takes to

complete. It may make sense to let the groups start the cloning operations if there are

dedicated devices for cloning and the backup window is not impacted.

Start the clone operations after all the backups have ended if there are a limited number

of devices. Wait until all the Groups have ended before starting any cloning when using

physical tape devices for backup and cloning operations. Consider throughput and

storage node resources such as Fibre, iSCSI, CPU, memory, etc. and ensure that they

are sufficient to handle backups and cloning at the same time even if there are plenty of

devices to perform backup and cloning simultaneously.

Cloning using nsrclone

Start the GUI manually or using the nsrclone command from the command line to start

the cloning operation after all backups have completed. Using nsrclone in a .bat file on

Windows or a shell script on Unix or Linux is a good option since it allows the Windows

Task Scheduler or cron to start the clone scripts automatically.

The basic form of the nsrclone command is to specify the unique ssid of a save set that

needs to be cloned such as:

# nsrclone –S ssid

Where ssid is the save set identification number of the save set that needs to be cloned.


However, there are other variations of the command, and some possible switches. A

description of each follows.

Valid nsrclone option Description of option

-b pool_name Clone data to destination pool “pool_name”. When cloning data, the destination copy needs to be sent to a clone type pool.

-B pool_name Clone data from source pool “pool_name.”

-C number Clone only save sets with copies less than “number.”

-f file_name Clone only those save sets or volumes listed in the file “file_name”. Each entry for save sets to be the ssid listed each on separate lines.

- y retention_date Set clone retention (clretent) to the date specified as “retention_date” to be in acceptable nsr_getdate format.

-w browse_date Set clone retention (ssbrowse) to the date specified as “browse_date” to be in acceptable nsr_getdate format.

-n Do not actually execute the clone operation but instead list out all save sets that will be cloned.

-t start_query_time Set a start time for a query window from the media database where “start_query_time” is the beginning of a window of time to select save sets from for cloning specified in nsr_getdate format.

-e end_query_time Set an end time for a query window from the media database where “end_query_time” is the ending of a window of time to select save sets for cloning specified in nsr_getdate format.

-c client_name Clone only those save sets belonging to client with name “client_name”.

- g group_name Clone only those save sets that were backed up in the group of name “group_name”.

-F If included, will tell NetWorker to skip cloning invalid save sets.


Some of the options mentioned in the chart reference nsr_getdate(1m) format. The term

nsr_getdate is a reference man page on UNIX and can be found in the NetWorker

Command Reference Guide. The commands that use date formats in NetWorker

support the described date formats in nsr_getdate(1m). The acceptable date formats

include hours, days, weeks, months , years as well as MM/DD/YYY or “MM/DD/YYYY

HH:MM:SS”. When a period of time in the past needs to be specified, the word “ago”

can be attached to the end such as “1 week ago.”

Some examples and their descriptions:

nsr_getdate format Period of time description

3 days Three days into the future

3 months Three months into the future

1 quarter Three months into the future

1 year One year into the future

7 days ago 7 days or 168 hours into the past

9 months ago 9 months into the past

now The current time

today Midnight of the current day

yesterday Midnight of the previous day

7yesterday 7 times yesterday or 7 days into the past

Examples using nsrclone

This following section shows some full nsrclone command examples with switches,

sample pool names, and date formats. The browse policy would never be set to less

than the clone retention or the save set retention.

This first example shows the command to clone all save sets that were backed up in the

last week to a pool named “Default clone.” Keep the copies for one year and exclude

save sets that were already cloned.

nsrclone –b “Default clone” –y “1 year” –C 2 –t “7 days ago” –e “now”


This second example shows how to clone all save sets backed up in the last day from

the Daily Backup pool to the Daily Clone pool, that have not yet been cloned. It also

shows how to set a browse policy of three months and retention policy of two years.

nsrclone –B “Daily Backup” –b “Daily Clone” –w “3 months” –y “2 years” –C 2 –t “24

hours ago”

This third example shows how to clone all save sets backed up in the last two days from

the Nightly Group backup to the Daily Clone pool, that have not yet been cloned. It also

shows how to set a browse policy of one month and retention policy of one month.

nsrclone –g “Nightly Group” –b “Daily Clone” –b “1 month” –y “1 month” –C 2 –t “48

hours ago” –e “18 hours ago”

This last example shows how to clone all save sets that have not yet been cloned and

belong to client with hostname “database-client” to a destination pool named “Tape

pool.”

nsrclone –c “database-client” –b “Tape pool” –C 2

Retentions on Cloned Save Sets

You can specify retention for a clone copy of a save set in a command option of nsrclone

or in the Pool resource for a clone pool. When you specify retention in the pool resource

for a clone pool, it will automatically be applied to all save sets copied to that pool.

Browse and retention times for clone save sets can also be shortened or extended using

the nsrmm command with the –w option to set the browse expiration time and the –e

option to set the retention. Values of time can be specified in nsr_getdate format. If a

specific clone instance of a save set is to be modified, after the –S option at the end of

the nsrmm command, the save set id and clone id will both need to be specified.


For example,

nsrmm –w “3 months” –e “1 year” –S 12345678/98765432

where 12345678 is the save set id number of the save set to be modified and the

98765432 is the clone id of that instance of the save set to be modified. This example

command would update the browse policy to three months and the retention one year

from the current date and time.

Using nsrclone with mminfo

You can also use the nsrclone command in conjunction with mminfo. Mminfo is a

command used to query the media database and report back information. The long form

of the command uses a “-q” option for the query constraint and the “-r” option to report

information to the display where the command is being run.

Use the mminfo command on any NetWorker server to generate a list of save sets as

ssid numbers and have the output redirected to a file. Then, use the nsrclone command

with the “-f” option to read back the file mminfo created to clone all save sets listed in the

file. Various options can be added or removed from the mminfo queries to get specific

save set lists in different files, and have multiple nsrclone commands started at one time

to make use of multiple pairs of devices for cloning. The overall cloning operations will

complete faster by having multiple cloning operations running at once.

You can combine the mminfo and nsrclone commands in a .bat or script file with some

wrapper commands and an external scheduler like Windows Task Scheduler or cron to

automate the cloning operation. An example mminfo and nsrclone command

combination is shown here.

mminfo -q “savetime>=”date/time”,copies<2,!incomplete' -r -ssid > mminfo.txt

nsrclone -b Default -S -f mminfo.txt

The example mminfo command will build a list of save sets that were backed up after

“date/time” (date/time to be replaced by a valid time specified in nsr_getdate format),

have not yet been cloned (copies<2), and are completed save sets (for example,

exclude save sets that are aborted or not yet complete) and write this list to the


mminfo.txt file. The nsrclone command will read the list of save set ids from the

mminfo.txt file and begin cloning them sequentially.

Cloning by Volume

You can start clone operations by selecting volumes in the GUI or volume names from

the command line. When starting a clone by volume, NetWorker will look at a volume

and build a list of save sets that begin on that volume or are completely contained on

that volume and clone all of those save sets. Since it is building a list of save sets to

clone anyways, it is usually better to focus on one of the methods that allow you to select

save sets on criteria related to the importance of the data rather than the pieces of

media. There are more variations of switches that you can use with the mminfo and

nsrclone commands to include or exclude save sets that make it more convenient to

focus on selecting data to a clone based on save sets rather than volume names.

Recovering from a Cloned Save Set

If you would like NetWorker to recover from a clone copy of a save set and the original

still exists (it hasn’t expired or been overwritten), the original save set or volume can be

marked as ‘suspect.’ That will force NetWorker to skip over it when performing the

recovery and look for a clone copy of that same save set. You can mark volumes

‘suspect’ easily in the Volumes list in the NetWorker GUI Media section. Both volumes

and save sets can be changed to a status of ‘suspect’ by using the nsrmm command on

the NetWorker server and using the lower-case O option (nsrmm –o).

Overview of Staging

Staging moves backup data from one media type to another. It can be used in

conjunction with adv_file media type (local disk, SAN disk, or CIFS/NFS). Move the

backup data to other media once certain criteria are met. When a staging process is

started, NetWorker uses cloning to copy the save set to a new location and then the

original copy is removed and moved to the new location. Like cloning, staging will

require at least two devices. One is used to read the original backup and the second to

create a copy at the new location. With staging, NetWorker can move the backup data to

any type of pool, unlike cloning that requires a clone pool as a destination.


Staging data can be accomplished by using two different approaches, either through a

Staging resource in the NetWorker GUI or through the nsrstage command from the

command line.

Staging Resource

Confiugre a Staging resource in the NetWorker GUI if you are planning to stage from a

file type device (file) or advanced file type device (adv_file). The resource is used to

monitor the directory where the file or adv_file device exists and to automatically move

data to another destination. You can configure the staging resource to use criteria such

as amount of data written or length of time data has been stored to decide when to move

data to the new destination as configured through the “Destination pool“ attribute. There

are some attribute settings in the Staging resource labeled as “High water mark” and

“Low water mark” that are used to decide when to begin a staging operation based on

data storage. If the disk is more than the percentage specified in the “High water mark”

value when a “File system check” is performed, NetWorker will kick off the staging

operation to begin moving save sets to free up space for new backup data. This

operation will run until the “Low water mark” setting is reached. If using an advanced file

type device, the staging operation can occur for an existing save set even if a backup of

a newer save set is still running to the device.

The “Max storage period” attribute in the staging resource is used to specify when to

move data off the device based on a length of time. Once a save set has been on the file

device longer than the “Max storage period,” NetWorker will move it to the selected

“Destination pool.” This will free up space for new backups and can be started

automatically once a save set has passed the Max storage period and the “Recover

space interval” check has been performed. Backup data that has an expired retention

policy will be removed when the “Recover space interval” check runs and this will also

free up space for new backups.

Using nsrstage command

You will start the staging operation from the command line or script using the nsrstage

command if you are using staging to move data from virtual tape to physical tape or from


physical tape to physical tape. Staging can also be useful if you need to move one or

more save sets that may have been written to an incorrect piece of media.

Other ways to schedule scripts in NetWorker

There are some other unique settings in NetWorker that can be used to call scripts for

scheduling. The “Backup command” attribute in a client resource is one. Any script that

is called from here must have its name begin with nsr or save (for example,

nsr_custom_script) and will be run instead of the save command.

The savpnpc command can be used to call a pre-script before the backup and a post-

script after the backup. The script name to be called is configured in the groupname.res

file on the client in the /nsr/res directory, wherever it is located.

The owner notification in the client resource and any of the Notification resources are

other places that NetWorker calls commands or scripts that can be customized. The

owner notification command will be called after a client is backed up. It is typically used

to send an e-mail alert of what was backed up on a client. You can also customize the

Notification resources to call needed commands such as sending e-mail alerts or other

custom commands or scripts.

Conclusion

This article provides some insight on different ways to schedule operations in NetWorker

including strategies for configuration in the NetWorker Management Console GUI as well

as different ways to use commands and call commands that are in scripts. The goal of

this article was to document methods that are not as clearly documented in the typical

NetWorker references, and provide some insight for NetWorker administrators who may

be new to the product. It is also useful for experienced NetWorker administrators who

may be rethinking their backup strategy.

scheduling operations in networker - dell · so if the requirement was to perform a backup every 2...

Documents