Archive for December 2013

14 December 2013

Tivoli Storage Manager (TSM) 7.1 known issues and problems in UNIX and LINUX TSM Clients

09:49 No Comments
IBM has released its next and new version of Tivoli Storage Manager which is IBM TSM V7.1 for their customers from 13th December 2013. This TSM V7.1 can be downloaded from Passport Advantage or FTP sites if you are a IBM customer. Nevertheless, how many new versions with new fixes/patches are released for any kind of software it still consists some bugs and issues, so does TSM V7.1. IBM has shared some of the known issues in this new TSM version 7.1. In this post we will see some of the known issues observed in new TSM Version 7.1 Backup-Archive Clients on UNIX and LINUX platforms. Go to this link for known TSM V7.1 issues in Windows clients.

Limitations and Problems in IBM TSM V7.1 on all UNIX & LINUX Platforms

1) Files that have been backed up or archived with the V7.1 Tivoli Storage Manager API cannot be restored or retrieved with a previous version API, regardless of the server level. The data must be restored or retrieved by a V7.1 level or higher API.

2) A writable log file is required for backup and restore operations. The DSMI_LOG environment variable or the dsm.sys file (on UNIX and Linux clients) or the dsm.opt file (on Windows clients) errorlogname option must be set to a path that the user of the application can write to. If the user performing the backup or restore does not have write permissions to the log file, the process will fail. The TSM API return code for this situation is 106 (Access to the specified file or directory is denied). 

3) If you back up or archive data from a Tivoli Storage Manager Version 7.1 client, you cannot restore or retrieve that data using a Tivoli Storage Manager Version 6.4 or earlier client. 

4) When trying to take journal based backup with -absolute option it shows the below error

ANS7559E The absolute option requires specifying the NoJournal option when performing a Journal Based Backup for backing up fs 

The backup works but all files are not backed up. To fix this problem, add the -nojournal option to the command.

5) When Journal Based Backup (JBB) is configured and running during TSM client upgrade, the first backup after the upgrade will be progressive incremental, i.e. it will not use the journal. Incremental backup can take more time comparing to JBB. To prevent this stop the journal daemon before upgrading the client. Restart the daemon when the upgrade is complete.

6) When you are trying to do a client node open registration with the passwordaccess generate is set in the client option file and if an invalid password is given at the first attempt, the second attempt will also fail even if you give the correct password. You have to close the session and retry the open registration again.

7) If you are using TSM client 7.1 GUI for the initial configuration, the configuration wizard may reports a protocol violation. In addition to that the Domain panel will not list the file systems to select for the domain option. You can ignore these errors and continue with the configuration. Once you successfully completes the initial configuration, you can use the preference editor to set the Domain option.

8) Most of us generally use CTRL-C frequently to abort or cancel the command, but in TSM 7.1 it will give some serious problems. pressing CTRL-C during command line client operation might result in a TSM client program exception or other unexpected behavior. You can use 'Q' key instead of CTRL-C to abort any command. Stop using CTRL-C from IBM TSM 7.1.

9) If the specified maximum error log file size is greater than the available free space on the specified file system and the log is being transitioned from a non-wrapped log to a wrapped log, the following error message will be issued

ANS1521E Failure writing to a Tivoli Storage Manager log or log-related file: <LOG FILE NAME>, errno = 28, There is not enough space in the file system

This is correct behavior. However, the log header record might be incomplete or there may be no "END OF DATA" text marker at the end of the error log. After space has been made available, the TSM client will subsequently treat the log as unwrapped because a valid header record is not found. A new log will be created and this partial log will be written to the prune file. If there is insufficient space in the file system to append an entry to the log, the TSM client will continue to run, but the error message will not be logged.

10) When defining a client schedule in the TSM Server, you should use single quotations ('') and double quotations (''") correctly for OBJECTS & OPTIONS parameter if the file specifications contains blank spaces. For example

objects=’"c:\Users\user1\Documents\Some file.doc" "c:\Users\user5\Documents\ Another file.txt" c:\Users\user3\Documents\noblanks.txt’ 

objects=’"c:\home\proj1\Some file.doc"’

options='-preschedulecmd="c:\home\me\my files\bin\myscript" -postschedulecmd="c:\home\me\my files\bin\mypostscript" -quiet'

11) Data that has been backed up from Mac OS versions prior to Mac OS X will not have the correct file owner or permissions when restored on Mac OS X. After the restore is complete, use the "sudo chown" and "sudo chmod" commands to set them.

12) Files backed up or archived by the z/OS Unix System Services client cannot be restored or retrieved by any other TSM Unix client. Files backed up or archived by any other TSM Unix client cannot be restored or retrieved by the z/OS Unix System Services client.

13) Files backed up or archived with DES-56 encryption on Intel platforms such as Linux86 cannot be restored or retrieved on non-Intel platforms (for example, AIX), and vice versa. AES-128 is the recommended encryption method.

14) If you want to restore or retrieve to a destination path that contains a symbolic link to a directory, use the "FOLLOWSYMBOLIC Yes" option setting either during the operation or in your client options file (dsm.opt). This setting allows you to restore or retrieve the original directory tree underneath the destination path. Otherwise, you will get the ANS4029E error during restore or retrieve.

15) When you select an item in the filelist tree of Backup window, the drop-down box for backup type will be permanently disabled. To re-enable this, exit the Backup window and reload the window. Disable Java caching from the Java Control Panel to help improve performance on the Web client on the machine running the browser. 

16) Client-side data deduplication might require additional memory during backup or archive processing. You might need to increase the system limit on the size of the data area (segment). To check current setting of the limit, run the following command

    ulimit -d

To update the limit, run the following command: 

    ulimit -d <new value>
or 

    ulimit -d unlimited

17) In AIX, If you use the memoryefficient=diskcachemethod option for full incremental backup, you might need large amounts of disk space. Ensure that large file support is enabled on the file system that is being used for the disk cache file.

18) In AIX, When running ssh localhost dsmc command, the TSM client uses the server stanza defined in the dsm.opt file in the default installation directory, /usr/tivoli/tsm/client/ba/bin64 for the 64-bit client, ignoring the environment variable DSM_CONFIG, if it is set. Ensure that the dsm.opt file in the default installation directory has the correct server stanza when you invoke dsmc with ssh.

19) In AIX, If the COMMMethod or LANFREECommmethod options are set to SHAREdmem, the AIX environment variable EXTSHM needs to be turned on to enable extended shared memory. In order to turn on EXTSHM setting, do one of the following

To make this a permanent change, add the following line to the /etc/environment file: 
EXTSHM=ON
To make the change in the immediate shell, enter: 
EXTSHM=ON
export EXTSHM
 This change is effective until you log out of this shell. 

20) In HP-UX, Raw logical volume backup does not support devices other than logical volumes, such as /dev/dsk/c0t0d1. Logical volume devices usually take the form /dev/vgXY/lvolAB.

21) In HP-UX , When doing image backup directly to tape, the RESOURCEUTILIZATION option value cannot exceed the value of the MAXNUMMP on the server for that node. If it does,the backup can fail with an Unknown System Error message.

22) In HP-UX , During image backup and restore over LANFree, pressing Ctrl-C can lead to a core dump of the client. Such operations must be avoided during image backup or restore.

23) In LINUX, When restoring to NFS-mounted file systems, remount them using the mount option "noac" to prevent NFS-related errors that are not detectable by the client. The performance can be raised for NFS by using larger values for the options rsize and wsize (for example 8192).

24) In LINUX, When restoring an image with XFS file system to another location, you might see the message appear on the screen

"ANS1066E The restore operation completed successfully, but the file system could not be remounted"  

The mount of XFS file system is not possible because by default XFS does not allow the mount of several XFS file systems with the same FS UUID simultaneously. Use the XFS mount option "nouuid" to mount XFS file systems with the same UUID or manually change FS UUID using xfs_admin utility.

25) In LINUX, If the host name shown with the hostname command is not specified in the /etc/hosts file, there will be no GUID support available. If you want to have GUID support, you must add the host name into the /etc/hosts file.

26) In LINUX X86, If you want to start the TSM scheduler using inittab, ensure that the required language environment is set. You can do this by starting a script that first sets the environment and then starts the scheduler. See also the section that handles SBCS in this file.

27) In Solaris, If the DISABLENQR YES (Classic Restore) option is set, the restore performance of the Solaris client is slower when compared to previous releases. Depending on how much load you put on the system, the restore may take up to 25% more time.

28) For image backup, the file systems are unmounted and re-mounted read-only if you use COPYSERIALIZATION=STATIC in order to prohibit access during the backup. However, you might still be able to change the size of the file system by using a volume manager for all values of copy serialization. Such operations must be avoided during image backup.

Tivoli Storage Manager (TSM) 7.1 known issues and problems in Windows clients

09:19 No Comments
IBM has released its next and new version of Tivoli Storage Manager which is IBM TSM Version 7.1 for their customers from 13th December 2013. This TSM V7.1 can be downloaded from Passport Advantage or FTP sites if you are a IBM customer. Nevertheless, how many new versions with new fixes/patches are released for any kind of software it still consists some bugs and issues, so does TSM V7.1. IBM has shared some of the known issues in this new TSM version 7.1 and some fix-packs APARs to the TSM previous versions recently. In this post we will see some of the known issues observed in new TSM V7.1 Backup-Archive Clients in Windows. Go to this link for known TSM V7.1 issues in UNIX and LINUX BA clients.

Limitations and problems with IBM TSM 7.1 on Windows platforms

1) If you backup or archive data by using Tivoli Storage Manager Version 7.1 client, you cannot restore or retrieve that data using TSM V 6.4 or earlier TSM clients. Check the TSM V 7.1 client compatibility with your existing TSM Server before installing and configuring it. For more information on TSM 7.1 clients and TSM servers compatibility click this link.

2) When trying to take journal based backup with -absolute option it shows the below error 

ANS7559E The absolute option requires specifying the NoJournal option when performing a Journal Based Backup for backing up fs 

The backup works but all files are not backed up. To fix this problem, add the -nojournal option to the command.

3) When you are trying to do a client node open registration with the passwordaccess generate is set in the client option file and if an invalid password is given at the first attempt, the second attempt will also fail even if you give the correct password. You have to close the session and retry the open registration again.

4) If you are using TSM client 7.1 GUI for the initial configuration, the configuration wizard may reports a protocol violation. In addition to that the Domain panel will not list the file systems to select for the domain option. You can ignore these errors and continue with the configuration. Once you successfully completes the initial configuration, you can use the preference editor to set or edit the Domain option.

5) Most of us generally use CTRL-C frequently to abort or cancel the command, but in TSM 7.1 it will give some serious problems. Pressing CTRL-C during command line client operation might result in a TSM client program exception or other unexpected behavior. You can use 'Q' key instead of CTRL-C to abort any command. Stop using CTRL-C from IBM TSM version 7.1.

6) If any log file (dsmerror.log, dsmsched.log, or dsmwebcl.log) runs out of space during a session, writing to that log ceases, but other processing continues. End of processing return codes will reflect all errors and conditions, not just those we were able to log. To prevent this problem, set the ERRORLOGMAXSIZE (for dsmerror.log) or SCHEDLOGMAXSIZE (for dsmsched.log and dsmwebcl.log) options to limit the log size to available space.

7) If the specified maximum error log file size is greater than the available free space on the specified file system and the log is being transitioned from a non-wrapped log to a wrapped log, the following error message will be issued

ANS1521E Failure writing to a Tivoli Storage Manager log or log-related file: <LOG FILE NAME>, errno = 28, There is not enough space in the file system

This is correct behavior. However, the log header record might be incomplete or there may be no "END OF DATA" text marker at the end of the error log. After space has been made available, the TSM client will subsequently treat the log as unwrapped because a valid header record is not found. A new log will be created and this partial log will be written to the prune file. If there is insufficient space in the file system to append an entry to the log, the TSM client will continue to run, but the error message will not be logged.

8) If you try to cancel or abort a command line (encrypted enabled) backup or archive operation when prompted for an encryption key, the whole operation will stops immediately with RC 12 even if there are other files eligible for backup which do not require encryption. So always know the encryption password before starting an encrypted command line backup or archive.

9) When defining a client schedule in the TSM Server, you should use single quotations ('') and double quotations (''") correctly for OBJECTS & OPTIONS parameter if the file specifications contains blank spaces. For example

objects=’"c:\Users\user1\Documents\Some file.doc" "c:\Users\user5\Documents\ Another file.txt" c:\Users\user3\Documents\noblanks.txt’ 

objects=’"c:\home\proj1\Some file.doc"’

options='-preschedulecmd="c:\home\me\my files\bin\myscript" -postschedulecmd="c:\home\me\my files\bin\mypostscript" -quiet'


10) Windows 2008 must be restored using Windows PE 2.0. Windows 7 and Windows 2008 R2 require Windows PE 3.0 for ASR restore.

11) When running an incremental backup with a UNC file specification, omit the wildcard "*" at the end of the specification. Otherwise the root directory will not be backed up.

12) VSS image backup might fail after a VSS image restore on small volumes (about 500 MB). On Windows Vista and later operating systems, VSS stores snapshot data on the volume on which the snapshot is being taken. This data located in the "System Volume Information" directory. On a 500 MB disk it takes about 350 MB to create the snapshot. TSM backs up only used blocks during the image backup. The data internal to VSS is also saved as part of the image. During an image restore, TSM recovers all image data. In this case, a second VSS image backup is not possible because of the lack of free space on the hard disk. VSS cannot create a snapshot for TSM due to the lack of space.

13) When installing the TSM scheduler service while the TSM server is down or otherwise unreachable, the node password will not be stored on the client computer. Without a stored password, the scheduler service cannot connect to the TSM server. To resolve this, after the TSM server becomes available, use the command line or GUI client to connect to the server, and enter the password when prompted. This will cause the password to be stored on the client computer, allowing the scheduler service to authenticate with the server. This problem might also occur when you are setting up a scheduler running through a firewall using the server-initiated sessions. In this case, start the scheduler from the command line (dsmc schedule) and enter the node's password when prompted. After the password is updated, stop the command line scheduler (press the 'q' key twice), then restart the scheduler service.

14) The Windows GUI Configuration Wizard for creating a service does not properly set up a service when the user selects the option to run the service under a different user account. The issue is that on Windows, the user that runs the service needs to be granted the "Logon as a service" security right and that is not being setup. You can avoid this problem by going into the MMC services panel and re-entering the users password, after which Windows automatically grants the required right to that user.

15) There is a Synchronization problem between 32-bit and 64-bit client node passwords. Windows 32-bit applications have a different HKEY_LOCAL_MACHINE\SOFTWARE registry tree from 64-bit applications. This means that the 32-bit and 64-bit TSM API are mutually exclusive and their passwords will be stored in different locations in the registry tree. Thus the 32-bit API will not be able to access the password entries generated by the 64-bit API. For example

The 64-bit BA client package will only install a 64-bit scheduler. But another application could be using the 32-bit API. If the scheduler and application share the same node name, then the generated passwords stored in the registry in encrypted form could become out of sync. If the application changes the password, the 32-bit portion of the registry will be updated with the new password, but the 64-bit portion of the registry will not reflect this change. To alleviate password synchronization issues between 32-bit and 64-bit Windows clients, use a separate node name for each client.

Monitoring and Managing Tivoli Storage Manager (TSM) Expire Inventory Process

07:48 No Comments
Tivoli Storage Manager Expire Inventory command might take several minutes to hours some times which results in slowness of the TSM Server. This process will removes client backup and archive file copies from server storage based on policy specified in the backup and archive copy groups of the management classes to which the files are bound. This command should be scheduled to run only when the TSM server is not busy. There are several ways to minimize the processing time of this EXPIRE INVENTORY command to make sure the TSM Server runs smoothly. With SQL queries you can actually know how many files are eligible for expiration at a particular time and also you can use NODE, DOMAIN, TYPE, DURATION, AND RESOURCE parameters along with this command according to your convenience. 

Managing IBM Tivoli Storage Manager EXPIRE INVENTORY Process

Some important tips to monitor or manager TSM Expire Inventory Process

1) It is always a best method to stop automatic TSM Expire Inventory Process everytime any client files are eligible for expiration. set EXPINTERVAL server option to zero to stop automatic expiration and define an admin schedule at times when the TSM server is less busy.
Setopt expinterval 0

2) You can kill a running EXPIRE INVENTORY process by using CANCEL PROCESS command and you can also start it manually by using EXPIRE INV command.

cancel pr 451

3) To run expiration only for one hour, use this command

expire inventory duration=60

4) If you want to know how many files are eligible for expiration, run the following sql command.

select activity,cast ((end_time) as date) as "Date",(examined/cast ((end_time-start_time) seconds as decimal (18,13))*3600) "Obj/Hr" from summary where activity='EXPIRATION' and days (end_time) - days (start_time) = 0

Output:

ACTIVITY                     Date                                       Obj/Hr
------------------                 ----------                       ---------------------------------
EXPIRATION             2013-11-14                               2829600
EXPIRATION             2013-11-14                               1224000
EXPIRATION             2013-11-15                               2984400

5) If you dont want to expire the directories but expire all the files in it, use the following command.

expire inventory skipdirs=yes

This will prevent deletion of directories, and expiration processing can occur more quickly

6) If you want to expire only particular nodes data or all nodes data in a domain, use

expire inventory node=nodename
expire inventory domain=domainname

7) If you want to expire inventory of all the archive data of all nodes in a particular domain, use 

expire inventory domain=domainname type=archive

8) When expiring multiple nodes data you can set parallel expire inventory threads to minimize the processing time by using RESOURCE parameter. Multiple parallel threads will run by the server within the single expiration process (expiration still runs as a single process)

expire inventory node=node1,node2,node3 resource=3

The above expiration processing will start 3 threads within a single EXPIRE INVENTORY process.

5 December 2013

What is IBM AIX (Advance Interactive eXecutive) Operating System ?

04:05 No Comments
A computer consists of many hardware devices that the users of a computer system want to use. For example, they want to print documents or they want to play a game from a CD-ROM. To control these hardware devices and to share them between multiple users an operating system must be loaded during the system startup.

AIX Operating System Basics:

In the case of the AIX operating system, there is one special program which interfaces directly to the hardware devices - the AIX Kernel. The Kernel controls the access to the devices. On the other hand the users start different programs, for example, a program that prints a document or removes a file. These programs that run in AIX processes are also controlled by the AIX Kernel.

 To say it in easy words: The AIX Kernel is the heart of your operating system.

Working on AIX:

AIX is a multi-user system. Before a user can work with AIX, an authentication process takes place. The user must log in with his username and password. After a successful authentication AIX starts a certain program for the user, a shell. The shell is a command interpreter that waits for input and executes the commands and programs the user types in. As you will learn in this course, the shell is not only a command
interpreter - it offers great flexibility.

Multiple users can work at the same time on an AIX system or in a network. One of the basic tasks in your daily work is to communicate with other users on a system or in the network. In this course you will learn different commands that allow communication with other users.

 AIX offers a wide range of tools and commands. To get help about these commands, AIX offers different possibilities, for example, the man command or the AIX Online.

One of the major tasks of a computer system is to read and write data. In order to do this AIX uses a hierarchical file tree that consists of directories, subdirectories and files. The top level directory is called the root (/) directory that has many subdirectories. Each of these subdirectories can contain files or other subdirectories. Compare a directory with a document folder in which you put certain documents.

 The file tree is mounted during the system startup. AIX supports different file system types, which are all mounted to one big file tree. This is shown on the visual. Parts of this file tree reside on a disk, other parts may reside on a CD-ROM or are mounted from another computer in a network.

 This course explains how to work with directories and files on a user level. You will learn how to navigate in the file tree and how to manage directories. You will learn how to copy, move, delete and print files, how to edit files using the common UNIX editor vi. Another topic will be how to specify correct file permissions.

The SHELL: User Interface to AIX

When you log in successfully to an AIX system a special program is started for you - the shell. The shell waits for input and executes the commands and programs you type in. In other words the shell is a command interpreter.

 The shell offers many features (like wildcards to match filenames, variables, command line editing) that help the user in his daily work. We will discuss all these features in this course. The shell is customizable, that means the user interface may be tailored according to the requirements of each user. Customizing the user environment is another topic in this course.

 The shell offers different ways to control processes. In this course we explain how a user can control his processes. Besides all these properties the shell is a programming language. You can write shell scripts to create and tailor commands. Writing simple shell scripts is one additional topic in this course.

AIX Utilities:

The components that you use on AIX are files and directories. To work with these components AIX offers a wide range of utilities:
  • The find command to search for specific files.
  • The grep command to search for patterns in files. 
  • Commands to compare files and directories. 
  • Commands to compress and uncompress files to save disk space.
Note that this list is not complete. Besides these utilities the course introduces additional tools that are useful for your work.

AIX Graphical User Interfaces:

Modern operating systems are based on graphical desktops. These desktops consist of multiple Windows where you can start different applications, icons that are minimized windows to manage the screen space, and further controls.

 To execute certain actions on the desktop, you have to use the mouse attached to the system. AIX offers two different graphical user interfaces:
  • AIXwindows
  • Common Desktop Environment (CDE)
Using and customizing these desktops are major topics in this course. If you install the AIX Toolbox for Linux applications, two more graphical user interfaces are supported. These are the KDE and Gnome desktops.

Summary

  • The AIX KERNEL inerfaces to hardware devices and controlls processes running in the AIX system.
  • The user's inerface to AIX is the SHELL. The shell is a command interpreter that offers a great flexibility.
  • To store data AIX uses a hierarchial FILE TREE that consists of FILES and DIRECTORIES.
  • AIX offers wide range of utilities.

4 December 2013

IBM TDP for SQL VSS backup basic introduction and features

00:23 No Comments
The IBM Tivoli Data Protection for Microsoft SQL also known as TDP SQL can be used to take two kinds of SQL database backups - Legacy Backups & Volume Shadow copy Services (VSS) Backup. In this post we will see what is TDP SQL VSS backup and its advantages. VSS backup uses Microsoft Volume Shadow Copy Service technology to produce an online snapshot (point-in-time consistent copy) of SQL data.
 

TDP SQL VSS Backup Introduction

VSS is a set of Microsoft COM APIs that implement a storage management architecture that enables volume-level snapshots to be performed while the applications that contain data on those volumes remain online and continue to write. This architecture can be used for backing up your applications by creating point-in-time snapshots of your data. VSS provides the backup infrastructure as well as the mechanism for creating consistent point-in-time copies of data known as shadow copies. VSS produces consistent shadow copies by coordinating with business applications, file-system services, backup applications, fast-recovery solutions, and storage hardware. VSS Backups can be stored on Tivoli Storage Manager server storage or local VSS shadow volumes. Both of these storage destinations require that sufficient space be available for the snapshot. VSS Backups stored locally on VSS shadow volumes are directly accessible by the SQL system as long as sufficient space is available for the snapshot.

These types of local VSS Backups are faster for a couple of reasons. They are faster because the way the snapshots are managed and because the data is not placed into Tivoli Storage Manager server storage. Restoring these backups is also fast because the SQL data is not transferred from Tivoli Storage Manager server storage over the network.

Requirements for TDP SQL VSS Backups

For local VSS Backups you must have a licensed version of IBM Tivoli Storage FlashCopy Manager or IBM Tivoli Storage Manager for Copy Services installed on your system.

When performing VSS Backups and moving data to Tivoli Storage Manager server storage, sufficient space on local snapshot volumes is still temporarily required to hold the snapshot. For SQL data backed up to Tivoli Storage Manager server storage, the SQL data on the snapshot volume is sent to the Tivoli Storage Manager server. After the data transfer to the server is complete, the snapshot volume is made available for reuse. If you are storing VSS Backups locally and the maximum number of local backup versions to be maintained (as specified by the Tivoli Storage Manager policy) is reached, the oldest backup version is expired to create the snapshot for the backup to Tivoli Storage Manager server storage.

For SQL data backed up to local VSS shadow volumes, the snapshot backup resides on the shadow copy volume. For SQL data backed up to both destinations, a local snapshot backup is performed and the SQL data on the local snapshot volume is sent to the Tivoli Storage Manager server. The local snapshot volume is retained as a local backup.

The Tivoli Storage Manager backup-archive client serves as the VSS requestor that communicates with VSS to access the SQL data to create shadow copies of SQL databases. Data Protection for SQL serves as a front end for VSS Backup operations. Because of the role that the backup-archive client performs as the VSS requestor, features such as LAN-free backup, client-side deduplication, data encryption, and data compression require that options related to these features be specified in the backup-archive client options file for VSS operations. With the IBM Tivoli Storage Manager for Copy Services Microsoft SQL VSS Integration Module, you can manage local persistent snapshots and perform offloaded VSS backups.

Volume Shadow copy Services Overview

The VSS service manages and directs three VSS software applications that are used during VSS operations. These are the VSS software applications that the VSS Service manages and directs

VSS writer
The VSS writer for the Microsoft SQL Server is the SqlServerWriter. The SqlServerWriter is provided by the SQL Server VSS Writer service.
 
VSS requestor
This application initiates a snapshot operation. The application sends a command to the VSS service to create a shadow copy of a specified volume. The VSS requestor is the Tivoli Storage Manager backup-archive client.
VSS provider
This application produces the shadow copy and also manages the volumes where the SQL data resides. A provider can be a system provider (such as the one included with the Microsoft Windows operating system), a software provider, or a hardware provider (such as one that is included with a storage system). VSS hardware providers require installation and configuration, including the installation of all required fix packages.

VSS system provider overview
A VSS system provider assists with creating and maintaining copies on local shadow volumes. The VSS system provider refers to the default VSS provider that is available with Windows Server. If you are using the Windows VSS system provider, no configuration is required. However, you can make some configuration changes using the VSSADMIN commands.
 
VSS software or hardware provider overview
A software or hardware provider acts as an interface during VSS processing at the software or hardware level, respectively. If you use a software or hardware provider, consider the following requirements when planning for VSS Backups
  • Place database files for each database or group of databases that are to be backed up and restored together as a unit on a separate and dedicated logical volume.
  • Place logs for each database on a separate logical volume.
  • Do not place non-SQL data on storage volumes that are dedicated to SQL data.
  • When using hardware snapshot providers, do not share storage group LUNs with other databases or applications.
  • Make sure to read and follow specific installation and configuration instructions in the documentation provided by your VSS provider vendor.
  • If you use XIV, you must install and configure IBM XIV Provider for Microsoft Windows Volume Shadow Copy Service (xProv) Version 2.3.0 or later.
  • If a hardware provider is used, configure the disks that store SQL data and log files as basic disks.
 

Features of TDP SQL VSS Backup

Some VSS Backup characteristics are different from Legacy backup characteristics. Examples of these differences are the backup characteristics for types supported, the backup granularity, and the backup storage location options. The following characteristics are true of VSS Backups
  • Full backups are supported. The copy-only full backup type is supported for VSS and Legacy backups. Log, differential, file, group, and set backup types are not supported. Legacy differential and Legacy log backups can be applied after a full VSS Backup is restored.
  • Backup granularity is at the database level only.
  • Backups are managed through Tivoli Storage Manager server policy.
  • Backups can be stored on local shadow volumes, Tivoli Storage Manager server storage, or both locations.
  • Different policy settings can be defined for each storage location, backup method, and backup type (FULL or COPY).
  • Backups to Tivoli Storage Manager server storage can be offloaded to an alternate machine as resource relief for production servers.
  • Backups can be run in a Microsoft Windows Failover Clustering or Veritas Cluster Server (VCS) environments.
 
VSS Backup Planning Requirements
Plan a VSS Backup strategy to optimize your backup operations performance and avoid potential problems. Consider the following requirements when planning for VSS Backups
  • When running VSS operations, make sure you have at least 200 MB of free disk space on your Windows System Drive. This space is used to hold the metadata files for Data Protection for SQL.
  • Continue to schedule and perform Legacy backups in your strategy.
  • Make sure you have a well-defined and tested recovery plan that meets your service level objectives.
  • Use basic disks.
  • If you plan to keep some VSS snapshot backups on local shadow volumes only, make sure to consider the VSS provider-specific implementation and configuration options when setting up your strategy. For example, if your VSS hardware provider supports a full-copy snapshot versus a copy-on-write (COW) snapshot mechanism, be aware that full-copy type implementations have greater disk storage requirements but are less risky because they do not rely on the original volume to restore the data. COW implementations require much less disk storage but rely completely on the original volume to perform a restore. Since these implementations are entirely controlled by the VSS provider and not Data Protection for SQL, make sure to consult your VSS provider documentation for a complete understanding of your VSS implementation.
  • If you perform parallel VSS Backups, stagger the start of the backups by at least ten minutes. This interval ensures the snapshot operations do not overlap. If you do not stagger the snapshots, errors can occur. In addition, configure the parallel instance backups so they do not take snapshots of the same volumes. Make sure that parallel backups do not snapshot the same LUN.
  • Do not place multiple volumes on the same LUN. Microsoft recommends that you configure a single volume, single partition, and single LUN as 1 to 1 to 1.
  • Do not set the ASNODENAME option in the dsm.opt file when using Data Protection for SQL. Setting ASNODENAME can cause VSS Backups and VSS Restores to fail.
  • Specific database, log, file, and LUN settings are required for IBM System Storage. The DS8000®, SAN Volume Controller, Storwize V7000, and XIV storage subsystems require special settings when planning for VSS Backups


Offloaded VSS Backups
An offloaded backup uses another machine to move the data to the Tivoli Storage Manager server. This type of backup shifts the backup load from the production machine to another machine. An offloaded VSS Backup requires a VSS hardware provider that supports transportable shadow copy volumes is installed on the production and secondary machines.
Offloaded VSS Backups require a Tivoli Storage FlashCopy Manager license.

 

Best Practices for Configuring TDP SQL VSS Backup and Restore

Choose a suitable backup and restore strategy based on your predefined service level requirements. Consider these general factors as they can affect strategy

  • When creating policy for VSS Backups to be stored on local volumes, use the VEREXISTS and VERDELETED management class settings. That is because the number of versions you keep should be based on the numbers of volumes your subsystem has available for shadow copies. The general rule is that if you want to keep N local backups, you need to specify VEREXISTS=N+1. You need to specify N+1 because VSS requires pre-allocated space for taking a snapshot.
  • Many service level agreements base the retention of application server backups stored on the Tivoli Storage Manager server according to specific time periods, not versioning. If that is true in your environment, use the RETEXTRA and RETONLY management class settings instead of VEREXISTS and VERDELETED when setting policy for the backups stored on the Tivoli Storage Manager server.
  • Use different management classes if each storage group has its own retention requirements.
  • Legacy backup management policy remains the same as before. However, adding VSS Backups to your backup strategy might influence you to re-evaluate the frequency and retention of the legacy backups in order to best exploit both legacy and VSS Backup capabilities to meet your service level requirements.
  • Make sure to provision enough local storage on your disk subsystem to meet the management class settings.
  • Use Tivoli Storage Manager policy management for the different VSS Backup types. You can assign separate policy management classes for the different VSS Backup destination types and different storage groups or databases. Set these to meet your short-term and long term recovery objectives.
  • Automated (scheduled) backups: You can use the Tivoli Storage Manager scheduler to back up your application data automatically. Define different client schedules for VSS and legacy backups. Make sure backup schedules do not overlap.

 

Legacy backup versus VSS Backup
Each backup method may provide different types of functions and features. Evaluate the functions and features available with both methods and determine how they might affect service level requirements for your overall backup and restore strategy.
 
VSS Backup to TSM versus VSS Backup to local
Compared to VSS Backup to local, VSS Backup to TSM requires a longer processing time for backup and recovery; however, VSS Backup to TSM stores the data on Tivoli Storage Manager server tape or disk media for long-term retention (e.g. for regulatory requirements). After a VSS Backup to TSM is complete, the target volume is released for use by future snapshots. On the other hand, VSS Backup to local might be a better choice for recovering large databases in a shorter time frame because VSS Fast Restore or VSS Instant Restore can be used. These two restore methods typically complete much quicker than when restoring a VSS snapshot from Tivoli Storage Manager storage. However, these two restore methods require more target volume space for snapshots.
 

Tivoli Storage Manager VSS instant restore considerations

VSS Instant Restore is a hardware-assisted, volume level restore which performs a very quick copy of the target volumes back to the original source volumes. This is the quickest restore method in most cases. Consider these recommendations when using VSS Instant Restore
  • VSS Instant Restore is not part of the generic VSS infrastructure provided by Microsoft. It requires specialized code to directly communicate with the disk subsystem. As of the publication date of this document, VSS Instant Restore is supported on IBM DS6000, IBM DS8000, and IBM SAN Volume Controller subsystems.
  • Carefully plan the layout of the application data files. A VSS Instant Restore will overwrite all data on the production source volumes. If multiple objects that can be restored individually are placed on a single volume, those objects must be restored as a unit if a VSS Instant Restore is performed. Be aware that Tivoli Storage Manager does provide an option to disable VSS Instant Restore should it be necessary.
  • If you want to preserve any data on the original source volumes, you must manually copy the desired data to a temporary location and replace it after the VSS Instant Restore operation is complete.

 

Tivoli Storage Manager VSS offloaded backup considerations
Setting up your environment for VSS offloaded backups requires the following components:
  • A secondary machine that has access to the storage area network (SAN).
  • A VSS hardware provider that support transportable shadow copies.
  • The Tivoli Storage Manager nodes are configured in a proxy relationship.

 

IBM TDP SQL Performance Tuning

There are several performance considerations to evaluate when performing VSS Backups and VSS Restores for large databases
  • The amount of overhead involved for VSS snapshots depends on the VSS Provider implementation for the disk subsystem. For example, if the VSS Provider uses copy-on-write, it may be able to create snapshots almost instantaneously; however, this method relies heavily on the original volumes to restore the data. If the VSS Provider uses full volume copy, it may not require the original volumes to restore data but some operations might take a longer time to complete.
  • During a VSS backup, some application servers might require additional processing that can extend the time required for the VSS Backup to complete. For example, Exchange Server VSS Backups require an integrity check that must scan each page of the database and log files.
  • When performing a VSS Backup to local volumes, there is low overhead since there are no resources needed to send the application data to Tivoli Storage Manager Server storage. In addition, the application server will be in backup mode for a very short period of time.
  • VSS offloaded backups may provide better overall performance of the production application server machine since the movement of data to Tivoli Storage Manager Server storage is offloaded to the second machine.
 
For more detailed instructions follow this link

2 December 2013

Basic Introduction to IBM Tivoli Data Protection for MS SQL (TDPSQL) and its features

03:44 No Comments
IBM Tivoli Data Protection for Microsoft SQL Database is one of the IBM TSM family products which can be used to backup & restore all SQL databases including log files to Tivoli Storage Manager (TSM) server storage. It can be used to backup either individual or group of databases. Data Protection for SQL provides several methods of backing up small sized or large sized SQL Server data. 

IBM Tivoli Data Protection for Microsoft SQL Database Introduction (TDPSql)

There are two types of backups available in IBM Tivoli Data Protection for MS SQL, they are LEGACY TDP SQL and VSS TDP SQL backups. Legacy backups are a stream of bytes that Data Protection for SQL stores on the Tivoli Storage Manager server. VSS Backups differ since they are at the volume and file-level. When a SQL Server database is not fully allocated, a Legacy backup might transfer a smaller amount of data for a Tivoli Storage Manager backup than for a VSS Backup because a VSS Backup transfers the entire file, regardless of its allocation.

TDP SQL Legacy Backup

A Legacy backup creates a copy of all or part of a SQL database or logs on Tivoli Storage Manager storage media. Data Protection for SQL provides selection mechanisms and the logic that are required to back up and restore SQL data.  When a backup is performed, Tivoli Storage Manager server retains information about the SQL Server and database. This information is available for query and restore operations after the backup is completed. Data Protection for SQL software can either compress or instruct the SQL Server to compress the SQL data before sending it to the Tivoli Storage Manager server. When you initiate a Legacy backup operation, Data Protection for SQL completes the following actions

1. First TDP SQL begins a session with a Tivoli Storage Manager server using the Tivoli Storage Manager API and information contained in a client options file (dsm.opt).

2. Starts a session with the SQL Server using the SQL-SMO interface.

3. Instructs the SQL Server using the SQL VDI interface to begin a backup of the
selected or all database objects.

4. Receives data from the SQL Server and sends it to the Tivoli Storage Manager server.

5. Informs the SQL Server that the backup is complete.

6. Ends the Tivoli Storage Manager server and SQL Server sessions.

TDP SQL VSS Backup

A VSS Backup uses Microsoft Volume Shadow Copy Service technology to produce an online snapshot (point-in-time consistent copy) of SQL data. A VSS Backup means that the SQL Server is not in backup mode for an extended period of time. The length of time to perform the snapshot is usually measured in seconds, not hours. Ideally, it reduces the pressure/load on the production system where database resides by taking a snapshot of production database and place the snapshot on other server or volumes and then start the backup from the secondary volumes to reduce the offload, these secondary volumes are often referred as backup servers or VSS shadow volumes. In addition, a VSS Backup allows a snapshot of large amounts of data at one time because the snapshot works at the volume level. VSS Backups can be stored on Tivoli Storage Manager server storage or local VSS shadow volumes. Both of these storage destinations require that sufficient space be available for the snapshot. VSS Backups stored locally on VSS shadow volumes are directly accessible by the SQL system as long as sufficient space is available for the snapshot.

These types of local VSS Backups are faster for a couple of reasons. They are faster because the way the snapshots are managed and because the data is not placed into Tivoli Storage Manager server storage. Restoring these backups is also fast because the SQL data is not transferred from Tivoli Storage Manager server storage over the network. For local VSS Backups you must have a licensed version of IBM Tivoli Storage FlashCopy Manager or IBM Tivoli Storage Manager for Copy Services installed on your system.

Backing up Microsoft SQL Server 2012 Databases

You can use Data Protection for SQL (check version requirements & compatibility) to protect Microsoft SQL Server 2012 data.  Microsoft SQL Server 2012 uses AlwaysOn Availability Groups to provide high availability and disaster recovery capabilities. Data Protection for SQL protects availability databases in an AlwaysOn Availability Group and AlwaysOn Failover Cluster Instances to provide high availability and disaster recovery at the SQL Server database level and SQL Server instance level. The following types of VSS backup operations are supported
  • Full VSS backups of the primary availability replica
  • VSS copy-only full backups of availability replicas

The following types of legacy backup operations are supported:
  • On the primary replica, legacy full, differential, file, set, group, and log backups are supported.
  • On the secondary replica, legacy full, file, set, group, and log backups are supported.
  • VSS and legacy copy-only full backups, legacy copy-only file, set, or group backups, and legacy copy-only and normal log backups are supported.
  •  For all backup operations of secondary availability replicas, the secondary replicas must be in the synchronized or synchronizing state.
 

Restoring Microsoft SQL Server 2012 availability Databases

You can use Legacy restore or VSS restore operations to restore the MS SQL 2012 availability database but with some restrictions and limitations.

Legacy restore

You can restore an availability database on either a primary or secondary replica. During the restore process, the restored database is removed from the availability group. When a database is removed from the availability group, the database becomes a local database on that replica. The database is restored as a local database. After this restore is complete, manually add the database back to the availability group. However, before adding the database to the availability group, verify that the data on all replicas is transactionally consistent. For example, the database and any log files must be restored so the files are all at the same level. After verifying the data is transactionally consistent, the database can be added to the availability group.

VSS restore

Because of a SQL Server limitation, you cannot restore a VSS backup to an alternative SQL server instance. Therefore, VSS backups must be restored to the same SQL server instance where the snapshot was taken.

Tivoli Data Protection for MS SQL Database Backup Types

Data Protection for SQL offers an expanded range of backup types that allows flexibility for your environment and processing needs. With TDP SQL you can take

Full database backup (Legacy and VSS)
Data Protection for SQL backs up an entire SQL Server database and the portion of the transaction log necessary to provide a consistent database state. With both full and differential backups, the copy includes enough information from any associated transaction logs to make a backup consistent with itself. The portion of the log included contains only the transactions that occur from the beginning of the backup until its completion.

Copy-only full backup (Legacy and VSS)
A copy-only full backup is a type of backup that is independent of the sequence of conventional SQL Server backups. The copy-only full backup does not disturb the sequence for a differential backup. The differential backup is not associated with the copy-full backup, but is associated with the prior full backup that was completed. This type of backup can be used for special purpose backups that do not affect existing backup and restore procedures. In addition, when compared to conventional backups, this type of backup can be used for longer term retention. An example of a special purpose backup is a backup of a log before an online file restore. In this scenario, the copy-only full backup is used one time. After the backup is used, it is deleted.

Differential backup (Legacy only)
Data Protection for SQL backs up only the data pages in a SQL Server database instance that changed after the last full backup and a portion of the transaction log. Differential backup is associated with the last full backup that was performed. The last full backup might be completed by Data Protection for SQL or another tool or product. For example, if you run a full backup with SQL Server to disk backup, and run a differential backup with Data Protection for SQL, the differential backup is associated with the SQL Server disk backup. (Microsoft SQL Server 2012 only) Differential backup is not supported for databases on the secondary replica.

Log backup (Legacy only)
Data Protection for SQL backs up only the contents of a SQL Server database transaction log since the last successful log backup. Before the first log backup, complete either a full backup or an equivalent type of backup. Log backups normally follow full backups. The portion of the log included in full and differential backups is not equivalent to a log backup. Additionally, in full and differential backups, the log is not truncated as it is during a log backup. However, a log backup following a full or differential backup includes the same transactions as a full or differential backup. Log backups are not cumulative as are differential; they must be applied against a base backup and in the correct order. A log backup in SQL Server terms is not equivalent to an incremental backup in Tivoli Storage Manager terms.

File backup (Legacy only)
Data Protection for SQL backs up only the contents of a specified SQL Server logical file. This type of backup can ease the scheduling for backing up large databases. You can back up different sets of files during different scheduled backups. File, group, and set backups must be followed by a log backup, but a full is not required.

Group backup (Legacy only)
Data Protection for SQL backs up only the contents of a specified SQL Server file group. This backup allows you to back up the set of database tables and indexes within a specific group of files. The "group" is specified as part of the setup within the SQL Server when you define the database files. If no group is specified and all the database files are part of the "primary" group, it is not possible to back up or restore just part of the database by using the group.

Set backup (Legacy only)
Data Protection for SQL backs up the contents of specified SQL Server file groups and files as a unit.

Tivoli Data Protection for MS SQL Database Restore Types

Data Protection for SQL provides several methods of restoring SQL Server data. You can restore the SQL server data both with Legacy and VSS methods but with some limitations and restrictions.

Legacy restore
A legacy restore obtains backup copies of SQL databases from Tivoli Storage Manager server storage and restores them to their original location. Like a Legacy backup, it uses a specialized API restore that functions with the SQL Server. A complete restore of a database involves restoring a full backup or the equivalent thereof (from group, file, or set backups) and restoring all transaction logs since the last full backup.

VSS Restore
A VSS Restore restores VSS Backups (SQL database files and log files) that reside on Tivoli Storage Manager server storage to their original location.  The limitations and restrictions of VSS Restores are
  • You can only restore SQL Server VSS Backups to the same SQL Server instance. 
  • Full and copy-only full backup types can be restored. Differential, individual file groups, individual files, and set backups are not supported by VSS and therefore, cannot be restored. However, Legacy differential and Legacy log backups can be applied after a full VSS Backup has been restored.
  • VSS Restore granularity is at the database level.
  • Supports restoring one or more databases from a VSS snapshot backup located on Tivoli Storage Manager server storage.
  • Restores can be performed in a Microsoft Windows Failover Clustering or Veritas Cluster Server (VCS) environment.
  • Supports restoring a VSS Backup (directly from Tivoli Storage Manager server storage) to an alternate location using the /relocatedir option.

VSS Fast Restore
A VSS Fast Restore restores data from a local snapshot. The snapshot is the VSS backup that resides on a local shadow volume. The restore operation retrieves the data by using a file-level copy method. The following characteristics are true of VSS Fast Restores
  • Full and copy-only full backup types can be restored. Differential, individual file groups, individual files, and set backups are not supported by VSS and therefore, cannot be restored. However, Legacy differential and Legacy log backups can be applied after a full VSS Backup has been restored.
  • You can only restore SQL Server VSS Backups to the same SQL Server instance. v VSS Backups can be restored to an alternate location by using the /relocatedir option.
  • Restore granularity is at the database level.
  • Restores can be performed in a Microsoft Windows Failover Clustering or Veritas Cluster Server environment.

VSS Instant Restore
A VSS Instant Restore restores data from a local snapshot. The snapshot is the VSS backup that resides on a local shadow volume. The restore operation retrieves the data by using a hardware assisted restore method (for example, a FlashCopy operation). A VSS Instant Restore is only possible when all of the data from the storage group or database that is specified for restore resides on storage subsystems that are supported by the VSS Instant Restore. If part of the data being restored, including the log files and full-text index files, resides on a local disk, a VSS Fast Restore is performed.

When performing VSS Instant Restores, a best practice is to make sure that any previous background copies (that involve the volumes being restored) are completed prior to initiating the VSS Instant Restore. However, this check is not necessary for XIV, SAN Volume Controller, or Storwize V7000 with space-efficient target volumes. Check TDP SQL document for configuration information on how to perform VSS Instant restores using DS8000, SAN Volume Controller, Storwize V7000 storage systems.

General Backup Scenarios

Different backup strategies are available depending on specific requirements regarding network traffic, backup window and acceptable restore times. Strategies defined by backup type. Some commonly used strategies (based upon backup type) are described as follows

Full backup only (Legacy and VSS)
This approach is best for SQL databases that are relatively small because it implies that the entire database is backed up each time. Each full backup takes longer to perform, but the restore process is most efficient because only the most recent (or other appropriate) full backup need be restored. This is the appropriate strategy for system databases such as master, model, and msdb due to their normally small size.

Full plus log backup (Legacy and VSS)
A full plus transaction log backup strategy is commonly used when the normal backup window or network capacity cannot support a full backup each time. In such cases, a periodic full backup followed by a series of log backups allows the backup window and network traffic to be minimized. For example, you can perform full backups on the weekend and log backups during the week. The restore process becomes more complex, however, because a full backup, as well as subsequent log backups, must be restored. It is also possible to do a point-in-time restore to restore a transaction log to a specified date and time.

Full plus differential backup (Legacy and VSS)
This strategy can be used between full backups. A differential database backup can save both time and space. Space is saved because the backup consists of only the changed portions of a database since the last full backup (it is cumulative). Time is saved because you can avoid applying all individual log backups within that time to the operation. This applies to restore operations as well; only the last differential backup (latest version) need be restored. Although VSS supports full backups only, Legacy differential backups can be applied to the VSS full backup.

Full plus differential plus log backup (Legacy and VSS)
This strategy allows for a faster restore scenario by reducing the number of transactions that may need to be restored and applied. If, for example, a full Legacy or VSS backup is done weekly, a differential nightly, and a log backup every four hours, the restore would involve the full backup, a differential, and at most five log backups.

File or group backups (Legacy only)
When a group is created on the SQL Server, database files are identified with that group. The group used for the group backup is dependent on the group to which the database files are defined. Use a file backup strategy when it is impractical to backup an entire database because of size and accompanying time and performance issues. When performing restore operations for a file or file group, provide a separate backup of the transaction log.
Subscribe to: Posts (Atom)