Archive for February 2015

21 February 2015

Different types of VMWare backups available in TSM for Virtual Environments and their advantages and disadvantages

06:38 No Comments
You can take different types of virtual machines backups by using TSM BA client and TSM for VE softwares. From BA client command line,  Backup VM command can be used to back up both Microsoft Hyper-V virtual machines and VMware virtual machines. One or more virtual machines can be backed-up using the Tivoli Storage Manager data mover node. Data mover node is the name given to a configuration where the Backup-Archive Client runs on a vStorage backup server and is configured to protect the virtual machines in a Virtual Center or ESX/ESXi server. You can also use the -vmbackuptype and -mode options to indicate how the backups are to be performed. 

-vmbackuptype

Use the -vmbackuptype option with the backup VM command  to specify the type of virtual machine backup to complete. You can also place this option in the client options file (dsm.opt), or on the command line. Different options available for this parameter are

FIle
Specifies a file-level backup to be performed.

FUllvm
Specifies that a full VM backup to be performed. This is the default backup type.

HYPERVFULL
Required for Hyper-V systems. Specifies a full virtual machine backup of all virtual machines that are defined on the Hyper-V host machine. If the vmbackuptype=hypervfull option is specified, most of the other options associated with backing up files from a vStorage backup server are not allowed, they are ignored.


-mode

Use -mode option with the backup VM command to indicate how the VM backups are to be performed. Different types of parameters available are

Full 
Full mode specifies that you want to perform an image backup of all objects on a VMware virtual machine’s disks. 

Incremental
Incremental mode specifies that you want to create an image backup of only the objects that have changed since the last backup. You must have license to use Tivoli Storage Manager for Virtual Environments to use this mode.

IFFull 
IFFull stands for incremental forever full VM backup. In this mode, a snapshot of all used blocks on a VMware virtual machine’s disks are backed up to the server. You must have license to use Tivoli Storage Manager for Virtual Environments to use this mode.

IFIncremental
IFIncremental stands for incremental forever, incremental. In this mode, a snapshot is created of the blocks that have changed since the last backup. You must have license to use Tivoli Storage Manager for Virtual Environments to use this mode.

Virtual machines full backups

Full virtual machine backup processing backs up a virtual machine from a virtual machine-based host. A full VM backup stores a backup copy of all virtual disk images and configuration information for a virtual machine. Full VM backups enable a complete restore of a virtual machine, but they take more time and more server space than a file-level or incremental backup. For example

dsmc backup vm vmname -vmbackuptype=fullvm -mode=full

How it works ?

Full virtual machine backup processing stores a backup copy of all virtual disk images and configuration information for a virtual machine.

Advantages

With full virtual machine backup processing, you get faster data movement than a file-level backup.

Disadvantages
  • Backups are not granular.
  • Full virtual machine backup operations enable a complete restore of a virtual machine, but they take more time and more server space than a file-level or incremental backup.
  • You can restore individual files from a full virtual machine backup only with IBM Tivoli Storage Manager for Virtual Environments.
  • This method is only available on Linux and Windows clients.

VMware incremental backup

You can run an incremental backup of a virtual machine from a VMware ESX or ESXi-based host. Incremental backup processing requires a Tivoli Storage Manager for Virtual Environments license. For example

dsmc backup vm vmname -vmbackuptype=fullvm -mode=incremental

How it works ?

An incremental backup of a virtual machine backs up all changes that occurred since the previous backup of the virtual machine, whether the backup was a full backup, or another incremental backup.

Advantages

Incremental backup processing backs up changes to virtual machines between full virtual machine backups.

Disadvantages
  • The size of incremental backups can increase if you do not run a full backup regularly.
  • It is inefficient to restore data from incremental backups because the process must automatically complete the following tasks:
                   – Restore the most recent full backup.
                   – Restore each incremental backup up to the specified recovery point.
  • This method is available only on Linux and Windows clients.


Virtual machine incremental-forever-full backup

Incremental-forever-full virtual machine backup processing backs up all the used blocks on a virtual machine's disks. To run this type of backup, the system that hosts the Hyper-V server must run Windows Server 2012 R2 or a newer Windows Server operating system. For example

dsmc backup vm vmname -vmbackuptype=fullvm -mode=IFFull

How it works ?

The following processes occur during incremental-forever-full virtual machine backup processing
  • A full virtual machine backup is required only one time.
  • Data from incremental backups is combined with data from the full backup to create a synthetic full backup image. This type of full backup is called a synthetic backup because it is created from the data that is stored on the server and not from reading the used blocks on the production disks.
  • Each incremental-forever-full virtual machine backup operation reads and copies all of the used blocks, whether the blocks are changed or not since the previous backup.
Advantages
  • Periodic full backups are no longer necessary.
  • During a restore operation, you can specify options for a point in time and date to recover data. The data is restored from the original full backup and all of the changed blocks that are associated with the data.
Disadvantages
  • If one or more of the progressive incremental backups is corrupted on the server, you might not be able to fully recover a virtual machine. To ensure that you can fully recover a virtual machine, periodically run a full virtual machine backup.
  • This method is available only on Linux and Windows clients.

Virtual machine incremental-forever-incremental backup

Incremental-forever-incremental backup processing backs up only the disk blocks that have changed since the last backup. For example

dsmc backup vm vmname -vmbackuptype=fullvm -mode=IFincremental

How it works ?

The following processes occur during incremental-forever-incremental backup processing of a virtual machine
  • A full virtual machine backup is required only one time.
  • A full virtual machine backup operation copies all of the used disk blocks that are owned by a virtual machine to the Tivoli Storage Manager server.
  • After the initial full backup, all subsequent backup operations of the virtual machine are incremental-forever-incremental backups.
  • This method copies only the blocks that changed since the previous backup, regardless of the type of the previous backup.
  • The server uses a grouping technology that associates the changed blocks from the most recent backup with data that is already stored on the server from previous backups.
  • A new full backup is then effectively created each time changed blocks are copied to the server by an incremental-forever-incremental backup.
Advantages
  • Periodic full virtual machine backups are no longer necessary.
  • This method reduces the amount of data that goes across the network.
  • This method reduces data growth because all incremental backups contain only the blocks that changed since the previous backup.
  • No comparison with the backup target is required since only changed blocks are identified.
  • Impact to the client system is minimized.
  • The length of the backup window is reduced.
  • Data restore operations are simplified.
  • This method optimizes data restore operations.
Disadvantages
  • If one or more of the progressive incremental backups is corrupted on the server, you might not be able to fully recover a virtual machine. To ensure that you can fully recover a virtual machine, periodically run a full virtual machine backup.
  • It is available only on Linux and Windows clients.

VMware file-level backup on Windows

On Windows systems, you can use the backup-archive client to create file-level backups of VMware virtual machines. For example

dsmc backup vm vmname -vmbackuptype=file

How it works ?

The following processes occur during file-level backup processing of a virtual machine
  • A VMware snapshot is taken of the virtual machine to be backed up.
  • The file systems of the virtual machine are remotely mapped to the vStorage backup server.
  • A file-level progressive incremental backup is run for all of the file systems.
  • The data is stored under the node name that matches the host name of the virtual machine.
  • The data for each virtual machine is stored in the node that is associated with the virtual machine.
  • The file system mappings are removed and the snapshot is removed.
When to use ?

Use file-level virtual machine backup processing if you want to restore individual files from a virtual machine but you do not have a license for Tivoli Storage Manager for Virtual Environments.

Advantages
  • You can use include and exclude rules to identify the files to back up.
  • Files are backed up as individual files rather than as an image backup.
Disadvantages
  • File-level restores must be made from a backup-archive client that is installed directly on a virtual machine guest.


Parallel backups of virtual machines

You can improve performance of virtual machine backups by running parallel backups of multiple virtual machines by using a single instance of the backup-archive client. For example

dsmc backup vm vmname1,vmname2 -vmbackuptype=file

How it works ?

The following processes occur during parallel backup processing of virtual machines
  • A single Tivoli Storage Manager data mover node can be used to concurrently back up multiple virtual machines.
  • When the backups are initiated, the client establishes parallel sessions to copy the data to the Tivoli Storage Manager server.
Advantages
  • You can optimize the backups so that they do not adversely affect the servers that are hosting the virtual machines.
Disadvantages

You must optimize the parallel backups. The number of virtual machines that you can back up in parallel depends on the following factors
  • The processing power of the server that the Tivoli Storage Manager data mover node runs on.
  • The performance of I/O between the client and the Tivoli Storage Manager server.

18 February 2015

How to take TSM Database backup when the TSM Server is down

06:17 No Comments
You can take TSM DB backup by using DB2 commands even after TSM server is down. There are many situations where you need to take TSM DB backup offline to start the TSM server. For example some of the reasons are if
  • You havent took TSM DB backup for while and your TSM server got crashed.
  • Your TSM server got crashed while taking TSM DB backup.
  • If your DB backup command gets hanged you need to manually restart TSM server.

In the above situations if  you try to restart your TSM server with dsmserv command, it does not start and might show you the below error

ANR7801I Subsystem process ID is 4117.
ANR0900I Processing options file /home/tsminst1/dsmserv.opt.
ANR7814I Using instance directory /home/tsminst1.
ANR4726I The ICC support module has been loaded.
ANR0990I Server restart-recovery in progress.
ANR0172I rdbdb.c(2356): Error encountered performing action ActivateDatabase.
ANR0162W Supplemental database diagnostic information:  -1035:SQLSTATE 57019: The statement
was not successful, because of a problem with a resource.
:-1035 (SQL1035N  The database is
currently in use.  SQLSTATE=57019

and in the db2diag.log file, it will show the following error message. The location of the log is /home/tsminst1/sqllib/db2dump directory.

2015-02-17-18.40.11.234235+330 I35531942E481       LEVEL: Event
PID     : 2515                 TID  : 140736896952064PROC : db2sysc 0
INSTANCE: tsminst1             NODE : 000
EDUID   : 1                    EDUNAME: db2sysc 0
FUNCTION: DB2 UDB, base sys utilities, sqeAppServices::ExecuteStopForce, probe:1000
DATA #1 : String, 47 bytes
[Force]->Number of applications to be forced :
DATA #2 : Hexdump, 4 bytes
0x00007FFFDCBEF1D4 : 0200 0000                                  ....
[tsminstn@DCMBACKUP db2dump]$ tail -20 tsminstn.0.nfy
ADM10502W  Health indicator "Database Backup Required" ("db.db_backup_req") is
in state "Manual backup required" on "database" "tsminst1.TSMDB1  ".

2015-02-17-18.00.44.237755   Instance:tsminst1   Node:000
PID:2509(db2star2)   TID:3969238816   Appid:none
base sys utilities  DB2StartMain Probe:911


In the above situation, you cannot restart your TSM server unless you take the DB backup. and we dont have any dsmserv utilities to take the TSM DB backup offline. 

Solution:

Since TSM uses DB2 database, you can take TSM DB backup using DB2 commands offline and then restart your TSM server. Please note that this backup cannot be used for DRM purpose, and you have to take TSM DB backup as soon as the server starts running fine. So here are the simple steps which might help you

1) First connect to DB2 database through command-line, Go to /home/tsminst1/sqllib/db2dump directory and run 

$ db2start
02/17/2015 18:42:59     0   0   SQL1063N  DB2START processing was successful.
SQL1063N  DB2START processing was successful.

2) Now, you have to take the DB backup by using the following command, in the below command tsmdb1 is the DB name and DBBACKUP2/dbb2 is the location where backup will be stored. make sure you have enough place for the backup to be stored in the directory.

$ db2 backup db tsmdb1 to /DBBACKUP2/dbb2

Backup successful. The timestamp for this backup image is : 20150217184833

3) After the backup is successfully completed, you can run the dsmserv command to start the TSM server as usual.

$ su - tsminst1
$ /opt/tivoli/tsm/server/bin/dsmserv

11 February 2015

Different types of image backups and their advantages and disadvantages

07:52 No Comments

Different types of Image backup techniques in TSM

If the different variations of progressive incremental backups and file backup operations do not complete successfully or do not reach your SLA requirements, consider running an image backup to reduce the backup window. Just as we have different variations of incremental backups, we also have different types of TSM image backups.

Normal Image Backup

Image backup processing backs up your file system as a single object.

How it works ?

During image backup processing, the client sends a logical block image of a file system to the Tivoli Storage Manager server.

When to use ?
  • When you cannot resolve system memory problems or progressive incremental backup is otherwise unusable.
  • There are too many changes in the file system (greater than 1,000,000 objects) for journal-based backup.
  • Most of the file system contains small files (average size less than 1 MB).
  • You must have a faster recovery time than what can be achieved with file-level restore.
  • For AIX, HP-UX, Linux, and Solaris clients:
            - When the file system is at least 60% full.
            - Online (tdp) image backup cannot be taken, and you can unmount the file system.

Advantages
  • Backups are faster.
  • No scan time is required to determine what changed.
  • Overall data movement is faster.
  • Restore times are faster.
Disadvantages
  • You cannot restore individual files directly from the Tivoli Storage Manager server
There are 3 different variations of image backups available in TSM

Offline (static) image backup processing
  • The volumes to be backed up are mounted read-only.
  • This method is available for AIX, HP-UX, Linux x86, Solaris, and Windows operating systems.
  • This method is the most effective backup method for FlashCopy operations.

Online (dynamic) image backup processing
  • The volumes to be backed up remain online.
  • Fuzzy backup processing occurs when the data is changed during the image backup processing.
Online image backup by using snapshots
  • The volumes to be backed up remain online.
  • The image backup is made at a single point in time.
  • It is available only for AIX JFS2, Linux x86, and Windows operating systems

Image plus incremental-by-date image backup

Image backup plus incremental-by-date image backup processing is one of two methods that you can use to run efficient incremental backups of your file system.

How it works ?

The following processes occur during image plus incremental-by-date image backup processing
  • During a full image backup (for example, when you issue the dsmc backup image command), the client sends a logical block image of a file system to the server.
  • Subsequent backups are incremental-by-date image backups (for example, when you issue the dsmc backup image -mode=incremental command), in which the client queries the server for the last backup of the entire file system.
  • The server sends the time stamp of last backup of the entire file system to the client.
  • The client scans and compares the time stamp with the local file system, and backs up the new and changed files.
During an image plus incremental-by-date restore operation, the following processes occur:
  • The client requests an incremental image restore.
  • The server sends the base image to the client.
  • The server returns more files that must be applied to the base image to satisfy the recovery point.
When to use

Run image plus incremental-by-date image backup processing in the following situations
  • You need faster backups.
  • You must be able to restore files to a specific point in time.
Periodically run full image backups to maintain a file system image that is close to what existed at the time of the last incremental-by-date image backup. When you periodically run a full image backup, it can also improve restore time.

Advantages
  • Backups are faster.
  • No scan time is required to determine what changed.
  • Overall data movement is faster.
  • Restore times are faster.
  • Protection of files that changed after the image backup was created.
  • In some cases, recovery time and recovery point objectives are improved.
Disadvantages

Image plus incremental-by-date image backup processing has the following limitations:
  • This method reduces the flexibility of the scope of the backup operation. You must back up the entire file system.
  • The files are not backed up if the changes do not affect the date (for example, attribute, mode, ACL, rename, copy, move, and security changes).
  • The deleted files are not expired on the server.
  • Policy rebinding does not take place.
  • The entire file system must be scanned.
  • This method cannot be used if the client and server clocks are set to different times or are not in the same time zone.
  • Deleted files are not reconciled. Deleted files are not expired on the server. Therefore, when you restore an image with the incremental option, files that were deleted after the original image backup are present after the restore.
  • More storage space is required on the Tivoli Storage Manager server.

Image plus incremental backup

Image backup plus file system incremental backup processing is the second method that you can use to run efficient incremental backups of your file system.

How it works ?

The following processes during image plus incremental backup processing:
  • During a full image backup (for example, when you issue the dsmc backup image command), the client sends a logical block image of a file system to the server.
  • Subsequent backups are progressive incremental backups in which the client queries server for the active backup version metadata.
  • The server returns list of active backup versions for the entire file system.
  • The client scans and compares the list with the local file system.
  • The client backs up the new and changed files.
During an image plus progressive incremental restore operation, the following processes occur:
  • The client requests an incremental image restore.
  • The server returns the base image.
  • The server returns more files that must be applied to the base image to satisfy the recovery point.
  • The server optionally returns the list of files that must be deleted from the base image.
 When to use ?
  • When you need faster backups.
  • When you want to restore files to a specific point in time.
  • When you want to be able to reconcile deleted files.
Run incremental backups of the file system periodically to ensure that the server records additions and deletions accurately. Run an image backup periodically to ensure faster restores.

 Advantages
  • Backups are faster.
  • No scan time is required to determine what changed.
  • Overall data movement is faster.
  • Restore times are faster.
  • Protection of files that changed after the image backup was created.
  • In some cases, recovery time and recovery point objectives are improved.
 Disadvantages
  • More time is required to periodically create image backups.
  • More storage space is required on the Tivoli Storage Manager server.

Snapshot differential backup

If you are backing up NetApp filer or vFiler volumes or N-Series file server volumes, you can use a snapshot differential backup to streamline the incremental backup process.

How it works ?

The following processes occur during snapshot differential backup processing:
  • The first time that you run an incremental backup with the snapdiff option, a snapshot is created (the base snapshot) and a traditional incremental backup is run by using this snapshot as the source. The name of the snapshot that is created is recorded in the Tivoli Storage Manager database.
  • The second time an incremental backup is run with the snapdiff option, a newer snapshot is either created, or an existing one is used to find the differences between these two snapshots. The second snapshot is called diffsnapshot. The client then incrementally backs up the files that are reported as changed by NetApp to the Tivoli Storage Manager server.
When to use ?

Use this method to back up NetApp filer or vFiler volumes or N-Series file server volumes on Windows, AIX 64-bit, and Linux x86/86_64 clients.

Advantages

Snapshot differential backup processing can save you time by not having to scan the whole volume for changed files.

Disadvantages

Snapshot differential backup processing has the following limitations:
  • On Windows systems, it does not work for any of the NetApp predefined shares, including C$, because the client cannot determine their mount points programmatically.
  • You must periodically take a new base snapshot with the createnewbase option to back up any files that might have been skipped

2 February 2015

Tips for troubleshooting slow performance and long running incremental or full backups

07:36 No Comments
Even after trying all the incremental backup types you might have still see a slow incremental backups or sometimes the incremental/full backups get failed in the middle due to insufficient memory (RAM) resources. These kind of issues are generally seen in the windows (mostly 2003) environment if you have large number of files to be backed up in less time. To address this issue, you can use these below methods.

It is recommended to test these backup methods one after another and you should decide which one suits best for your backup window. Also consider the advantages & disadvantages of these methods before taking any decision.

Memory-efficient backup method

Ideally you should first try this option and then try the next methods. Some times the performance of incremental backups can degrade if the system is memory-constrained before the backup begins. Run incremental backup with the memoryefficientbackup yes option in the client options file (dsm.opt). This setting causes the client to process only one directory at a time during incremental backups, which reduces memory consumption but increases backup time.

How it works ?
  • First, the client queries the server for the metadata of active backup versions for the first directory to be backed up.
  • The server returns a list of active backup versions for the directory.
  • The client scans the list and compares it with the local file system, and backs up the new and changed files.
  • The client queries the server for the next directory and repeats the process for all directories.

When to use memory-efficient backup method ?

Use memory-efficient backup when your system has a low amount of memory available for incremental backups.

Advantages

Memory-efficient backup is a comprehensive incremental backup with a smaller backup memory usage.

Disadvantages
  • The backup run time will be increased because it has to check each directory one by one.
  • This method does not work for a single directory that contains a large number of files.
  • If the system is enough memory to run regular incremental backups, running memory-efficient backup can degrade the backup performance.


Memory-efficient backup with disk caching

If the client system is memory-constrained and incremental backups still cannot complete successfully even with the memoryefficientbackup yes setting, run incremental backups with the memoryefficientbackup diskcachemethod option. This setting causes the client to use less memory but requires more disk space on the client system. 

How it works ?

This method is similar to incremental backup processing but the client temporarily stores active backup version metadata on disk instead of memory. You have to give the the diskcachelocation c:\ in the option file.

When to use  memory-efficient backup with disk caching ?
  • The client is running out of memory with incremental backups and memory-efficient backup is not sufficient.
  • Journal-based backup is not available on the operating system.

Advantages

Memory-efficient backup with disk caching is a comprehensive incremental backup operation with a smaller backup memory usage.

Disadvantages
  • The backup processing time might be longer because the active backup inventory is on disk instead of in memory. however, in most cases backup will be successfull.
  • Gigabytes of free disk space are required to temporarily cache the active backup inventory


Journal-based backup

Journal-based backup is an alternative form of incremental backup that uses a change journal that is maintained by the Tivoli Storage Manager journal process. On Windows clients, the change journal is maintained by a journal service. On AIX and Linux clients, the change journal is maintained by a journal daemon process.

How it works ?
  • Journal-based backup processing uses real-time monitoring of a file system for changed files all the day.
  • The names of the changed files are then logged to the journal database.
  • During backup processing, the client queries the journal for the list of changed files, and then backs up the changed files.

When to use Journal based backup ?
  • When the scheduled backups are not completed within the allotted time even after trying memory efficient backup methods.
  • When there are less than 1,000,000 files and a small number of changes between backups (less than 1,000,000).
  • When there are less than 10,000,000 objects with 10-15% velocity of change. The velocity of change means the rate at which files are changed over a short amount of time (such 1 or 2 seconds).

Advantages

Journal-based backup can often greatly reduce the time that it takes to determine which files changed.

Disadvantages
  • You must still run incremental backups periodically.
  • Journal-based backups are not suitable for file systems where large numbers of files can change over a short time interval, such as changing hundreds or thousands of files in 1 or 2 seconds.
  • This method is available only on Windows, AIX, and Linux clients.
Subscribe to: Posts (Atom)