Archive for March 2014

31 March 2014

How to troubleshoot and improve the IBM TDP domino (performance tuning) backup and restore performance

00:48 Posted by Anil K Y Ommi No Comments

If your backup and restore process are running very slow, you need to find the reason for this slow process and take necessary troubleshooting actions to prevent this. In most scenarios the reason for this problem is because of not following best practices for installing and configuring the products. According to IBM, many factors can affect the backup and restore performance of your Domino Server databases. Some of these, such as hardware configuration, network type, and capacity are beyond the control of Data Protection for Domino. However, some parameters that are related to Data Protection for Domino can be tuned for optimum performance.

1) By running and checking these commnads might give you an idea where to start the troubleshoot.

domdsmc query adsm

domdsmc query domino

domdsmc query preferences

2) Below are some of the performance tuning parameters that might help you to troubleshoot your slow backup and restore process. You can find these parameters in dsm.opt or domdsm.cfg files which are located in the default TDP Domino installation directory.

/buffers parameter

Data Protection for Domino uses multiple data buffers when transferring data between the Domino and Tivoli Storage Manager servers. The number and size of the buffers can be specified using the /buffers parameter. The number and size of buffers that are allocated by default can be configured through the set command or by selecting the Preferences item from the Edit menu on the Data Protection for Domino GUI. The default number of buffers is 3 and the default buffer size is 1024 KB.

The statistics option

The statistics option logs performance information about an individual database at the backup or restore level. Data Protection for Domino processing is performed under two threads: a producer process (which reads the data) and a consumer process (which sends the data). During a backup, the producer reads the database and the consumer sends this data to the Tivoli Storage Manager server. During a restore, the producer receives the data from the Tivoli Storage Manager server and the consumer writes the restored database. The statistics option logs this information to assist in tuning Data Protection for Domino for optimal. The statistics option provides performance information at the individual database backup or restore level. Statistics are logged to the Data Protection for Domino log file (domdsm.log by default). Make sure this option is specified in the Data Protection for Domino preferences file (domdsm.cfg by default) during backup and restore processing.

The sessions option

The sessions option allows a specified number of TCP/IP sessions to be made available for communication with the Tivoli Storage Manager server when backing up Domino NSF databases. Since more than one TCP/IP session is made available for backup processing, improvements in performance are possible performance.

The DOMTXNBYTELIMIT option

The DOMTXNBYTELIMIT=number option specifies the number of bytes sent between Data Protection for Domino and the Tivoli Storage Manager server in a single transaction. The default value is 0, which indicates no limit, and the maximum value is 2097152. This number is multiplied by 1024 to calculate the limit in bytes.

The DOMTXNGROUPmax option

The DOMTXNGROUPmax=number option specifies the number of individual objects sent to the Tivoli Storage Manager server in a single transaction. Two objects are sent to the Tivoli Storage Manager server for each database backup so the default value of this option is 2. The maximum value is 65000. The DOMTXNGROUPmax option can be overridden by the Tivoli Storage Manager server TXNGRPMAX option. However, when domtxngroupmax is set, the minimum of the two values is used

To reduce query processing time when querying the Tivoli Storage Manager server for databases to restore via the Data Protection for Domino Windows native GUI,

specify a database name using letters and a wildcard character (*) in the By

Database Name field.

3) To improve throughput for backup and restore operations, run multiple sessions in parallel. This is most effective when work is partitioned by physical volume. For example, one Data Protection for Domino session backs up all databases on one physical volume while a second Data Protection for Domino session backs up all databases on another volume. To improve throughput for backup operations, run multiple sessions in parallel.

4) You can also specify tcpnodelay yes in the Data Protection for Domino options file (dsm.opt) to improve backup and restore performance. Instead of buffering the data, this option sends the data as successive small packets across the network without delay.

5) Data Protection for Domino logs information, by default, to the domdsm.log file in the directory where Data Protection for Domino is installed. This file indicates the date and time of a backup, data backed up, and any error messages or completion codes. This file is very important and should be monitored daily.

The Tivoli Storage Manager API logs API error information, by default, to the dsierror.log file in the directory where Data Protection for Domino is installed. This file does not contain backup statistics.

The Domino server logs information to the Windows Domino Event Log. Domino server error information can be obtained by viewing the Windows Domino Event Log.

The Tivoli Storage Manager scheduler logs information to both the dsmsched.log and the dsmerror.log files. By default, these files are located in the directory where the Tivoli Storage Manager backup-archive client is installed.

How to backup and restore Lotus Domino mail servers using IBM TSM for Mail - TDP Domino

00:20 Posted by Anil K Y Ommi No Comments

To backup and restore your Lotus Domino Mail Servers using IBM Tivoli Data Protection for Domino, you should first install & configure compatible TDP Domino & TSM BA Client , configure and register a node in TSM Server accordingly. It is recommended to check the compatibility between your Lotus Domino, operating system and TDP Domino Versions. If possible try to install same version of BA Client & TDP Domino packages to prevent annoying errors.

Backup and Restore Lotus Domino Mail Server using IBM TDP Domino

1) You can start taking backup of your domino server either by command-line or through GUI. Open TDP SQL GUI by going to START>PROGRAMMS>TIVOLI STORAGE MANAGER>TDPdomino GUI. You can also double click domdsmc.exe file to open GUI in C:Program Files/Tivoli/tsm/domino folder.

2) Go to the BACKUP tab and select the mail server you want to backup as shown below.

3) You can select indivdual .nsf files for backup or you can backup group of .nsf files.

4) To take backup through commandline, go to the domino directory and run domdsmc i command for incremental backup as shown below.

5) If you want to check the status of previous backups, run the command domdsmc query dbbackup as shown below.

6) Similarly, you can go to the RESTORE tab on TDP Domino GUI and select the files which you want to restore. Alternatively to restore the domino backups through command line , run the command domdsmc restore as shown below

30 March 2014

How to install and configure Tivoli Data Protection for IBM Lotus Domino Mail Server (TDP Domino)

23:42 Posted by Anil K Y Ommi No Comments

Before you install Tivoli Data Protection for Lotus Domino Mail Server (TDP for Domino) in a Windows environment, make sure your system meets the minimum hardware, software and operating system requirements. You need to install compatible TSM Backup Archive Client and API software before installing TDP Domino Software. Data Protection for Domino must be installed from an account that is a member of the local Administrators group for the machine on which the Domino server is running.

Installing Tivoli Data Protection for Lotus Domino

1) First, Log on as an administrator. Go to the path where the TDP Domino executable file is located and double click the .exe setup files.

2) Select the language in which to perform the installation as shown below.

Click OK to start the installation program.

3) Read the License Agreement. You must accept the License Agreement in order for Data Protection for Domino to install successfully.

4) After accepting the License Agreement, click Next and choose the folder where you would like to install Data Protection for Domino. It is recommended to choose the default path only unless you have different configuration requirements.

The default installation directory is C:\Program Files\Tivoli\TSM\Domino\

5) New, select the setup type, default is TYPICAL. If you want to install only selected products choose CUSTOM option and then click NEXT.

6) On the next screen click INSTALL to start the installation process.

Then click Finish to complete installation.

7) You can also upgrade from a previous version of the software by using the same procedure. After installing or upgrading the IBM Tivoli Data Protection for Lotus Domino, the next step is to configure the TDP Domino to run a manual or scheduled Domino Mail server backups.

Configuring IBM Tivoli Data Protection for Lotus Domino

1) Data Protection for Domino communicates with several product APIs in order to complete various functions. The Tivoli Storage Manager API is accessed in order for Data Protection for Domino to communicate with the Tivoli Storage Manager server. The Domino API is accessed in order to communicate with the Domino server during database operations, and DB2 enabled Notes data is accessed by communicating with the DB2 Recovery API. The communication protocols and option parameters are specified in the dsm.opt options file.

2) Change to the C:\Program Files\Tivoli\TSM\domino directory. Rename the dsm.smp file to dsm.opt. Edit the dsm.opt file to include these options

3) Register the TDP node to the Tivoli Storage Manager server with the following command

REG NODE <tdpdomnode> password

4) To register the licenses during configuration, copy the license file (domclient.lic) to the default installation location.

5) If a password file for the Domino user ID does not exist then you need to store the TDP node password. The easy way to store the PASSWORD locally is through GUI, open TDP DOMINO GUI or by using command “domdsm” from default installation path as shown below.

or through GUI

6) Verify that you can communicate with the Domino Server by running the domdsmc query domino command.

7) Verify that you can communicate with the Tivoli Storage Manager server by running the domdsmc query adsmserver command as shown below.

8) Data Protection for Domino is now ready for backup and restore processing. You can use either GUI or command line to take backup or restore.

9) The important configuration files for TDP Domino are shown below. These are located in the default installation directory.

20 March 2014

IBM Tivoli Data Protection (TDP) for Domino Basic Introduction and Features

05:51 Posted by Anil K Y Ommi No Comments

IBM Tivoli Data Protection (TDP) for Domino is an application that backs up and restores Lotus Domino databases and transaction logs. When archival logging is used on the Domino server, it archives transaction log files and retrieves them as required for a database recovery. Database backups and archived transaction log files are stored on Tivoli Storage Manager server.

Data Protection for Domino communicates with a Tivoli Storage Manager server using the Tivoli Storage Manager application programming interface (API).

Data Protection for Domino communicates with a Domino server using the Lotus Domino API.

IBM Tivoli Data Protection for Lotus Domino Server Features

Data Protection for Domino helps protect and manage Lotus Domino server data by allowing you to perform the following actions

Back up Lotus Domino NSF databases.

Back up DB2 enabled Notes databases when a DB2-enabled Domino server is available.

Restore DB2 enabled Notes databases when a DB2-enabled Domino server is available.

Maintain multiple backup versions of Domino databases.

Archive Lotus Domino transaction log files when archival logging is in effect.

Restore backup versions of a Lotus Domino database and apply changes since the last backup from the transaction log.

Restore Domino databases to a specific point in time.

Restore one or more archived transaction log files.

Expire database backups automatically based on version limit and retention period.

Expire archived transaction log files when no longer needed.

Obtain online context-sensitive, task, and conceptual help.

View online documentation for Data Protection for Domino.

Automate scheduled backups.

Restore Domino databases to an alternate server or partition.

Access Data Protection for Domino remotely using the Tivoli Storage Manager Web client.

Access Data Protection for Domino using the client GUI based on Oracle Javatm.

Access Data Protection for Domino using the command-line interface.

Backup only Domino NSF databases using TDP Domino

1) The backup and recovery API in Domino provides the capability to perform these tasks

Online full backups of individual databases.
Archives of the transaction log when archival logging is in effect.

2) Data Protection for Domino provides two types of database backups and an archive log function

Incremental Backup
Selective Backup
Archive Log Backup

3) The backup strategy you choose should be depending on your specific requirements regarding network traffic, backup window, and acceptable restore times. Your choice of strategy includes selecting the type of backup commands to use and the type of transaction logging to be done on the Domino server.

4) Data Protection for Domino can only back up transaction logs from a Domino server that has archival logging in effect. Transaction logs cannot be backed up from a Domino server that has circular or linear loop logging in effect.

5) Archival logging allows transaction log data to be archived on the Tivoli Storage Manager server so that changes to logged databases can be stored on the Tivoli Storage Manager server without having to perform a full backup. This allows a strategy with less frequent full database backups because changes to logged databases are available for restore in the archived transaction log files.

6) You can either take only Full backups or Full backup plus regular transaction log archives as a backup plan according to your requirements and available storage resources.

Backup only DB2 enabled Notes databases using TDP Domino

1) The following list provides a brief overview of key DB2 enabled Notes database backup features

The entire Domino DB2 database or separate DB2 Groups can be backed up.
The backup can be restored to an alternate database.
In a disaster recovery situation, the backup can be restored to the original Domino DB2 database.
Individual DB2 enabled Notes databases are copied from the alternate DB2 database to the Domino DB2 database.

2) DB2 enabled Notes databases are stored in a DB2 database and managed by a DB2 server. Data Protection for Domino provides the ability to back up the Domino 8 DB2 database and DB2 Groups (table space).

3) Note that a DB2 enabled Notes backup is significantly different than an NSF backup. An NSF database is backed up directly. A DB2 enabled Notes database is backed up indirectly as a DB2 Group. A DB2 Group (or DB2 table space) is really a collection of one or more DB2 enabled Notes databases.

4) DB2 provides a Tivoli Storage Manager Agent and a utility program (db2adutl) that interfaces with the DB2 Recovery API in order to manage Tivoli Storage Manager objects created on the DB2 server. Data Protection for Domino uses the DB2 Tivoli Storage Manager Agent through the DB2 Recovery API to back up and restore the Domino DB2 database and DB2 Groups (table space).

5) Data Protection for Domino provides three types of database backups:

DB2 Database Backup - Data Protection for Domino DB2 database backups create a selective backup image that can be used for disaster recovery of the Domino 8 DB2 database or for restoring individual DB2 enabled Notes databases. Only selective backup (db2selective) is provided for DB2 enabled Notes databases.

DB2 Group (table space) Backup - Data Protection for Domino DB2 Group backups create a selective table space backup image. This type of backup can only be performed after the DB2 database is enabled for roll-forward recovery.

Full DB2 Database and NSF Database Backup - Data Protection for Domino can perform a selective NSF database backup and a full Domino DB2 database backup in a single operation.

6) The following backup strategies can be used for DB2 enabled Notes database backup.

Full DB2 database backups only
Full DB2 database backups plus DB2 Group backups

Backup both NSF and DB2 enabled Notes databases using TDP Domino

Domino 8 environments that contain both NSF and DB2 enabled Notes databases can implement the following backup strategy

Perform full DB2 database backups and NSF selective backups on a regular basis.

Perform routine incremental backups of NSF databases to inactivate backup copies that have been deleted from the Domino server.

Perform regular DB2 Group backups if the DB2 database is enabled for rollforward recovery.

Perform routine archiving of the transaction log files if archival transaction logging is enabled on the Domino server.

Routinely inactivate the Domino server log file and routinely inactivate (and delete) DB2 objects from the Tivoli Storage Manager server.

Restoring NSF and DB2 enabled Notes databases using TDP Domino

Restoring NSF and DB2 enabled Notes databases using TDP Domino is a two-stage recovery process, RESTORE & ACTIVATION.

Restore

This is the first step you should do for a complete recovery of NSF and DB2 enabled Notes databases using TDP Domino. In this step it restores a single database or group of databases from Tivoli Storage Manager storage to the Domino server.

You can restore the database to a different database file name or to a different Domino server.

You can also restore a group of databases to a different directory and preserve existing file names. In addition, if you specify a point in time on the restore command, the most recent backup version prior to that time is restored.

To restore a database without applying updates from the transaction log, the two steps can be combined into one step by specifying /activate=yes during the restore command.

Activation

This is the second step of the two stage recovery process. This function brings restored databases online for use by the Domino server.

You can optionally apply transactions from the transaction log to update the database. Transactions can be applied up to a specific point in time or up through the most recent changes recorded in the transaction log. If archival logging is in effect, Data Protection for Domino automatically restores archived transaction log files as needed.

13 March 2014

Differences between IBM Tivoli Storage Manager (TSM) and EMC Avamar backup software technologies

00:33 Posted by Anil K Y Ommi No Comments

Working as a IBM TSM Administrator, it is our job to know the major advantages of Tivoli Storage Manager (TSM) on other backup software tools that are available in the market. Knowing these details will also helps your future career growth and prospects. In this post I will try to put some major advantages of IBM Tivoli Storage Manager over EMC Avamar. This comparison is made between IBM TSM V6 and EMC Avamar V5. There may be changes in these differences (IBM TSM vs EMC Avamar) when the new versions of both the backup tools are released.

EMC AVAMAR Architecture Basic Introduction

Avamar is a backup software tool acquired by EMC in November, 2006. This was EMC’s second major acquisition of enterprise level backup software, having acquired Legato in July of 2003.

Avamar uses an incremental forever architecture supported by policy based management with versioning, expiration and reclamation. It can simulate a traditional backup plan that appears to require full periodic backups.

It encrypts data in transit with 128 or 256 AES bit or proprietary encryption for data in transit and optionally at rest, compression, central administration and a dashboard.

The Avamar server runs on a single server for small environments or scales up by adding additional systems and redistributing workload with automatic load balancing.

Avamar is based on a grid architecture that stores only a single instance of any unique data for all clients that use the grid servers. The Avamar client (on the source server that hosts the application) scans its filesystem to identify new or changed files. The client then segments the data and uniquely identifies it using a SHA-1 hashing algorithm. If the client identifies that data segment is new the client sends the hash, called a segment ID, to the Avamar server where it checks if the data segment is new or already stored. If the data is new to the Avamar server it asks the client to send the data.

The Avamar server grid stores the data on a node within the grid based on the unique ID. Avamar is a distributed index based architecture and does not maintain a single meta database, rather it uses the self describing organization of the unique ID’s. The Avamar server takes checkpoints which are snapshots in case a roll back to a previous state is required.

Networker 7.4 SP 1 integrates with Avamar by offering a single agent that can be used for traditional backup using Networker or as an Avamar agent for backup with deduplication. Networker also provides configuration and management of the Avamar client through the Networker Management Console.

When an Avamar agent is used the data is backed up to an Avamar server, it is not backed up to a Networker server. When the traditional Networker client is used to back data up to a Networker server Avamar is not used as a backend data store for Networker. Avamar integration with Networker is currently limited to agent integration and administration.

Differences between IBM TSM and EMC AVAMAR Backup Software Tools

Performance

1) EMC Avamar is best supported for small and medium sized enterprises. It does not perform well in large enterprises or in stress full enterprises. Any organization which meet below setup would leads to very slow backup performance.

Large, high transaction databases >1TB
>10% daily change rates
Encrypted, Compressed, or “Noisy” Data
Image Files, Videos, X-rays

Tivoli Storage Manager is proven in its ability to backup large high transaction databases and “difficult” data including data that is encrypted, compressed, image file, videos and X-rays. Many of TSM’s most satisfied customers are heath care, energy and media companies with data intensity levels well beyond typical commercial applications.

2) Avamar has a performance impact on the customer’s application servers as it is a client side only implementation. Whereas TSM server side deduplication does not impact the application servers performance. TSM client side deduplication where the application server has adequate cycles to devote to deduplication. TSM can support both client and server deduplication to the same storage pool, getting the benefit of deduplication between application servers that have time to dedupe for themselves and those that don’t.

Hardware Administration

Avamar requires additional hardware and administrative expense for a second backup infrastructure assuming tape support is required either for lower cost disaster recovery or off site storage for long term archive. Whereas IBM TSM’s integrated tape support eliminates the need to create and maintain a second infrastructure to archive to tape, a capability used in most installations.

Avamar requires a minimum of a three server configuration for full function. Separate dedicated systems are required as “Access Nodes” for the database search function and also a separate dedicated system is required for the NDMP Accelerator for backup of Network Appliance filers and it appears an additional dedicated system is also required for backup of EMC Celerra. Whereas TSM only needs a single server to provide its services and can take advantage of servers with multi-core processors for higher levels of scalability and performance.

Scalabiity

Scaling Avamar requires substantial additional systems. The system that stores the de-duplicated data (“storage node”) is limited to storing 3.3TB of data and is limited to an 18 node system with 52.8TBs of data in standard form. One sophisticated and well supported Avamar user has reported that Amamar tops out at 12 to 13 TB in a cluster (Avamar ‘grid”).

TSM scalability of its disk storage pools has been demonstrated to 30TB using a single TSM server. Differences in the environment, such as average file size and dedupe chunk size, can vary scalability substantially but in this fairly typical TSM environment a TSM metadatabase of 368 GB db supported the 30TB of data in a deduplicated storage pool. In typical operational environments a single TSM server using deduplication is estimated to have scalability several times this size.

Tape Support

Avamar cannot directly write to tape. Avamar itself does not write data directly to tape, rather it uses its Data Transport function to send data to a third party backup product that can then write the data to tape. The Data Transport function is limited to the use of either Networker or Symantec NetBackup running only on Solaris, Linux or Windows servers (no AIX or HP-UX support) to write the data to tape, requiring a second backup infrastructure to buy, plan, install, operate and maintain. Data Transport is limited to Avamar grids up to 22 TB, which reduces the theoretical scale down for an Avamar Grid from 52.8 TB by nearly 60%.

TSM has robust tape support from its many server platforms and its use does not severely limit its scalability and, unlike Avamar, provides the function on TSM avoiding the need for a buy, plan, install, operate and maintain a second backup infrastructure.

Hierarchal Storage Management (HSM) Support

Avamar has no HSM to offset its scalability limits. Avamar is a disk only backup/restore process and cannot minimize the amount of tier 1 disk used to store unique data. The capacity of the storage node could be stretched by limiting the number of versions of files kept, reducing the ability to recover further back in time.

TSM has built in hierarchical storage management and allows the use of multiple levels of disk, tape and optical storage to provide different levels of service at the minimum expense for storage media.

Archival Support

Avamar has no built-in Archive support, instead of providing built-in support for archiving Avamar has an extra cost option that allows Avamar to be the back end data store for EMC Centera, which is of limited value since Centera has native file level deduplication.

TSM has built in archive support and does not require an expensive additional piece of hardware, such as EMC Centera, to deliver archive support.

Information Lifecycle Management Support

Avamar lacks the ability to actively participate as part of an automated Information Lifecycle Management Process. Applications server and workstations often build up unneeded files over time that consume disk space on both the primary and backup server.

TSM can use automation to archive files identified by Tivoli Storage Productivity Center (TPC) so they don’t take up space on the application system.

Database checkpoints Frequency

Avamar has a less than robust core infrastructure, database checkpoints are taken twice a day and allow roll backs to a previous point in time but there is no database transaction logging with a roll back/forward recovery process. Falling back to the last checkpoint, where only two checkpoints are taken per day, creates a potentially large recovery exposure where the latest backups are unavailable.

TSM uses DB2 and provides the highest level of data integrity to maintain its meta data base with transaction logging and a roll back/forward recovery processes.

Disaster Recovery Strategies

1) Avamar has limited disaster recovery options for the Avamar server. Replication to a second Avamar server is an optional module and is expensive. Traditional backup to Networker or other backup software is only practical for at most a daily backup.

2) TSM is legendary for the range of choices and depth of its disaster recovery capabilities

Vaulting of TSM database, recovery log, volume history information, device configuration information, DRP file (if using Disaster Recovery. Manager) and copy pools for storage at an offsite location.

Use of TSM server-to-server communications to enable enterprise configuration (multiple TSM servers), enterprise event logging and monitoring, and command routing.

TSM servers installed at multiple locations, optionally setup as peer to peer servers (that is, each server able to recover at the alternate site).

Use of TSM virtual volumes over TCP/IP connection to allow storage of TSM entities (TSM database backups, recovery log backups, and primary and copy storage pools, DRM plan files) on remote target servers.

Use of high bandwidth connections and data replication technology (such as IBM PPRC, EMC SRDF) to support asyschronous/sychronous data replication of TSM databases backups, recovery log backups, TSM database and recovery log mirrors, and storage pools.

Use of remote electronic tape vaulting of TSM database and recovery log backups, primary or copy storage pools. Extended distances can be achieved by using distance technologies, for example, extended SAN, DWDM, IP/WAN channel extenders.

Application Support

Avamar lacks application support for SAP and Informix.

TSM has wide application support including support for SAP and Informix.

FlashCopy Support

Avamar lacks support for disk array copy services (EMC snap or IBM flashcopy).

TSM supports IBM Flashcopy for Exchange, DB2 and MySAP on IBM San Volume Controller (SVC), IBM DS6000 and DS8000.

VMWare Support

Avamar recommends their customers to use the newest API to backup VMware. The use of the new VMware vStorage API for backup was needed by Avamar to overcome problems with it prior options for VMware support. Many customers have only recently just shifted to the use of the VMware Consolidated Backup (VCB) for backups and are again likely to take a conservative approach to adoption of the new technology, preferring instead to let bleeding-edge customers pilot the new vStorage API for Data Protection from VMware. Like other backup products Avamar is carrying forward the use of the popular VMware Consolidated Backup (VCB) API but it creates the unacceptable administrative burden of requiring that each virtual machine (VM) must have its own group to schedule VCB backups which must be setup and maintained one at a time. At one point to bypass the VCB admin issue EMC was recommending the use of the Image backup based on an agent running in an ESX service console, even though this posed a serious performance issue for the service console.

TSM supports the new VMware vStorage API. TSM also continues to support the more popular VMware VCB without the administrative burden.

Data Security

Avamar is a security exposure. The Avamar File System is not able to limit file access on a user-by-user basis. All files stored on a server are viewable by any user with access to Avamar File System.

TSM has client level access for restores to provide data security.

Centralized Client Maintenance

1) Avamar lacks automated and centralized client maintenance , just offering “support” for the push install of the clients on Windows and Mac clients by common systems management tools, such as Microsoft’s Systems Management Server 2003 (SMS).

2) Like Avamar, TSM does not automate the initial client install. Unlike Avamar TSM through the TSM Admin Center provides a rich set of client software maintenance which can

discover client maintenance levels on the TSM FTP site
download needed packages and store them on the TSM server
manage packages stored on the TSM server (retention, etc.)
select a maintenance level and distribute to a list of clients. The code will then be distributed based on a predefined schedule.
review the client distribution status

Data Deduplication

The superficial integration that Avamar has with Networker is an attempt to cover up the inherent disadvantages of having two separate code bases that must be installed and maintained: a legacy product that couldn’t reasonably he enhanced to support built in deduplication and a deduplication product that does not scale well enough to even be used as a back end data store to provide deduplication for the legacy enterprise data center product. The agents are packaged together and not integrated. Avamar needs its own management facilities and can’t run just using the Networker management function. The monitoring and reporting function Networker provides was so weak that EMC had to acquire Wysdm to support Avamar. Contrast that with Tivoli’s built-in deduplication which is truly integrated with a multitude of advantages starting with eliminating the need to understand, deploy and manage two completely different infrastructures and ending with eliminating the need to bring in yet a third product to do centralized reporting.

Avamar deduplication to tape has many limitations. EMC recommends its use only for infrequent long term data retention use cases for performance reasons. It does not match the ability of TSM to mix the use of tape and disk in normal operational environments where restores are likely or frequent.

Avamar itself does not store the data to tape, it requires the use of either Networker or Symantec NetBackup running only on Solaris, Linux or Windows servers (no AIX or HP-UX support) to write the data to tape after it is deduplicated by Avamar, requiring a second backup infrastructure to buy, plan, install, operate and maintain.

CPU Throttling

Avamar has CPU throttling to control resource use on the application server. Avamar is limited to the use of client side deduplication and uses CPU throttling in an attempt to not impact the application server. TSM server side deduplication is a better choice when performance concerns exist over using application server resources for deduplication.

Continuous Data Protection (CDP)

Avamar Lacks continuous data protection. Avamar has been positioned by EMC to backup frequent point in time copies by CLARiiON disk snapshot (like IBM Flashcopy) the solution is extremely expensive and still does not provide the frequency of data protection provided by TSM FastBack.

Tivoli FastBack has (real time replication) continuous data protection which saves data throughout the day, greatly reducing the amount of data lost compared to the typical use case of nightly backup by Avamar.

Time for Restoration

Avamar restore time is lengthy and is not appropriate for application that require higher levels of availability.

Tivoli FastBack’s near instant restore does not require the restoration of data from the backup copy to a different disk accusable by the application server, rather with FastBack, the backup data becomes the production data, nearly instantly.

Bare Machine Recover (BMR) Support

Avamar lacks integrated bare machine recovery, requiring the use of a separate product such as EMC HomeBase for restoring the Avamar server in case of disaster.

TSM FastBack has integrated Bare Machine Recovery.

10 March 2014

How to improve the SAP database backup and restore performance using TDP for ERP (SAP for oracle) ?

22:53 Posted by Anil K Y Ommi No Comments

The performance of SAP database backup using Data Protection for SAP for Oracle depends upon on the three components. These three components have the greatest impact on data transfer rates

The type of disks on which the database resides.

The network capabilities accessed by the database host and the Tivoli Storage Manager server.

The type of storage device that contains the backup.

A continuous stream of data is generated among these components during a backup or restore operation. The weakest component in this stream decreases the overall data transfer rate.

Tivoli Data Protection (TDP) for SAP for Oracle backup Performance Tuning

Data Protection for SAP for Oracle is configured (by default) to send uncompressed data to the Tivoli Storage Manager server using a single session. Consider to modify these below components to optimize the SAP database backup or restore performance. It is recommended to modify these component parameters one at a time depending upon your environment to identify the individual components performance. These performance tuning parameters can be find in either initSID.utl, initSID.sap and dsm.sys files.

Parallel (Multiple) Sessions

Data Protection for SAP can back up or restore data to multiple tape drives in parallel. Parallelism is achieved by using more than one session to send data to a backup server.

Multiplexing

Multiplexing simultaneously transfers data from different files through one session (MULTIPLEXING) in order to maximize tape performance. Multiplexing is useful for tape storage since tape drives often have higher data transfer rates than the disks. Combining multiplexing and parallel sessions can optimize overall backup and restore performance.

Disk Sorting

Data Protection for SAP uses Adaptive File Sequencing during backup processing. This feature sorts database files in sequential order to avoid simultaneously reading files located on the same disk. As a result, backup processing time is reduced.

Multiple (Parallel) Network Paths and Multiple (Parallel) Servers

Improve performance by configuring Data Protection for SAP to distribute a database backup across two or more Tivoli Storage Manager servers. In addition, you can balance network traffic by providing two (or more) separate network connections between the SAP database host and the Tivoli Storage Manager server.

Incremental Backup

Data Protection for SAP supports incremental RMAN backup of a SAP databases. Depending on the system environment, incremental backups might decrease backup processing time.

Individual Tablespace Locking

Data Protection for SAP provides a backup profile parameter (util_file_online) that minimizes the number of archived redo logs backed up during online backup operations. This parameter informs the SAP database utilities of the files (and related table spaces) to be backed up. The SAP utilities then switch those table spaces into backup mode. After the files are backed up, the table spaces are released and a new cycle starts.

Buffer Copies (BUFFCOPY)

Data Protection for SAP for Oracle uses internal buffers to store and exchange data with Tivoli Storage Manager. When sending data from one component to another, data buffers are copied (by default). Data Protection for SAP can prevent copying the data buffers by sending the original data buffers. This reduces the CPU load of the database server. However, if client compression or client encryption are specified in the Tivoli Storage Manager options file then the original data buffers are sent.

Buffer Size (BUFFSIZE)

Data Protection for SAP for Oracle allows the size of the internal data buffers to be adjusted. These buffers are used for both reading the disk and sending data to the Tivoli Storage Manager client API. The default values typically produce acceptable performance. It is recommended to optimize the buffer size for disk I/O. For disk subsystems, the best transfer rates have been achieved when the buffer size was set equal to the stripe size. Before increasing the size of internal buffers, however, make sure that sufficient storage is available for the number of buffers specified by Data Protection for SAP. This number correlates to the number of sessions requested. Be aware that number of buffers doubles when compression is specified.

Compression (RL_COMPRESSION)

Data Protection for SAP for Oracle can decrease the amount of data sent to the Tivoli Storage Manager server by compressing zero-byte blocks. Although compression can increase the CPU load on the database server, it can improve performance in situations when the network is the point of constraint. Compression is most effective with database files that contain large portions of null blocks.

Multiple Servers (SERVER)

Data Protection for SAP for Oracle supports multiple servers which can distribute backup data among two (or more) backup servers. This feature helps eliminate constraints that are frequently encountered among backup servers. A server statement must be entered in the Data Protection for SAP profile for each adapter of the backup server as described for the SERVER keyword.

Multiple Sessions

Data Protection for SAP for Oracle allows use of multiple tape drives simultaneously in order to increase the transfer rate to or from the Tivoli Storage Manager server. The keywords MAX_SESSIONS, MAX_BACK_SESSIONS, MAX_ARCH_SESSIONS and MAX_RESTORE_SESSIONS are used for defining the number of parallel sessions to be established with the Tivoli Storage Manager server for database backup, archive (backup of log files) and restore. The parameter specified in the MAX_SESSIONS keyword must match the number of tape drives used simultaneously. The number of storage pools that are required depends on the number of backup copies requested for a log file.

Multiplexing (MULTIPLEXING)

Multiplexing is using parallel access to data on the database server. This is recommended when using a tape drive during database backup operations on the backup server. The value of keyword MULTIPLEXING defines the number of files read in parallel within a single session. Appropriate MULTIPLEXING values are expected in the range of 1 to 4. The best value for your environment depends on the I/O rate of your disks, the location of your data on the disks, the network capacity, the throughput rate of the storage media, and the compression setting. If the MULTIPLEXING value is too high, a thread management overhead may be occur that might offset any performance gain.

Multiple Network Paths

Data Protection for SAP for Oracle allows you to use multiple network connections (paths) for data transfer between the database server and the backup server. Parallel paths can be used to eliminate network points of constraint. For each additional path, additional network adapters are required on both the production and the backup server. A server statement must be entered in the Data Protection for SAP profile for each adapter of the backup server. The value of the MAX_SESSIONS keyword is not greater than the sum of all SESSION values specified for the SERVER statements used concurrently.

The SERVER <server 1..n> statement denotes Tivoli Storage Manager servers defined in the Data Protection for SAP profile. This corresponds to the statement SERVERNAME server 1..n in the Tivoli Storage Manager client option file(s). These servers are identified by their TCPSERVERADDRESS and can be located on one system or several systems (multiple servers). SESSIONS denotes the number of parallel session that Data Protection for SAP schedules for the given path. If only one path is used, SESSIONS must be equal to MAX_SESSIONS, which specifies the total number of parallel sessions to be used. Data Protection for SAP attempts to communicate with the Tivoli Storage Manager server using the first path in the profile. If this proves successful, Data Protection for SAP starts the number of parallel sessions as specified for this path. If the attempt was unsuccessful, this path is skipped and Data Protection for SAP continues to the next path. This process continues until as many sessions are active as were specified in the total session number (MAX_SESSIONS). If this number is never reached (for example, because several paths were inactive), Data Protection for SAP terminates the backup job.

9 March 2014

How to configure SAP Database backup using TSM for ERP (TDP SAP for oracle) and RMAN utility

22:46 Posted by Anil K Y Ommi 1 Comments

To take SAP database backup using IBM TSM for ERP SAP, you can use the backup and recovery tools which are provided with SAP. Backup and recovery tools are either an integrated component of a relational database system, such as Oracle Recovery Manager (RMAN) or may be installed as an independent software package developed by a third party such as SAP BR*Tools for Oracle. In this post we will see how to take SAP database and archive logs backup using TDP for ERP and Oracle RMAN utility.

Oracle RMAN Utility Features

1) Recovery Manager (RMAN) is the Oracle proprietary backup and recovery management tool. RMAN is included as a part of Oracle RDBMS and provides users with backup and restore functionality as well as other database management tasks. RMAN can back up data files, archived redo logs, control files, and configuration files (such as the init file, password file, tnsnames.ora, and sqlnet.ora). RMAN supports functions for online or offline and partial or full backup. It also supports an incremental and cumulative incremental backup. RMAN operations can be controlled by the command-line interface or from the Oracle Enterprise Manager GUI.

2) You can take backup SAP database either through RMAN only or integrating RMAN with brtools. RMAN provides for the following basic backup functions

Online full backup
Offline full backup
Partial (tablespace or datafile) backup
Incremental backup
Controlfile and configuration files backup

3) You can also back up archived redo logs with the RMAN command BACKUP ARCHIVELOG. Additionally, you can instruct RMAN to back up archived logs after backing up data files and control files by specifying BACKUP ... PLUS ARCHIVELOG. By archiving the redo logs immediately after the online backup, you ensure that you have a full set of archived logs through the time of the backup.

4) When performing database restore or recovery from the Tivoli Storage Manager server, you first have to allocate the RMAN channel based on the Tivoli Storage Manager SBT adapter. As soon as the channel is allocated, you can instruct RMAN to perform the restore or recovery of a database or the restore of control files. RMAN provides the following basic restore and recovery options

Database full restore
Database partial restore
Control file restore from auto backup
Database recovery

Integration of BR*Tools with Oracle RMAN

You can configure Oracle RMAN as a backup agent sitting between BR*Tools and the Tivoli Storage Manager server. In such a configuration, BR*Tools calls RMAN to perform the backup and restore operations on the database. In this configuration, BR*Tools is instructed to dynamically allocate the RMAN SBTAPI channel. The channel is then used to transfer data to and from the Tivoli Storage Manager server.

The integration of BR*Tools and RMAN enables BR*Tools to exploit the advantages of RMAN backup and restore functions, such as block checking during the backup and online backup, without needing to change the mode of data files (using the alter tablespace begin backup command). This configuration also enables BR*Tools for incremental backups, which are not supported when using the other BR*Tools interfaces (BACKINT and dd).

If you perform incremental backup, it can significantly reduce the requirements on the capacity of the backup storage media. Depending on the system environment, this may also lead to a significant decrease in backup times. To implement such a configuration, you have to configure RMAN as a BR*Tools backup interface.

The advantages of integrating RMAN with BR*Tools are Database blocks are checked for logical errors during the backup, Only the used database blocks are backed up, Tablespaces do not need to be set to BEGIN BACKUP mode so the amount of logs generated during the backup is reduced, BRBACKUP integrated with RMAN supports cumulative incremental backup at level 1.

RMAN and BRTOOLS use below process to take the SAP database backup

BR*Tools utility BRBACKUP informs Oracle RMAN what data has to be backed up. The RMAN process puts the database into the proper backup state, online or offline.

The Oracle server process loads Data Protection for SAP and interacts with it through the Media Management API.

Data Protection for SAP reads the requested data from the database and reports back to BRBACKUP. BRBACKUP adds this data to the repository containing all processed backups.

Data Protection for SAP saves the data to a Tivoli Storage Manager server.

BR*Tools updates the repository containing information about the status of the data. RMAN has its own repository, with a control file for a separate recovery catalog database.

Steps to configure IBM TDP for SAP for taking SAP database backup using RMAN and BR*Tools utilities

A database backup consists of two steps, Initially BRBACKUP calls the backup function of Oracle RMAN. RMAN reads the database blocks to be backed up and sends it to the Tivoli Storage Manager media management library. The Tivoli Storage Manager media management library calls the Tivoli Storage Manager client API to send backup data to the

Tivoli Storage Manager server. After the database backup is completed, BRBACKUP sends the Oracle controlfile, BR*Tools initialization profiles, and BR*Tools log files to the Tivoli Storage Manager server using the BACKINT adapter.

You can perform the following steps to configure BR*Tools to use the Oracle RMAN utility with Tivoli Storage Manager for ERP

Steps to be performed on the Tivoli Storage Manager server

Define a policy domain with two management classes that will be used to transfer data and logs.

Define an archive management class within each of the management classes. If the retention control will be performed at the Tivoli Storage Manager server, specify RETVER= number of days for each archive copy group. If the retention control will be performed at Tivoli Storage Manager for ERP, specify RETVER=nolimit.

Update the parameter MAXNUMMP for the Tivoli Storage Manager node to the value according to your environment MAXNUMMP= value.

Steps to be performed on the client node

Update or create DSM.OPT and DSM.SYS to configure the Tivoli Storage Manager API client. The parameter PASSWORDACCESS must be set to PROMPT in this configuration.

Set up the environment values DSMI_DIR and DSMI_LOG for the Oracle OS user.

Install Tivoli Storage Manager for ERP (Oracle) on the Oracle server, with SAP already installed.

Configure the client resources for Oracle server in the Tivoli Storage Manager for ERP configuration file (<ORACLE_HOME>\dbs\init<SID>.utl). Check the defined Tivoli Storage Manager node name and Tivoli Storage Manager management classes to be used for the backup of offline redo log files and data files.

Make sure that the SERVER parameter refers to an existing stanza in the dsm.sys file. If the retention control will be driven by Tivoli Storage Manager for ERP, set the MAX_VERSIONS parameter.

Switch to the Oracle instance owner and update the Tivoli Storage Manager node password for Oracle using the following command:

cd <ORACLE_HOME>\dbs

backint -p init<SID>.utl -f password

Make sure that RMAN can access the Tivoli Storage Manager SBTAPI. The following links must exist

ln -s /usr/tivoli/tsm/tdp_r3/ora/libtdp_r3.<ext> /usr/lib/libobk.<ext> ln -s /usr/lib/libobk.<ext>

Now the important step, Instruct BR*Tools to use RMAN by setting the backup_dev_type and rman_parms in the SAP initialization file (init<SID>.sap) as follows:

backup_dev_type = rman_util

rman_parms="ENV=(XINT_PROFILE=<ORACLE_HOME>/dbs/init<SID>.utl,PRO

LE_PORT=<portnumber>,&BR_INFO)"

Instruct BR*Tools to use the file init<SID>.utl for Tivoli Storage Manager specific parameters by setting the util_par_file parameter in the SAP initialization file:

util_par_file= /path/init<SID>.utl

You should see a prole process running in the background after all these configuration. The purpose of this entry is to start a daemon process for ProLE. This process listens on the Data Protection for SAP for Oracle port tdpr3ora64 for backint and RMAN connections and sends performance-related information to the Administration Assistant Server component

Now Finally after all these configuration steps, you can either manually call BRTOOLS manually or schedule it through DB13 from SAP to start taking the backup. To call BRTOOLS manually, log in as oracle user and run brtools command as shown below.

Use, options 4 & 5 and follow the steps after to take backup or restore of SAP Database and archive redo-logs using SAP BRTOOLS utility.

Storage & Backup Tutorials

Categories

Follow us on Google+