Archive for May 2014

13 May 2014

Troubleshooting TDP SAP backup error "BKI4008E: File cannot be opened. Reason: errno(22) A system call received a parameter that is not valid"

08:40 No Comments
The following error will be seen while trying to take the SAP Oracle database backup using Tivoli Data Protection (TDP) for SAP with Oracle software through brtools and backint interface.

BR0280I BRCONNECT time stamp: 2014-05-08 17.10.09
#BEGIN /oracle/QDI/sapdata1/sysaux_1/sysaux.data1
#BEGIN /oracle/QDI/sapdata1/system_1/system.data1
BR0315I 'Alter tablespace SYSAUX begin backup' successful
BR0315I 'Alter tablespace SYSTEM begin backup' successful
BR0280I BRCONNECT time stamp: 2014-05-08 17.10.09

BKI4008E: File '/oracle/QDI/sapdata1/sysaux_1/sysaux.data1' cannot be opened. Reason:  errno(22)  A system call received a parameter that is not valid.

BR0280I BRCONNECT time stamp: 2014-05-08 17.10.13
#END /oracle/QDI/sapdata1/sysaux_1/sysaux.data1
#END /oracle/QDI/sapdata1/system_1/system.data1
BR0317I 'Alter tablespace SYSAUX end backup' successful
BR0317I 'Alter tablespace SYSTEM end backup' successful
BR0280I BRCONNECT time stamp: 2014-05-08 17.10.13

BKI1208W: The object '/oracle/QDI/sapdata1/sysaux_1/sysaux.data1' will be retried [3].
BKI1208W: The object '/oracle/QDI/sapdata1/system_1/system.data1' will be retried [3]

The above error doesnt show much information about the error and it will be quite annoying if you have no idea about how SAP and AIX works together. You will see this kind of error if your SAP database filesystems are mounted with the CIO (Concurrent I/O) option enabled. Concurrent I/O (CIO) for JFS2 filesystems gains the benefits of Direct I/O without the locking serialization. It allows a process writing to a file exclusive write access to that file. 

You need to remount the filesystems with CIO option disabled to fix this problem. Apparently, this requires the downtime of your SAP system.

Follow these below steps to troubleshoot this BKI4008E error permanently

1) First, STOP your SAP system (database & application).

2) Check which filesystems have CIO enabled by using lsfs command

      # lsfs

/dev/fslv03     --         /oracle/PRD/origlogA      jfs2  1048576 rw,cio         yes  no
/dev/fslv04     --         /oracle/PRD/origlogB      jfs2  1048576 rw,cio         yes  no
/dev/fslv12     --         /oracle/PRD/oraarch        jfs2  20971520 rw              yes  no
/dev/fslv13     --         /oracle/PRD/sapreorg      jfs2  2097152 rw               yes  no
/dev/fslv14     --         /oracle/PRD/sapdata1      jfs2  41943040 rw,cio       yes  no
/dev/fslv15     --         /oracle/PRD/sapdata2      jfs2  94371840 rw,cio       yes  no
/dev/fslv16     --         /oracle/PRD/sapdata3      jfs2  4194304 rw,cio         yes  no
/dev/fslv17     --         /oracle/PRD/sapdata4      jfs2  115343360 rw,cio     yes  no
/dev/fslv21     --         /eccerp6                           jfs2  4194304 rw                   yes  no

3) After making sure that SAP is down, unmount all the filesystems which have CIO enabled one by one. I am showing only for 2 filesystems, you need to change all the filesystems which have CIO enabled.

      # umount /oracle/PRD/origlogA
      # umount /oracle/PRD/sapdata2

4) Disable the CIO option for the filesystems by using chfs command as shown below

     # chfs -a options=rw /oracle/PRD/origlogA
    # chfs -a options=rw /oracle/PRD/sapdata2

5) Then remount the filesystems. You should disable CIO option for all the SAP filesystems.

      # mount /oracle/PRD/origlogA
     # mount /oracle/PRD/sapdata2

6) Now, if you run lsfs command, the output should be somewhat like

/dev/fslv03     --         /oracle/PRD/origlogA      jfs2  1048576 rw               yes  no
/dev/fslv04     --         /oracle/PRD/origlogB      jfs2  1048576 rw               yes  no
/dev/fslv12     --         /oracle/PRD/oraarch        jfs2  20971520 rw              yes  no
/dev/fslv13     --         /oracle/PRD/sapreorg      jfs2  2097152 rw                yes  no
/dev/fslv14     --         /oracle/PRD/sapdata1      jfs2  41943040 rw              yes  no
/dev/fslv15     --         /oracle/PRD/sapdata2      jfs2  94371840 rw              yes  no
/dev/fslv16     --         /oracle/PRD/sapdata3      jfs2  4194304 rw                yes  no
/dev/fslv17     --         /oracle/PRD/sapdata4      jfs2  115343360 rw            yes  no
/dev/fslv21     --         /eccerp6                              jfs2  4194304 rw                yes  no

7) After making sure all the SAP filesystems are correctly mounted with CIO disabled, START SAP application and database.

8) Finally, now you can restart your SAP database backup using SAP brtools and backint interface and the backup should complete successfully.

Check this below IBM technote for more info

10 May 2014

Steps to restore TDP SAP for Oracle production (PRD) database to another SAP oracle database (DEV or QTY)

00:13 No Comments
You might required to restore the current SAP oracle database to another SAP database according to the clients requirement. For example, it is required to restore the SAP production database to SAP development/quality database for testing any upgrade/modifications on the existing database before making it live. It is the TSM administrators job to restore the database within the given downtime. TSM administrator should also make sure that the offline backup of SAP production database is successful before going for restoration process. In this post we will see how to restore Production (PRD) SAP database to Quality (QTY) SAP database.

How to restore SAP Oracle database backup to another SAP system ?

Step 1: First, you need to plan for a offline backup of production SAP database and it should be successful. Make sure that the tapes used for this backup are secure and are inside the library slots for immediate mounting. 

Step 2: Make all the SAP file systems on QTY server like PRD server with  same size (or +1GB % utilize). If filesystems already exist then clean or empty them. Any SAP filesystems will be in the format of 

/oracle/SID/sapdata1
/oracle/SID/orilog1
/oracle/SID/sapreorg
/oracle/SID/saparch1
/oracle/SID/saptrace/usertrace
/oracle/SID/saptrace/background
/oracle/SID/sapcheck
/oracle/SID/sapbackup

Here SID is the Oracle SID - ideally PRD for production & QTY for quality systems. You have to remove any existing data in these filesystems by using rm -rf command (command depends on OS).

Step 3: In this step you have to copy all the TDP SAP configuration files from Production system to the Quality system. The TDP SAP configuration files and their locations are 
  • /oracle/PRD/102_64/dbs/initPRD.sap
  • /oracle/PRD/102_64/dbs/initPRD.utl

You need to copy these files and paste them in the same location in the Quality System by changing their name as shown below

  • /oracle/QTY/102_64/dbs/initQTY.sap
  • /oracle/QTY/102_64/dbs/initQTY.utl
If files with the same name already exists, then rename the old files with another name before copying above files.

Step 4: In TDP SAP, after the database backup (online & offline) completes it generates the log file. For offline the log file extension is (.aff) and for online backup the log file extension is (.anf). Because we are restoring latest offline backup we need to copy the latest .aff file  from production system to quality system. The file location is shown below.

 Copy from /oracle/PRD/sapbackup/brnnfghrk.aff and paste it in /oracle/QTY/sapbackup filesystem.

Step 5: From the same location, we also have to copy the backPRD.log file to the same location in the quality system.

Step 6: Next, we need to change permissions and ownership (dba user) of all the files which are copied in the above steps.

chown oraqty:dba initQTY.sap initQTY.utl
chmod 777 initQTY.sap initQTY.utl

chown oraqty:dba brnnfghrk.aff backPRD.log 
chmod 777 brnnfghrk.aff backPRD.log

Step 7: In this step, we need to open initQTY.sap & initQTY.utl files and change the PRD to QTY. Make sure you change these below important parameters

In initQTY.sap file
util_par_file=/oracle/QTY/102_64/dbs/initQTY.utl    (PRD  → QTY)
compress_dir = /oracle/QTY/sapreorg

In initQTY.utl file
CONFIG_FILE       /oracle/QTY/102_64/dbs/initQTY.bki    (PRD  → QTY)

Step 8: Next, we need to change the node in the /usr/tivoli/tsm/client/api/bin64/dsm.sys file. use the same node name which you have used for taking the production database backup.  You have to make sure that the SERVERNAME in all the files dsm.opt, initQTY.utl and dsm.sys are same.

cd /usr/tivoli/tsm/client/api/bin64/dsm.sys
vi dsm.sys
nodename PRDnode

Step 9: Next, we need to authenticate the TDP node with TSM server. This TDP node name is nothing but the production TDP node name, we have to authenticate PRD TDP node from QTY system.To do this you have to login using oracle user

su - oraqty
cd /oracle/QTY/102_64/dbs
./backint -p initQTY.utl  -f password
or 
su - oraqty
cd /oracle/QTY/102_64/dbs
./usr/tivoli/tsm/tdp_r3/ora64/backint -p initQTY.utl  -f password

Type the TDP node password and press enter. It should successfully authenticate the password for the production node through which you have taken the offline backup.

Note: backint utility is by default located in /usr/tivoli/tsm/tdp_r3/ora64 directory. You can either make a soft link to the /oracle/PRD/102_64/dbs directory or use the whole path name for authenticating TDP node as above.

Step 10: Finally, restoring the backup data, you have to login as oracle user and run the following command to restore whole database.

su - oraqty
cd /oracle/QTY/102_64/dbs
brrestore –m full –b brnnfghrk.aff –p initQTY.sap –c

Note: You can run brrestore utility from anywhere if the environment variable setting is enabled for this utility. Otherwise you need to create soft link from /usr/sap/QTY/sys/exe/run to the /oracle/QTY/102_64/dbs directory.

Commands to restore single database and archivelog files

1) For restoring only single file use the following command

brretore –m /oracle/QTY/sapdata7/btabd_250/btabd.data250 –b brnnfghrk.aff  –p initQTY.sap  –c

2) For restoring one database file use the following command

brretore –m /oracle/PRD/sapdata38/btabd_250/btabd.data250 –b  brnnfghrk.aff   –p initQTY.sap -c

3) For restoring archive files online use below steps
  • Copy archPRD.log to QTY server  
  • Change permission and owner as above
  • su - oraQTY
3.1) Restoring one archive file
brrestore -u system/passw0rd -a 519407  (No. of archive file)

3.2) Restoring multiple archive files
 brrestore -u system/passw0rd -a '519408-519498'  (range of archive files)
Subscribe to: Posts (Atom)