Monitoring RMAN Restore and Recovery Progress

 While we start a restore and recovery process for database or datafiles, next step is to monitor the progress of restore and recovery process. This is particularly important when end user or customer is keen to learn the approximate completion time of recovery when a critical system is down because of database/datafile corruption. In this article, I would explain how we monitor the recovery of database, and approximate time of completion for the recovery process.

Renaming Datafile(s) in Standby Database

 Renaming a datafile on standby database is a quite simple process. If you are using ASM diskgroups as storage, the primary and standby databases already have different locations and names, and you can use RMAN command “switch datafile/database to copy” to move a datafile to a new location. This method is explained here as part of standby database incremental restore. If you are using file system for datafiles storage, you might sometimes need to relocate one or more datafiles to a different location; for example if file system is full. Following are the steps that you can follow to relocate a datafile in this case.

CRS-4535: While Starting CRS

Follow this document to understand the process of stopping and starting the CRS or Cluster. But, there is a possibility that CRS got stopped automatically because of some reason (for example: if file system where CRS/Grid Infrastructure is installed gets full that mames CRS to crash). In this case, if you try to start, stop or check the status of CRS, you may be returned CRS-4535. There could be other reasons for this error message as well.

ORA-16541: database is not enabled

 Accidentally or purposely, we sometimes need to disable a database from DG broker configuration. This is done by using command on DGMGRL.

[oracle:DB_DG ~]$ dgmgrl / DGMGRL for Linux: Version 11.2.0.4.0 - 64bit Production Copyright (c) 2000, 2009, Oracle. All rights reserved. Welcome to DGMGRL, type "help" for information. Connected.

Error: ORA-16765: Redo Apply is running

DGMGRL for Linux: Version 11.2.0.4.0 - 64bit Production Copyright (c) 2000, 2009, Oracle. All rights reserved. Welcome to DGMGRL, type "help" for information. Connected. DGMGRL> show configuration   Configuration - TEST DG configuration

Error: ORA-16724: cannot resolve gap for one or more standby databases

 If there is an archive gap and dataguard database is not fully in synch with the primary database, initiating DGMGRL and connecting to the database would immediately show ORA-16724 with the configuration. This can be seen bellow.

Standby Database Archived Redo Logs Gap and Incremental Restore

Archive log gap is a very common issue that DBAs face every now and then. Normally standby instance would automatically fetch the required archived logs in case of any gap, but there could be scenarios when missing archived logs cannot be found, restored or fetched. In some scenarios, standby database me be several weeks behind the primary and restoring and applying huge number of archived logs could take very long time. In such case, we can take incremental backup of primary database and restore it on the standby to make in in sync with primary. I will explain this procedure in this article.

ORA-17627: ORA-01017: invalid username/password; logon denied

 If you are doing a duplicate database from active database and commands fails with a combination of ORA-17627: ORA-01017, as follows

RMAN> duplicate target database to dupdb from active database;  Starting Duplicate Db at 23-FEB-18 using target database control file instead of recovery catalog allocated channel: ORA_AUX_DISK_1 channel ORA_AUX_DISK_1: SID=4201 device type=DISK allocated channel: ORA_AUX_DISK_2 channel ORA_AUX_DISK_2: SID=4306 device type=DISK allocated channel: ORA_AUX_DISK_3 channel ORA_AUX_DISK_3: SID=4411 device type=DISK allocated channel: ORA_AUX_DISK_4 channel ORA_AUX_DISK_4: SID=4516 device type=DISK current log archived at primary database

Checking Redo Log Switch and Redo Generating by Session

Focus of this article is to check the redo log switch frequency as well as the sessions causing huge redo logs generation. A sudden surge in archived log generation can make DBA look into the issue to find out since when redo log generation has spiked. Following is the query we can easily find everywhere on the internet to find out the redo logs switch frequency. Of course this is not written by me. I do not know who is the original writer of this query therefore I can refer to that great guy or website.

Dataguard out of sync OR Log shipping not working

 You can seek help from this (single node) and this (RAC) document for setting up dataguard. But if dataguard has become out of sync, there are several things that you might need to look at before you fix it. In the following I am providing a checklist that you can take a look to find out the issue and thus implement a solution.