Wednesday, July 23, 2014

ORA-29702 While Starting 10.2.0.4 Instance(s) on 11.2.0.3 RAC/Clusterware

I was in process of upgrading my 3 nodes 10.2.0.4 RAC database to 11.2.0.3 RAC on Windows 2008. Since we had new hardware, we decided to build a new 11.2.0.3 RAC and create a 10.2.0.4 standby on the new RAC and then do a switch-over and immediately after that, perform the upgrade of switched over database. To accomplish this task, we followed following steps

1) Build a new 11.2.03 RAC by installing 11.2.0.3 GI and RDBMS
2) Install 10.2.0.4 on this new RAC
3) Build standby database on this new RAC setup
4) Perform switchover to this new RAC standby
5) Upgrade 10.2.0.4 to 11.2.0.3
6) Perform any post upgrade tasks

Problem
On step 3 while building standby database, when I tried to start the instances, it failed to start with error "ORA-29702: error occurred in Cluster Group Service operation".

Reason
In Oracle 11.2 RAC, there is a concept of "pinning of nodes". By default, RAC nodes are not pinned, 
click here for the details.
10g is an older version in which, by default, nodes are actually "pinned" (having consistent configuration information), whereas in 11g; functionality is different and this error was appearing because of this reason.

Solution
Pinning of nodes resolved this issue. Following are the steps to check current nodes configuration and how to pin the nodes.

#####Getting information

%GRID_HOME%\bin\olsnodes -t
node01 Unpinned 
node02 Unpinned
node02 Unpinned 

#####Pinning the nodes by executing following from node1
crsctl pin css -n node01 node02 node03 
#####Getting information
%GRID_HOME%\bin\olsnodes -t
node01 1 Pinned
node02 2 Pinned
node02 3 Pinned 

After pinning the nodes, I was able to start all standby instances.

No comments:

Post a Comment