Saturday, June 30, 2018

Installing Oracle 12c R2 Grid Infrastructure on Linux 6 and Linux 7

You need to install 12c R2 Grid Infrastructure (for a standalone server) software in order to use ASM and/or Oracle Restart feature. In the following I will explain how to install GI for a standalone server on Linux 6 and Linux 7. You must use at least Linux 6.4 for installation on Linux 6. For this installation, I am using Oracle Linux 7

There are only a couple of differences between the installation procedures on Linux 6 and Linux 7 which I will mention along the way.

Download and unzip Oracle Grid Infrastructure 
Download 12c R2 Grid Infrastructure.
Release 2 (12.2) can be downloaded from 
www.oracle.com. For this installation, I have downloaded the latest release available at the time of writing this article which has downloadable zip file named as linuxx64_12201_grid_home.zip. I have copied it under /u01 directory on the server.

1)
Make sure that you have entries in the /etc/host file for the name resolution.
127.0.0.1   localhost localhost.localdomain localhost
192.231.231.40 salman11.salman.com salman11


2)
Edit /etc/selinux/config and set value for SELINUX to either “permissive” or “disabled”

           SELINUX=permissive


3)
As root user, configure Shared Memory File System. Add following line in /etc/fstab file for shared memory file system. Modify the value of “size” based on the amount of memory you will be using for the SGA (of database instance you will run on this server). An appropriate value is needed to avoid ORA-00845.
tmpfs                                   /dev/shm                tmpfs   rw,exec,size=8g        0 0


4)
Disable the firewall.
Use following steps for Linux 6
[root@salman11 ~]# service iptables stop
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Unloading modules:                               [  OK  ]

[root@salman1 ~]# chkconfig iptables off

Use following steps for Linux 7
[root@salman11 ~]# systemctl start firewalld.service
[root@salman11 ~]# systemctl stop firewalld.service
[root@salman11~]# systemctl disable firewalld.service
rm '/etc/systemd/system/basic.target.wants/firewalld.service'
rm '/etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service' 

5)
Reboot the host (Optional)

6)
We can perform automatic configuration of the server using “yum” command. If you want to do manual configuration, skip this step and go to next step (step 7).
Automatic configuration would perform following tasks
- Installation of required RPM packages
- Setup kernel parameters in /etc/sysctl.conf file
- Creation of OS groups (oinstall, dba) and OS user (oracle)
- Setting limits for installation user “oracle”

For Oracle Linux, follow the steps mentioned in the following documents to access the online yum repository.
http://public-yum.oracle.com/

[root@salman11 ~]# yum install oracle-database-server-12cR2-preinstall.x86_64 preinstall -y
As already mentioned, above command will install all required packages which are needed for grid infrastructure and/or RDBMS software installation.
I have noticed that even if we do automatic configuration, 32 bit RPM packages still don’t get installed. Check if following 32-bit packages have been installed, and install manually if they are not.
For Linux 6, install following RPMs manually if not already installed.

compat-libstdc++-33-3.2.3-69.el6 (i686)
glibc-2.12-1.7.el6 (i686)
glibc-devel-2.12-1.7.el6 (i686)
libgcc-4.4.4-13.el6 (i686)
libstdc++-4.4.4-13.el6 (i686)
libstdc++-devel-4.4.4-13.el6 (i686)
libaio-0.3.107-10.el6 (i686)
libaio-devel-0.3.107-10.el6 (i686)
libXtst-1.0.99.2 (i686)
libX11-1.5.0-4.el6 (i686)
libXau-1.0.6-4.el6 (i686)
libxcb-1.8.1-1.el6 (i686)
libXi-1.3 (i686)

Example (Checking 32-bit version of glibc-devel). 32-bit version is not installed
[root@salman11 ~]# rpm -q glibc-devel
glibc-devel-2.17-157.el7_3.1.x86_64

Example (installing 32-bit version of glibc-devel using yum)
[root@salman11 ~]# yum install glibc-devel.i686


For Linux 7, install following (or latest version) 32-bit RPMs manually if not already installed.
compat-libstdc++-33-3.2.3-71.el7 (i686)
glibc-2.17-36.el7 (i686)
glibc-devel-2.17-36.el7 (i686)
libaio-0.3.109-9.el7 (i686)
libaio-devel-0.3.109-9.el7 (i686)
libX11-1.6.0-2.1.el7 (i686)
libXau-1.0.8-2.1.el7 (i686)
libXi-1.7.2-1.el7 (i686)
libXtst-1.2.2-1.el7 (i686)
libgcc-4.8.2-3.el7 (i686)
libstdc++-4.8.2-3.el7 (i686)
libstdc++-devel-4.8.2-3.el7 (i686)
libxcb-1.9-5.el7 (i686)

Example (Checking 32-bit version of glibc-devel). 32-bit version is not installed
[root@salman11 ~]# rpm -q glibc-devel
glibc-devel-2.17-157.el7_3.1.x86_64

Example (installing 32-bit version of glibc-devel using yum)
[root@salman11 ~]# yum install glibc-devel.i686


If you have performed this step; skip next step and go to step 8

7)
For Linux 6, install following RPM packages (or latest version) from either yum repository or from Linux 6 media
binutils-2.20.51.0.2-5.36.el6 (x86_64)
compat-libcap1-1.10-1 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (x86_64)
compat-libstdc++-33-3.2.3-69.el6 (i686)
e2fsprogs-1.41.12-14.el6 (x86_64)
e2fsprogs-libs-1.41.12-14.el6 (x86_64)
glibc-2.12-1.7.el6 (i686)
glibc-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (x86_64)
glibc-devel-2.12-1.7.el6 (i686)
ksh
libgcc-4.4.4-13.el6 (i686)
libgcc-4.4.4-13.el6 (x86_64)
libs-1.42.8-1.0.2.el6.x86_64
libstdc++-4.4.4-13.el6 (x86_64)
libstdc++-4.4.4-13.el6 (i686)
libstdc++-devel-4.4.4-13.el6 (x86_64)
libstdc++-devel-4.4.4-13.el6 (i686)
libaio-0.3.107-10.el6 (x86_64)
libaio-0.3.107-10.el6 (i686)
libaio-devel-0.3.107-10.el6 (x86_64)
libaio-devel-0.3.107-10.el6 (i686)
libXtst-1.0.99.2 (x86_64)
libXtst-1.0.99.2 (i686)
libX11-1.5.0-4.el6 (i686)
libX11-1.5.0-4.el6 (x86_64)
libXau-1.0.6-4.el6 (i686)
libXau-1.0.6-4.el6 (x86_64)
libxcb-1.8.1-1.el6 (i686)
libxcb-1.8.1-1.el6 (x86_64)
libXi-1.3 (x86_64)
libXi-1.3 (i686)
make-3.81-19.el6
net-tools-1.60-110.el6_2.x86_64 (for Oracle RAC and Oracle Clusterware)
nfs-utils-1.2.3-15.0.1 (for Oracle ACFS)
sysstat-9.0.4-11.el6 (x86_64)
smartmontools-5.43-1.el6.x86_64

Example (yum)
[root@salman11 ~]# yum install glibc
Example (Linux Media)
[root@salman11 ~]# rpm -i glibc
Example (Check after install)
[root@salman11 ~]# rpm -q glibc
glibc-2.12-1.7.el6.x86_64

For Linux 7, install following RPM packages (or latest version) from either yum repository or from Linux 7 media
binutils-2.23.52.0.1-12.el7 (x86_64)
compat-libcap1-1.10-3.el7 (x86_64)
compat-libstdc++-33-3.2.3-71.el7 (i686)
compat-libstdc++-33-3.2.3-71.el7 (x86_64)
glibc-2.17-36.el7 (i686)
glibc-2.17-36.el7 (x86_64)
glibc-devel-2.17-36.el7 (i686)
glibc-devel-2.17-36.el7 (x86_64)
ksh
libaio-0.3.109-9.el7 (i686)
libaio-0.3.109-9.el7 (x86_64)
libaio-devel-0.3.109-9.el7 (i686)
libaio-devel-0.3.109-9.el7 (x86_64)
libX11-1.6.0-2.1.el7 (i686)
libX11-1.6.0-2.1.el7 (x86_64)
libXau-1.0.8-2.1.el7 (i686)
libXau-1.0.8-2.1.el7 (x86_64)
libXi-1.7.2-1.el7 (i686)
libXi-1.7.2-1.el7 (x86_64)
libXtst-1.2.2-1.el7 (i686)
libXtst-1.2.2-1.el7 (x86_64)
libgcc-4.8.2-3.el7 (i686)
libgcc-4.8.2-3.el7 (x86_64)
libstdc++-4.8.2-3.el7 (i686)
libstdc++-4.8.2-3.el7 (x86_64)
libstdc++-devel-4.8.2-3.el7 (i686)
libstdc++-devel-4.8.2-3.el7 (x86_64)
libxcb-1.9-5.el7 (i686)
libxcb-1.9-5.el7 (x86_64)
make-3.82-19.el7 (x86_64)
nfs-utils-1.3.0-0.21.el7.x86_64 (for Oracle ACFS)
net-tools-2.0-0.17.20131004git.el7 (x86_64) (for Oracle RAC and Oracle Clusterware)
smartmontools-6.2-4.el7 (x86_64)
sysstat-10.1.5-1.el7 (x86_64)
Example (yum)
[root@salman1 ~]# yum install glibc
Example (Linux Media)
[root@salman1 ~]# rpm -i glibc
Example (Check after install)
[root@salman1 ~]# rpm -q glibc
glibc-2.17-55.el7.x86_64

Edit /etc/sysctl.conf add following entries to set kernel parameters
fs.aio-max-nr = 1048576
fs.file-max = 6815744
kernel.shmall = 2097152
kernel.shmmax = 4294967295
kernel.shmmni = 4096
kernel.sem = 250 32000 100 128
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 262144
net.core.rmem_max = 4194304
net.core.wmem_default = 262144
net.core.wmem_max = 1048576
net.ipv4.ip_local_port_range = 9000 65500

Execute following command after adding above lines
/sbin/sysctl –p


8)
Since I will use “grid” user as GI software owner, I would need to add OS groups and also “grid” user manually. Automatic configuration done in step 6 above does not create “grid” user and ASM related OS groups. If you don’t want to use “grid” user and continue with “oracle” user, you don’t need to create “grid” user in this step, but still you would need to add “asmdba, admadmin, asmoper” groups, and make “oracle” user member of these groups.
Add groups
[root@salman11 ~]#  groupadd -g 54325 asmdba
[root@salman11 ~]#  groupadd -g 54328 asmadmin
[root@salman11 ~]#  groupadd -g 54329 asmoper
Add user
[root@salman11 ~]# useradd -u 54322 -g oinstall -G dba,asmdba,asmadmin,asmoper grid
Set passwords for the grid user
[root@salman11 ~]# passwd grid

9)
If you want to continue with “oracle” user as GI software owner, replace “grid” with “oracle” while adding following limits in the .conf file.
As root, create a file under /etc/security/limits.d directory and set limits for user “grid”. You can choose any file name, for example grid_user.conf. Add following lines in this file
grid soft nofile 1024
grid hard nofile 65536
grid soft nproc 16384
grid hard nproc 16384
grid soft stack 10240
grid hard stack 32768
grid hard memlock 134217728
grid soft memlock 134217728
In case of auto configuration (as mentioned in step 6), a file oracle-database-server-12cR2-preinstall.conf is created under /etc/security/limits.d that contains limits already set for user “oracle”. Since we are using grid user for GI installation, so we will need to set limits for grid user manually. Alternatively, these limits for grid user can also be specified in the file oracle-database-server-12cR2-preinstall.conf by pasting above values at the bottom of this file

10)
As per Oracle’s recommendation, we need to
disable Transparent Huge Pages. Disable if not already disabled.

11) 

Starting 12.1.0.2 we can use ASM Filter Driver for our ASM disks labeling and configuration, and ASMLib configuration would be no longer required. If you want to use ASMLib (as I used in this installation), you can continue, otherwise you can skip details on this step and can straight away go to next step (step 12). I will explain later how to configure ASM Filter Driver.

In order to use ASMLib, perform following configuration.
Install oracleasm-support and oracleasmlib, and then configure oracleasm.
oracleasm kernel driver is built in Oracle Linux and does not need to be installed. After installing oracleasm-support and oracleasmlib packages, oracleasm driver starts working.
If you are using some other flavor of Linux, for example RedHat Linux, then you would need to install all 3 packages (oracleasm driver, oracleasm-support and oracleasmlib).

For Oracle Linux, as root, install oracleasm-support from yum repository or from Linux media, then download oracleasmlib package and install it.
For Linux 6, download form the following URL.
http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol6-1709075.html

For Linux 7, download from the following URL.

http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol7-2352094.html


Install oracleasmlib and oracleasm-support
[root@salman11 ~]#  yum install oracleasm-support
[root@salman11 ~]#  rpm -i oracleasm-lib*
Configure oracleasm (highlighted in red are the inputs during configuration)
[root@salman11 ~]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []:
grid
Default group to own the driver interface []:
asmadmin
Start Oracle ASM library driver on boot (y/n) [n]:
y
Scan for Oracle ASM disks on boot (y/n) [y]:
y
Writing Oracle ASM library driver configuration: done
Initializing the Oracle ASMLib driver:                     [  OK  ]
Scanning the system for Oracle ASMLib disks:               [  OK  ]
Check Configuration
[root@salman11 ~]# /usr/sbin/oracleasm configure
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

12)
As root user, add following lines in /etc/pam.d/login file if not present already
session required /lib64/security/pam_limits.so
session required pam_limits.so

13) 
As root, partition your disks that you will be using for CRS and other diskgroups (to be used later for database) and create ASM disks using oracleASMLib. You can find out here how to create disk partitions in Linux, and this article
explains how to create ASM disks using ASILib.
For this installation I have created 1 ASM disk with name "DATA" to create a diskgroup "DATA" that will store OCR.

14)
Oracle recommends to using “Deadline IO scheduler” for optimum performance of ASM disks. Following command will tell if Deadline IO scheduler is already configured or not. Here /sdb/ is the disk I used to create ASM disk in the previous step. You may check for all of your disks that you will be using.
[root@salman11 ~]# cat /sys/block/sdb/queue/scheduler
noop [deadline] cfq
For me, Deadline scheduler is already configured. If not, following is the procedure to configuring Deadline scheduler.

--Create a new rules file
[root@salman11 ~]# vi /etc/udev/rules.d/60-oracle-schedulers.rules

--Add following line into this rules file
ACTION=="add|change", KERNEL=="sd[a-z]", ATTR{queue/rotational}=="0", ATTR{queue/scheduler}="deadline"

--Reload the rules file
[root@salman11 ~]# udevadm control --reload-rules

15)
Add the following in /home/<username>/.bash_profile file of grid user. At this point, we do no need to set other environment variables i.e. ORACLE_BASE or ORACLE_HOME.
umask 022

16)
Now we will creating oracle base and oracle home directories for grid infrastructure installation.  As root, create directories and change ownership to grid user. Oracle recommends creating separate ORACLE_BASE for GI and RDBMS installations. Thus, we need to create a separate ORACLE_BASE for RDBMS installation that will be owned by user “oracle”, and we can do it right now, or during RDBMS installation.
[root@salman11 ~]# mkdir -p /u01/app/12.2.0/grid
[root@salman11 ~]# mkdir -p /u01/app/grid
[root@salman11 ~]# mkdir -p /u01/app/oracle
[root@salman11 ~]# chown -R grid:oinstall /u01

17)
Now we need to extract the downloaded zip file for GI installation. Starting 12.2, we have image based installation for Grid Infrastructure. It means that we need to extract the zip file under ORACLE_HOME for GI, and software is installation is completed. After extracting the zip file (installation), still we need to run the setup to perform all configurations and other required tasks.
After logging in as grid user, extract the zip file under GI home directory that we already have created.
[grid@salman11 grid] cd /u01
[grid@salman11 u01] unzip /u01/linuxx64_12201_grid_home.zip –d /u01/app/12.2.0/grid


Now log in as grid user using Linux desktop, or you may also use any X server software. Start setup by executing “gridSetup.sh” (unlike runInstaller in previously releases)

$ cd /u01/app/12.2.0/grid
$ ./gridSetup.sh

18)
Select second option for installing GI for standalone server. Click Next

Oracle 12c grid infrastructure installation wizard screen 1





























19) 
Installation wizard will create an ASM diskgroup by using ASM disk we have already created, as I mentioned above. If disk(s) is/are not listed here, click on “Change Discovery Path” to specify the correct discovery path. For example, I also saw here no disk listed and after clicking on “Disk Recovery Path”, I found out that discovery path was set to “/dev/sd*”, whereas the ASM disk that I created using ASMLib needs to be discovered using discovery path “ORCL:*”. So after I changed the discovery path to “ORCL:*”, wizard was able to discover the disk.


This screen also allows you to configure ASM disks using ASM Filter Driver (ASMAFD). If you skipped step 11 above and have not configured ASMLib, follow steps explained here to label disk for ASMFD before GI installation.

Oracle 12c grid infrastructure installation wizard screen 2






























Since I have a single disk, I am selecting “Normal” redundancy for my diskgroup, and selecting the disk by clicking checkbox. Click Next.
Oracle 12c grid infrastructure installation wizard screen 3




























20) 
Provide passwords, click Next
Oracle 12c grid infrastructure installation wizard screen 4




























 21)
Click Next

Oracle 12c grid infrastructure installation wizard screen 5




























22)
Click Next
 
Oracle 12c grid infrastructure installation wizard screen 6




























23) 
Specify the oracle base directory that we created above. Click Next (Ignore any warning if you see after clicking Next)
Oracle 12c grid infrastructure installation wizard screen 7




























24)
Click Next
 
Oracle 12c grid infrastructure installation wizard screen 8





























25)
We have option to execute configuration script manually after the installation, or if we provide the root credentials (or sudo credentials), scripts can be execute automatically by the installer. Here I chose to execute scripts automatically by providing root password. Click Next.

Oracle 12c grid infrastructure installation wizard screen 9




























26) 
Solve, if prerequisites check shows any warning. For me, it is showing low RAM which I can ignore because I am using a virtual machine and I do not have sufficient RAM available on my host OS. Second warning is because cluvfy is not installed and I can also ignore this warning. Click Next.

Oracle 12c grid infrastructure installation wizard screen 10

27)
Click on Install from Summary screen.
Oracle 12c grid infrastructure installation wizard screen 11

28)
During the installation installer will prompt for the execution of scripts. Since root privileges have already been provided, click Yes
Oracle 12c grid infrastructure installation wizard screen 12

29)
Click Close after configuration completes.
Oracle 12c grid infrastructure installation wizard screen 13
30)
Check the status of GI services.
[grid@salman11 ~]$ /u01/app/12.2.0/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
               ONLINE  ONLINE       salman11                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       salman11                 STABLE
ora.asm
               ONLINE  ONLINE       salman11                 Started,STABLE
ora.ons
               OFFLINE OFFLINE      salman11                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.cssd
      1        ONLINE  ONLINE       salman11                 STABLE
ora.diskmon
      1        OFFLINE OFFLINE                               STABLE
ora.evmd
      1        ONLINE  ONLINE       salman11                 STABLE
--------------------------------------------------------------------------------

31)
You can add following environment variables in .bash_profile file.
ORACLE_SID=+ASM; export ORACLE_SID
ORACLE_BASE=/u01/app/grid; export ORACLE_BASE
ORACLE_HOME=/u01/app/12.2.0/grid; export ORACLE_HOME
PATH=$ORACLE_HOME/bin:$PATH; export PATH

Installation has completed. You may continue with Oracle 12c R2 Database software installation.

No comments:

Post a Comment

Popular Posts - All Times