Friday, November 25, 2011

Oracle 11g Clusterware Installation Requirements

Server (node) requirements

each node in the cluster must meed the following requirements
  1. A minimum of 2 GB RAM is required for the complete Grid Infrastructure and RAC software installation.
  2. A minimum of 1 GB free space is required in the /tmp directory.
  3. A minimum of 10 GB free space for the entire Clusterware/Grid and RAC software homes is required.
  4. shared storage  to place Clusterware critical components: Oracle Cluster Registry (OCR) and Voting Disk.
  5. ensure that you have adequate shared storage disks for ASM if you consider using ASM to host the datafiles.
  6. Configure the server swap size to be 1.5 time if the RAM on the server  is <=2 GB. Otherwise, configure the sever swap size to be equal to the  RAM size. Each server must have a minimum of two network adapters  or network interface cards (NIC) configured to be able to support the public and private networks.

Network requirements
  1. every node in the cluster must be configured with a minimum of two NIC interfaces (cards):
  2. One NIC is for the public communication and another for the private communication. 
  3. You also need to secure three IP addresses, Public, Private, and VIP respectively for each individual node.
  4. You will need to ensure that the public network interface supports TCP/IP and that the private network adapter supports UDP.
These IP addresses with their fully qualified hostname and domain name either need to be configured on the Domain Name Server (DNS) or added to the /etc/hosts file

SCAN:
  1. Beginning with Oracle 11g R2 a maximum of three additional IP addresses in a round-robin fashion are needed for a Single Client Access Name (SCAN).
  2. Unlike the other three IP (Public, Private, and VIP) addresses and hostnames, the SCAN must be assigned a unique name across all nodes and it must remain on the same subnet as a public and VIP address.
Kernel parameters

Oracle recommended kernel parameters are configured or added in the /etc/sysctl.conf file with the values suggested for a successful installation, you need to be the root user to perform this action

For Oracle 11g R1:
  
   kernel.shmall = 2097152
   kernel.shmmax = 2147483648
   kernel.shmmni = 4096
   kernel.sem = 250 32000 100 128
   fs.aio-max-size =1048576
   fs.file-max = 6815744
   net.ipv4.ip_local_port_range = 1024 65000
   net.core.rmem_default = 262144
   net.core.rmem_max = 4194304
   net.core.wmem_default = 262144
   net.core.wmem_max = 1048576

For Oracle 11g R2:

   kernel.shmall = 2097152
   kernel.shmmax = 536870912
   kernel.shmmni = 4096
   kernel.sem = 250 32000 100 128
   fs.aio-max-size =1048576
   fs.file-max = 6815744
   net.ipv4.ip_local_port_range = 9000 65500
   net.core.rmem_default = 262144
   net.core.rmem_max = 4194304
   net.core.wmem_default = 262144
   net.core.wmem_max = 1048576

Note: In 11g R2, when the kernel parameters fail to meet the minimum requirements, they can be dynamically fixed during the installation phase by using the fixup option.

Operating system packages

Apart from the basic packages (RPM) that were installed as part of the typical operating system installation, the following additional Oracle Recommended Packages (RPM) need to be configured on each server. The required packages can be found in the OS source (DVD) or you can also download them from the Internet.

For Oracle 11g R1:
   
   binutils-2.17.50.0.6-2.el5
   compat-libstdc++-33-3.2.3-61
   elfutils-libelf-0.125.el5
   elfutils-libelf-devel-0.125
   glibc-2.5-12
   glibc-common-2.5-12
   glibc-devel-2.5-12
   glibc-headers-2.3.4-2
   gcc-4.1.1-52
   gcc-c++-4.1.1-52
   libaio-0.3.106
   libaio-devel-0.3.106 
   libgcc-4.1.1-52
   libstdc++-4.1.1 
   libstdc++-devel-4.1.1-52.e15
   make-3.81-1.1
   sysstat-7.0.0
   unixODBC-2.2.11
   unixODBC-devel-2.2.11
   numactl-7.0.0

For Oracle 11g R2:
   
   binutils-2.17.50.0.6
   compat-libstdc++-33-3.2.3
   compat-libstdc++-33-3.2.3 (32 bit)
   elfutils-libelf-0.125
   elfutils-libelf-devel-0.125
   elfutils-libelf-devel-static-0.125
   gcc-4.1.2
   gcc-c++-4.1.2
   glibc-2.5-24
   glibc-2.5-24 (32 bit)
   glibc-common-2.5
   glibc-devel-2.5
   glibc-devel-2.5 (32 bit)
   glibc-headers-2.5
   ksh-20060214
   libaio-0.3.106
   libaio-0.3.106 (32 bit)
   libaio-devel-0.3.106
   libaio-devel-0.3.106 (32 bit)
   libgcc-4.1.2
   libgcc-4.1.2 (32 bit)
   libstdc++-4.1.2
   libstdc++-4.1.2 (32 bit)
   libstdc++-devel 4.1.2
   make-3.81
   sysstat-7.0.2
   unixODBC-2.2.11
   unixODBC-2.2.11 (32 bit)
   unixODBC-devel-2.2.11
   unixODBC-devel-2.2.11 (32 bit)
   numactl-devel-0.9.8.i386

OS groups and users

The following mandatory OS groups and Oracle software owners (user) are required for each node. As a matter of fact, in the cluster environment, the user equivalences such as group/user name and their IDs must remain the same across all nodes of a cluster:

   Groups : oinstall, dba, oper, asmadmin
   User   : oracle 

As the root user, use the following commands to create OS groups and a user:
   groupadd –g 500 oinstall
   groupadd –g 501 dba
   groupadd –g 502 oper
   groupadd –g 503 asmadmin
   useradd –u 1000 –g oinstall –G dba,oper,asmadmin oracle
   passwd oracle

As per Oracle recommendation, the oinstall group should be configured as the  primary group and the remaining groups should be configured as secondary groups for the Oracle/Cluster software users (typically, the oracle user) on the server.

OS user settings
Once you have managed to create the necessary OS groups and users, the subsequent task is to set the limits for the user in order to improve the shell limits. You will need to add the following user limits in the /etc/security/limits.conf file:

   oracle               soft    nproc   2047
   oracle               hard    nproc   16384
   oracle               soft    nofile  1024
   oracle               hard    nofile  65536
   oracle               soft    stack   10240

Next, add the following user session limit in the /etc/pam.d/login file:

The following limits and shell settings need to be added in the user profile file, .bash_profile. In the following example, presuming that the korn shell is being used in your environment, log in as Oracle user, edit the hidden .bash_profile file, and add the following lines; if another shell is being used, modify the following 
example accordingly:

   if [ $USER = "oracle" ]; then
              if [ $SHELL = "/bin/ksh" ]; then
                 ulimit -p 16384
                 ulimit -n 65536
           else
              ulimit -u 16384 -n 65536
           fi
              umask 022
   fi

Configuring Secure Shell (SSH)

After you have successfully completed the preceding necessary settings, the subsequent mandatory task is to configure the Secure Shell (SSH) passwordless connectivity between the nodes. Bear in mind that SSH connectivity configuration is essential between the nodes of a cluster as the OUI uses the ssh and scp commands internally during the installation phase to perform remote operations in which it copies the software from the local to other nodes.

You need to generate RSA or DSA keys on each node as part of the SSH configuration. 
For that, use the following set of commands to configure SSH:

1.  Log in as Oracle user on the first node of the cluster (in our example, it will be on the raclinux1), and execute the following sequence of commands:



2.  When prompted for the key location, accept the default settings by hitting  the Enter key. Once this setup is done on the first node, repeat the same  steps on the rest of the nodes (in our example, repeat the same step on the raclinux2 node).

3.  Now, switch back to the first node (raclinux1) and execute the following command to add authorization keys. After adding the authorization keys, copy the authorized_keys file to the other nodes using the scp command, and enter Yes when prompted.

4.  Now, switch back to the second node, (raclinux2) and run the  following command:



5.  After adding the authorization keys on the second node, copy the file to the other nodes using the scp command, as demonstrated in the preceding example, and enter Yes when prompted.

6.  After configuring the SSH setup successfully across all nodes, let's run the following test on each node, starting from the first node, to establish a passwordless connection between all the nodes to meet the Oracle
recommendations:

   ssh raclinux1 date
   ssh raclinux2 date
   ssh raclinux1-priv date
   ssh raclinux2-priv date
   ssh raclinux1-sjh.com date
   ssh raclinux2-sjh.com date


Fortunately, with Oracle 11g R2, you are no longer required to perform the manual SSH connectivity configuration, as it can be configured automatically by the OUI during the installation phase. You will be allowed to configure SSH by the OUI interface during the installation phase. Nevertheless, it will be the same set of actions that we have manually performed earlier in this chapter, but an automated one.


Reference: ORACLE 11g R1.R2 Real Application Clusters Essentials


No comments:

Post a Comment