You must ensure that you have a certified combination of the operating system and the Oracle Database software.

For a list of the certified operating systems and Oracle Database software, refer to My Oracle Support certification, which is located at the following website

https://support.oracle.com

You can find certification information by selecting the Certifications tab. You can also search for support notes that contain instructions on how to locate the Certification information for your platform.

Each node you make part of your cluster, or Oracle Clusterware and Oracle RAC installation, must satisfy the minimum hardware requirements of the software.

The minimum hardware requirements are:

• At least 4 GB of RAM for Oracle Grid Infrastructure for a Cluster installations, including installations where you plan to install Oracle RAC

• An amount of swap space that is at least equal to the amount of RAM

• At least two network switches, each at least 1 GbE, to support the public and private networks

• Servers should be either in runlevel 3 or runlevel 5.

• Temporary space (at least 1 GB) available in /tmp

• A processor type (CPU) that is certified with the release of the Oracle software being installed (64-bit)

• A minimum of 1024 x 786 display resolution, so that Oracle Universal Installer (OUI) displays correctly

• All servers in the cluster have the same chip architecture, for example, all 64-bit processors

• Access to either Storage Area Network (SAN) or Network-Attached Storage (NAS).

You must also have sufficient disk space in the software installation locations to store the Oracle software, as described in the following table.

Oracle Grid Infrastructure and Oracle RAC require access to shared storage, so each node and instance in the cluster can access the same set of files.

An Oracle RAC database is a shared everything database. All data files, control files, redo log files, and the server parameter file (SPFILE) used by the Oracle RAC database must reside on shared storage that is accessible by all the Oracle RAC database instances. The Oracle RAC installation demonstrated in this guide uses Oracle ASM for the shared storage for Oracle Clusterware and Oracle Database files.

Oracle Clusterware achieves superior scalability and high availability by using the following components:

• Voting disk–Manages cluster membership and arbitrates cluster ownership between the nodes in case of network failures. The voting disk is a file that resides on shared storage. For high availability, Oracle recommends that you have multiple voting disks, and that you have an odd number of voting disks. If you define a single voting disk, then use mirroring at the file system level for redundancy.

• Oracle Cluster Registry (OCR)–Maintains cluster configuration information and configuration information about any cluster database within the cluster. The OCR contains information such as which database instances run on which nodes and which services run on which databases. The OCR also stores information about processes that Oracle Clusterware controls. The OCR resides on shared storage that is accessible by all the nodes in your cluster. Oracle Clusterware can multiplex, or maintain multiple copies of, the OCR and Oracle recommends that you use this feature to ensure high availability.

• Grid Infrastructure Management Repository–Supports Oracle Database Quality of Service management, Memory Guard, and Cluster Health Monitor

These Oracle Clusterware components require the following disk space on a shared file system:

• Three Oracle Clusterware Registry (OCR) files, a least 400 MB for each volume, or 1.2 GB total disk space

• Three voting disk files, 300 MB each volume, or 900 MB total disk space

• The Grid Infrastructure Management Repository requires an additional 5.9 GB of disk space and is stored in an OCR volume

You must have the hardware to support communication, both public and private, between all the nodes in the cluster.

Oracle Clusterware requires that you connect the nodes in the cluster to a private network by way of a private interconnect . The private interconnect is a separate network that you configure between cluster nodes. The interconnect serves as the communication path between nodes in the cluster. This interconnect should be a private interconnect, meaning it is not accessible to nodes that are not members of the cluster.

The interconnect used by Oracle RAC is the same interconnect that Oracle Clusterware uses. Each cluster database instance uses the interconnect for messaging to synchronize the use of shared resources by each instance. Oracle RAC also uses the interconnect to transmit data blocks that are shared between the instances.

When you configure the network for Oracle RAC and Oracle Clusterware, each node in the cluster must meet the following requirements:

• Each node must have at least two network interface cards (NICs), or network adapters. One adapter is for the public network interface and the other adapter is for the private network interface (the interconnect). Install additional network adapters on a node if that node meets either of the following conditions:

  • Does not have at least two network adapters

  • Has two network interface cards but is using network attached storage (NAS). You should have a separate network adapter for NAS.

  • Has two network cards, but you want to use the Redundant Interconnect Usage feature, which allows multiple network adapters to be accessed as a single adapter. Redundant Interconnect Usage enables load-balancing and high availability across multiple (up to 4) private networks (also known as interconnects).

• For easier management, public interface names should be the same for all nodes. If the public interface on one node uses the network adapter eth0, then you should configure eth0 as the public interface on all nodes. Network interface names are case-sensitive.

• The network adapter for the public interface must support TCP/IP.

• The network adapter for the private interface must support the user datagram protocol (UDP) using high-speed network adapters and a network switch that supports TCP/IP (Gigabit Ethernet or better).

  Note:

  • UDP is the default interface protocol for Oracle RAC and Oracle Clusterware.

  • You must use a switch for the interconnect. Oracle recommends that you use a dedicated network switch. Token-rings or crossover cables are not supported for the interconnect.

  • Loopback devices are not supported.

• For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. Every node in the cluster must be able to connect to every private network interface in the cluster.

• The host name of each node must conform to the RFC 952 standard, which permits alphanumeric characters. Host names using underscores ("_") are not allowed.

Oracle Grid Infrastructure and Oracle RAC use a variety of IP addresses for communication.

When performing an advanced installation of Oracle Grid Infrastructure for a cluster, you can chose to use Grid Naming Service (GNS) and Dynamic Host Configuration Protocol (DHCP) for virtual IPs (VIPs). GNS uses multicast Domain Name Server (mDNS) to enable the cluster to assign host names and IP addresses dynamically as nodes are added and removed from the cluster, without requiring additional network address configuration in the domain name server (DNS).

You can configure the public network interfaces for cluster nodes to use IPv4, IPv6, or both types of IP addresses. During installation, the nodes can be connected to networks that have both address types, but you cannot configure the VIPs to use both address types during installation. After installation, you can configure cluster member nodes with a mixture of IPv4 and IPv6 addresses. Oracle Grid Infrastructure and Oracle RAC support the standard IPv6 address notations specified by RFC 2732.

This guide documents how to perform a typical installation, which does not use GNS and uses only IPv4 addresses. You must configure the following addresses manually in your corporate DNS:

• A fixed public IP address for each node

• A virtual IP address for each node

• Three single client access name (SCAN) addresses for the cluster

During installation a SCAN for the cluster is configured, which is a domain name that resolves to all the SCAN addresses allocated for the cluster. The public IP addresses, SCAN addresses, and VIP addresses must all be on the same subnet. The SCAN must be unique within your network. The SCAN addresses should not respond to ping commands before installation. In a Typical installation, the SCAN is also the name of the cluster, so the SCAN must meet the same requirements as the cluster name and can be no longer than 15 characters.

During installation of the Oracle Grid Infrastructure for a cluster, a listener is created for each of the SCAN addresses. Clients that access the Oracle RAC database should use the SCAN or SCAN address, not the VIP name or address. If an application uses a SCAN to connect to the cluster database, then the network configuration files on the client computer do not have to be modified when nodes are added to or removed from the cluster. The SCAN and its associated IP addresses provide a stable name for clients to use for connections, independent of the nodes that form the cluster. Clients can connect to the cluster database using the easy connect naming method and the SCAN.

Ensure your system meets the operating system version and other software requirements.

Refer to the Oracle Grid Infrastructure Installation and Upgrade Guide and the Oracle Real Application Clusters Installation Guide for your platform for information about specific requirements for your environment.

Topics:

• About Operating System and Software Requirements
  Before installing the Oracle software, check that your operating system meets the requirements.
• About Installation Fixup Scripts
  Oracle Universal Installer (OUI) detects when the minimum requirements for an installation are not met, and creates shell scripts, called Fixup scripts, to finish incomplete system configuration steps.
• Checking the Current Operating System Configuration
  Instead of waiting for the installation to notify you of an incorrect configuration, you can use operating system commands to manually check the operating system configuration before installation. This helps you determine if additional time will be needed to update the operating system before starting the Oracle Grid Infrastructure installation.

Depending on whether this is the first time Oracle software is installed on this server, you may have to create operating system groups and users.

Topics:

• Required Operating System Users and Groups
  To install Oracle Grid Infrastructure for a cluster and Oracle Real Application Clusters (Oracle RAC), you must create certain operating system groups and users.
• Separate Operating System Users and Groups for Oracle Software Installations on UNIX and Linux Systems
  Instead of using a single operating system user as the owner of every Oracle software installation, you can use multiple users, each owning one or more Oracle software installations.
• Separate Operating System Users and Groups for Oracle Software Installations on Windows Systems
  On the Windows platform, management of operating system users and groups is different from Linux and UNIX Systems.
• Optional Operating System Users and Groups
  You can use additional users and groups to divide administrative access privileges to the Oracle Grid Infrastructure for a cluster installation from other administrative users and groups associated with other Oracle installations.

This task describes how to create the grid and oracle users prior to installing the software.

1. Determine the groups that exist on your server by listing the contents of the /etc/group file.

   cat /etc/group
   

2. If this is the first time Oracle software has been installed on your server, and the Oracle Inventory group does not exist, then create the Oracle Inventory group (oinstall) with a group ID that is currently not in use on all the nodes in your cluster. Enter a command as the root user that is similar to the following:

   # /usr/sbin/groupadd -g 1000 oinstall
   

3. Create an OSDBA (dba) group with a group ID that is currently not in use on all the nodes in your cluster by entering a command as the root user that is similar to the following:

   # /usr/sbin/groupadd -g 1001 dba
   

4. Create an OSDBA group for Oracle ASM (asmdba) with a group ID that is currently not in use on all the nodes in your cluster by entering a command as the root user that is similar to the following:

   # /usr/sbin/groupadd -g 1002 asmdba
   

5. If the user that owns the Oracle software (grid or oracle) does not exist on your server, then you must create the user. Select a user ID (UID) that is currently not in use on all the nodes in your cluster. To determine which users have been created on your server, list the contents of the /etc/passwd file using the command:

   cat /etc/passwd
   

   The following command shows how to create the grid user and the user's home directory (/home/grid) with the default group as oinstall and the secondary groups as dba and asmdba, using a UID of 1100:

   # /usr/sbin/useradd -u 1100 –g oinstall -G dba,asmdba -d /home/grid -r grid
   

   The following command shows how to create the oracle user and the user's home directory (/home/oracle) with the default group as oinstall and the secondary group as dba, using a UID of 1100:

   # /usr/sbin/useradd -u 1200 –g oinstall -G dba,asmdba -d /home/oracle -r oracle
   

6. Set the password for the grid and oracle accounts using the following command. Replace password with your own password.

   passwd grid
   Changing password for user grid.
   New UNIX password: password
   retype new UNIX password: password
   passwd:  all authentication tokens updated successfully.
   passwd oracle
   Changing password for user oracle.
   New UNIX password: password
   retype new UNIX password: password
   passwd:  all authentication tokens updated successfully.

7. Repeat Step 1 through Step 6 on each node in your cluster.
8. Verify that the attributes of the user grid and oracle are identical on each node of your cluster:

   id grid
   id oracle 
   

   The command output should be similar to the following:

   uid=1100(grid) gid=1000(oinstall) groups=1000(oinstall),1001(dba),1002(asmdba)
   uid=1200(oracle) gid=1000(oinstall) groups=1000(oinstall),1001(dba),1002(asmdba)

To install Oracle software, Secure Shell (SSH) connectivity must be set up between all cluster member nodes.

OUI uses the ssh and scp commands during installation to run remote commands on and copy files to the other cluster nodes. You must configure SSH so that these commands do not prompt for a password. SSH is also used by the configuration assistants, Enterprise Manager, and when adding nodes to the cluster.

You can configure SSH from the Oracle Universal Installer (OUI) interface during installation for the user account running the installation. The automatic configuration creates passwordless SSH connectivity between all cluster member nodes.

To enable the script to run, you must remove stty commands from the profiles of any Oracle software installation owners, and remove other security measures that are triggered during a login, and that generate messages to the terminal. These messages, mail checks, and other displays prevent Oracle software installation owners from using the SSH configuration script that is built into the Oracle Universal Installer. If they are not disabled, then SSH must be configured manually before an installation can be run.

On Linux systems, to enable Oracle Universal Installer to use the ssh and scp commands without being prompted for a pass phrase, you must have user equivalency in the cluster. User equivalency exists in a cluster when the following conditions exist on all nodes in the cluster:

• A given user has the same user name, user ID (UID), and password.

• A given user belongs to the same groups.

• A given group has the same group ID (GID).

On Oracle Linux, you run Oracle Universal Installer (OUI) from the grid account. OUI obtains information from the environment variables configured for the grid user.

Before running OUI, you must make the following changes to the shell startup file for the software owner of Oracle Grid Infrastructure for a cluster:

• Set the default file mode creation mask (umask) of the installation user (grid) to 022 in the shell startup file. Setting the mask to 022 ensures that the user performing the software installation creates files with 644 permissions.

• Set ulimit settings for file descriptors (nofile) and processes (nproc) for the installation user (grid).

• Set the software owner's environment variable DISPLAY environment variables in preparation for the Oracle Grid Infrastructure for a cluster installation.

• Remove any lines in the file that set values for ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variables.

After you have saved your changes, run the shell startup script to configure the environment.

Also, if the /tmp directory has less than 1 GB of available disk space, but you have identified a different, non-shared file system that has at least 1 GB of available space, then you can set the TEMP and TMPDIR environment variables to specify the alternate temporary directory on this file system.

To review your current environment settings, use the env | more command as the grid user.

After you have configured the network, perform verification tests to make sure it is configured properly. If there are problems with the network connection between nodes in the cluster, then the Oracle Clusterware installation fails.

1. As the root user, verify the configuration of the public and private networks. Verify that the interfaces are configured on the same network (either private or public) on all nodes in your cluster.

   In this example, eth0 is used for the public network and eth1 is used for the private network on each node.

   # /sbin/ifconfig
    
   eth0      Link encap:Ethernet  HWaddr 00:0E:0C:08:67:A9  
             inet addr: 192.0.2.100   Bcast:192.0.2.255   Mask:255.255.240.0
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:270332689 errors:0 dropped:0 overruns:0 frame:0
             TX packets:112346591 errors:2 dropped:0 overruns:0 carrier:2
             collisions:202 txqueuelen:1000 
             RX bytes:622032739 (593.2 MB)  TX bytes:2846589958 (2714.7 MB)
             Base address:0x2840 Memory:fe7e0000-fe800000 
    
   eth1      Link encap:Ethernet  HWaddr 00:04:23:A6:CD:59  
             inet addr: 10.10.10.11   Bcast: 10.10.10.255   Mask:255.255.240.0   
             UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
             RX packets:21567028 errors:0 dropped:0 overruns:0 frame:0
             TX packets:15259945 errors:0 dropped:0 overruns:0 carrier:0
             collisions:0 txqueuelen:1000 
             RX bytes:4091201649 (3901.6 MB)  TX bytes:377502797 (360.0 MB)
             Base address:0x2800 Memory:fe880000-fe8a0000 

2. As the root user, verify the network configuration by using the ping command to test the connection from each node in your cluster to all the other nodes. For example, as the root user, you might run the following commands on each node:

   # ping -c 3 racnode1.example.com
   # ping -c 3 racnode1
    
   # ping -c 3 racnode2.example.com
   # ping -c 3 racnode2
   

   You should not get a response from the nodes using the ping command for the virtual IPs (racnode1-vip, racnode2-vip) or the SCAN IPs until after Oracle Clusterware is installed and running. If the ping commands for the public addresses fail, then resolve the issue before you proceed.

3. Ensure that you can access the default gateway with a ping command. To identify the default gateway, use the route command, as described in the Oracle Linux Help utility.

Before starting the installation, ensure that the date and time settings on all the cluster nodes are set as closely as possible to the same date and time.

A cluster time synchronization mechanism ensures that the internal clocks of all the cluster members are synchronized. For Oracle RAC on Linux, you can use either the Network Time Protocol (NTP) or the Oracle Cluster Time Synchronization Service.

NTP is a protocol designed to synchronize the clocks of servers connected by a network. When using NTP, each server on the network runs client software to periodically make timing requests to one or more servers, referred to as reference NTP servers. The information returned by the timing request is used to adjust the server's clock. All the nodes in your cluster should use the same reference NTP server.

If you do not configure NTP, then Oracle configures and uses the Cluster Time Synchronization Service (CTSS). CTSS can also be used to synchronize the internal clocks of all the members in the cluster. CTSS keeps the member nodes of the cluster synchronized. CTSS designates the first node in the cluster as the master and then synchronizes all other nodes in the cluster to have the same time as the master node. CTSS does not use any external clock for synchronization.

OUI checks the current settings for various kernel parameters to ensure they meet the minimum requirements for deploying Oracle RAC. For production database systems, Oracle recommends that you tune the settings to optimize the performance of your particular system.

You may be required to perform special configuration steps that are specific to the operating system on which you are installing Oracle RAC, or for the components used with your cluster.

The following list provides examples of operating-specific installation tasks:

• Configure the use of Huge Pages on SUSE Linux, Red Hat Enterprise Linux, or Oracle Linux.

• Set shell limits for the oracle user on Red Hat Linux or Oracle Linux systems to increase the number of files and processes available to Oracle Clusterware and Oracle RAC.

• Create X library symbolic links on HP-UX.

• Configure network tuning parameters on AIX Based Systems.

If you have an existing Oracle Inventory, then ensure that you use the same Oracle Inventory for all Oracle software installations, and ensure that all Oracle software users you intend to use for installation have permissions to write to this directory.

1. Run the following command to check for an existing Oracle Inventory directory:

   # more /etc/oraInst.loc
   

   • If the oraInst.loc file exists, then the output from this command is similar to the following:

     inventory_loc=/u01/app/oraInventory
     inst_group=oinstall
     

     In the previous output example:

     • The inventory_loc group shows the location of the Oracle Inventory.

     • The inst_group parameter shows the name of the Oracle Inventory group (in this example, oinstall).

   • If the Oracle Inventory directory does not exist, then the command returns an error message indicating that the file or directory does not exist.

2. If the Oracle Inventory directory does not exist, then you do not have to create one before installing the Oracle software.

During installation, you are prompted to provide a path to a home directory in which to place the software for Oracle Grid Infrastructure for a cluster. OUI installs Oracle Clusterware and Oracle ASM into a directory referred to as Grid_home.

# mkdir -p /u01/app/12.2.0/grid
# chown -R grid:oinstall /u01/app/12.2.0/grid

Before installing any Oracle software, you must configure an Oracle base directory. OUI uses the Oracle base directory to determine the location of the Oracle Inventory directory, and also for installing Oracle RAC.

/mount_point/app/user

1. Log in as the root user.
2. Use the mkdir command to create the path to the Oracle base directory.

   # mkdir -p /u01/app/oracle/
   

3. Change the ownership of the Oracle base path to the Oracle software owner, oracle.

   # chown -R oracle:oinstall /u01/app/oracle/
   

4. Change the permissions on the Oracle base directory to 775.

   # chmod -R 775 /u01/app/oracle/
   

The Oracle home directory is the location in which the Oracle RAC software is installed.

You can use an Oracle home directory created in the local file system, for example, /u01/app/oracle/product/12.2.0/dbhome_1. The same directory must exist on every node in the cluster. You do not have to create these directories before installation. By default, the installer suggests a subdirectory of the Oracle base directory for the Oracle home.

You can also use a shared Oracle home. The location of the shared Oracle home can be on network storage, or a supported cluster file system such as Oracle Automatic Storage Management Cluster File System (Oracle ACFS). For more information about Oracle ACFS, see Oracle Automatic Storage Management Administrator's Guide.

If you use the local file system for the Oracle home directory, and you want to install a different release of Oracle RAC or Oracle Database on the same server, then you must use a separate Oracle home directory for each software installation. Multiple releases of the same product or different products can run from different Oracle homes concurrently. Products installed in one Oracle home do not conflict or interact with products installed in another Oracle home.

Using different Oracle homes for your installed software enables you to perform maintenance operations on the Oracle software in one home without effecting the software in another Oracle home. However, it also increases your software maintenance costs, because each Oracle home must be upgraded or patched separately.

Each node in a cluster requires external shared disks for storing the Oracle Clusterware (Oracle Cluster Registry and voting disk) files, and Oracle Database files.

The supported types of shared storage depend upon the platform you are using, for example:

• Oracle Automatic Storage Management (strongly recommended)

• A supported cluster file system, such as OCFS2 for Linux, or General Parallel File System (GPFS) on IBM platforms

• Network file system (NFS), which is not supported on Linux on POWER or on IBM zSeries Based Linux

• (Upgrades only) Shared disk partitions consisting of block devices. Block devices are disk partitions that are not mounted using the Linux file system. Oracle Clusterware and Oracle RAC write to these partitions directly.

  Note:

  You cannot use OUI to install Oracle Clusterware files on block or raw devices. You cannot put Oracle Clusterware binaries and files on Oracle Automatic Storage Management Cluster File System (Oracle ACFS).

If you decide to use OCFS2 to store the Oracle Clusterware files, then you must use the proper version of OCFS2 for your operating system version. OCFS2 works with Oracle Linux and Red Hat Linux kernel version 2.6.

For all installations, you must choose the storage option to use for Oracle Clusterware files and Oracle Database files. The examples in this guide use Oracle ASM to store the Oracle Clusterware and Oracle Database files. The Oracle Grid Infrastructure for a cluster and Oracle RAC software is installed on disks local to each node, not on a shared file system.

This guide describes two different methods of configuring the shared disks for use with Oracle ASM:

• Creating Files on a NAS Device for Use with Oracle Automatic Storage Management

• Using Oracle ASMFD to Configure Disks for Oracle ASM