2 Pre-Installation Tasks for RAC on AIX

Checking the Software Requirements

Check that the required software and patches are installed on the system.

Check for Required Software

Depending on the products that you intend to install, verify that the following software is installed on the system. The procedure following the table describes how to check these requirements.

bos.adt.base
bos.adt.lib
bos.adt.libm
bos.perf.libperfstat
bos.perf.perfstat
bos.perf.proctools
rsct.basic.rte

X11.motif.lib

mqm.Client.Bnd
mqm.Server.Bnd

To ensure that the system meets these requirements, follow these steps:

1. To determine which version of AIX is installed, enter the following command:

   # oslevel -r
   
   

   If the operating system version is lower than AIX 5.2.0.0 Maintenance Level 1 (5200-01), upgrade your operating system to this level. AIX 5L version 5.2 maintenance packages are available from the following Web site:

   http://www-912.ibm.com/eserver/support/fixes/
   
   

2. To determine whether the required filesets are installed and committed, enter a command similar to the following:

   # lslpp -l bos.adt.base bos.adt.lib bos.adt.libm bos.perf.perfstat \
    bos.perf.libperfstat bos.perf.proctools rsct.basic.rte
   
   

   If a fileset is not installed and committed, then install it. Refer to your operating system or software documentation for information about installing filesets.

Check for Required Patches

Depending on the products that you intend to install, verify that the following patches are installed on the system. The procedure following the table describes how to check these requirements.

To ensure that the system meets these requirements, follow these steps:

1. To determine whether an APAR is installed, enter a command similar to the following:

   # /usr/sbin/instfix -i -k "IY43980 IY44810 IY45462 IY45707 IY46214 IY46605 \ 
     IY48525 IY51801 IY56024"
   
   

   If an APAR is not installed, download it from the following Web site and install it:

   http://www-912.ibm.com/eserver/support/fixes/
   
   

2. To determine whether a PTF is installed, enter a command similar to the following:

   # lslpp -l -B U489726 U485561 ...
   
   

   If a PTF is not installed, download it from the following Web site and install it:

   http://www-912.ibm.com/eserver/support/fixes/
   
   

3. If you require a CSD for WebSphere MQ, refer to the following Web site for download and installation information:

   http://www.ibm.com/software/integration/mqfamily/support/summary/aix.html
   
   

Creating the Oracle Inventory Group

You must create the Oracle Inventory group if it does not already exist. The following subsections describe how to determine the Oracle Inventory group name, if it exists, and how to create it if necessary.

Determine Whether the Oracle Inventory Group Exists

When you install Oracle software on the system for the first time, the Installer creates the oraInst.loc file. This file identifies the name of the Oracle Inventory group and the path of the Oracle Inventory directory. To determine whether the Oracle Inventory group exists, enter the following command:

# more /etc/oraInst.loc

If the oraInst.loc file exists, the output from this command is similar to the following:

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

The inst_group parameter shows the name of the Oracle Inventory group (oinstall).

Create the Oracle Inventory Group

If the oraInst.loc file does not exist, create the Oracle Inventory group as follows:

1. Enter the following command:

   # smit security
   
   

2. Choose the appropriate menu items to create the oinstall group.

3. Press F10 to exit.

Creating the OSDBA Group

You must create an OSDBA group in the following circumstances:

• An OSDBA group does not exist, for example, if this is the first installation of Oracle Database software on the system

• An OSDBA group exists, but you want to give a different group of UNIX users database administrative privileges in a new Oracle installation

If the OSDBA group does not exist or if you require a new OSDBA group, create it as follows. In the following procedure, use the group name dba unless a group with that name already exists.

1. Enter the following command:

   # smit security
   
   

2. Choose the appropriate menu items to create the dba group.

3. Press F10 to exit.

Creating an OSOPER Group (Optional)

Create an OSOPER group only if you want to identify a group of UNIX users with a limited set of database administrative privileges (SYSOPER operator privileges). For most installations, it is sufficient to create only the OSDBA group. If you want to use an OSOPER group, you must create it in the following circumstances:

• If an OSOPER group does not exist, for example, if this is the first installation of Oracle Database software on the system

• If an OSOPER group exists, but you want to give a different group of UNIX users database operator privileges in a new Oracle installation

If you require an new OSOPER group, create it as follows. In the following procedure, use the group name oper unless a group with that name already exists.

1. Enter the following command:

   # smit security
   
   

2. Choose the appropriate menu items to create the oper group.

3. Press F10 to exit.

Creating the Oracle Software Owner User

You must create an Oracle software owner user in the following circumstances:

• If an Oracle software owner user does not exist, for example, if this is the first installation of Oracle software on the system

• If an Oracle software owner user exists, but you want to use a different UNIX user, with different group membership, to give database administrative privileges to those groups in a new Oracle Database installation

Determine Whether an Existing Oracle Software Owner User Exists

To determine whether an Oracle software owner user named oracle exists, enter the following command:

# id oracle

If the oracle user exists, the output from this command is similar to the following:

uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

If the user exists, determine whether you want to use the existing user or create a new user. If you want to use the existing user, ensure that the user's primary group is the Oracle Inventory group and that it is a member of the appropriate OSDBA and OSOPER groups. Refer to one of the following sections for more information:

• If you want to use the existing Oracle software owner user, and the user's primary group is the Oracle Inventory group, refer to the "Verifying that the UNIX User nobody Exists" section.

• To modify an existing user, refer to the "Modify an Existing Oracle Software Owner User" section.

• To create a new user, refer to the following section.

Create a New Oracle Software Owner User

If the Oracle software owner user does not exist or if you require a new Oracle software owner user, create it as follows. Use the user name oracle unless a user with that name already exists.

1. Enter the following command:

   # smit security
   
   

2. Choose the appropriate menu items to create the oracle user, specifying the following information:

   • In the Primary GROUP field, specify the Oracle Inventory group, for example oinstall.

   • In the Group SET field, specify the OSDBA group and if required, the OSOPER group. For example: dba or dba,oper.

   
   

   Note: The UID for the oracle user must be less than 65536.

   
   

3. Press F10 to exit.

4. Set the password of the oracle user:

   # passwd oracle
   
   

Refer to the "Verifying that the UNIX User nobody Exists" section to continue.

Modify an Existing Oracle Software Owner User

If the oracle user exists, but its primary group is not oinstall or it is not a member of the appropriate OSDBA or OSOPER groups, you can modify it as follows:

1. Enter the following command:

   # smit security
   
   

2. Choose the appropriate menu items to modify the oracle user.

3. In the Primary GROUP field, specify the Oracle Inventory group, for example oinstall.

4. In the Group SET field, specify the required secondary groups, for example dba and oper.

5. Press F10 to exit.

Verifying that the UNIX User nobody Exists

Before installing the software, verify that the UNIX user nobody exists on the system:

1. To determine whether the user exists, enter the following command:

   # id nobody
   
   

   If this command displays information about the nobody user, you do not have to create that user.

2. If the nobody user does not exist, enter the following command to create it:

   # smit security
   
   

3. Repeat this procedure on all of the other cluster nodes.

Create Identical Users and Groups on Other Cluster Nodes

The Oracle software owner user and the Oracle Inventory, OSDBA, and OSOPER groups must exist and be identical on all cluster nodes. To create these identical users and groups, you must identify the user ID and group IDs assigned them on the node where you created them, then create the user and groups with the same name and ID on the other cluster nodes.

Identifying the User and Group IDs

To determine the user ID (UID) of the Oracle software owner user and the group IDs (GID) of the Oracle Inventory, OSDBA, and OSOPER groups, follow these steps:

1. Enter following command:

   # id oracle
   
   

   The output from this command is similar to the following:

   uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)
   
   

2. From the output, identify the UID for the oracle user and the GIDs for the groups to which it belongs.

Create the User and Groups on the Other Cluster Nodes

To create the user and groups on the other cluster nodes, repeat the following procedure on each node:

1. Log in to the next cluster node as root.

2. Enter the following command:

   # smit security
   
   

3. Choose the appropriate menu items to create or modify the oinstall and dba groups, and if required, the oper group.

   In the Group NAME and Group ID fields, specify the group names and GIDs that you identified previously.

   
   

   Note: If you cannot use the same group ID for a particular group on this node, identify a group ID that is available on all nodes and specify that ID for the group on all of the nodes.

   
   

4. Choose the appropriate menu items to create or modify the oracle user, specifying the following information:

   • In the User ID field, specify the user ID that you identified in the previous subsection.

   • In the Primary GROUP field, specify the Oracle Inventory group, for example oinstall.

   • In the Group SET field, specify the OSDBA group and if necessary, the OSOPER group, for example dba or dba,oper.

     
     

     Note: If you cannot use the same user ID for the oracle user on this node, identify a user ID that is available on all nodes and specify that ID for the user on all of the nodes.

     
     

5. Press F10 to exit.

6. Set the password of the oracle user:

   # passwd oracle
   
   

Set Up User Equivalence for rsh and rcp on All Cluster Nodes

Before you install and use Oracle Real Application clusters, you must set up user equivalence on all cluster nodes, as follows:

1. On all cluster nodes, edit the /etc/hosts.equiv file in any text editor and add entries similar to the following for all nodes in the cluster:

   node1-public oracle
   node1-public.domain oracle
   node2-public oracle
   node2-public.domain oracle
      .
      .
      .
   

   In this example:

   • noden_public is the host name associated with the primary public network interface

   • domain is the domain name of the system, for example mydomain.com

   
   

   Note: Alternatively, you can create a .rhosts file using the same format in the oracle user's home directory on all nodes in the cluster.

   
   

2. To test user equivalence on all cluster nodes, including the first node, enter commands similar to the following from the node where you will run the Installer:

   # su oracle
   $ rsh node1-public date
   $ rsh node2-public date
       .
       .
       .
   $ exit
   
   

   Each command should return only the date setting on the specified node. If a node prompts for a password or displays any other text, then user equivalence is not set up correctly on that node. In this case, verify that the /etc/hosts.equiv (or ~oracle/.rhosts) file has the correct entries.

Configuring Shell Limits, System Configuration, and Network Tuning Parameters

Oracle recommends that you set shell limits, system configuration parameters, and network tuning parameters as described in this section on all cluster nodes.

Configure Shell Limits

Verify that the shell limits shown in the following table are set to the values shown. The procedure following the table describes how to verify and set the values.

To view the current value specified for these shell limits, and to change them if necessary, follow these steps:

1. Enter the following command:

   # smit chuser
   
   

2. In the User NAME field, enter the user name of the Oracle software owner, for example oracle.

3. Scroll down the list and verify that the value shown for the soft limits listed in the previous table is -1.

   If necessary, edit the existing value.

4. When you have finished making changes, press F10 to exit.

Configure System Configuration Parameters

Verify that the maximum number of processes allowed per user is set to 2048 or greater:

1. Enter the following command:

   # smit chgsys
   
   

2. Verify that the value shown for Maximum number of PROCESSES allowed per user is greater than or equal to 2048.

   If necessary, edit the existing value.

3. When you have finished making changes, press F10 to exit.

Configure Network Tuning Parameters

Verify that the network tuning parameters shown in the following table are set to the values shown or higher values. The procedure following the table describes how to verify and set the values.

To view the current value specified for these parameters, and to change them if necessary, follow these steps:

1. To check the current values of the network tuning parameters, enter commands similar to the following:

   # /usr/sbin/no -a | more
   
   

2. If you must change the value of any parameter, enter the following command to determine whether the system is running in compatibility mode:

   # /usr/sbin/lsattr -E -l sys0 -a pre520tune
   
   

   If the system is running in compatibility mode, the output is similar to the following, showing that the value of the pre520tune attribute is enable:

   pre520tune enable Pre-520 tuning compatibility mode True
   
   

3. If the system is running in compatibility mode, follow these steps to change the parameter values:

   1. Enter commands similar to the following to change the value of each parameter:

      # /usr/sbin/no -o parameter_name=value
      
      

      For example:

      # /usr/sbin/no -o udp_recvspace=655360
      
      

   2. Add entries similar to the following to the /etc/rc.net file for each parameter that you changed in the previous step:

      if [ -f /usr/sbin/no ] ; then
         /usr/sbin/no -o udp_sendspace=65536
         /usr/sbin/no -o udp_recvspace=655360
         /usr/sbin/no -o tcp_sendspace=65536
         /usr/sbin/no -o tcp_recvspace=65536
         /usr/sbin/no -o rfc1323=1
         /usr/sbin/no -o sb_max=2*655360
         /usr/sbin/no -o ipqmaxlen=512
      fi
      
      

      By adding these lines to the /etc/rc.net file, the values persist when the system restarts.

4. If the system is not running in compatibility mode, enter commands similar to the following to change the parameter values:

   • ipqmaxlen parameter:

     /usr/sbin/no -r -o ipqmaxlen=512
     
     

   • Other parameter:

     /usr/sbin/no -p -o parameter=value
     
     

   
   

   Note: If you modify the ipqmaxlen parameter, you must restart the system.

   
   

   These commands modify the /etc/tunables/nextboot file, causing the attribute values to persist when the system restarts.

Identifying Storage Requirements for ASM

To identify the storage requirements for using ASM, you must determine how many devices and the amount of free disk space that you require. To complete this task, follow these steps:

1. Determine whether you want to use ASM for Oracle database files, recovery files, or both.

   
   

   Note: You do not have to use the same storage mechanism for database files and recovery files. You can use the file system for one file type and ASM for the other. For RAC installations, if you choose to enable automated backups and you do not have a shared file system available, you must choose ASM for recovery file storage.

   
   

   If you enable automated backups during the installation, you can choose ASM as the storage mechanism for recovery files by specifying an ASM disk group for the flash recovery area. Depending on how you choose to create a database during the installation, you have the following options:

   • If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to use the same ASM disk group for database files and recovery files, or you can choose to use different disk groups for each file type.

     The same choice is available to you if you use DBCA after the installation to create a database.

   • If you select an installation method that runs DBCA in non-interactive mode, you must use the same ASM disk group for database files and recovery files.

2. Choose the ASM redundancy level that you want to use for the ASM disk group.

   The redundancy level that you choose for the ASM disk group determines how ASM mirrors files in the disk group and determines the number of disks and amount of disk space that you require, as follows:

   • External redundancy

     An external redundancy disk group requires a minimum of one disk device. The effective disk space in an external redundancy disk group is the sum of the disk space in all of its devices.

     Because ASM does not mirror data in an external redundancy disk group, Oracle recommends that you use only RAID or similar devices that provide their own data protection mechanisms as disk devices in this type of disk group.

   • Normal redundancy

     In a normal redundancy disk group, ASM uses two-way mirroring by default, to increase performance and reliability. A normal redundancy disk group requires a minimum of two disk devices (or two failure groups). The effective disk space in a normal redundancy disk group is half the sum of the disk space in all of its devices.

     For most installations, Oracle recommends that you use normal redundancy disk groups.

   • High redundancy

     In a high redundancy disk group, ASM uses three-way mirroring to increase performance and provide the highest level of reliability. A high redundancy disk group requires a minimum of three disk devices (or three failure groups). The effective disk space in a high redundancy disk group is one-third the sum of the disk space in all of its devices.

     While high redundancy disk groups do provide a high level of data protection, you must consider the higher cost of additional storage devices before deciding to use this redundancy level.

3. Determine the total amount of disk space that you require for the database files and recovery files.

   Use the following table to determine the minimum number of disks and the minimum disk space requirements for the installation:

   Redundancy Level	Minimum Number of Disks	Database Files	Recovery Files	Both File Types
   External	1	1.15 GB	2.3 GB	3.45 GB
   Normal	2	2.3 GB	4.6 GB	6.9 GB
   High	3	3.45 GB	6.9 GB	10.35 GB

   
   

   For RAC installations, you must also add additional disk space for the ASM metadata. You can use the following formula to calculate the additional disk space requirements (in MB):

   15 + (2 * number_of_disks) + (126 * number_of_ASM_instances)

   For example, for a four-node RAC installation, using three disks in a high redundancy disk group, you require an additional 525 MB of disk space:

   15 + (2 * 3) + (126 * 4) = 525

   If an ASM instance is already running on the system, you can use an existing disk group to meet these storage requirements. If necessary, you can add disks to an existing disk group during the installation.

   The following section describes how to identify existing disk groups and determine the free disk space that they contain.

4. Optionally identify failure groups for the ASM disk group devices.

   
   

   Note: You need to complete this step only if you intend to use an installation method that runs DBCA in interactive mode, for example, if you intend to choose the Custom installation type or the Advanced database configuration option. Other installation types do not enable you to specify failure groups.

   
   

   If you intend to use a normal or high redundancy disk group, you can further protect your database against hardware failure by associating a set of disk devices in a custom failure group. By default, each device comprises its own failure group. However, if two disk devices in a normal redundancy disk group are attached to the same SCSI controller, the disk group becomes unavailable if the controller fails. The controller in this example is a single point of failure.

   To protect against failures of this type, you could use two SCSI controllers, each with two disks, and define a failure group for the disks attached to each controller. This configuration would enable the disk group to tolerate the failure of one SCSI controller.

   
   

   Note: If you define custom failure groups, you must specify a minimum of two failure groups for normal redundancy disk groups and three failure groups for high redundancy disk groups.

   
   

5. If you are sure that a suitable disk group does not exist on the system, install or identify appropriate disk devices to add to a new disk group. Use the following guidelines when identifying appropriate disk devices:

   • All of the devices in an ASM disk group should be the same size and have the same performance characteristics.

   • Do not specify more than one partition on a single physical disk as a disk group device. ASM expects each disk group device to be on a separate physical disk.

   • Although you can specify a logical volume as a device in an ASM disk group, Oracle does not recommend their use. Logical volume managers can hide the physical disk architecture, preventing ASM from optimizing I/O across the physical devices.

   For information about completing this task, refer to the "Configuring Disks for ASM" section.

Using an Existing ASM Disk Group

If you want to store either database or recovery files in an existing ASM disk group, you have the following choices, depending on the installation method that you select:

• If you select an installation method that runs DBCA in interactive mode, by choosing the Advanced database configuration option for example, you can decide whether you want to create a new disk group or use an existing one.

  The same choice is available to you if you use DBCA after the installation to create a database.

• If you select an installation method that runs DBCA in non-interactive mode, you must choose an existing disk group for the new database; you cannot create a new disk group. However, you can add disk devices to an existing disk group if it has insufficient free space for your requirements.

To determine whether an existing ASM disk group exists, or to determine whether there is sufficient disk space in a disk group, you can use Oracle Enterprise Manager Grid Control or Database Control. Alternatively, you can use the following procedure:

1. View the contents of the oratab file to determine whether an ASM instance is configured on the system:

   # more /etc/oratab
   
   

   If an ASM instance is configured on the system, the oratab file should contain a line similar to the following:

   +ASM:oracle_home_path:N
   
   

   In this example, +ASM is the system identifier (SID) of the ASM instance and oracle_home_path is the Oracle home directory where it is installed. By convention, the SID for an ASM instance begins with a plus sign.

2. Set the ORACLE_SID and ORACLE_HOME environment variables to specify the appropriate values for the ASM instance that you want to use.

3. Connect to the ASM instance as the SYS user with SYSDBA privilege and start the instance if necessary:

   # $ORACLE_HOME/bin/sqlplus "SYS/SYS_password as SYSDBA"
   SQL> STARTUP
   
   

4. Enter the following command to view the existing disk groups, their redundancy level, and the amount of free disk space in each one:

   SQL> SELECT NAME,TYPE,TOTAL_MB,FREE_MB FROM V$ASM_DISKGROUP;
   
   

5. From the output, identify a disk group with the appropriate redundancy level and note the free space that it contains.

6. If necessary, install or identify the additional disk devices required to meet the storage requirements listed in the previous section.

   
   

   Note: If you are adding devices to an existing disk group, Oracle recommends that you use devices that have the same size and performance characteristics as the existing devices in that disk group.

   
   

Configuring Disks for ASM

To configure disks for use with ASM on AIX, follow these steps:

1. If necessary, install the shared disks that you intend to use for the ASM disk group and restart the system.

2. To make sure that the disks are available, enter the following command on every node:

   # /usr/sbin/lsdev -Cc disk
   
   

   The output from this command is similar to the following:

   hdisk0 Available 1A-09-00-8,0  16 Bit LVD SCSI Disk Drive
   hdisk1 Available 1A-09-00-9,0  16 Bit LVD SCSI Disk Drive
   hdisk2 Available 17-08-L       SSA Logical Disk Drive
   
   

3. If a disk is not listed as available on any node, enter the following command to configure the new disks:

   # /usr/sbin/cfgmgr
   
   

4. Enter the following command on any node to identify the device names for the physical disks that you want to use:

   # /usr/sbin/lspv | grep -i none
   
   

   This command displays information similar to the following for each disk that is not configured in a volume group:

   hdisk2     0000078752249812   None
   
   

   In this example, hdisk2 is the device name of the disk and 0000078752249812 is the physical volume ID (PVID). The disks that you want to use might have a PVID, but they must not belong to a volume group.

5. If a disk that you want to use for the disk group does not have a PVID, enter a command similar to the following to assign one to it:

   # /usr/sbin/chdev -l hdiskn -a pv=yes
   
   

6. On each of the other nodes, enter a command similar to the following to identify the device name associated with each PVID on that node:

   # /usr/sbin/lspv | grep -i 0000078752249812
   
   

   Depending on how each node is configured, the device names might differ between nodes.

7. To enable simultaneous access to a disk device from multiple nodes, you must set the appropriate Object Data Manager (ODM) attribute listed in the following table to the value shown, depending on the disk type:

   Disk Type	Attribute	Value
   SSA or FAStT disks	reserve_lock	no
   ESS, EMC, HDS, CLARiiON, or MPIO-capable disks	reserve_policy	no_reserve

   
   

   To determine whether the attribute has the correct value, enter a command similar to the following on all cluster nodes for each disk device that you want to use:

   # /usr/sbin/lsattr -E -l hdiskn
   
   

   If the required attribute is not set to the correct value on any node, enter a command similar to one of the following on that node:

   • SSA and FAStT devices:

     # /usr/sbin/chdev -l hdiskn  -a reserve_lock=no
     
     

   • ESS, EMC, HDS, CLARiiON, and MPIO-capable devices:

     # /usr/sbin/chdev -l hdiskn  -a reserve_policy=no_reserve
     
     

8. Enter commands similar to the following on any node to clear the PVID from each disk device that you want to use:

   # /usr/sbin/chdev -l hdiskn -a pv=clear
   
   

9. Enter commands similar to the following on every node to change the owner, group, and permissions on the character raw device file for each disk that you want to add to the disk group:

   # chown oracle:dba /dev/rhdiskn
   # chmod 660 /dev/rhdiskn
   

   
   

   Note: If you are using a multi-pathing disk driver with ASM, make sure that you set the permissions only on the correct logical device name for the disk. The device name associated with a disk might be different on other nodes. Make sure that you specify the correct device name on each node.

   
   

10. If you also want to use raw devices for storage, refer to the "Configure Raw Devices" section.

    Otherwise, refer to the "Verify the Cluster Software Configuration" section.

Configuring Raw Disk Devices or Raw Logical Volumes

The following subsections describe how to configure raw disk devices or raw logical volumes on AIX:

• Configuring Raw Disk Devices for Oracle CRS or Database File Storage for Installations Without HACMP or GPFS

  Follow the procedures described in this section if you are installing RAC on a cluster without HACMP or GPFS and you intend to use raw devices for CRS or database file storage.

• Configuring Raw Logical Volumes for Oracle CRS or Database File Storage

  Follow the procedures described in this section only if you are using HACMP.

Configuring Raw Disk Devices for Oracle CRS or Database File Storage for Installations Without HACMP or GPFS

If you are installing RAC on an AIX cluster without HACMP or GPFS, you must use shared raw disk devices for the Cluster Ready Services (CRS) files. You can also use shared raw disk devices for database file storage, however, Oracle recommends that you use ASM to store database files in this situation. This section describes how to configure the shared raw disk devices for Oracle CRS files (Oracle Cluster Registry and CRS voting disk) and database files.

Table 2-1 lists the number and size of the raw disk devices that you must configure for database files. Table 2-2 lists the number and size of the raw disk devices that you must configure for CRS files.

dbname_system_raw_500m

dbname_sysaux_raw_800m

dbname_undotbsn_raw_500m

dbname_temp_raw_250m

dbname_example_raw_160m

dbname_users_raw_120m

dbname_redon_m_raw_120m

dbname_control{1|2}_raw_110m

dbname_spfile_raw_5m

dbname_pwdfile_raw_5m

ora_ocr_raw_100m

ora_vote_raw_20m

To configure shared raw disk devices for CRS files, database files, or both, follow these steps:

1. If you intend to use raw disk devices for database file storage, choose a name for the database that you want to create.

   The name that you choose must start with a letter and have no more than four characters, for example, orcl.

2. Identify or configure the required disk devices.

   The disk devices must be shared on all of the cluster nodes.

3. As the root user, enter the following command on any node to identify the device names for the disk devices that you want to use:

   # /usr/sbin/lspv | grep -i none 
   
   

   This command displays information similar to the following for each disk device that is not configured in a volume group:

   hdisk17         0009005fb9c23648                    None  
   
   

   In this example, hdisk17 is the device name of the disk and 0009005fb9c23648 is the physical volume ID (PVID).

4. If a disk device that you want to use does not have a PVID, enter a command similar to the following to assign one to it:

   # /usr/sbin/chdev -l hdiskn -a pv=yes
   
   

5. On each of the other nodes, enter a command similar to the following to identify the device name associated with each PVID on that node:

   # /usr/sbin/lspv | grep -i "0009005fb9c23648"
   
   

   The output from this command should be similar to the following:

   hdisk18         0009005fb9c23648                    None  
   
   

   In this example, the device name associated with the disk device (hdisk18) is different on this node.

6. If the device names are the same on all nodes, enter commands similar to the following on all nodes to change the owner, group, and permissions on the character raw device files for the disk devices:

   • OCR device:

     # chown root:oinstall /dev/rhdiskn
     # chmod 640 /dev/rhdiskn
     
     

   • Other devices:

     # chown oracle:dba /dev/rhdiskn
     # chmod 660 /dev/rhdiskn
     
     

7. If the device name associated with the PVID for a disk that you want to use is different on any node, you must create a new device file for the disk on each of the nodes using a common unused name.

   For the new device files, choose an alternative device file name that identifies the purpose of the disk device. The previous table suggests alternative device file names for each file. For database files, replace dbname in the alternative device file name with the name that you chose for the database in step 1.

   
   

   Note: Alternatively, you could choose a name that contains a number that will never be used on any of the nodes, for example hdisk99.

   
   

   To create a new common device file for a disk device on all nodes, follow these steps on each node:

   1. Enter the following command to determine the device major and minor numbers that identify the disk device, where n is the disk number for the disk device on this node:

      # ls -alF /dev/*hdiskn
      
      

      The output from this command is similar to the following:

      brw------- 1 root system    24,8192 Dec 05 2001  /dev/hdiskn
      crw------- 1 root system    24,8192 Dec 05 2001  /dev/rhdiskn
      
      

      In this example, the device file /dev/rhdiskn represents the character raw device, 24 is the device major number, and 8192 is the device minor number.

   2. Enter a command similar to the following to create the new device file, specifying the new device file name and the device major and minor numbers that you identified in the previous step:

      
      

      Note: In the following example, you must specify the character c to create a character raw device file.

      
      

      # mknod /dev/ora_ocr_raw_100m c 24 8192
      
      

   3. Enter commands similar to the following to change the owner, group, and permissions on the character raw device file for the disk:

      • OCR:

        # chown root:oinstall /dev/ora_ocr_raw_100m
        # chmod 640 /dev/ora_ocr_raw_100m
        
        

      • CRS voting disk or database files:

        # chown oracle:dba /dev/ora_vote_raw_20m
        # chmod 660 /dev/ora_vote_raw_20m
        
        

   4. Enter a command similar to the following to verify that you have created the new device file successfully:

      # ls -alF /dev | grep "24,8192"
      
      

      The output should be similar to the following:

      brw------- 1 root   system   24,8192 Dec 05 2001  /dev/hdiskn
      crw-r----- 1 root   oinstall 24,8192 Dec 05 2001  /dev/ora_ocr_raw_100m
      crw------- 1 root   system   24,8192 Dec 05 2001  /dev/rhdiskn
      
      

8. To enable simultaneous access to a disk device from multiple nodes, you must set the appropriate Object Data Manager (ODM) attribute listed in the following table to the value shown, depending on the disk type:

   Disk Type	Attribute	Value
   SSA or FAStT disks	reserve_lock	no
   ESS, EMC, HDS, CLARiiON, or MPIO-capable disks	reserve_policy	no_reserve

   
   

   To determine whether the attribute has the correct value, enter a command similar to the following on all cluster nodes for each disk device that you want to use:

   # /usr/sbin/lsattr -E -l hdiskn
   
   

   If the required attribute is not set to the correct value on any node, enter a command similar to one of the following on that node:

   • SSA and FAStT devices

     # /usr/sbin/chdev -l hdiskn  -a reserve_lock=no
     
     

   • ESS, EMC, HDS, CLARiiON, and MPIO-capable devices

     # /usr/sbin/chdev -l hdiskn  -a reserve_policy=no_reserve
     
     

9. Enter commands similar to the following on any node to clear the PVID from each disk device that you want to use:

   # /usr/sbin/chdev -l hdiskn -a pv=clear
   
   

10. If you are using raw disk devices for database files, follow these steps to create the DBCA raw device mapping file:

    
    

    Note: You must complete this procedure only if you are using raw devices for database files. The DBCA raw device mapping file enables the DBCA to identify the appropriate raw disk device for each database file. You do not specify the raw devices for the Oracle CRS files in the DBCA raw device mapping file.

    
    

    1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously:

       • Bourne, Bash, or Korn shell:

         $ ORACLE_BASE=/u01/app/oracle ; export ORACLE_BASE
         
         

       • C shell:

         % setenv ORACLE_BASE /u01/app/oracle
         
         

    2. Create a database file subdirectory under the Oracle base directory and set the appropriate owner, group, and permissions on it:

       # mkdir -p $ORACLE_BASE/oradata/dbname
       # chown -R oracle:oinstall $ORACLE_BASE/oradata
       # chmod -R 775 $ORACLE_BASE/oradata
       
       

       In this example, dbname is the name of the database that you chose previously.

    3. Change directory to the $ORACLE_BASE/oradata/dbname directory.

    4. Using any text editor, create a text file similar to the following that identifies the disk device file name associated with each database file.

       Oracle recommends that you use a file name similar to dbname_raw.conf for this file.

       
       

       Note: The following example shows a sample mapping file for a two-instance RAC cluster. Some of the devices use alternative disk device file names. Make sure that the device file name that you specify identifies the same disk device on all nodes.

       
       

       system=/dev/rhdisk11
       sysaux=/dev/rhdisk12
       example=/dev/rhdisk13
       users=/dev/rhdisk14
       temp=/dev/rhdisk15
       undotbs1=/dev/rhdisk16
       undotbs2=/dev/rhdisk17
       redo1_1=/dev/rhdisk18
       redo1_2=/dev/rhdisk19
       redo2_1=/dev/rhdisk20
       redo2_2=/dev/rhdisk22
       control1=/dev/rhdisk23
       control2=/dev/rhdisk24
       spfile=/dev/dbname_spfile_raw_5m
       pwdfile=/dev/dbname_pwdfile_raw_5m
       
       

       In this example, dbname is the name of the database.

       Use the following guidelines when creating or editing this file:

       • Each line in the file must have the following format:

         database_object_identifier=device_file_name
         
         

         The alternative device file names suggested in the previous table include the database object identifier that you must use in this mapping file. For example, in the following alternative disk device name, redo1_1 is the database object identifier:

         /dev/rac_redo1_1_raw_120m
         
         

       • For a RAC database, the file must specify one automatic undo tablespace datafile (undotbsn) and two redo log files (redon_1, redon_2) for each instance.

       • Specify at least two control files (control1, control2).

       • To use manual instead of automatic undo management, specify a single RBS tablespace datafile (rbs) instead of the automatic undo management tablespace datafiles.

    5. Save the file and note the file name that you specified.

    6. When you are configuring the oracle user's environment later in this chapter, set the DBCA_RAW_CONFIG environment variable to specify the full path to this file.

11. When you are installing Oracle Cluster Ready Services, you must enter the paths to the appropriate device files when prompted for the path of the OCR and CRS voting disk, for example:

    /dev/rhdisk10
    
    

12. To continue the installation, refer to the "Verify the Cluster Software Configuration"section

Configuring Raw Logical Volumes for Oracle CRS or Database File Storage

This section describes how to configure raw logical volumes for Oracle CRS and database file storage. The procedures in this section describe how to create a new volume group that contains the logical volumes required for both types of files.

Before you continue, review the following guidelines which contain important information about using volume groups with this release of Oracle Real Application Clusters:

• You must use concurrent-capable volume groups for Oracle CRS and database files.

• If you specify raw logical volumes for the Oracle Cluster Registry and Oracle CRS voting disk during the Oracle CRS installation, the installation process configures the Oracle CRS startup scripts to activate the volume group that contains these logical volumes when Oracle CRS starts.

  
  

  Note: For this reason, you must not configure the volume group that contains the Oracle CRS files in a HACMP concurrent resource group.

  
  

• The Oracle CRS files require less than 200 MB of disk space. To make efficient use of the disk space in a volume group, Oracle recommends that you use the same volume group for the logical volumes for both the CRS files and the database files.

• If you are upgrading an existing Oracle9i release 2 RAC installation that uses raw logical volumes, you can use the existing SRVM configuration repository logical volume for the OCR and create a new logical volume in the same volume group for the Oracle CRS voting disk. However, you must remove this volume group from the HACMP concurrent resource group that activates it before you install Oracle CRS.

  
  

  See Also: For information about removing a volume group from a concurrent resource group, refer to the HACMP documentation.

  
  
  
  

  Note: If you are upgrading a database, you must also create a new logical volume for the SYSAUX tablespace. Refer to the "Create Raw Logical Volumes in the New Volume Group" section for more information about the requirements for the Oracle CRS voting disk and SYSAUX logical volumes.

  
  

• You must use a HACMP concurrent resource group to activate new or existing volume groups that contain only database files (not Oracle CRS files).

  
  

  See Also: For information about adding a volume group to a new or existing concurrent resource group, refer to the HACMP documentation.

  
  

• All volume groups that you intend to use for Oracle CRS or database files must be activated in concurrent mode before you start the installation.

• The procedures in this section describe how to create basic volumes groups and volumes. If you want to configure more complex volumes, using mirroring for example, use this section in conjunction with the HACMP documentation.

Create a Volume Group

To create a volume group for the Oracle database files and the Oracle CRS files, follow these steps:

1. If necessary, install the shared disks that you intend to use.

2. To make sure that the disks are available, enter the following command on every node:

   # /usr/sbin/lsdev -Cc disk
   
   

   The output from this command is similar to the following:

   hdisk0 Available 1A-09-00-8,0  16 Bit LVD SCSI Disk Drive
   hdisk1 Available 1A-09-00-9,0  16 Bit LVD SCSI Disk Drive
   hdisk2 Available 17-08-L       SSA Logical Disk Drive
   
   

3. If a disk is not listed as available on any node, enter the following command to configure the new disks:

   # /usr/sbin/cfgmgr
   
   

4. Enter the following command on any node to identify the device names and any associated volume group for each disk:

   # /usr/sbin/lspv
   
   

   The output from this command is similar to the following:

   hdisk0     0000078752249812   rootvg
   hdisk1     none               none
   hdisk4     00034b6fd4ac1d71   ccvg1
   
   

   For each disk, this command shows:

   • The disk device name

   • Either the 16 character physical volume identifier (PVID) if the disk has one, or none

   • Either the volume group to which the disk belongs, or none

   The disks that you want to use might have a PVID, but they must not belong to existing volume groups.

5. If a disk that you want to use for the volume group does not have a PVID, enter a command similar to the following to assign one to it:

   # /usr/sbin/chdev -l hdiskn -a pv=yes
   
   

6. To identify used device major numbers, enter the following command on each node of the cluster:

   # ls -la /dev | more
   
   

   This command displays information about all configured devices, similar to the following:

   crw-rw----   1 root     system    45,  0 Jul 19 11:56 vg1
   
   

   In this example, 45 is the major number of the vg1 volume group device.

7. Identify an appropriate major number that is unused on all nodes in the cluster.

8. To create a volume group, enter a command similar to the following, or use SMIT (smit mkvg):

   # /usr/sbin/mkvg -y VGname -B -s PPsize -V majornum -n \
   -C PhysicalVolumes
   
   

   The following table describes the options and variables used in this example. Refer to the mkvg man page for more information about these options.

   Command Option	SMIT Field	Sample Value and Description
   -y VGname	VOLUME GROUP name	oracle_vg1 Specify the name for the volume group. The name that you specify could be a generic name, as shown, or for a database volume group, it could specify the name of the database that you intend to create.
   -B	Create a big VG format Volume Group	Specify this option to create a big VG format volume group. Note: If you are using SMIT, choose yes for this field.
   -s PPsize	Physical partition SIZE in megabytes	32 Specify the size of the physical partitions for the database. The sample value shown enables you to include a disk up to 32 GB in size (32 MB * 1016).
   -V Majornum	Volume Group MAJOR NUMBER	46 Specify the device major number for the volume group that you identified in Step 7.
   -n	Activate volume group AUTOMATICALLY at system restart	Specify this option to prevent the volume group from being activated at system restart. Note: If you are using SMIT, choose no for this field.
   -C	Create VG Concurrent Capable	Specify this option to create a concurrent capable volume group. Note: If you are using SMIT, choose yes for this field.
   PhysicalVolumes	PHYSICAL VOLUME names	hdisk3 hdisk4 Specify the device names of the disks that you want to add to the volume group.

   
   

9. Enter a command similar to the following to vary on the volume group that you created:

   # /usr/sbin/varyonvg VGname
   

Create Raw Logical Volumes in the New Volume Group

To create the required raw logical volumes in the new volume group, follow these steps:

1. Choose a name for the database that you want to create.

   The name that you choose must start with a letter and have no more than four characters, for example, orcl.

2. Identify the logical volumes that you must create.

   Table 2-3 lists the number and size of the logical volumes that you must create for database files. Table 2-4 lists the number and size of the logical volumes that you must create for CRS files.

   Table 2-3 Raw Logical Volumes Required for Database Files on AIX

   Number	Size (MB)	Purpose and Sample Logical Volume Name
   1	500	SYSTEM tablespace: dbname_system_raw_500m
   1	300 + (Number of instances * 250)	SYSAUX tablespace: dbname_sysaux_raw_800m
   Number of instances	500	UNDOTBSn tablespace (One tablespace for each instance, where n is the number of the instance): dbname_undotbsn_raw_500m
   1	250	TEMP tablespace: dbname_temp_raw_250m
   1	160	EXAMPLE tablespace: dbname_example_raw_160m
   1	120	USERS tablespace: dbname_users_raw_120m
   2 * number of instances	120	Two online redo log files for each instance (where n is the number of the instance and m is the log number, 1 or 2): dbname_redon_m_raw_120m
   2	110	First and second control files: dbname_control{1|2}_raw_110m
   1	5	Server parameter file (SPFILE): dbname_spfile_raw_5m
   1	5	Password file: dbname_pwdfile_raw_5m

   
   

   Table 2-4 Raw Logical Volumes Required for CRS Files on AIX

   Number	Size (MB)	Purpose and Sample Logical Volume Name
   1	100	Oracle Cluster Registry: ora_ocr_raw_100m Note: You need to create this raw logical volume only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle Cluster Registry (OCR). If you are upgrading from Oracle9i Release 2, you can continue to use the raw device that you used for the SRVM configuration repository instead of creating this new logical volume.
   1	20	Oracle CRS voting disk: ora_vote_raw_20m Note: You need to create this raw logical volume only once on the cluster. If you create more than one database on the cluster, they all share the same Oracle CRS voting disk.

   
   

3. To create each required logical volume, enter a command similar to the following (or use smit mklv):

   # /usr/sbin/mklv -y LVname -T O -w n -s n -r n VGname NumPPs
   
   

   In this example:

   • LVname is the name of the logical volume that you want to create

   • The -T O option specifies that the device subtype should be z, which causes Oracle to use a zero offset when accessing this raw logical volume

   • VGname is the name of the volume group where you want to create the logical volume

   • NumPPs is the number of physical partitions to use

     To determine the value to use for NumPPs, divide the required size of the logical volume by the size of the physical partition and round the value up to an integer. For example, if the size of the physical partition is 32 MB and you want to create a 500 MB logical volume, you should specify 16 for the NumPPs (500/32 = 15.625).

   The following example shows the command used to create a logical volume for the SYSAUX tablespace of the test database in the oracle_vg1 volume group with a physical partition size of 32 MB (800/32 = 25):

   # /usr/sbin/mklv -y test_sysaux_raw_800m -T O -w n -s n -r n oracle_vg1 25
   
   

4. Change the owner, group, and permissions on the character device files associated with the logical volumes that you created, as follows:

   
   

   Note: The device file associated with the Oracle Cluster Registry must be owned by root. All other device files must be owned by the Oracle software owner user (oracle).

   
   

   • Oracle Database files:

     # chown oracle:dba /dev/rdbname*
     # chmod 660 /dev/rdbname*
     
     

   • Oracle CRS files:

     # chown oracle:dba /dev/rora_vote_raw_20m
     # chmod 660 /dev/rora_vote_raw_20m
     # chown root:oinstall /dev/rora_ocr_raw_100m
     # chmod 640 /dev/rora_ocr_raw_100m
     
     

Import the Volume Group on the Other Cluster Nodes

To make the volume group available to all nodes in the cluster, you must import it on each node, as follows:

1. Because the physical volume names might be different on the other nodes, enter the following command to determine the PVID of the physical volumes used by the volume group:

   # /usr/sbin/lspv
   
   

2. Note the PVIDs of the physical devices used by the volume group.

3. To vary off the volume group that you want to use, enter a command similar to the following on the node where you created it:

   # /usr/sbin/varyoffvg VGname
   
   

4. On each cluster node, complete the following steps:

   1. Enter the following command to determine the physical volume names associated with the PVIDs you noted previously:

      # /usr/sbin/lspv
      
      

   2. On each node of the cluster, enter commands similar to the following to import the volume group definitions:

      # /usr/sbin/importvg -y VGname -V MajorNumber PhysicalVolume
      
      

      In this example, MajorNumber is the device major number for the volume group and PhysicalVolume is the name of one of the physical volumes in the volume group.

      For example, to import the definition of the oracle_vg1 volume group with device major number 45 on the hdisk3 and hdisk4 physical volumes, enter the following command:

      # /usr/sbin/importvg -y oracle_vg1 -V 45 hdisk3
      
      

   3. Change the owner, group, and permissions on the character device files associated with the logical volumes you created, as follows:

      • Oracle Database files:

        # chown oracle:dba /dev/rdbname*
        # chmod 660 /dev/rdbname*
        
        

      • Oracle CRS files:

        # chown oracle:dba /dev/rora_vote_raw_20m
        # chmod 660 /dev/rora_vote_raw_20m
        # chown root:oinstall /dev/rora_ocr_raw_100m
        # chmod 640 /dev/rora_ocr_raw_100m
        
        

   4. Enter the following command to ensure that the volume group will not be activated by the operating system when the node boots:

      # /usr/sbin/chvg -a n VGname
      

Activate the Volume Group in Concurrent Mode on All Cluster Nodes

To activate the volume group in concurrent mode on all cluster nodes, enter the following command on each node:

# /usr/sbin/varyonvg -c VGname

Create the DBCA Raw Device Mapping File

To enable Database Configuration Assistant (DBCA) to identify the appropriate raw device for each database file, you must create a raw device mapping file, as follows:

1. Set the ORACLE_BASE environment variable to specify the Oracle base directory that you identified or created previously:

   • Bourne, Bash, or Korn shell:

     $ ORACLE_BASE=/u01/app/oracle ; export ORACLE_BASE
     
     

   • C shell:

     % setenv ORACLE_BASE /u01/app/oracle
     
     

2. Create a database file subdirectory under the Oracle base directory and set the appropriate owner, group, and permissions on it:

   # mkdir -p $ORACLE_BASE/oradata/dbname
   # chown -R oracle:oinstall $ORACLE_BASE/oradata
   # chmod -R 775 $ORACLE_BASE/oradata
   
   

   In this example, dbname is the name of the database that you chose previously.

3. Change directory to the $ORACLE_BASE/oradata/dbname directory.

4. Enter the following command to create a text file that you can use to create the raw device mapping file:

   # find /dev -user oracle -name 'r*' -print > dbname_raw.conf
   
   

5. Edit the dbname_raw.conf file in any text editor to create a file similar to the following:

   
   

   Note: The following example shows a sample mapping file for a two-instance RAC cluster.

   
   

   system=/dev/rdbname_system_raw_500m
   sysaux=/dev/rdbname_sysaux_raw_800m
   example=/dev/rdbname_example_raw_160m
   users=/dev/rdbname_users_raw_120m
   temp=/dev/rdbname_temp_raw_250m
   undotbs1=/dev/rdbname_undotbs1_raw_500m
   undotbs2=/dev/rdbname_undotbs2_raw_500m
   redo1_1=/dev/rdbname_redo1_1_raw_120m
   redo1_2=/dev/rdbname_redo1_2_raw_120m
   redo2_1=/dev/rdbname_redo2_1_raw_120m
   redo2_2=/dev/rdbname_redo2_2_raw_120m
   control1=/dev/rdbname_control1_raw_110m
   control2=/dev/rdbname_control2_raw_110m
   spfile=/dev/rdbname_spfile_raw_5m
   pwdfile=/dev/rdbname_pwdfile_raw_5m
   
   

   In this example, dbname is the name of the database.

   Use the following guidelines when creating or editing this file:

   • Each line in the file must have the following format:

     database_object_identifier=logical_volume
     
     

     The logical volume names suggested in this manual include the database object identifier that you must use in this mapping file. For example, in the following logical volume name, redo1_1 is the database object identifier:

     /dev/rac_redo1_1_raw_120m
     
     

   • For a RAC database, the file must specify one automatic undo tablespace datafile (undotbsn) and two redo log files (redon_1, redon_2) for each instance.

   • Specify at least two control files (control1, control2).

   • To use manual instead of automatic undo management, specify a single RBS tablespace datafile (rbs) instead of the automatic undo management tablespace datafiles.

6. Save the file and note the file name that you specified.

7. When you are configuring the oracle user's environment later in this chapter, set the DBCA_RAW_CONFIG environment variable to specify the full path to this file.