Skip Navigation Links
Exit Print View
| Oracle Solaris Cluster Software Installation Guide Oracle Solaris Cluster 4.1

Search Scope:

- This Document
- Entire Library

## Planning the Oracle Solaris Cluster Environment

This section provides guidelines for planning and preparing the following components for Oracle Solaris Cluster software installation and configuration:

- Licensing

- Software Updates

- Public-Network IP Addresses

- Console-Access Devices

- Public Network Configuration

- Quorum Server Configuration

- NFS Guidelines

- Service Restrictions

- Network Time Protocol (NTP)

- Oracle Solaris Cluster Configurable Components

- Zone Clusters

For detailed information about Oracle Solaris Cluster components, see the Oracle Solaris Cluster Concepts Guide.

### Licensing

Ensure that you have available all necessary license certificates before you begin software installation. Oracle Solaris Cluster software does not require a license certificate, but each node installed with Oracle Solaris Cluster software must be covered under your Oracle Solaris Cluster software license agreement.

For licensing requirements for volume-manager software and applications software, see the installation documentation for those products.

### Software Updates

After installing each software product, you must also install any required software updates. For proper cluster operation, ensure that all cluster nodes maintain the same update level.

For general guidelines and procedures for applying software updates, see Chapter 11, Updating Your Software, in Oracle Solaris Cluster System Administration Guide.

### Public-Network IP Addresses

For information about the use of public networks by the cluster, see Public Network Adapters and IP Network Multipathing in Oracle Solaris Cluster Concepts Guide.

You must set up a number of public-network IP addresses for various Oracle Solaris Cluster components. The number of addresses that you need depends on which components you include in your cluster configuration. Each Oracle Solaris host in the cluster configuration must have at least one public-network connection to the same set of public subnets.

The following table lists the components that need public-network IP addresses assigned. Add these IP addresses to the following locations:

- Any naming services that are used

- The local `/etc/inet/hosts` file on each global-cluster node, after you install Oracle Solaris software

- The local `/etc/inet/hosts` file on any exclusive-IP non-global zone

Table 1-2 Oracle Solaris Cluster Components That Use Public-Network IP Addresses

Component | Number of IP Addresses Needed
--- | ---
Administrative console | 1 IP address per subnet
Global-cluster nodes | 1 IP address per node, per subnet
Zone-cluster nodes | 1 IP address per node, per subnet
Domain console network interface | 1 IP address per domain
(Optional) Non-global zones | 1 IP address per subnet
Console-access device | 1 IP address
Logical addresses | 1 IP address per logical host resource, per subnet

For more information about planning IP addresses, see Chapter 1, Planning the Network Deployment, in Configuring and Administering Oracle Solaris 11.1 Networks.

### Console-Access Devices

You must have console access to all cluster nodes. A service processor (SP) is used to communicate between the administrative console and the global-cluster node consoles.

For more information about console access, see the Oracle Solaris Cluster Concepts Guide.

You can use the Oracle Solaris `pconsole` utility to connect to the cluster nodes. The utility also provides a master console window from which you can propagate your input to all connections that you opened. For more information, see the `pconsole`(1) man page that is available when you install the Oracle Solaris 11 `terminal/pconsole` package.

### Public Network Configuration

Public networks communicate outside the cluster. Consider the following points when you plan your public network configuration:

- Separation of public and private network – Public networks and the private network (cluster interconnect) must use separate adapters, or you must configure tagged VLAN on tagged-VLAN capable adapters and VLAN-capable switches to use the same adapter for both the private interconnect and the public network.

  Alternatively, create virtual NICs on the same physical interface and assign different virtual NICs to the private and public networks.

- Minimum – All cluster nodes must be connected to at least one public network. Public-network connections can use different subnets for different nodes.

- Maximum – You can have as many additional public-network connections as your hardware configuration allows.

- Scalable services – All nodes that run a scalable service must either use the same subnet or set of subnets or use different subnets that are routable among themselves.

- Logical addresses – Each data-service resource group that uses a logical address must have a hostname specified for each public network from which the logical address can be accessed. For additional information about data services and resources, also see the Oracle Solaris Cluster Concepts Guide.

- IPv4 – Oracle Solaris Cluster software supports IPv4 addresses on the public network.

- IPv6 – Oracle Solaris Cluster software supports IPv6 addresses on the public network for both failover and scalable data services.

- IPMP groups – Each public-network adapter that is used for data-service traffic must belong to an IP network multipathing (IPMP) group. If a public-network adapter is not used for data-service traffic, you do not have to configure it in an IPMP group.

  Unless there are one or more non-link-local IPv6 public network interfaces in the public network configuration, the `scinstall` utility automatically configures a multiple-adapter IPMP group for each set of public-network adapters in the cluster that uses the same subnet. These groups are link-based with transitive probes.

  If the configuration contains any non-link-local IPv6 public network interfaces, you must manually configure in IPMP groups all interfaces that will be used for data-service traffic. You can configure the IPMP groups either before or after the cluster is established.

  The `scinstall` utility ignores adapters that are already configured in an IPMP group. You can use probe-based IPMP groups or link-based IPMP groups in a cluster. Probe-based IPMP groups, which test the target IP address, provide the most protection by recognizing more conditions that might compromise availability.

  If any adapter in an IPMP group that the `scinstall` utility configures will not be used for data-service traffic, you can remove that adapter from the group.

  For guidelines on IPMP groups, see Chapter 5, Introduction to IPMP, in Managing Oracle Solaris 11.1 Network Performance. To modify IPMP groups after cluster installation, follow the guidelines in How to Administer IP Network Multipathing Groups in a Cluster in Oracle Solaris Cluster System Administration Guide and procedures in Chapter 6, Administering IPMP (Tasks), in Managing Oracle Solaris 11.1 Network Performance.

- Local MAC address support – All public-network adapters must use network interface cards (NICs) that support local MAC address assignment. Local MAC address assignment is a requirement of IPMP.

- `local-mac-address` setting – The `local-mac-address?` variable must use the default value `true` for Ethernet adapters. Oracle Solaris Cluster software does not support a `local-mac-address?` value of `false` for Ethernet adapters.

For more information about public-network interfaces, see Oracle Solaris Cluster Concepts Guide.

### Quorum Server Configuration

You can use Oracle Solaris Cluster Quorum Server software to configure a machine as a quorum server and then configure the quorum server as your cluster's quorum device. You can use a quorum server instead of or in addition to shared disks and NAS filers.

Consider the following points when you plan the use of a quorum server in an Oracle Solaris Cluster configuration.

- Network connection – The quorum-server computer connects to your cluster through the public network.

- Supported hardware – The supported hardware platforms for a quorum server are the same as for a global-cluster node.

- Operating system – Oracle Solaris software requirements for Oracle Solaris Cluster software apply as well to Quorum Server software.

- Restriction for non-global zones – In the Oracle Solaris Cluster 4.1 release, a quorum server cannot be installed and configured in a non-global zone.

- Service to multiple clusters – You can configure a quorum server as a quorum device to more than one cluster.

- Mixed hardware and software – You do not have to configure a quorum server on the same hardware and software platform as the cluster or clusters for which it provides quorum. For example, a SPARC based machine that runs the Oracle Solaris 10 OS can be configured as a quorum server for an x86 based cluster that runs the Oracle Solaris 11 OS.

- Spanning tree algorithm – You must disable the spanning tree algorithm on the Ethernet switches for the ports that are connected to the cluster public network where the quorum server will run.

- Using a cluster node as a quorum server – You can configure a quorum server on a cluster node to provide quorum for clusters other than the cluster that the node belongs to. However, a quorum server that is configured on a cluster node is not highly available.

### NFS Guidelines

Consider the following points when you plan the use of Network File System (NFS) in an Oracle Solaris Cluster configuration:

- NFS client – No Oracle Solaris Cluster node can be an NFS client of an HA for NFS exported file system that is being mastered on a node in the same cluster. Such cross-mounting of HA for NFS is prohibited. Use the cluster file system to share files among global-cluster nodes.

- NFSv3 protocol – If you are mounting file systems on the cluster nodes from external NFS servers, such as NAS filers, and you are using the NFSv3 protocol, you cannot run NFS client mounts and the HA for NFS data service on the same cluster node. If you do, certain HA for NFS data-service activities might cause the NFS daemons to stop and restart, interrupting NFS services. However, you can safely run the HA for NFS data service if you use the NFSv4 protocol to mount external NFS file systems on the cluster nodes.

- Locking – Applications that run locally on the cluster must not lock files on a file system that is exported through NFS. Otherwise, local blocking (for example, `flock` or `fcntl`) might interfere with the ability to restart the lock manager ( `lockd`). During restart, a blocked local process might be granted a lock which might be intended to be reclaimed by a remote client. This situation would cause unpredictable behavior.

- NFS security features – Oracle Solaris Cluster software does not support the following options of the `share_nfs`(1M) command:

  - `secure`

  - `sec=dh`

  However, Oracle Solaris Cluster software does support the following security feature for NFS:

  - The use of secure ports for NFS. You enable secure ports for NFS by adding the entry set `nfssrv:nfs_portmon=1` to the `/etc/system` file on cluster nodes.

- Fencing – Zone clusters support fencing for all supported NAS devices, shared disks, and storage arrays.

### Service Restrictions

Observe the following service restrictions for Oracle Solaris Cluster configurations:

- Routers – Do not configure cluster nodes as routers (gateways) due to the following reasons:

  - Routing protocols might inadvertently broadcast the cluster interconnect as a publicly reachable network to other routers, despite the setting of the `IFF_PRIVATE` flag on the interconnect interfaces.

  - Routing protocols might interfere with the failover of IP addresses across cluster nodes that impact client accessibility.

  - Routing protocols might compromise proper functionality of scalable services by accepting client network packets and dropping them, instead of forwarding the packets to other cluster nodes.

- NIS+ servers – Do not configure cluster nodes as NIS or NIS+ servers. There is no data service available for NIS or NIS+. However, cluster nodes can be NIS or NIS+ clients.

- Install servers – Do not use an Oracle Solaris Cluster configuration to provide a highly available installation service on client systems.

- RARP – Do not use an Oracle Solaris Cluster configuration to provide an `rarpd` service.

- Remote procedure call (RPC) program numbers – If you install an RPC service on the cluster, the service must not use any of the following program numbers:

  - 100141

  - 100142

  - 100248

  These numbers are reserved for the Oracle Solaris Cluster daemons `rgmd_receptionist`, `fed`, and `pmfd`, respectively.

  If the RPC service that you install also uses one of these program numbers, you must change that RPC service to use a different program number.

- Scheduling classes – Oracle Solaris Cluster software does not support the running of high-priority process scheduling classes on cluster nodes. Do not run either of the following types of processes on cluster nodes:

  - Processes that run in the time-sharing scheduling class with a high priority

  - Processes that run in the real-time scheduling class

  Oracle Solaris Cluster software relies on kernel threads that do not run in the real-time scheduling class. Other time-sharing processes that run at higher-than-normal priority or real-time processes can prevent the Oracle Solaris Cluster kernel threads from acquiring needed CPU cycles.

### Network Time Protocol (NTP)

Observe the following guidelines for NTP:

- Synchronization – The primary requirement when you configure NTP, or any time synchronization facility within the cluster, is that all cluster nodes must be synchronized to the same time.

- Accuracy – Consider accuracy of time on individual nodes to be of secondary importance to the synchronization of time among nodes. You are free to configure NTP as best meets your individual needs if this basic requirement for synchronization is met.

See the Oracle Solaris Cluster Concepts Guide for further information about cluster time. For more information about NTP, see the `ntpd`(1M) man page that is delivered in the Oracle Solaris 11 `service/network/ntp` package.

### Oracle Solaris Cluster Configurable Components

This section provides guidelines for the following Oracle Solaris Cluster components that you configure:

- Global-Cluster Name

- Global-Cluster Node Names and Node IDs

- Private Network Configuration

- Private Hostnames

- Cluster Interconnect

- Global Fencing

- Quorum Devices

#### Global-Cluster Name

Specify a name for the global cluster during Oracle Solaris Cluster configuration. The global cluster name should be unique throughout the enterprise.

For information about naming a zone cluster, see Zone Clusters.

#### Global-Cluster Node Names and Node IDs

The name of a node in a global cluster is the same name that you assign to the physical or virtual host when you install it with the Oracle Solaris OS. See the `hosts`(4) man page for information about naming requirements.

In single-host cluster installations, the default cluster name is the name of the node.

During Oracle Solaris Cluster configuration, you specify the names of all nodes that you are installing in the global cluster. Node names must be unique throughout the cluster.

A node ID number is assigned to each cluster node for intracluster use, beginning with the number 1. Node ID numbers are assigned to each cluster node in the order that the node becomes a cluster member. If you configure all cluster nodes in one operation, the node from which you run the `scinstall` utility is the last node assigned a node ID number. You cannot change a node ID number after it is assigned to a cluster node.

A node that becomes a cluster member is assigned the lowest available node ID number. If a node is removed from the cluster, its node ID becomes available for assignment to a new node. For example, if in a four-node cluster the node that is assigned node ID `3` is removed and a new node is added, the new node is assigned node ID `3`, not node ID `5`.

If you want the assigned node ID numbers to correspond to certain cluster nodes, configure the cluster nodes one node at a time in the order that you want the node ID numbers to be assigned. For example, to have the cluster software assign node ID `1` to `phys-schost-1`, configure that node as the sponsoring node of the cluster. If you next add `phys-schost-2` to the cluster established by `phys-schost-1`, `phys-schost-2` is assigned node ID `2`.

For information about node names in a zone cluster, see Zone Clusters.

#### Private Network Configuration

Note - You do not need to configure a private network for a single-host global cluster. The `scinstall` utility automatically assigns the default private-network address and netmask even though a private network is not used by the cluster.

Oracle Solaris Cluster software uses the private network for internal communication among nodes and among non-global zones that are managed by Oracle Solaris Cluster software. An Oracle Solaris Cluster configuration requires at least two connections to the cluster interconnect on the private network. When you configure Oracle Solaris Cluster software on the first node of the cluster, you specify the private-network address and netmask in one of the following ways:

- Accept the default private-network address (`172.16.0.0`) and default netmask (`255.255.240.0`). This IP address range supports a combined maximum of 64 nodes and non-global zones, a maximum of 12 zone clusters, and a maximum of 10 private networks.

  Note - The maximum number of nodes that an IP address range can support does not reflect the maximum number of nodes that the hardware or software configuration can currently support.

- Specify a different allowable private-network address and accept the default netmask.

- Accept the default private-network address and specify a different netmask.

- Specify both a different private-network address and a different netmask.

If you choose to specify a different netmask, the `scinstall` utility prompts you for the number of nodes and the number of private networks that you want the IP address range to support. The utility also prompts you for the number of zone clusters that you want to support. The number of global-cluster nodes that you specify should also include the expected number of unclustered non-global zones that will use the private network.

The utility calculates the netmask for the minimum IP address range that will support the number of nodes, zone clusters, and private networks that you specified. The calculated netmask might support more than the supplied number of nodes, including non-global zones, zone clusters, and private networks. The `scinstall` utility also calculates a second netmask that would be the minimum to support twice the number of nodes, zone clusters, and private networks. This second netmask would enable the cluster to accommodate future growth without the need to reconfigure the IP address range.

The utility then asks you what netmask to choose. You can specify either of the calculated netmasks or provide a different one. The netmask that you specify must minimally support the number of nodes and private networks that you specified to the utility.

Note - Changing the cluster private IP address range might be necessary to support the addition of nodes, non-global zones, zone clusters, or private networks.

To change the private-network address and netmask after the cluster is established, see How to Change the Private Network Address or Address Range of an Existing Cluster in Oracle Solaris Cluster System Administration Guide. You must bring down the cluster to make these changes.

However, the cluster can remain in cluster mode if you use the `cluster set-netprops` command to change only the netmask. For any zone cluster that is already configured in the cluster, the private IP subnets and the corresponding private IP addresses that are allocated for that zone cluster will also be updated.

If you specify a private-network address other than the default, the address must meet the following requirements:

- Address and netmask sizes – The private network address cannot be smaller than the netmask. For example, you can use a private network address of `172.16.10.0` with a netmask of `255.255.255.0`. However, you cannot use a private network address of `172.16.10.0` with a netmask of `255.255.0.0`.

- Acceptable addresses – The address must be included in the block of addresses that RFC 1918 reserves for use in private networks. You can contact the InterNIC to obtain copies of RFCs or view RFCs online at http://www.rfcs.org.

- Use in multiple clusters – You can use the same private-network address in more than one cluster provided that the clusters are on different private networks. Private IP network addresses are not accessible from outside the physical cluster.

- Oracle VM Server for SPARC - When guest domains are created on the same physical machine and are connected to the same virtual switch, the private network is shared by such guest domains and is visible to all these domains. Proceed with caution before you specify a private-network IP address range to the `scinstall` utility for use by a cluster of guest domains. Ensure that the address range is not already in use by another guest domain that exists on the same physical machine and shares its virtual switch.

- VLANs shared by multiple clusters – Oracle Solaris Cluster configurations support the sharing of the same private-interconnect VLAN among multiple clusters. You do not have to configure a separate VLAN for each cluster. However, for the highest level of fault isolation and interconnect resilience, limit the use of a VLAN to a single cluster.

- IPv6 – Oracle Solaris Cluster software does not support IPv6 addresses for the private interconnect. The system does configure IPv6 addresses on the private-network adapters to support scalable services that use IPv6 addresses. However, internode communication on the private network does not use these IPv6 addresses.

See Chapter 1, Planning the Network Deployment, in Configuring and Administering Oracle Solaris 11.1 Networks for more information about private networks.

#### Private Hostnames

The private hostname is the name that is used for internode communication over the private-network interface. Private hostnames are automatically created during Oracle Solaris Cluster configuration of a global cluster or a zone cluster. These private hostnames follow the naming convention `clusternode` node-id `-priv`, where node-id is the numeral of the internal node ID. During Oracle Solaris Cluster configuration, the node ID number is automatically assigned to each node when the node becomes a cluster member. A node of the global cluster and a node of a zone cluster can both have the same private hostname, but each hostname resolves to a different private-network IP address.

After a global cluster is configured, you can rename its private hostnames by using the `clsetup`(1CL) utility. Currently, you cannot rename the private hostname of a zone-cluster node.

The creation of a private hostname for a non-global zone is optional. There is no required naming convention for the private hostname of a non-global zone.

#### Cluster Interconnect

The cluster interconnects provide the hardware pathways for private-network communication between cluster nodes. Each interconnect consists of a cable that is connected in one of the following ways:

- Between two transport adapters

- Between a transport adapter and a transport switch

For more information about the purpose and function of the cluster interconnect, see Cluster Interconnect in Oracle Solaris Cluster Concepts Guide.

Note - You do not need to configure a cluster interconnect for a single-host cluster. However, if you anticipate eventually adding more nodes to a single-host cluster configuration, you might want to configure the cluster interconnect for future use.

During Oracle Solaris Cluster configuration, you specify configuration information for one or two cluster interconnects.

- If the number of available adapter ports is limited, you can use tagged VLANs to share the same adapter with both the private and public network. For more information, see the guidelines for tagged VLAN adapters in Transport Adapters.

- You can set up from one to six cluster interconnects in a cluster. While a single cluster interconnect reduces the number of adapter ports that are used for the private interconnect, it provides no redundancy and less availability. If a single interconnect fails, the cluster is at a higher risk of having to perform automatic recovery. Whenever possible, install two or more cluster interconnects to provide redundancy and scalability, and therefore higher availability, by avoiding a single point of failure.

You can configure additional cluster interconnects, up to six interconnects total, after the cluster is established by using the `clsetup` utility.

For guidelines about cluster interconnect hardware, see Interconnect Requirements and Restrictions in Oracle Solaris Cluster 4.1 Hardware Administration Manual. For general information about the cluster interconnect, see Cluster Interconnect in Oracle Solaris Cluster Concepts Guide.

##### Transport Adapters

For the transport adapters, such as ports on network interfaces, specify the transport adapter names and transport type. If your configuration is a two-host cluster, you also specify whether your interconnect is a point-to-point connection (adapter to adapter) or uses a transport switch.

Consider the following guidelines and restrictions:

- IPv6 – Oracle Solaris Cluster software does not support IPv6 communications over the private interconnects.

- Local MAC address assignment – All private network adapters must use network interface cards (NICs) that support local MAC address assignment. Link-local IPv6 addresses, which are required on private-network adapters to support IPv6 public-network addresses for scalable data services, are derived from the local MAC addresses.

- Tagged VLAN adapters – Oracle Solaris Cluster software supports tagged Virtual Local Area Networks (VLANs) to share an adapter between the private cluster interconnect and the public network. You must use the `dladm create-vlan` command to configure the adapter as a tagged VLAN adapter before you configure it with the cluster.

  To configure a tagged VLAN adapter for the cluster interconnect, specify the adapter by its VLAN virtual device name. This name is composed of the adapter name plus the VLAN instance number. The VLAN instance number is derived from the formula (1000*V)+N, where V is the VID number and N is the PPA.

  As an example, for VID 73 on adapter `net2`, the VLAN instance number would be calculated as (1000*73)+2. You would therefore specify the adapter name as `net73002` to indicate that it is part of a shared virtual LAN.

  For information about configuring VLAN in a cluster, see Configuring VLANs as Private Interconnect Networks in Oracle Solaris Cluster 4.1 Hardware Administration Manual. For information about creating and administering VLANs, see the `dladm`(1M) man page and Chapter 3, Working With VLANs, in Managing Oracle Solaris 11.1 Network Performance.

- SPARC: Oracle VM Server for SPARC guest domains – Specify adapter names by their virtual names, `vnet` N, such as `vnet0` and `vnet1`. Virtual adapter names are recorded in the `/etc/path_to_inst` file.

- Logical network interfaces – Logical network interfaces are reserved for use by Oracle Solaris Cluster software.

##### Transport Switches

If you use transport switches, such as a network switch, specify a transport switch name for each interconnect. You can use the default name `switch` N, where N is a number that is automatically assigned during configuration, or create another name.

Also specify the switch port name or accept the default name. The default port name is the same as the internal node ID number of the Oracle Solaris host that hosts the adapter end of the cable. However, you cannot use the default port name for certain adapter types.

Clusters with three or more nodes must use transport switches. Direct connection between cluster nodes is supported only for two-host clusters. If your two-host cluster is direct connected, you can still specify a transport switch for the interconnect.

Tip - If you specify a transport switch, you can more easily add another node to the cluster in the future.

#### Global Fencing

Fencing is a mechanism that is used by the cluster to protect the data integrity of a shared disk during split-brain situations. By default, the `scinstall` utility in Typical Mode leaves global fencing enabled, and each shared disk in the configuration uses the default global fencing setting of `prefer3`. With the `prefer3` setting, the SCSI-3 protocol is used.

If any device is unable to use the SCSI-3 protocol, the `pathcount` setting should be used instead, where the fencing protocol for the shared disk is chosen based on the number of DID paths that are attached to the disk. Non-SCSI-3 capable devices are limited to two DID device paths within the cluster. Fencing can be turned off for devices which do not support either SCSI-3 or SCSI-2 fencing. However, data integrity for such devices cannot be guaranteed during split-brain situations.

In Custom Mode, the `scinstall` utility prompts you whether to disable global fencing. For most situations, respond `No` to keep global fencing enabled. However, you can disable global fencing in certain situations.

| Caution - If you disable fencing under other situations than the ones described, your data might be vulnerable to corruption during application failover. Examine this data corruption possibility carefully when you consider turning off fencing.

The situations in which you can disable global fencing are as follows:

- The shared storage does not support SCSI reservations.

  If you turn off fencing for a shared disk that you then configure as a quorum device, the device uses the software quorum protocol. This is true regardless of whether the disk supports SCSI-2 or SCSI-3 protocols. Software quorum is a protocol in Oracle Solaris Cluster software that emulates a form of SCSI Persistent Group Reservations (PGR).

- You want to enable systems that are outside the cluster to gain access to storage that is attached to the cluster.

If you disable global fencing during cluster configuration, fencing is turned off for all shared disks in the cluster. After the cluster is configured, you can change the global fencing protocol or override the fencing protocol of individual shared disks. However, to change the fencing protocol of a quorum device, you must first unconfigure the quorum device. Then set the new fencing protocol of the disk and reconfigure it as a quorum device.

For more information about fencing behavior, see Failfast Mechanism in Oracle Solaris Cluster Concepts Guide. For more information about setting the fencing protocol of individual shared disks, see the `cldevice`(1CL) man page. For more information about the global fencing setting, see the `cluster`(1CL) man page.

#### Quorum Devices

Oracle Solaris Cluster configurations use quorum devices to maintain data and resource integrity. If the cluster temporarily loses connection to a node, the quorum device prevents amnesia or split-brain problems when the cluster node attempts to rejoin the cluster. For more information about the purpose and function of quorum devices, see Quorum and Quorum Devices in Oracle Solaris Cluster Concepts Guide.

During Oracle Solaris Cluster installation of a two-host cluster, you can choose to have the `scinstall` utility automatically configure an available shared disk in the configuration as a quorum device. The `scinstall` utility assumes that all available shared disks are supported as quorum devices.

If you want to use a quorum server or a Sun ZFS Storage Appliance NAS device from Oracle as the quorum device, you configure it after `scinstall` processing is completed.

After installation, you can also configure additional quorum devices by using the `clsetup` utility.

Note - You do not need to configure quorum devices for a single-host cluster.

If your cluster configuration includes third-party shared storage devices that are not supported for use as quorum devices, you must use the `clsetup` utility to configure quorum manually.

Consider the following points when you plan quorum devices:

- Minimum – A two-host cluster must have at least one quorum device, which can be a shared disk, a quorum server, or a NAS device. For other topologies, quorum devices are optional.

- Odd-number rule – If more than one quorum device is configured in a two-host cluster or in a pair of hosts directly connected to the quorum device, configure an odd number of quorum devices. This configuration ensures that the quorum devices have completely independent failure pathways.

- Distribution of quorum votes – For highest availability of the cluster, ensure that the total number of votes that are contributed by quorum devices is less than the total number of votes that are contributed by nodes. Otherwise, the nodes cannot form a cluster if all quorum devices are unavailable even if all nodes are functioning.

- Connection – You must connect a quorum device to at least two nodes.

- SCSI fencing protocol – When a SCSI shared-disk quorum device is configured, its fencing protocol is automatically set to SCSI-2 in a two-host cluster or SCSI-3 in a cluster with three or more nodes.

- Changing the fencing protocol of quorum devices – For SCSI disks that are configured as a quorum device, you must unconfigure the quorum device before you can enable or disable its SCSI fencing protocol.

- Software quorum protocol – You can configure supported shared disks that do not support SCSI protocol, such as SATA disks, as quorum devices. You must disable fencing for such disks. The disks would then use the software quorum protocol, which emulates SCSI PGR.

  The software quorum protocol would also be used by SCSI-shared disks if fencing is disabled for such disks.

- Replicated devices – Oracle Solaris Cluster software does not support replicated devices as quorum devices.

- ZFS storage pools – Do not add a configured quorum device to a ZFS storage pool. When a configured quorum device is added to a ZFS storage pool, the disk is relabeled as an EFI disk and quorum configuration information is lost. The disk can then no longer provide a quorum vote to the cluster.

  After a disk is in a storage pool, you can configure that disk as a quorum device. Or, you can unconfigure the quorum device, add it to the storage pool, then reconfigure the disk as a quorum device.

For more information about quorum devices, see Quorum and Quorum Devices in Oracle Solaris Cluster Concepts Guide.

### Zone Clusters

A zone cluster is a cluster of Oracle Solaris non-global zones. You can use the `clsetup` utility to create a zone cluster and add a network address, file system, ZFS storage pool, or storage device. You can also use a command line interface (the `clzonecluster` utility) to create a zone cluster, make configuration changes, and remove a zone cluster. For more information about using the `clzonecluster` utility, see the `clzonecluster`(1CL) man page.

Supported brands for zone clusters are `solaris`, `solaris10`, and `labeled`. The `labeled` brand is used exclusively in a Trusted Extensions environment. To use the Trusted Extensions feature of Oracle Solaris, you must configure the Trusted Extensions feature for use in a zone cluster. No other use of Trusted Extensions is supported in an Oracle Solaris Cluster configuration.

You can also specify a shared-IP zone cluster or an exclusive-IP zone cluster when you run the `clsetup` utility.

- Shared-IP zone clusters work with `solaris` or `solaris10` brand zones. A shared-IP zone cluster shares a single IP stack between all the zones on the node, and each zone is allocated an IP address.

- Exclusive-IP zone clusters work only with `solaris` brand zones, and do not work with `solaris10` brand zones. An exclusive-IP zone cluster supports a separate IP instance stack.

Consider the following points when you plan the creation of a zone cluster:

- Global-Cluster Requirements and Guidelines

- Zone-Cluster Requirements and Guidelines

- Guidelines for Trusted Extensions in a Zone Cluster

#### Global-Cluster Requirements and Guidelines

- Global cluster – The zone cluster must be configured on a global Oracle Solaris Cluster configuration. A zone cluster cannot be configured without an underlying global cluster.

- Cluster mode – The global-cluster node from which you create or modify a zone cluster must be in cluster mode. If any other nodes are in noncluster mode when you administer a zone cluster, the changes that you make are propagated to those nodes when they return to cluster mode.

- Adequate private-IP addresses – The private IP-address range of the global cluster must have enough free IP-address subnets for use by the new zone cluster. If the number of available subnets is insufficient, the creation of the zone cluster fails.

- Changes to the private IP-address range – The private IP subnets and the corresponding private IP-addresses that are available for zone clusters are automatically updated if the global cluster's private IP-address range is changed. If a zone cluster is deleted, the cluster infrastructure frees the private IP-addresses that were used by that zone cluster, making the addresses available for other use within the global cluster and by any other zone clusters that depend on the global cluster.

- Supported devices – Devices that are supported with Oracle Solaris zones can be exported to a zone cluster. Such devices include the following:

  - Oracle Solaris disk devices (`c` N `t` X `d` Y `s` Z)

  - DID devices (`/dev/did/*dsk/d` N)

  - Solaris Volume Manager and Solaris Volume Manager for Sun Cluster multi-owner disk sets (`/dev/md/`setname`/*dsk/d` N)

#### Zone-Cluster Requirements and Guidelines

- Distribution of nodes – You cannot host multiple nodes of the same zone cluster on the same host machine. A host can support multiple zone-cluster nodes as long as each zone-cluster node on that host is a member of a different zone cluster.

- Node creation – You must create at least one zone-cluster node at the time that you create the zone cluster. You can use the `clsetup` utility or the `clzonecluster` command to create the zone cluster. The name of the zone-cluster node must be unique within the zone cluster. The infrastructure automatically creates an underlying non-global zone on each host that supports the zone cluster. Each non-global zone is given the same zone name, which is derived from, and identical to, the name that you assign to the zone cluster when you create the cluster. For example, if you create a zone cluster that is named `zc1`, the corresponding non-global zone name on each host that supports the zone cluster is also `zc1`.

- Cluster name – Each zone-cluster name must be unique throughout the cluster of machines that host the global cluster. The zone-cluster name cannot also be used by a non-global zone elsewhere in the cluster of machines, nor can the zone-cluster name be the same as that of a global-cluster node. You cannot use “all” or “global” as a zone-cluster name, because these are reserved names.

- Public-network IP addresses – You can optionally assign a specific public-network IP address to each zone-cluster node.

  Note - If you do not configure an IP address for each zone cluster node, two things will occur:

  - That specific zone cluster will not be able to configure NAS devices for use in the zone cluster. The cluster uses the IP address of the zone cluster node when communicating with the NAS device, so not having an IP address prevents cluster support for fencing NAS devices.

  - The cluster software will activate any Logical Host IP address on any NIC.

- Private hostnames – During creation of the zone cluster, a private hostname is automatically created for each node of the zone cluster, in the same way that hostnames are created in global clusters. Currently, you cannot rename the private hostname of a zone-cluster node. For more information about private hostnames, see Private Hostnames.

- Oracle Solaris Zones brands – All nodes of a zone cluster are configured as non-global zones of the `solaris`, `solaris10` , or `labeled` brand that is set with the `cluster` attribute. No other brand types are permitted in a zone cluster.

  For Trusted Extensions, you must use only the `labeled` brand.

- IP type - You can create a zone cluster that is either the `shared` IP type or the `exclusive` IP type. If the IP type is not specified, a shared-IP zone cluster is created by default.

- `Global_zone=TRUE` resource-type property – To register a resource type that uses the `Global_zone=TRUE` resource-type property, the resource-type file must reside in the `/usr/cluster/global/rgm/rtreg/` directory of the zone cluster. If that resource-type file resides in any other location, the command to register the resource type is rejected.

- Conversion to a zone-cluster node – You cannot add to a zone cluster a non-global zone that resides outside that zone cluster. You must use only the `clzonecluster` command to add new nodes to a zone cluster.

- File systems – You can use the `clsetup` utility or the `clzonecluster` command to add the following types of file systems for use by the zone cluster. A file system is exported to a zone cluster by using either a direct mount or a loopback mount. Adding a file system with the `clsetup` utility is done in cluster scope, which affects the entire zone cluster.

  - By direct mount:

    - UFS local file system

    - Oracle Solaris ZFS (exported as a data set)

    - NFS from supported NAS devices

  - By loopback mount:

    - UFS local file system

    - UFS cluster file system

  You configure an HAStoragePlus or `ScalMountPoint` resource to manage the mounting of the file system.

- Fencing – Zone clusters support fencing for all supported NAS devices, shared disks, and storage arrays.

#### Guidelines for Trusted Extensions in a Zone Cluster

Consider the following points when you use the Trusted Extensions feature of Oracle Solaris in a zone cluster:

- Only zone-cluster support – In an Oracle Solaris Cluster configuration with Trusted Extensions enabled, applications must run only in a zone cluster. No other non-global zones can be used on the cluster. You must use only the `clzonecluster` command to create a zone cluster. Do not use the `txzonemgr` command to create a non-global zone on a cluster that has Trusted Extensions enabled.

- Trusted Extensions scope – You can either enable or disable Trusted Extensions for the entire cluster configuration. When Trusted Extensions is enabled, all non-global zones in the cluster configuration must belong to one of the zone clusters. You cannot configure any other kind of non-global zone without compromising security.

- IP addresses – Each zone cluster that uses Trusted Extensions must use its own IP addresses. The special networking feature in Trusted Extensions that enables an IP address to be shared between multiple non-global zones is not supported with Oracle Solaris Cluster software.

- Loopback mounts – You cannot use loopback mounts that have write permissions in a zone cluster that uses Trusted Extensions. Use only direct mounts of file systems that permit write access, or use loopback mounts that have only read permissions.

- File systems – Do not configure in the zone cluster the global device that underlies a file system. Configure only the file system itself in the zone cluster.

- Storage device name – Do not add an individual slice of a storage device to a zone cluster. You must add the entire device to a single zone cluster. The use of slices of the same storage device in different zone clusters compromises the security of those zone clusters.

- Application installation – Install applications only in the zone cluster or in the global cluster and then export to the zone cluster by using read-only loopback mounts.

- Zone cluster isolation – When Trusted Extensions is used, the name of a zone cluster is a security label. In some cases, the security label itself might be information that cannot be disclosed, and the name of a resource or resource group might be a sensitive piece of information that cannot be disclosed. When an inter-cluster resource dependency or inter-cluster resource-group affinity is configured, the name of the other cluster becomes visible as well as the name of any affected resource or resource group. Therefore, before you establish any inter-cluster relationships, evaluate whether this information can be made visible according to the your requirements.

Copyright © 2000, 2012, Oracle and/or its affiliates. All rights reserved. Legal Notices |