Skip Headers

Oracle® Application Server Containers for J2EE User's Guide 10g Release 2 (10.1.2) Part No. B14011-02 |
| Library | Product | Contents | Index

Previous | Next |

# 4 OC4J Clustering

This chapter discusses OC4J clustering, and provides instructions on how to manage OC4J processes and applications within a cluster.

It contains the following topics:

- The OC4J Instance in a Cluster

- Instance-Specific Parameters

- OC4J Clustering Examples

- OC4J Cluster Configuration

The full picture of clustering within Oracle Application Server is discussed in the Oracle Application Server High Availability Guide.

## 4.1 The OC4J Instance in a Cluster

The OC4J instance is the entity to which J2EE applications are deployed and configured. It defines how many OC4J processes exist within the application server and the configuration for these OC4J processes. The OC4J process is what executes the J2EE applications for the OC4J instance.

The OC4J instance has the following features:

- The configuration of the OC4J instance is valid for one or more OC4J executable processes. This way, you can duplicate the configuration for multiple OC4J processes by managing these processes in the OC4J instance construct. When you modify the cluster-wide configuration within the OC4J instance, the modifications are valid for all OC4J processes.

- Each OC4J instance can be configured with one or more OC4J processes.

- When you deploy an application to an OC4J instance, the OC4J instance deploys the application to all OC4J processes defined in the OC4J instance. The OC4J instance is also responsible for replicating the state of its applications.

- The number of OC4J processes is specific to each OC4J instance. This must be manually configured for each application server instance in the cluster. The OC4J process configuration provides flexibility to tune according to the specific hardware capabilities of the host. By default, each OC4J instance is instantiated with a single OC4J process.

Within the application sever instance, you can configure multiple OC4J instances, each with its own number of OC4J processes. The advantage for this is for configuration management and application deployment for separate OC4J processes in your cluster.

Figure 4-1 demonstrates the `home` default OC4J instance. In the context of a cluster, the OC4J instance configuration is part of the cluster-wide configuration. Thus, the `home` instance, configured on the first application instance, is replicated on all other application server instances.

The number of processes in each `home` instance is an instance-specific parameter, so you must configure the `home` instance separately on each application server instance for the number of OC4J processes that exist on each application server instance. Figure 4-1 shows that the `home` instance on application server instance 1 contains two OC4J processes; the `home` instance on application server instance 2 contains only one OC4J process. Each OC4J instance defaults to having one OC4J process.

Figure 4-1 OC4J Processes in a Cluster

Description of "Figure 4-1 OC4J Processes in a Cluster"

### 4.1.1 The OC4J Process in a Cluster

The OC4J process is the JVM process that executes J2EE applications. Each OC4J process is associated to an OC4J instance and inherits its configuration from that OC4J instance. All applications deployed to an OC4J instance are started in all the OC4J processes associated to that OC4J instance.

You can define one or more OC4J processes within an OC4J instance, so that J2EE requests can be load balanced and have failover capabilities.

The configuration for the number of OC4J processes is instance-specific. Thus, you must configure each OC4J instance in each application server instance with the number of OC4J processes you want to start up for that OC4J instance. The default is one OC4J process.

Each host that you install the application server instances on has different capabilities. To maximize the hardware capabilities, configure the number of OC4J processes in each OC4J instance that will use these capabilities properly. For example, you can configure a single OC4J process on host A and five OC4J processes on host B.

When you define multiple OC4J processes, you enable the following:

- You can serve multiple users with multiple OC4J processes.

- You can provide failover if the state of the application is replicated across multiple OC4J processes.

- OHS provides load balancing for all OC4J processes in the OC4J instance. The OPMN component notifies each OHS when a new OC4J process is initiated. Thus, each OHS in the cluster knows of each OC4J process in the cluster.

  Note: `mod_oc4j`, which sits between OHS and OC4J, is able to provide metric-based load balancing of requests to available OC4J instances. See the discussion of the `<metric-collector>` element in "Elements in the server.xml File" for instructions on implementing this feature.

#### 4.1.1.1 Replicating Application State

The OC4J processes involved in the cluster can replicate application state to all OC4J processes. Once you configure replication, OC4J handles the propagation of the application state for you.

If one OC4J process fails, then another OC4J process—which has had the application state replicated to it—takes over the application request. When an OC4J process fails during a stateful request, the OHS forwards the request in the following order:

1. If another OC4J process is active within the same application server instance, OHS forwards the request to this process.

2. Otherwise, OHS forwards the state request to an OC4J process in another application server instance in the cluster.

There are two types of failure that you want to protect against: software failure and hardware failure.

Failure Type | Avoidance Technique
--- | ---
Software failure occurs when the OC4J process fails. | Multiple OC4J processes in the same OC4J instance. When one OC4J process fails, the OHS forwards the request to another OC4J process in the same OC4J instance.
Hardware failure occurs when the host goes down. | OC4J processes in the cluster configured on separate hosts. When the first host dies, the OC4J process on another host can take over the request. This requires that you have installed an application server instance on another host, which is a part of the cluster, and the OC4J instance has at least one OC4J process.

### 4.1.2 Islands

An island is a logical grouping of OC4J processes that allows you to determine which OC4J processes will replicate state.

In each OC4J instance, you can have more than one OC4J process. If we consider state replication in a situation where all OC4J processes tried to replicate state, then the CPU load can significantly increase. To avoid a performance degradation, the OC4J instance enables you to subgroup your OC4J processes. The subgroup is called an island.

To ensure that the CPU load is partitioned among the processes, the OC4J processes of an OC4J instance can be partitioned into islands. The state for application requests is replicated only to OC4J processes that are grouped within the same island. All applications are still deployed to all OC4J processes in the OC4J instance. The only difference is that the state for these applications is confined to only a subset of these OC4J processes.

The island configuration is instance-specific. The name of the island must be identical in each OC4J instance, where you want the island to exist. When you configure the number of OC4J processes on each application server instance, you can also subgroup them into separate islands. The OC4J processes are grouped across application server instances by the name of the island. Thus, the application state is replicated to all OC4J processes within the island of the same name spanning application server instances.

The grouping of OC4J processes for the state replication is different for EJB applications than for Web applications. Web applications replicate state within the island sub-grouping. EJB applications replicate state between all OC4J processes in the OC4J instance and do not use the island sub-grouping.

Figure 4-2 demonstrates OC4J processes in islands within the cluster. Two islands are configured in the `home` instance: `default_island` and `second_island`. One OC4J process is configured in each island on each application server instance. The OC4J islands, designated within the shaded area, span application server instances.

Figure 4-2 Island Description

Description of "Figure 4-2 Island Description"

### 4.1.3 J2EE Applications Involved in a Cluster

J2EE applications are deployed in all cases to the OC4J instance—whether the application server instance is included in a cluster or not. However, when the application is deployed to an OC4J instance that is in a cluster, certain configuration details must be accomplished:

- Multicast host and port—The state of the applications is replicated from one OC4J process to another over a multicast address. In the case of an EJB application, you must also specify a username and password. You can either accept the defaults for the multicast address or configure it through the Enterprise Manager.

- State replication request—You request state replication for all applications through the Enterprise Manager.

- XML deployment descriptor elements—Both Web and EJB applications require an additional configuration in their respective XML deployment descriptors.

- Island definition—Web applications use the island sub-grouping for its state replication. EJB applications ignore the island sub-grouping and use all OC4J processes for its state replication.

## 4.2 Instance-Specific Parameters

The following parameters are not replicated across the cluster; thus, each must be set at the OC4J instance level on each application server.

- Islands and number of OC4J processes—While you want to keep the names of the islands consistent across the application server instances, the definition of the islands and the number of OC4J processes is configured independently. The host on which you install each application server instance has different capabilities. On each host, you can tune the number of OC4J processes to match the host capabilities. Remember that the state is replicated in islands across application boundaries. So the island names must be the same in each OC4J instance.

- Port numbers—The RMI, JMS, and AJP port numbers can be different for each host.

- Command line options—The command line options you use can be different for each host.

## 4.3 OC4J Clustering Examples

No matter how many application server instances you add within the cluster, the cluster-wide configuration is replicated within the cluster. You control protecting against software and hardware failure with how you configure island and OC4J processes, which are instance-specific parameters.

### 4.3.1 Software Failure

Suppose you configure more than one OC4J process within your OC4J instance, then if one of these processes fails, another process can take over the work load of the failed process. Figure 4-3 shows application server instance 1, which is involved in the cluster. Within this application server instance, there are two OC4J processes defined in the `default_island` in the `home` instance. If the first OC4J process fails, the other can pick up the work load.

Both of these OC4J processes are on the same host; so, if the host goes down, both OC4J processes fail and the client cannot continue processing.

Figure 4-3 Software Failure Demonstration

Description of "Figure 4-3 Software Failure Demonstration"

### 4.3.2 Hardware Failure

To protect against hardware failure, you must configure OC4J processes in the same OC4J instance across hosts. Figure 4-4 shows `home` instance in application server instance 1 and 2. Within the `default_island`, two OC4J processes are configured on application server instance 1 and three are configured in application server instance 2. If a client is interacting with one of the OC4J processes in application server 1, which terminates abnormally, the client is redirected automatically to one of the OC4J processes in the `default_island` in application server 2. Thus, your client is protected against hardware failure.

Figure 4-4 Hardware Failure Demonstration

Description of "Figure 4-4 Hardware Failure Demonstration"

### 4.3.3 State Replication

If the client is a stateful application, then the state is replicated only within the same island. In the previous example, there is only a single island, so the state of the application would be preserved.

To enhance your performance, you want to divide up state replication among islands. However, you must also protect for hardware and software failure within these islands.

The optimal method of protecting against software and hardware failure, while maintaining state with the least number of OC4J processes, is to configure at least one OC4J process on more than one host in the same island. For example, if you have application server instance 1 and 2, within the `home` instance, you configure one OC4J process in the `default_island` on each application server instance. Thus, you are protected against hardware and software failure and your client maintains state if either failure occurs.

- If one of the OC4J processes fails, then the client request is redirected to the other OC4J process in the island. The state is preserved and the client does not notice any irregularity.

- If application server 1 terminates abnormally, then the client is redirected to the OC4J process in the `default_island` on application server 2. The state is preserved and the client does not notice any irregularity.

As demand increases, you will configure more OC4J processes. To guard against a performance slowdown, separate your OC4J processes into separate islands. For example, if fifteen OC4J processes utilize the hardware efficiently on the two hosts and serve the client demand appropriately, then you could divide these processes into at least two islands. The following shows the fifteen OC4J processes grouped into three islands:

Island Names | Application Server 1 | Application Server 2
--- | --- | ---
    default_island
| two | three
    second_island
| two | three
    third_island
| three | two

- The host where application server 1 is installed can handle seven OC4J processes; the host where application server 2 is installed can handle eight OC4J processes.

- Each island contains at least one OC4J process in each island across hosts to protect against software and hardware failure.

- The performance is maximized by dividing up the state replication across three islands.

## 4.4 OC4J Cluster Configuration

The following sections describe how to manage OC4J elements in the cluster: EJBs and Servlets. For directions on how to create and modify Oracle Application Server clusters, see the Oracle Application Server High Availability Guide.

- OC4J Instance Configuration

- Configuring OC4J Instance-Specific Parameters

  Note: As an alternative to using Enterprise Manager, you can create a cluster, add application server instances to the cluster, and manage the cluster using the DCM command-line tool. See the Distributed Configuration Management Administrator's Guide for information on the DCM command-line tool.

### 4.4.1 OC4J Instance Configuration

The following sections describe how to configure your OC4J Instance for clustering:

- Configuring Islands and Processes

- Configuring Web Application State Replication

- Configuring EJB Application State Replication

#### 4.4.1.1 Configuring Islands and Processes

To modify the islands and the number of processes each island contains, do the following:

1. Select the Administration page off the OC4J Home Page.

2. Select Server Properties in the Instance Properties column.

3. Scroll down to the Multiple VM Configuration section. This section defines the islands and the number of OC4J processes that should be started on this application server instance in each island.

4. Create any islands for this OC4J instance within the cluster by clicking Add Another Row. You can supply a name for each island within the Island ID field. You can designate how many OC4J processes should be started within each island by the number configured in the Number of Processes field.

Figure 4-5 displays the Multiple VM Configuration section.

Figure 4-5 Island and Process Configuration

Description of "Figure 4-5 Island and Process Configuration"

#### 4.4.1.2 Configuring Web Application State Replication

Configuring state replication for stateful applications is different for Web applications than for EJB applications. To configure state replication for Web applications, do the following:

1. Select the Administration page off of the OC4J Home Page.

2. Select Replication Properties in the Instance Properties column.

3. Scroll down to the Web Applications section. Figure 4-6 shows this section.

4. Select the Replicate session state checkbox.

5. Optionally, you can provide the multicast host IP address and port number. If you do not provide the host and port for the multicast address, it defaults to host IP address 230.230.0.1 and port number 9127. The host IP address must be between 224.0.0.2 through 239.255.255.255. Do not use the same multicast address for both HTTP and EJB multicast addresses.

   Note: When choosing a multicast address, ensure that it does not collide with the addresses listed in `http://www.iana.org/assignments/multicast-addresses`. Also, if the low order 23 bits of an address is the same as the local network control block (`224.0.0.0 - 224.0.0.255`), then a collision may occur. To avoid this, provide an address that does not have the same bits in the lower 23 bits of the address as the addresses in this range.

6. Add the `<distributable/>` tag to all `web.xml` files in all Web applications. If the Web application is serializable, you must add this tag to the `web.xml` file.

   The following shows an example of this tag added to `web.xml`:

       <web-app>
         <distributable/>
         <servlet>
         ...
         </servlet>
       </web-app>

Figure 4-6 Web State Replication Configuration

Description of "Figure 4-6 Web State Replication Configuration"

#### 4.4.1.3 Configuring EJB Application State Replication

To create an EJB cluster, you specify the OC4J instances that are to be involved in the cluster, configure each of them with the same multicast address, username, and password, and deploy the EJB, which is to be clustered, to each of the nodes in the cluster.

Unlike HTTP clustering, EJBs involved in a cluster cannot be sub-grouped in an island. Instead, all EJBs within the cluster are in one group. Also, only session beans are clustered.

The state of all beans is replicated at the end of every method call to all nodes in the cluster using a multicast topic. Each node included in the EJB cluster is configured to use the same multicast address.

The concepts for understanding how EJB object state is replicated within a cluster are described in the Oracle Application Server Containers for J2EE Enterprise JavaBeans Developer's Guide. To configure EJB replication, you configure the following:

1. Select the Administration page off of the OC4J Home Page.

2. Select Replication Properties in the Instance Properties column.

3. Scroll down to the EJB Applications section. Figure 4-7 shows this section.

4. Select the Replicate session state checkbox.

5. Provide the username and password, which is used to authenticate itself to other hosts in the cluster. If the username and password are different for other hosts in the cluster, they will fail to communicate. You can have multiple username and password combinations within a multicast address. Those with the same username/password combinations will be considered a unique cluster.

6. Optionally, you can provide the multicast host IP address and port number. If you do not provide the host and port for the multicast address, it defaults to host IP address 230.230.0.1 and port number 23791. The host IP address must be between 224.0.0.2 through 239.255.255.255. Do not use the same multicast address for both HTTP and EJB multicast addresses.

   Note: When choosing a multicast address, ensure that it does not collide with the addresses listed in `http://www.iana.org/assignments/multicast-addresses`. Also, if the low order 23 bits of an address is the same as the local network control block (`224.0.0.0 - 224.0.0.255`), then a collision may occur. To avoid this, provide an address that does not have the same bits in the lower 23 bits of the address as the addresses in this range.

7. Provide the host name where the OC4J Instance resides in the RMI Server Host field.

8. Configure the type of EJB replication within the `orion-ejb-jar.xml` file within the JAR file. See "Stateful Session Bean Replication Configuration in the Application JAR" for full details. You can configure these within the `orion-ejb-jar.xml` file before deployment or add this through the Enterprise Manager screens after deployment. If you add this after deployment, drill down to the JAR file from the application page.

Figure 4-7 EJB State Replication Configuration

Description of "Figure 4-7 EJB State Replication Configuration"

##### 4.4.1.3.1 Stateful Session Bean Replication Configuration in the Application JAR

For stateful session beans, you may have you modify the `orion-ejb-jar.xml` file to add the state replication configuration. Since you configure the replication type for the stateful session bean within the bean deployment descriptor, each bean can use a different type of replication.

Stateful session beans require state to be replicated among nodes. In fact, stateful session beans must send all their state between the nodes, which can have a noticeable effect on performance. Thus, the following replication modes are available to you to decide on how to manage the performance cost of replication:

##### 4.4.1.3.2 JVM Termination Replication

The state of the stateful session bean is replicated to only one other node in the cluster (with the same multicast address) when the JVM is terminating. Since this uses JDK 1.3 shutdown hooks, you must use JVM version 1.3 or later. This is the most performant option, because the state is replicated only once. However, it is not very reliable for the following reasons:

- Your state is not replicated if the power is shut off unexpectedly.

- The state of the bean exists only on a single node at any time; the depth of failure is equal to one node.

Set the `replication` attribute of the `<session-deployment>` tag in the `orion-ejb-jar.xml` file to "`VMTermination`". This is shown below:

    <session-deployment replication="VMTermination" .../>

##### 4.4.1.3.3 End of Call Replication

The state of the stateful session bean is replicated to all nodes in the cluster (with the same multicast address) at the end of each EJB method call. If the node loses power, then the state has already been replicated. The JVM termination replication mode does not guarantee state replication in the case of lost power.

Set the `replication` attribute of the `<session-deployment>` tag in the `orion-ejb-jar.xml` file to "`endOfCall`". This is shown below:

    <session-deployment replication="EndOfCall" .../>

### 4.4.2 Configuring OC4J Instance-Specific Parameters

The manageability feature of the cluster causes the configuration to be replicated across all application server instances in the cluster, which is defined as a cluster-wide configuration. However, there are certain parameters where it is necessary to configure them separately on each OC4J instance. These parameters are referred to as OC4J instance-specific.

The following parameters are OC4J instance-specific parameters, which are not replicated across the cluster. You must modify these parameters on each application server instance.

The following are instance-specific parameters within each OC4J instance:

- islands

- number of OC4J processes

- port numbers

- command-line options

All other parameters are part of the cluster-wide parameters, which are replicated across the cluster.

Figure 4-8 shows the section where these parameters are modified. These sections are located in the Server Properties on the Administration page.

Figure 4-8 Non-Replicated Configuration

Description of "Figure 4-8 Non-Replicated Configuration"