6 Configuring High Availability for Oracle ADF and WebCenter Applications

6.1.1.1 Oracle ADF Components

The core module in the framework is Oracle ADF Model, a declarative data binding facility that implements the JSR-227 specification. This specification provides an API for accessing declarative data binding metadata. The Oracle ADF Model layer enables a unified approach to bind any user interface to any business service, without the need to write code.

The other modules that make up a Fusion web application technology stack are:

• Oracle ADF Business Components, which simplifies building business services.

• Oracle ADF Faces, which offers a rich library of AJAX-enabled UI components for Web applications built with JavaServer Faces (JSF).

• Oracle ADF Controller, which integrates JSF with Oracle ADF Model. The ADF Controller extends the standard JSF controller by providing additional functionality, such as reusable task flows that pass control not only between JSF pages, but also between other activities, for instance method calls or other task flows.

Figure 6-2 illustrates where each Oracle ADF module fits in the Fusion web application architecture.

Figure 6-2 Simple Oracle ADF Architecture

Description of "Figure 6-2 Simple Oracle ADF Architecture"

6.1.1.2 Oracle ADF Single Node Architecture

You can install the Oracle ADF runtime to the Oracle WebLogic Server using either the Oracle JDeveloper Installer or the Oracle Fusion Middleware Application Developer Installer. The Application Developer Installer also lets you optionally install Fusion Middleware Control to provide web-based administration support for all Managed Servers in the domain. The Oracle JDeveloper installer does not install Fusion Middleware Control. Both of these options are described in the deployment chapter of the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

When you use the Application Developer Installer to install the Oracle ADF runtime, it results in the creation of an Oracle Application Developer home directory (by default, Oracle_APPDEV1) located under the Middleware home. After the administrator uses the domain configuration wizard to create an Application Developer domain (base_domain) based on the JRF domain template, the administrator can configure the topology of the server. In a typical set up, the domain contains an Administration server containing the WLS Administration Console and Fusion Middleware Control. Typically, the Oracle ADF runtime libraries (part of the Java Required Files) get deployed to the Managed Servers, in addition to the user-facing custom Fusion web applications. To provide customization and personalization features, an optional MDS repository may also be installed, and needs to be configured separately. Figure 6-3 shows a basic single-node Oracle ADF architecture.

Figure 6-3 Basic Single-Node Oracle ADF Architecture

Description of "Figure 6-3 Basic Single-Node Oracle ADF Architecture"

For more information about domains and servers, see the Oracle Fusion Middleware Administrator's Guide.

6.1.1.3 Oracle ADF External Dependencies

If the Fusion web application involves customization using Oracle Metadata Services (MDS), you should register your MDS repository with the Oracle WebLogic Server domain before you deploy the application. For information about registering MDS, see the Oracle Fusion Middleware Administrator's Guide.

Then, when you deploy the application, JDeveloper prompts you to choose the target metadata repository or shared metadata repository. You will be able to choose from the list of metadata repositories registered with the Oracle WebLogic Administration Server.

To ensure that you receive the metadata repository prompt, the application's adf-config.xml file must define a cust-config element in the mds-config section. This element specifies an ordered and named list of customization classes. A customization class is the interface that MDS uses to define which customization applies to the base definition metadata. In JDeveloper, you can use the overview editor for the adf-config.xml file to define a cust-config element.

For information about configuring the adf-config.xml file for MDS, see the chapter on customizing with MDS in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

For more information about the MDS architecture and metadata repositories (database and file-based) and archives (EAR, MAR), refer to the Oracle Fusion Middleware Administrator's Guide.

6.1.1.4 Oracle ADF Log File

The operations performed by the Fusion web application are logged directly to the WebLogic Managed Server where the application is running:

DOMAIN_HOME/servers/server_name/logs/server_name-diagnostic.log

The log files for the different WebLogic Managed Servers are also available from the Oracle WebLogic Server Administration Console. To verify the logs, access the Oracle WebLogic Server Administration Console http://<admin_server_host>:<port>/console and click on Diagnostics-Log Files.

This log's granularity and logging properties can be changed using Oracle Enterprise Manager Fusion Middleware Control (Fusion Middleware Control). Fusion Middleware Control is a Web browser-based, graphical user interface that you can use to monitor and administer a farm. To receive high availability warning diagnostic messages for Oracle ADF, the level should be set to FINE, as described in Section 6.1.4.3, "Troubleshooting Oracle ADF Replication and Failover Issues."

For more information about the level of diagnostics you can specify for Fusion web applications, see the chapter on testing and debugging in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

For details about using Fusion Middleware Control to change the log settings of WebLogic Managed Servers and Oracle ADF, see the Oracle Fusion Middleware Administrator's Guide.

6.1.2.1 Oracle ADF Scope and Session State

At runtime, ADF objects such as the binding container and managed beans are instantiated. Each of these objects has a defined life span set by its scope attribute.

There are six types of scopes in a Fusion web application:

• Application scope: The object is available for the duration of the application.

• Session scope: The object is available for the duration of the session.

• Page flow scope: The object is available for the duration of a bounded task flow.

• Request scope: The object is available from the time an HTTP request is made until a response is sent back to the client.

• Backing bean scope: Used for managed beans for page fragments and declarative components only, the object is available from the time an HTTP request is made until a response is sent back to the client.

• View scope: The object is available until the view ID for the current view activity changes. This scope can be used to hold values for a given page. However, unlike request scope, which can be used to store a value needed from one page to the next, anything stored in view scope will be lost once the view ID changes.

When the Fusion web application runs in a clustered environment, a portion of the application's state is serialized and copied to another server or a data store at the end of each request so that the state is available to other servers in the cluster.

When you are designing an application to run in a clustered environment, you must:

• Ensure that all managed beans with a life span longer than one request are serializable (that is, they implement the java.io.Serializable interface). Specifically, beans stored in session scope, page flow scope, and view scope need to be serializable.

• Ensure that Oracle ADF is aware of changes to managed beans stored in ADF scopes (view scope and page flow scope) and enable the tracking of changes to ADF memory scopes.

When a value within a managed bean in either view scope or page flow scope is modified, the application needs to notify Oracle ADF so that it can ensure the bean's new value is replicated.

In Example 6-1, an attribute of an object in view scope is modified.

Map<String, Object> viewScope =
    AdfFacesContext.getCurrentInstance().getViewScope();
MyObject obj = (MyObject)viewScope.get("myObjectName");
Obj.setFoo("newValue");

Without additional code, Oracle ADF will be unaware of this change and will not know that a new value needs to be replicated within the cluster. To inform Oracle ADF that an object in an ADF scope has been modified and that replication is needed, use the markScopeDirty() method, as shown in Example 6-2. The markScopeDirty() method accepts only viewScope and pageFlowScope as parameters.

controllerContext ctx = ControllerContext.getInstance();
ctx.markScopeDirty(viewScope);

This code is needed for any request that modifies an existing object in one of the ADF scopes. If the scope itself is modified by the scope's put(), remove(), or clear() methods, it is not necessary to notify Oracle ADF.

To enable ADF Controller to track changes to ADF memory scopes and replicate the page flow scope and view scope within the server cluster, you can enable the<adf-scope-ha-support> parameter in the adf-config.xml file, as described in Section 6.1.3.3, "Configuring adf-config.xml."

For more information about ADF object scopes, see the chapter on Fusion page lifecycle in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

6.1.2.2 Oracle ADF Failover and Expected Behavior

An Oracle WebLogic cluster provides application high availability. If one member of the cluster is unavailable, any other available member of the cluster is able to handle the request.

Session Failover Requirements

For seamless failover of a Fusion web application, the application must meet the following conditions:

• The application is in a cluster and at least one member of the application cluster is available to serve the request.

• For stateful applications, state replication is configured correctly as described in Section 6.1.3, "Configuring Oracle ADF for High Availability."

• If you are using Oracle HTTP Server, the server is configured with the WebLogicCluster directive to balance among all available application instances.

• If you are using a hardware load balancer, the load balancer is:

  • Routing traffic to all available instances

  • Configured correctly with a health monitor to mark unavailable instances

  • Configured to support persistence of session state

Expected Behavior for Application Failover

If the environment has been configured correctly, application users do not notice when an application instance in a cluster becomes unavailable. The sequence of events in an application failover is, for example, as follows:

1. A user makes a request and is routed by a hardware load balancer to instance A of the application.

2. Instance A of the application becomes unavailable because of node failure, process failure, or network failure.

3. The hardware load balancer marks instance A as unavailable.

4. The user makes a subsequent request. The request is routed to instance B.

5. Instance B is configured as a replication partner of Instance A and has the user's session state.

6. The application resumes using the session state on Instance B and the user continues working without interruption.

6.1.2.3 Oracle ADF Active Data Services

The Fusion technology stack includes the Active Data Service (ADS), which allows you to bind ADF Faces components to an active data source using the ADF Model layer. In JDeveloper, you configure individual components in your JSF pages to display active data. When you configure components to use active data, data is pushed to the client whenever a change event is raised by the data source. The data is pushed from the server to the client and displayed by the browser.

To support failover for pages configured to display active data, it is necessary to enable failover for the ADF Business Components application module and to enable ADF Controller to track changes to the ADF memory scopes. With these ADF settings configured, when failover occurs, the ADF server will detect the failover and request all pages configured to display active data to refresh themselves, after that ADS restarts and data is pushed to the client.

For details about how to enable failover for the Fusion web application, see Section 6.1.3, "Configuring Oracle ADF for High Availability."

For more information about using Oracle ADF Faces components with an active data service, see the chapter on ADS in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

6.1.2.4 Configuring the ADF Application Module for Oracle RAC

When configuring the ADF application module to access a highly available database system, such as redundant databases or Oracle Real Application Clusters (Oracle RAC) as the backend, the data source must be container-defined. In this scenario, it is required to use a multi data source; however, from the standpoint of the application module configuration, the naming convention for the multi data source is the same as it is an non-multi data source. This ensures that the correct data source will be used at runtime. For details about configuring multi data sources for high availability applications, see Section 4.1.3, "Configuring Multi Data Sources with Oracle RAC.".

6.1.3.1 Configuring Application Modules

An application module is the transactional component that UI clients use to work with application data. It defines an updateable data model and top-level procedures and functions (called service methods) related to a logical unit of work related to an end-user task. An application module supports passivating, or storing, its transaction state as a snapshot in the database. It also supports the reverse operation of activating the transaction state from one of these saved snapshots.

For more information on the management of application module state, see "Introduction to Fusion Web Application State Management" in Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

To enable support for ADF Business Components failover, set the jbo.dofailover parameter to true so that the application module state is saved on release. This allows Oracle ADF to restore the application module state from a snapshot saved from a previous checkin. By contrast, when the failover feature is disabled, which it is by default, then application module state is saved only when the application is reused by a subsequent user session and only when the application module pool cannot find an unused application module.

You can set this parameter in your application module configuration on the Pooling and Scalability tab of the Edit Business Components Configuration dialog.

To configure application modules for high availability:

1. Launch JDeveloper and open the application.

2. In the Application Navigator, expand the project that contains the data model and locate the application module.

3. Right-click the application module and select Configurations.

4. Click Edit.

5. Click the Pooling and Scalability tab.

6. Select the Failover Transaction State Upon Managed Release checkbox.

7. Click OK to close the Edit Business Components Configuration dialog.

8. Click OK to close the Manage Configurations dialog.

For more information about Oracle ADF state management facilities and how to use them, see the chapter on application state management in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

6.1.3.2 Configuring weblogic.xml

To enable support for replicating HTTP session state, you must assign a value to the persistent-store-type element in the Oracle WLS weblogic.xml file. The value replicated_if_clustered ensures that the in-effect persistent store type will be replicated so that sessions on the clustered environment are stored in accordance with the value set for the cluster of servers to which this server belongs.

To configure the weblogic.xml file for high availability:

1. Launch JDeveloper and open the application.

2. In the Application Navigator, expand the project that contains the web application and expand the WEB-INF folder.

3. Double-click the weblogic.xml file, and click the Source tab to edit the file.

4. In the file, add the persistent-store-type definition to the session-descriptor element:

   <weblogic-web-app>
       <session-descriptor>
       <persistent-store-type>
       replicated_if_clustered
       </persistent-store-type>
       </session-descriptor>
   </weblogic-web-app>
   

6.1.3.3 Configuring adf-config.xml

When you are designing an application to run in a clustered environment, you must make sure that Oracle ADF is aware of changes to managed beans stored in ADF scopes (view scope and page flow scope).

When a value within a managed bean in either view scope or page flow scope is modified, the application needs to notify Oracle ADF so that it can ensure the bean's new value is replicated.

To enable ADF Controller to track changes to ADF memory scopes and replicate the page flow scope and view scope within the server cluster, you must set the ADF Controller parameter <adf-scope-ha-support> in the application's adf-config.xml file to true. For example, when set to true for an application and that application adds or removes a bean from a page flow scope during a request, the change will automatically replicated within a cluster.

The adf-config.xml file is the central configuration file for all ADF components. The file contains sections to configure the runtime behavior for ADF components, including, ADF Controller.

To configure the adf-config.xml file for high availability:

1. Launch JDeveloper and open the application.

2. In the Application Navigator, expand the Application Resources.

3. Select Descriptors, and then select the ADF META-INF node.

4. Double-click the adf-config.xml file, and click the Source tab to edit the file.

5. Add the following to the file:

   <adf-controller-config xmlns="http://xmlns.oracle.com/adf/controller/config">
    <adf-scope-ha-support>true</adf-scope-ha-support>
   </adf-controller-config>
   

For more information about using the adf-config.xml file to configure ADF, see the chapter on creating complex task flows in the Oracle Fusion Middleware Fusion Developer's Guide for Oracle Application Development Framework.

6.1.3.4 Configuring org.apache.myfaces.trinidad.CHECK_FILE_MODIFICATION

The org.apache.myfaces.trinidad.CHECK_FILE_MODIFICATION parameter must not be set to true when running in a high availability environment. Setting this context parameter to true can lead to errors after failover occurs.

6.1.4.1 Troubleshooting Oracle ADF Development Issues

When you develop the Fusion web application in Oracle JDeveloper, the integrated development environment provides support for detecting potential High Availability issues. The warnings that JDeveloper provides are generated by the audit framework and will be triggered to display in the JDeveloper source editors. The warnings the editors display are based on the audit rules for High Availability applications.

The High Availability audit rules that JDeveloper enables by default are:

• ADF Controller Configuration - High Availability for ADF Scopes is not Enabled simply warns the developer that the adf-scope-ha-support flag in the adf-config.xml file is set is not set to true. This audit rule fires only when the <adf-controller-config> element is present the ADF application-level configuration file (adf-config.xml).

• ADF Page Flows - Bean in Scope Map is Modified warns the developer when the some code calls a setter method on a bean to indicate that the code did not subsequently call the ControllerContext.markScopeDirty() method. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.

• ADF Page Flows - EL Bean is Modified warns the developer when some code evaluates an EL expression that mutates a bean to indicate that the code did not subsequently call the ControllerContext.markScopeDirty() method. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.

• ADF Page Flows - Managed Bean Class Not Serializable warns the developer that a managed bean has a non-serializable class defined in viewScope, pageFlowScope, or sessionScope. This audit rule fire only when the adf-scope-ha-support flag in the adf-config.xml file is set to true.

You can modify the High Availability audit rule settings using the Preference dialog in JDeveloper. From the JDeveloper toolbar, choose Tools - Preferences, under Audit - Profiles expand ADF Controller Configuration or ADF Pages Flows and make the desired audit rule selections.

You can also trigger the audit by choosing Build - Audit project.jpr from the JDeveloper toolbar.

6.1.4.2 Troubleshooting Oracle ADF Deployment Issues

Fusion web applications are deployed when the managed server is first started. Use the Oracle WebLogic Server Administration Console first to check that all application deployments were successful:

Click Deployments in the left hand pane. The right hand pane shows the application deployments and their status. The state of all applications, assuming all the servers are running, should be ACTIVE.

If an application deployment has failed, the server logs may provide some indication of why the application was not deployed successfully. The server logs are located in the DOMAIN_HOME/servers/server_name/logs directory. Common issues include:

• Unavailability of external resources, such as database resources. Examine the error, fix it, and attempt to redeploy the application.

• The appropriate applications or libraries are not targeted correctly to the right managed server or Cluster.

6.1.4.3 Troubleshooting Oracle ADF Replication and Failover Issues

State Replication is most prominent in failover scenarios. A user working on one server may discover that, upon failover:

• Windows may be closed or state might be reset.

• Screens may require a reset.

• The application may be redirecting to the logon screen.

The following steps provide guidance in troubleshooting and diagnosing state replication issues.

1. Confirm that this is not a known replication issue.

   See Section 6.1.2.2, "Oracle ADF Failover and Expected Behavior" for possible expected behaviors. Before proceeding to further diagnose the issue, first confirm that the failover behavior is not an expected behavior.

2. Check load balancer settings.

   For replication and failover to function correctly, the load balancer must be configured with the appropriate persistence settings. For more details on configuring Hardware Load Balancers for Oracle WebLogic Server, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

3. Check the cluster status.

   Replication occurs within the context of a cluster. For failover to be successful, there must be at least one other healthy member of the cluster available. You can check cluster status in one of two ways:

   • Check the cluster status using the Oracle WebLogic Server Administration Console - In the Left-hand pane, click on Servers. Verify the state of all servers in the cluster.

   • Check the cluster status using weblogic.Admin utility The weblogic.Admin command can be used to query the state of all servers in a specific cluster. For example:

     $ java weblogic.Admin -url Adminhost:7001 -username <username> -password
      <password>  CLUSTERSTATE -clustername Spaces_Cluster
     

     This example returns:

     There are 2 server(s) in cluster: Spaces_Cluster
     The alive servers and their respective states are listed below:
     Application Server---RUNNING
     Managed Server---RUNNING
     

4. Check cluster communications.

   Although Cluster members may all be running, there may be communication issues which prevent them from communicating replication information to each other. There are two types of cluster communication configurations. Troubleshooting depends on the cluster type:

   • Checking Unicast cluster communications - For Unicast clusters, managed servers must be able to access each other's hosts and each other's default listening port.

     Ensure that all individual managed servers have their Listen Address set correctly. You can find this setting by selecting Configuration, General for each managed server.

   • Checking Multicast cluster communications - For multicast clusters, servers must be able to intercept the same multicast traffic. Ensure that multicast is configured correctly by running the WebLogic utility utils.MulticastTest on each machine. For example:

     $ java utils.MulticastTest -H
     

5. Confirm Oracle WLS application configuration.

   Oracle WLS is not configured by default for failover. In-memory replication takes place only with the proper setting in the weblogic.xml file:

   <session-descriptor>
   <persistent-store-type>replicated_if_clustered</persistent-store-type>
   </session-descriptor>
   

   A persistent-store-type of replicated is also acceptable. This setting can be made in JDeveloper, as described in Section 6.1.3.2, "Configuring weblogic.xml."

6. Confirm Oracle ADF Business Components configuration.

   Oracle ADF is not configured by default for failover. Failover is supported only with the proper setting in the ADF Business Components configuration file (bc4j.xcfg):

   <AppModuleConfig ...
    <AM-Pooling jbo.dofailover="true"/>
   </AppModuleConfig>
   

   This setting is made in JDeveloper through the Edit Business Components Configuration dialog, as described in Section 6.1.3.1, "Configuring Application Modules."

7. Confirm Oracle ADF Controller configuration.

   Oracle ADF is not configured by default to replicate changes to ADF objects in ADF memory scopes. ADF object replication is supported only with the proper setting in the ADF application-level configuration file (adf-config.xml):

   <adfc:adf-controller-config>
    <adfc:adf-scope-ha-support>true</adfc:adf-scope-ha-support>
   </adfc:adf-controller-config>
   

   This setting is made in JDeveloper through the source editor, as described in Section 6.1.3.1, "Configuring Application Modules."

8. Check default logger messages.

   By default the ADF log display high-level messages (INFO level). The default logging often reports problems with serialization and replication without the need to enable more detailed log messages. For more information about the log, see Section 6.1.1.4, "Oracle ADF Log File."

9. Enable log messages for ADF high availability applications.

   Configure the ADF logger to output runtime messages for high availability. By default the ADF log display high-level messages (INFO level). You enable high availability diagnostics for ADF Controller by setting the logging level in Fusion Middleware Control to FINE.

   When enabled, the logger outputs a warning if the adfc:adf-scope-ha-support setting in the adf-config.xml file is not set. For more information about the ADF logger, see Section 6.1.1.4, "Oracle ADF Log File."

10. Enable debug.

    Check the server logs for any unusual messages on managed server startup. In particular, if the managed server is unable to locate other members of the cluster. The server logs are located in the DOMAIN_HOME/servers/SERVER_NAME/logs directory.

    For further debugging, enable the flags DebugCluster, DebugClusterAnnouncements, DebugFailOver, DebugReplication, and DebugReplicationDetails. Each flag can be enabled with the weblogic.Admin utility:

    $ java weblogic.Admin -url Adminhost:7001 -username <username> -password
     <password> SET -type ServerDebug -property DebugCluster true
    

11. Enable component state serialization checking.

    Enable server checking to ensure no unserializable state content on session attributes is detected. This check is disabled by default to reduce runtime overhead. Serialization checking is supported by the Java server system property org.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION.

    Table 6-1 shows the options that can be used with the property. Use commas to delimit the options.

    Table 6-1 CHECK_STATE_SERIALIZATION Options

    Option	Description
    tree	Checks whether the entire component tree is serializable. This is the fastest component check. Most testing should be performed with this flag enabled.
    component	Checks each component individually for serializability. This option is much slower than "tree." It is typically turned on only after testing with "tree" has reported an error. This option is used to narrow down the problematic component.
    property	Checks each component attribute individually for serializability. This is slower than "component" and is used to narrow the down the specific problematic component attribute after a failure has been detected in the "tree" or "component" modes.
    session	Checks that all attributes in the JSF Session Map that are marked as Serializable are in fact serializable.
    application	Checks that all attributes in the JSF Application Map that are marked as Serializable are in fact serializable.
    beans	Checks that any serializable object in the appropriate map has been marked as dirty if the serializable content of the object has changed during the request.
    all	Checks everything.

    
    

    For high availability testing, start off by validating that the Session and JSF state is serializable by launching the application server with the system property:

    -Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=session,tree
    

    Add the beans option to check that any serializable object in the appropriate map has been marked as dirty if the serialized content of the object has changed during the request:

    -Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=session,tree,beans
    

    If a JSF state serialization failure is detected, relaunch the application server with the system property to enable component and property flags and rerun the test:

    -Dorg.apache.myfaces.trinidad.CHECK_STATE_SERIALIZATION=all
    

    Since these are Java system properties, they need to be specified when the application server is started.

6.2.2.1 Running RCU

Run RCU to install the required metadata for Oracle Fusion Middleware 11g:

1. Start up RCU using the following command:

   RCU_HOME/bin/rcu &
   

2. In the Welcome screen, click Next.

3. In the Create Repository screen, select Create to load component schemas into a database, and click Next.

4. In the Database Connection Details screen, enter connection information for your database:

   • Database Type: Select Oracle Database

   • Host Name: Enter the name of the node that is running the database. For an Oracle RAC database, specify the VIP name, or one of the node names as the host name: ADFDBHOST1VIRTUAL.

   • Port: The port number for the database: 1521

   • Service Name: Enter the service name of the database: adfha.mycompany.com

   • Username: SYS

   • Password: Enter the password of the SYS user.

   • Role: SYSDBA

   Click Next.

5. If you receive the following message, click Ignore or Stop:

   The database you are connecting is with non-UTF8 charset, if you are going to use this database for multilingual support, you may have data loss. If you are not using for multilingual support you can continue, otherwise, Oracle strongly recommends using a UTF-8 database.

6. In the Select Components screen, select Create a New Prefix, and enter a prefix to use for the database schemas, for example, ADFHA.

   Write down the schema names so they are available in later procedures.

   Select the following schemas:

   • AS Common Schemas

   • Metadata Services

   Click Next.

7. In the Schema Passwords screen, enter passwords for the main and additional (auxiliary) schema users, and click Next.

8. In the Map Tablespaces screen, choose the tablespaces for the selected components, and click Next.

9. In the Summary screen, click Create.

10. In the Completion Summary screen, click Close.

See the Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about installing RCU.

6.2.3.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by entering the following URL in a web browser:

WebHost1:7777/

If Oracle HTTP Server is set up correctly, the Hello World page appears in the browser.

6.2.4.1 Installing Oracle WebLogic Server

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the version of Oracle WebLogic Server to use with the latest version of Oracle Fusion Middleware.

To install Oracle WebLogic Server on all nodes in the application tier:

1. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

   If the /etc/oraInst.loc file does not exist, skip this step.

2. Start the Oracle WebLogic Server installer.

3. In the Welcome screen, click Next.

4. In the Choose Middleware Home Directory screen:

   • Select Create a New Middleware Home.

   • For the Middleware Home Directory field, enter:

     ORACLE_BASE/product/fmw
     

   Click Next.

5. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

6. In the Choose Install Type screen, select Custom, and click Next.

7. In the JDK Selection screen, select only Oracle JRockit 1.6.0_14 SDK, and click Next.

8. In the Choose Product Installation Directories screen, accept the following directory:

   ORACLE_BASE/product/fmw/wlserver_10.3
   

   Click Next.

9. In the Installation Summary screen, click Next.

10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

6.2.4.2 Installing Oracle Fusion Middleware for Oracle ADF Applications

To install Oracle Fusion Middleware for Oracle ADF, use the Application Developer Install and perform the following on all the nodes in the application tier:

1. Start the Oracle Fusion Middleware for Oracle Fusion Middleware 11g Application Developer installer:

   On UNIX (Linux used in this example):
   APPHOST1> runInstaller
   
   On Windows:
   APPHOST1> setup.exe
   

   When Oracle Fusion Middleware 11g Application Developer installer prompts you for a JRE/JDK location enter the Oracle SDK location created in the Oracle WebLogic Server installation in Section 6.2.4.1, "Installing Oracle WebLogic Server," for example:

   ORACLE_BASE/product/fmw/jrockit_160_14_R27.6.4-18
   

2. In the Welcome screen, click Next.

3. In the Prerequisite Check screen, verify that the checks complete successfully, and click Next.

4. In the Specify Installation Location screen:

   • For Middleware Home, enter: ORACLE_BASE/product/fmw

   • For Oracle Home Directory, enter the directory you want to use, for example: adf

   Click Next.

5. In the Installation Summary screen, click Install.

6. In the Installation Complete screen, click Finish.

6.2.6.1 Creating boot.properties for the Administration Server and Managed Servers on APPHOST1

This is an optional step for enabling the Administration Server to start up without prompting you for the administrator username and password. Create a boot.properties file for the Administration Server and for the managed servers on APPHOST1.

For the Administration Server, follow these steps:

1. Create the following directory:

   APPHOST1> mkdir -p MW_HOME/wls/user_projects/domains/adfdomain/servers/AdminServer/security
   

2. Use a text editor to create a file named boot.properties in the directory created in the previous step, and enter the following lines in the file:

   username=adminUser
   password=adminUserPassword
   

   For example:

   username=weblogic
   password=weblogic
   

   Note:

   When you start up the Administration Server or Managed Server, the username and password entries in the file are encrypted.

   For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

For the WLS_ADF Managed Servers, follow these steps:

1. Copy the file you created for the Administration Server to all servers:

   APPHOST1> mkdir -p servers/WLS_ADF1
   APPHOST1> mkdir -p servers/WLS_ADF1/security
   

2. Use a text editor to create a file named boot.properties in the directory created in the previous step, and enter the following lines in the file:

   username=adminUser
   password=adminUserPassword
   

   For example:

   username=weblogic
   password=weblogic
   

6.2.7.1 Starting the Administration Server on APPHOST1

To start the Administration Server on APPHOST1 run the following commands:

APPHOST1> cd ORACLE_BASE/product/fmw/user_projects/domains/adfdomain/bin
APPHOST1> ./startWebLogic.sh

6.2.7.2 Validating the Administration Server

To verify that the Administration Server is properly configured, follow these steps:

1. In a Web browser, go to http://VIP1:7001/console.

2. Log in as the administrator.

3. Verify that the WLS_ADF1 and WLS_ADF2 managed servers are listed.

4. Verify that the ADF_Cluster cluster is listed.

5. Verify that you can access Enterprise Manager at http://VIP1:7001/em.

6.2.7.3 Disabling Host Name Verification for the Administration Server and Managed Servers for APPHOST1 and APPHOST2

This step is required if you have not set up SSL communication between the Administration Server and the Node Manager. If SSL is not set up, you receive an error message unless you disable host name verification.

You can re-enable host name verification when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification on APPHOST1:

1. In Oracle WebLogic Server Administration Console, select Administration Server, SSL, and then Advanced.

2. In the Change Center, click Lock & Edit.

3. When prompted, save the changes and activate them.

4. Set Hostname Verification to None.

5. Select WLS_ADF1, SSL, and then Advanced.

6. Set Hostname Verification to None.

7. Restart the Administration Server and the WLS_ADF1 Managed Server.

To disable host name verification on APPHOST2:

1. In Oracle WebLogic Server Administration Console, select WLS_ADF2, SSL, and then Advanced.

2. Set Hostname Verification to None.

3. Restart the Administration Server and the WLS_ADF2 Managed Server.

6.2.7.4 Starting Node Manager on APPHOST1

Perform these steps to start Node Manager on APPHOST1:

1. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager:

   OAHOST1> cd ORACLE_COMMON_HOME/common/bin
   APPHOST1> ./setNMProps.sh
   

   Note:

   You must use the StartScriptEnabled property to avoid class loading failures and other problems.

2. Start Node Manager:

   APPHOST1> cd WL_HOME/server/bin
   APPHOST1> ./startNodeManager.sh
   

6.2.9.1 Creating boot.properties for the Administration Server and Managed Servers on APPHOST2

Create a boot.properties file for the Administration Server and for the Managed Servers on APPHOST2 by following these steps:

1. Create the following directories:

   APPHOST1> mkdir -p MW_HOME/wls/user_projects/domains/adfdomain/servers/WLS_ADF2
   APPHOST2> mkdir -p MW_HOME/wls/user_projects/domains/adfdomain/servers/WLS_ADF2/security
   

2. Use a text editor to create a file named boot.properties in the directory created in the previous step, and enter the following lines in the file:

   username=adminUser
   password=adminUserPassword
   

   For example:

   username=weblogic
   password=weblogic
   

   Note:

   When you start up the Administration Server or Managed Server, the username and password entries in the file are encrypted.

   For security reasons, minimize the time the entries in the file are left unencrypted. After you edit the file, start up the server as soon as possible in order for the entries to be encrypted.

6.2.9.2 Starting Node Manager on APPHOST2

To start the Node Manager on APPHOST2, repeat the steps from Section 6.2.7.4, "Starting Node Manager on APPHOST1" on APPHOST2.

6.2.9.3 Configuring the ADF Application for Replication

Use the procedures in this section to configure your application for replication.

Clustering Requirement

The application must be deployed to an Oracle WebLogic Cluster. This automatically establishes a replication channel for the multiple instances of the application.

Oracle ADF Replication

It is essential that Oracle ADF is configured properly. The following tag should be present in the adf-config.xml file, one of the Application Resources, for a stateful application:

<adfc:adf-controller-config><adfc:adf-scope-ha-support>true</adfc:adf-scope-ha-support></adfc:adf-controller-config>

Applications must also have replication enabled. Oracle WebLogic Server allows several types of persistent stores for replication. For more information on persistent stores, see the Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server manual.

The ADF application can be enabled by for this by default with the following setting in the weblogic.xml file:

<session-descriptor>
<persistent-store-type>replicated_if_clustered</persistent-store-type>
</session-descriptor>

The replicated_if_clustered setting disables replication for standalone application environments, and uses in-memory replication within a cluster environment.

Ensure that any custom application is configured for in-memory replication.

6.2.9.4 Deploying the ADF Application

After the cluster has been set up, the ADF application can be deployed. Be aware of the following:

• Deploy the ADF application to an EAR file and deploy using the Administration Server Console.

• Ensure that the deployment is to a cluster.

• If your application uses MDS, register the MDS with the domain using the instructions in the "Registering a Database-Based MDS Repository" section in the Oracle Fusion Middleware Administrator's Guide manual.

6.2.9.5 Configuring Oracle HTTP Server for the Administration Server and Oracle WebCenter Managed Servers

Enable Oracle HTTP Server to route to the Administration Server that contains Oracle WebCenter Managed Servers by setting the WebLogicCluster parameter to the list of nodes in the cluster. Follow these steps:

1. Add the following lines to the OHS_HOME/instances/ohs_instance1/config/OHS/ohs1/mod_wl_ohs.conf file:

   # Spaces
   <Location /applicationMountpoint>
       WebLogicCluster apphost1.com:8888,apphost2.com:8889
       SetHandler weblogic-handler
   </Location>
   

2. Restart Oracle HTTP Server on WEBHOST1:

   WEBHOST1> OHS_HOME/instances/ohs_instance1/bin/opmnctl restartproc ias-component=OHS_COMPONENT1
   

6.2.9.6 Validating Access through Oracle HTTP Server

Verify the URLs to ensure that appropriate routing and failover is working from the HTTP Server to Oracle WebCenter cluster. Follow these steps:

1. Start WC_Spaces1, WC_Spaces2, WC_Collaboration1, WC_Collaboration2, WC_Portlet1, and WC_Portlet2 from the WebLogic Server Administration Console as follows:

   1. Access the Administration Console at the following URL:

      http://APPHOST1/console
      

   2. Click on one of the Managed Servers, for example, WC_Spaces1.

   3. Select the Control tab.

   4. Select Start to start the Managed Server.

   5. Repeat the previous steps for each Managed Server.

   6. Verify direct access to the Managed Servers using the following URLs:

      apphost1:8888/applicationMountpoint
      
      apphost2:8888/applicationMountpoint
      
      apphost1:8889/applicationMountpoint
      
      apphost2:8889/applicationMountpoint
      

2. While WLS_ADF2 is running, stop WLS_ADF1 from Oracle WebLogic Server Administration Console.

3. Access the following URL and verify the appropriate functionality:

   WebHost1:7777/applicationMountpoint
   

4. Start WLS_ADF1 from the WebLogic Server Administration Console.

5. Stop WLS_ADF2.

6. Access the following URL and verify the appropriate functionality:

   WebHost2:7777/applicationMountpoint
   

6.2.10.1 Scaling Up the Topology (Adding Managed Servers to Existing Nodes)

In this case, you already have a node that runs a managed server configured with Oracle ADF. The node contains a Middleware home in shared storage.

You can use the existing installations (Middleware home, and domain directories) for creating new servers. There is no need to install Oracle Fusion Middleware binaries in a new location, or to run pack and unpack.

Follow these steps for scaling up the topology:

1. Using the Oracle WebLogic Server Administration Console, clone WLS_ADF1 into a new managed server. The source managed server to clone should be one that already exists on the node where you want to run the new managed server.

   To clone a managed server:

   1. Select Environment > Servers from the Administration Console.

   2. Select the managed server that you want to clone.

   3. Select Clone.

   Name the new managed server WLS_ADFn, where n is a number to identify the new managed server.

2. For the listen address, assign the host name or IP to use for this new managed server.

3. Ensure that the port number for this managed server is available on this node.

4. Reconfigure the Oracle HTTP Server module with the new member in the cluster. See Section 6.2.9.5, "Configuring Oracle HTTP Server for the Administration Server and Oracle WebCenter Managed Servers."

6.2.10.2 Scaling Out the Topology (Adding Managed Servers to New Nodes)

In scaling out your topology, you add new managed servers configured with Oracle ADF applications to new nodes.

Before performing the steps in this section, check that you meet these requirements:

• In your topology, there are existing nodes running managed servers configured with ADF applications.

• ADF managed servers are clustered and the new managed server will also be part of that cluster.

• The new node can access the existing home directories for WebLogic Server. You use the existing installations in shared storage for creating a new managed server. There is no need to install WebLogic Server binaries in a new location, although you need to run pack and unpack to create a managed server domain.

Follow these steps for scaling out the topology:

1. On the new node, mount the existing Middleware home, and ensure that the new node has access to this directory, just like the rest of the nodes in the domain.

2. To update the Middleware home list, create (or edit, if another WebLogic installation exists in the node) the MW_HOME/bea/beahomelist file and add ORACLE_BASE/product/fmw to it.

3. Log into the Oracle WebLogic Administration Console.

4. Create a new machine for the new node that will be used, and add the machine to the domain.

5. Update the machine's Node Manager's address to map the IP of the node that is being used for scale out.

6. Use the Oracle WebLogic Server Administration Console to clone WLS_ADF1 into a new managed server. Name it WLS_ADFn, where n is a number and assign it to the new machine.

7. For the listen address, assign the host name or IP to use for the new managed server.

   Perform these steps to set the managed server listen address:

   1. Log into the Oracle WebLogic Server Administration Console.

   2. In the Change Center, click Lock & Edit.

   3. Expand the Environment node in the Domain Structure window.

   4. Click Servers. The Summary of Servers page is displayed.

   5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

   6. Set the Listen Address to APPHOSTn where APPHOSTn is the DNS name of your new machine.

   7. Click Save.

   8. Save and activate the changes.

   The changes will not take effect until the managed server is restarted.

8. Run the pack command on APPHOST1 to create a template pack as follows:

   APPHOST1> cd ORACLE_COMMON_HOME/common/bin
   APPHOST1> ./pack.sh -managed=true -domain=MW_HOME/user_projects/domains/
   adfdomain/ -template=adfdomaintemplateScale.jar
   -template_name=adf_domain_templateScale
   

   Run the following command on APPHOST1 to copy the template file created to APPHOSTn:

   APPHOST1> scp adfdomaintemplateScale.jar oracle@APPHOSTN:/
   ORACLE_COMMON_HOME/common/bin
   

   Run the unpack command on APPHOSTn to unpack the template in the managed server domain directory as follows:

   APPHOSTn> cd ORACLE_COMMON_HOME/common/bin
   APPHOSTN> ./unpack.sh
   -domain=ORACLE_BASE/product/fmw/user_projects/domains/adfdomain/
   -template=adfdomaintemplateScale.jar
   

9. Start the Node Manager on the new node. To start the Node Manager, use the installation in shared storage from the existing nodes, and start Node Manager by passing the host name of the new node as a parameter as follows:

   APPHOSTn> WL_HOME/server/bin/startNodeManager <new_node_ip>
   

10. Start and test the new managed server from the Oracle WebLogic Server Administration Console:

    1. Shut down all the existing managed servers in the cluster.

    2. Ensure that the newly created managed server is running.

    3. Access the application on the newly created managed server. The application should be functional.

6.3.1.1 Oracle WebCenter Components

Oracle WebCenter includes the following components.

• Oracle WebCenter Spaces offers a single, integrated, Web-based environment for social networking, communication, collaboration, and personal productivity through a robust set of services and applications.

  Oracle WebCenter Spaces is built using JSF, Oracle ADF, Oracle WebCenter Framework, Oracle WebCenter services, and Oracle Composer. Oracle WebCenter Spaces provides:

  • A browser-based, community-focused application targeting the business user.

  • A personal space for each user, providing a private work area for storing personal content, keeping notes, viewing and responding to business process assignments, maintaining a list of online buddies, emailing, and so on. The focus of a personal space is personal productivity.

  • Group spaces, a rich team collaboration platform.

  • Threaded discussions, worklists, announcements, RSS, recent activities, search, and more.

• Oracle WebCenter Portlets Framework supports deployment and execution of both standards-based portlets (JSR 168, WSRP 1.0 and 2.0), and traditional Oracle PDK-Java based portlets. Oracle WebCenter provides several out-of-the-box producers, such as OmniPortlet, Web Clipping, and WSRP Tools.

• Oracle WebCenter Framework provides the following capabilities:

  • Runtime customization (you can make in-place changes to the application without redeploying it)

  • Support for JSR-168 standards-based WSRP portlets, and PDK-Java portlets

  • Content integration through JCR (JSR170) to content repositories such as Oracle Content Server, Oracle Portal, and file systems

  • JSF-Portlet Bridge, which lets you expose JSF pages and Oracle ADF task flows as standards-based portlets

• Oracle WebCenter Discussion Server provides the ability to integrate discussion forums and announcements into your applications.

• Oracle WebCenter Analytics provides users with the ability to view reports on the various user activities within an instance of Oracle WebCenter Spaces, for example:

  • Logins

  • Page views

  • Portlet views

  • Document views

  • Search metrics

  • Response times

  These reports can be broken out by parameters such as User properties, GroupSpaces, and time.

• Oracle WebCenter Activity Graph provides users with the ability to analyze the statistics collected by Oracle WebCenter Analytics. The output of an Oracle WebCenter Activity Graph analysis is the collected scores for objects and users, which are used to give recommendations. The scores are stored in the Oracle WebCenter Activities database.

• Oracle WebCenter Personalization Server is a lightweight service accessed by client applications via RESTful web services. New JDev tooling supports creation of the property definitions and scenarios used by Oracle WebCenter Personalization. Out-of-the-box integration with other Oracle WebCenter services (Oracle Activity Graph, CMIS, People Connections) is supported as well as a extensibility model for customers. Oracle WebCenter Personalization Server provides users with the ability to define user and applications property definitions and values, as well as implement structured scenarios. These can be used to personalize any kind of application or application logic for a user, group, or other scope.

Oracle WebCenter Spaces and custom WebCenter applications (Oracle WebCenter Portal Applications) can also integrate with the following Oracle WebCenter services:

• Announcements - Provides a means of posting announcements about important activities and events to members of a given group space.

• Discussions - Provides a means of creating threaded discussions, posting and responding to questions, and searching for answers-all within the context of a group space. Also provides an effective group communication mechanism for important activities and events.

• Documents - Provides content management and storage capabilities, including content upload, file and folder creation and management, file check out, versioning, and so on.

• Events - Available in Oracle WebCenter Spaces, the Events service provides a means of creating and maintaining a schedule of events relevant to a wider group of users. Events are published to all members of a group space.

• Instant Messaging and Presence (IMP) - Provides a means of observing the status of other authenticated users (whether online, offline, busy, or idle) and contacting them instantly

• Links - Provides viewing, accessing, and associating related information; for example you can link to a solution document from a discussion thread.

• Lists - Available in Oracle WebCenter Spaces, the Lists service provides the ability to create, publish, and manage lists and tables. Users can create lists from prebuilt structures or create their own custom lists.

• Mail - Provides easy integration with IMAP and SMTP mail servers to enable users to perform simple mail functions such as viewing, reading, creating, and deleting messages, creating messages with attachments, and replying to or forwarding existing messages.

• Notes - Available in Oracle WebCenter Spaces, the Notes service provides the ability to "jot down" and retain quick bits of personally relevant information.

• People Connections - Provides a means of connecting, interacting, and keeping track of other users through social networking applications, such as Message Board, Feedback, Profile, and Activity Stream.

• Recent Activities - Provides a summary view of recent changes to pages, documents, discussions, announcements, lists, and events.

• RSS - Provides the ability to access the content of many different Web sites from a single location-a news reader.

• Search - Provides a means of searching tags, services, the application, or an entire site. This includes integrating Oracle Secure Enterprise Search for Oracle WebCenter searches.

• Tags - Provides a means of assigning one or more personally relevant keywords to a given page or document.

• Worklists - Provides a means of viewing notifications from the various workflows established in your enterprise.

For information about how to configure these Oracle WebCenter services, see Oracle Fusion Middleware Administrator's Guide for Oracle WebCenter.

6.3.1.2 Oracle WebCenter Single-node Architecture

Oracle WebCenter installation creates a WebCenter directory under the Middleware home directory. The installation creates a WebCenter domain (wc_domain), which contains the Administration Server and four WebLogic Managed Servers: WC_Spaces1 (which hosts Oracle WebCenter Spaces), WC_Portlets (which hosts Oracle WebCenter Portlets), WC_Collaboration1 (which hosts Oracle WebCenter Discussion server, WC_Utilities1 (which hosts Oracle WebCenter Analytics, Oracle WebCenter Activity Graph, and Oracle WebCenter Personalization Server), and any additional WebCenter services that you choose to integrate).

An optional fifth managed server (an application server) must be used to run custom WebCenter applications (Oracle WebCenter Portal Applications). When you create additional managed servers, they are provisioned with the appropriate libraries to enable them to draw upon the same external resources as Oracle WebCenter Spaces. For more information about managed servers, see “Understanding Oracle Fusion Middleware Concepts” in the Oracle Fusion Middleware Administrator's Guide. Figure 6-4 shows a basic single-node Oracle WebCenter architecture.

Figure 6-4 Basic Single-Node Oracle WebCenter Architecture

Description of "Figure 6-4 Basic Single-Node Oracle WebCenter Architecture"

6.3.1.3 Oracle WebCenter State and Configuration Persistence

Oracle WebCenter Spaces runs as a Java EE application with application state and configuration persisted to the MDS repository. User session information within the application is held locally in memory. In a cluster environment, this state is replicated to other members of the cluster.

All of the out of the box Oracle WebCenter Portlets use the database to persist customizations.

Oracle WebCenter Discussion Server uses its own database persistence mechanisms for data and metadata.

Oracle Webcenter Analytics consists of two components: the Oracle Webcenter Analytics service, which consists of task flows and data controls, and the Oracle WebCenter Analytics Collector Service.

Events sent to the Oracle WebCenter Collector are placed into a configurable queue of incoming events for persisting to the database. In the event of a failure of an Oracle WebCenter Collector Service, these events will be lost. However, under typical usage the queue does not contain a large number of events.

The Oracle WebCenter Analytics Collector supports clustering and failover, but the queue is not shared between collectors.

The Oracle WebCenter Analytics service task flows, which provide the reporting UI for WebCenter are stateless, with the exception of Personalization and Customization features. These features are built on the standard Composer/ADF/MDS framework, which manages the state in the event of a failure.

Oracle WebCenter Activity Graph consists of two components: the Oracle WebCenter Activity Graph service, which consists of task flows and data controls, and the Oracle WebCenter Activity Graph engine.

The engine runs a batch data analysis process which updates tables in the database transactionally. Although it does not support clustering or failover, it can recover from failure.

The Oracle WebCenter Activity Graph service does not maintain any in-memory state. It is primarily a read-only process.

The Oracle WebCenter Spaces task flows query the Oracle WebCenter Activities database and render the result as a list of recommendations. The three places where state is updated are:

1. Personalization settings

2. Task flow configuration parameters

3. "Not-interested" feature

The first two are built on the standard Composer/ADF/MDS framework, which manages the state. The last is a feature where the user can indicate that they are not interested in a particular recommendation. This input is persisted synchronously in the Oracle WebCenter Activity Graph database.

An administrator uses the Oracle WebCenter Activity Graph user interface to set up and monitor the nightly schedule, but the user interface does not present the results of the analysis. The results of the analysis (the recommendations) are presented in Oracle WebCenter Spaces task flows. These task flows query the Activities database. If the Oracle WebCenter Activity Graph backend fails to run for a few days, this is not fatal, and it will not prevent Oracle WebCenter Activity Graph from being able to give recommendations. However, until the Oracle WebCenter Activity Graph backend does run again, some of the recommendations will be a bit stale.

Oracle WebCenter Activity Graph is a singleton application that has a background thread that wakes up periodically to check if it is time to run the nightly job, which can last several hours. The schedule is also persisted to the database. If the managed server fails, when it comes back up, the job will continue.

Oracle WebCenter Personalization Server is a stateless RESTful application. All state is managed in the client requests. Oracle WebCenter Personalization metadata produced by JDev tools (in support of the Property Service and Scenarios) is persisted in MDS. Additional configuration data (connection information) is persisted in domain scoped managed beans.

6.3.1.4 Oracle WebCenter External Dependencies

Oracle WebCenter Spaces is the client for Oracle WebCenter Analytics, Oracle WebCenter Activity Graph, and Oracle WebCenter Personalization Server.

Oracle WebCenter Analytics uses the database that includes the Activities schema. Oracle WebCenter Activity Graph uses the data stored in the Activities schema to provide recommendations. Oracle WebCenter Personalization Server uses a database for its Personalization schema and MDS schemas.

Clients connect to Oracle WebCenter Personalization Server using JDBC and UDP.

Oracle WebCenter Personalization Server uses REST, JDBC, the ADF Connection architecture, and MDS JMX.

Oracle WebCenter Activity Graph uses JDBC to mine statistical data in the Activities schema, and the client (Oracle WebCenter Spaces) also uses JDBC to connect to the Activities schema to present the data to users.

Oracle WebCenter Spaces uses the Java Client API that is based on Open Usage, and which uses multicasting over UDP, to communicate with Oracle WebCenter Analytics.

Table 6-2 shows the access type for each of the Oracle WebCenter components and services. The Configuration column lists the type of information provided to Oracle WebCenter to configure or initialize the connection. The Access column lists the protocol used in runtime access of the service.

The Oracle Discussion Server is a service provider to Oracle WebCenter Spaces. Oracle Portlets also exposes Portlet Producers as services.

Unavailability of these services does not prevent Oracle WebCenter Spaces application from starting, although application errors may be seen when running. Oracle WebCenter Spaces requires the MDS and WebCenter schemas.

Configure each of the external services independently for high availability. Oracle WebCenter's framework provides a single point of access for external services.

• For HTTP services, for example, direct the access URL to a load balancer which provides access to multiple service providers on the back-end.

  Instructions for configuring Oracle WebCenter Discussions servers for high availability are provided in Section 6.4.5, "Running Oracle Fusion Middleware Configuration Wizard on APPHOST1 to Create the WebLogic Server WebCenter Domain."

• For the MDS repository and schemas, Oracle recommends an Oracle Real Application Clusters (Oracle RAC) database as the back-end database.

  Instructions for configuring Oracle RAC as a database provider are in Section 6.4.5, "Running Oracle Fusion Middleware Configuration Wizard on APPHOST1 to Create the WebLogic Server WebCenter Domain."

For information about multi data source configuration with Oracle RAC and the MDS repository, see Section 4.1.2, "Using Multi Data Sources with Oracle RAC."

6.3.1.5 Oracle WebCenter Configuration Considerations

The main configuration files for Oracle WebCenter applications are listed and described in Table 6-3. Both these files are supplied within the Oracle WebCenter application deployment .EAR file.

Oracle WebCenter applications and Portlet Producers both use the Oracle Metadata Services (MDS) repository to store their configuration data; both access the MDS repository as a JDBC data source within the Oracle WebLogic framework.

The MDS repository stores post deployment configuration changes for Oracle WebCenter applications and Portlet Producers as customizations. MDS uses the original deployed versions of adf-config.xml and connection.xml as base documents and stores all subsequent customizations separately into MDS using a single customization layer.

When an Oracle WebCenter application starts up, customizations stored in MDS are applied to the appropriate base documents and the Oracle WebCenter application uses the merged documents (base documents with customizations) as the final set of configuration properties.

For Oracle WebCenter applications that are deployed to a server cluster, all members of a cluster read from the same location in the MDS repository.

Typically, there is no need for administrators to examine or manually change the content of base documents (or MDS customization data) for files such as adf-config.xml and connection.xml as Oracle provides several administration tools for post deployment configuration. For details, see the section "Oracle WebCenter Administration Tools" in the Oracle Fusion Middleware Administrator's Guide for Oracle WebCenter.

Oracle WebCenter applications store post-deployment configuration information in MDS but configuration information for Portlet Producers and Oracle WebCenter Discussions Server is stored in the file system (see Table 6-4).

The Oracle WebCenter Discussions Server stores configuration information in its database. Additionally, it stores startup configuration information in DOMAIN_HOME/config/fmwconfig/servers/SERVER_NAME/owc_discussions_11.1.1.4.0. This directory contains the jive_startup.xml file, jive.license files, and a \logs directory containing log files for the discussions server instance.

Oracle WebCenter Analytics uses WLST commands for the clients. JVM parameters can also be specified and will override the WLST values. If you specified JVM parameters in setDomainEnv.sh in previous releases of Oracle WebCenter, you should remove them if you plan to use Oracle WebCenter Analytics.

The Oracle WebCenter Activity Graph engine writes temporary files to disk, which are recreated if they are lost.

Oracle WebCenter Personalization Server has JMX beans for its provider connections, which get bootstrapped from provider-connections.xml in the classpath. WLST command scripts support configuration of Oracle WebCenter Personalization provider connections via JMX. These beans can also be managed via Oracle Enterprise Manager or JConsole.

6.3.1.6 Oracle WebCenter Log File Locations

Operations performed by Oracle WebCenter applications, Portlet Producers, and discussion servers, and so on, are logged directly to the WebLogic Managed Server where the application is running:

WLS_DOMAIN_HOME/servers/WLS_SERVER_NAME/logs/WLS_SERVER_NAME.log

You can view the log files for each WebLogic managed server from the Oracle WebLogic Server Administration Console. To view the logs, access the Oracle WebLogic Server Administration Console http://<admin_server_host>:<port>/console and click Diagnostics-Log Files.

Oracle WebCenter Analytics integrates with the current WebLogic Server logging framework by using Apache Commons logging in the Collector and ODL configuration files in the domain template used to install the Collector. The ODL logging configuration file is merged with the WebLogic Server default logging configuration file when the Collector is installed using the Configuration Wizard. When the ODL logging configuration file is merged with the WebLogic Server default logging configuration file, if you open the logging.xml file generated in the DOMAIN_HOME/config/fmwconfig/servers/WLS_Utilities folder, you will find logging settings for a log handler named collector-log-handler.

In addition, a diagnostic log is produced in the log directory for the managed server. This log's granularity and logging properties can be changed through the DOMAIN_HOME/config/fmwconfig/servers/SERVER_NAME/logging.xml file.

6.3.2.1 Oracle WebCenter Applications

During Oracle WebCenter installation, the managed servers are provisioned with system libraries and ADF libraries. Table 6-5 lists the managed servers and the applications which run on them.

6.3.2.2 Oracle WebCenter Startup Order

When a managed server is started, applications and libraries are started in the following order:

1. Oracle System libraries, known as the JRF libraries.

2. Oracle ADF libraries.

3. Instrumentation applications, such as Oracle DMS.

4. Oracle WebCenter applications, shown in Table 6-2

The startup order is also the order of dependency. If a dependent component does not deploy successfully, a later component may not function correctly. Oracle WebCenter application startup is not dependent of the availability of external services such as the Oracle Discussions server, or other back-end servers.

6.3.2.3 Deploying Oracle WebCenter Application on a Cluster

For an Oracle WebCenter cluster deployment, such as the one shown in Figure 6-5, follow these rules for the targeting of applications, libraries, and system resources:

• Target applications and libraries to the cluster target. For example, target the WebCenter application to the Spaces cluster.

• JDBC resources to the cluster target.

Oracle WebCenter applications and the Oracle WebCenter Discussions server are deployed as Oracle WebLogic stage applications. During the initial deployment of each application, the deployment files are received from the Administration Server and deployed locally.

Cluster Communications

By default, each Oracle WebCenter cluster is configured as unicast. To configure your Oracle WebCenter cluster for multicast, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

6.3.2.4 Oracle WebCenter Analytics Communications

Oracle WebCenter Analytics consists of individual Collectors which receive requests from Oracle WebCenter Analytics clients, such as Oracle WebCenter Spaces.

This section describes how an Oracle Analytics Collector cluster works.

In a clustered environment, producers (clients of the collector) and collectors are configured with a cluster-specific channel name. Each collector periodically broadcasts a heartbeat with its location to the cluster-specific channel. The producer listens to the channel for these collector heartbeats, and when it hears one, adds the collector to its list of known collectors. When the producer needs to send an event, it uses a round robin algorithm to select a collector from its list and sends the event to that collector. If a collector stops (either being stopped purposefully or failing), it stops broadcasting a heartbeat. When the producer stops hearing the heartbeat it removes the collector from its list and stops sending events to that collector. If the producer does not hear any collector heartbeats, it does not send any events.

Oracle WebCenter Analytics uses UDP and multicast on a configured set of ports to communicate with its clients. For a single node setup, the client is configured with a WLST command that has the server host/port location and simply transmits all events to this location via UDP. For a multiple node setup, the server is configured to broadcast (via UDP multicast) the location(s) of the various servers running on the cluster; the client is configured with the same WLST command, so it will receive the locations of these servers and keep the list of available servers in a list (which is persisted in memory.) Additionally, if a client does not receive a periodic heartbeat from a server to indicate it is functioning, it will remove that server from its list of known servers, and will cease sending events to it.

Oracle recommends that Collectors be configured as unicast, forming a 1-1 relationship with the Oracle WebCenter Analytics client.

6.3.2.5 Oracle WebCenter State Replication

Oracle WebCenter relies on Oracle ADF, which has several stateful components. Oracle WebCenter itself is also a stateful application. Therefore, you must configure state replication in cluster scenarios.

For more information on how state replication works in Oracle WebLogic Server, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

Oracle WebLogic Server supports two types of state replication:

• In-memory replication - Using in-memory replication, Oracle WebLogic Server copies a session state from one server instance to another. The primary server creates a primary session state on the server to which the client first connects, and a secondary replica on another WebLogic Server instance in the cluster. The replica is kept up to date so that it may be used if the server that hosts the servlet fails.

• JDBC-based persistence - In JDBC-based persistence, Oracle WebLogic Server maintains the HTTP session state of a servlet or JSP using file-based or database-based persistence.

Properly configuring state replication requires both configuring the environment and configuring the application properly. For information on state replication behavior under failover conditions see Section 6.3.2.8, "Expected Behavior for Application Failover."

For more information on diagnosing state replication issues see Section 6.3.2.5, "Oracle WebCenter State Replication."

6.3.2.6 Understanding the Distributed Java Object Cache

Oracle WebCenter applications use a distributed Java Object Cache for greater performance. Configure this cache across all Oracle WebCenter clusters, with one distributed cache per cluster.

Table 6-6 lists some examples of the type of objects which Oracle WebCenter places in the Java Object Cache.

Collaboration

Collaboration services cache objects in the Java Object Cache on a per user session basis. These cached user sessions are destroyed when the HTTP session is destroyed.

Worklist

Worklist caches the called items so that unless the refresh button is clicked, or the every fifteen minute refresh poll is triggered, the same items are displayed as the user changes the sort and group by settings. The display is also cached by group and sort order settings, updating when a change occurs. This is read only data which, if not present, is fetched and stored on the cache.

Oracle WebCenter Spaces

The list of all templates and public spaces are maintained in the Java Object Cache. This is shared by all users, in addition to the list of group spaces and templates that a particular user can access. These are cached per user. A template created on one JVM does not show up if the template cache has been initialized in another JVM, unless JOC distributes the data, or an administrator in the second JVM does an explicit refresh where the cache is rebuilt.

Documents Service

The Documents service uses Java Object Cache to cache provisioning and configuration checks for document services, as applies to Oracle WebCenter Spaces application configuration. For provisioning, cached objects are flagged as distributed. They are replicated by a correctly configured Java Object Cache in a high availability environment, the configuration cached state is kept locally. All cached objects are flagged to expire after one minute. Caching reduces the number of times UCM calls are made to check the state of the UCM Server, as Oracle WebCenter Spaces repeatedly checks provisioning and configuration to control service rendering in the UI.

Portlet Consumer

The portlet consumer caches portlet mark-up in the Java Object Cache according to the cache headers in the portlet response. The cache can be expires based or validation based.

Profile Management caches the lightweight user profile objects in Java Object Cache. If particular profile data is not found, it will queried from the backend and the cache would be populated. The number of objects stored have an upper bound of 1000 and are in the Java Object Cache for one hour.

Navigation caches the Navigation Model objects in the Java Object Cache on a per user session basis. These cached objects are destroyed when the HTTP session is destroyed.

Recent Activity

The list of recent activity results are cached for each user to prevent a requery of results each time the recent activity task flow or RSS feed is viewed. The cache is automatically refreshed every fifteen minutes, or can be manually refreshed using the refresh icon in the recent activity task flow.

For Java Object Cache configuration procedures, see Section 6.4.13, "Configuring the Java Object Cache."

6.3.2.7 Oracle WebCenter Protection from Failover and Expected Behavior

An Oracle WebLogic cluster provides application high availability. If one member of the cluster is unavailable, any other available member of the cluster is able to handle the request.

Session Failover Requirements

For seamless failover of an Oracle WebCenter application, the application must meet the following conditions:

• The application is in a cluster and at least one member of the application cluster is available to serve the request.

• For stateful applications, state replication is configured correctly as described in Section 6.4.15, "Configuring Oracle WebCenter for Replication."

• If you are using Oracle HTTP Server, the server configuration is configured with the WebLogicCluster directive to balance among all available application instances.

• If you are using a hardware load balancer, the load balancer is:

  • Routing traffic to all available instances

  • Configured correctly with a health monitor to mark unavailable instances

  • Configured to support persistence of session state

6.3.2.8 Expected Behavior for Application Failover

If the environment has been configured correctly, application users do not notice when an application instance in a cluster becomes unavailable. The sequence of events in an application failover is, for example, as follows:

1. A user makes a request and is routed by a hardware load balancer to instance A of the application.

2. Instance A of the application becomes unavailable because of node failure, process failure, or network failure.

3. The hardware load balancer marks instance A as unavailable.

4. The user makes a subsequent request. The request is routed to instance B.

5. Instance B is configured as a replication partner of Instance A and has the user's session state.

6. The application resumes using the session state on Instance B and the user continues working without interruption.

Exceptions to Expected Behavior

For Oracle WebCenter, the known exceptions to these expected behaviors are as follows:

• Oracle ADF Pop-ups - Open pop-ups are closed on failover. This affects many components which otherwise have no exceptions. The following components are affected:

  • Oracle Composer (property inspector popup)

  • Lists

  • Links (link deletion popup)

  When failover occurs, the action which led to the popup must be repeated in order to make it reappear. The specific ways in which this appears in WebCenter Spaces, as well as suggested remedies are listed in Table 6-7.

  Table 6-7 Oracle WebCenter Troubleshooting Scenarios

  Action Before Failover	After Failover	Suggested Remedy
  Go to any page and click Create Page.	Type in a name, select a theme, and click OK. When you select the theme, the page creation popup is closed.	Repeat the operation.
  Launch Manage Pages.	Perform any operation within the popup, except for closing the popup, for example, click Page Actions. When you perform any operation, the Manage Pages popup is closed.	Repeat the operation.
  Launch Manage Pages, click Page Actions against a page, and then the Delete option in the menu.	Click OK on the confirmation popup. Clicking OK not only closes the confirmation popup, but the Manage Pages popup also gets closed as part of the request, and the effect of the deletion (which may have gone through) is not visible among the tabs.	Relaunch the Manage Pages popup to see if the page has been deleted. If not, try deleting once again.
  Launch Personalize Applications popup. Perform any operation that sends a request, other than clicking OK, for example, expand the Applications node.	Perform any operation that sends a request, other than clicking OK, for example, expand the Applications node. When you perform any operation that sends a request to the server, the Personalize Applications popup is closed.	Repeat the operation
  Launch Preferences.	Switch between the Preferences tabs (General, Accounts, Messaging, Search). When you switch between the Preference tabs, the Preferences popup is closed.	Repeat the operation
  Launch Manage Favorites. Stop the server, perform any operation other than closing the popup, for example. expand a folder, click Edit favorite information.	Perform any operation other than closing the popup, for example, expand a folder, and click Edit favorite information. When you perform any operation, the Manage Favorites popup is closed.	Relaunch the Manage Favorites popup to see if the operation was successful. If not, retry the operation.
  Choose to edit applications and create a new folder	An MDS exception displays.	Retry the operation

  
  

• Component Specific Issues - Other issues which are specific to different components in Oracle WebCenter are listed in Table 6-8.

  Table 6-8 Oracle WebCenter Exceptions to Expected Behavior

  Oracle WebCenter Component	Exceptions to Expected Behavior
  Group Space Events	When changing certain fields (start or end date/hour/minute), the event form is closed when failover occurs.
  Portlets	Failures are fully transparent.
  Lists	Failures are fully transparent.
  Links	Failures are fully transparent.
  Search	In the midst of a long-running query. The results that come back are not guaranteed (note there is no "write" operation here), the user can just reissue the query.
  Tagging	Failures are fully transparent.
  Recent Activities	The open/close state of the tree nodes is not replicated. After after failover, the tree of results closes all of the nodes. The nodes need to be reopened.
  Worklist	Failures are fully transparent.
  Document Manager	When a user uploads a document and a document with the same name already exists, the user is taken to the Confirm new version screen. While on that screen, the file is stored in a temporary local location. If failover occurs at that moment, the uploaded file is lost and you must restart the upload process.

  
  

6.3.2.9 Monitoring Logging of Application Deployments

Use Oracle WebLogic Server Administration Console to check the status of the application deployments. You can also use Oracle WebLogic Server infrastructure and Enterprise Manager Fusion Middleware Control for starting stopping, and monitoring Oracle WebCenter processes.

6.3.2.10 Oracle WebCenter Cluster-wide Configuration Changes

For Oracle WebCenter Spaces and Portlet Producers, all configuration data is stored in the MDS repository and Portlet Producers. Additional cluster deployments automatically read the latest configuration upon application startup.

For Oracle Discussion Server, the configuration information should be moved over from an existing cluster server. This is done automatically by the pack/unpack utility of Oracle WebLogic Server. Oracle recommends this procedure.

6.3.2.11 Maintaining Configuration in a Clustered Environment

For Oracle WebCenter Spaces and Portlet Producers, all configuration data is stored in the MDS repository and Portlet Producers. Any changes made to the configuration of one server in the cluster are immediately visible to all other members.

Changes to Oracle WebCenter Discussion Server are not frequent, however, when they do occur, the changes must be reapplied to other members of the cluster. You can do this by connecting directly to each discussions server, instead of using the load balancer, and making the necessary administration changes.

6.4.1.1 Database Prerequisites

Oracle WebCenter requires the presence of a supported database and schemas.

To check if your database is certified or to see all certified databases, refer to the "Certified Databases" section in the Certification Document:

http://www.oracle.com/technology/software/products/ias/files/fusion_certification.html

To determine the database version, execute the following query:

SQL>select version from sys.product_component_version where product like 'Oracle%';

6.4.1.2 VIP and IP Prerequisites

To configure a virtual IP for the Administration Server, see Section 12.2.2.3, "Transforming the Administration Server for Cold Failover Cluster."

6.4.1.3 Installing and Configuring the Database Repository

This section describes how to install and configure the database repository.

Oracle Clusterware

• For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

• For 11g Release 1 (11.1) and later, see Oracle Clusterware Installation Guide.

Automatic Storage Management

• For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

• For 11g Release 1 (11.1) and later, see Oracle Real Application Clusters Installation Guide.

• When you run the installer, select the Configure Automatic Storage Management option in the Select Configuration page to create a separate Automatic Storage Management home.

Oracle Real Application Clusters

• For 10g Release 2 (10.2), see Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide.

• For 11g Release 1 (11.1) and later, see Oracle Real Application Clusters Installation Guide.

You must install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database before you install the Oracle Fusion Middleware components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database. You install RCU in its own, separate Middleware home.

Use the latest version of RCU to install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database.

See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about obtaining and running the latest version of RCU.

Database Initialization Parameters

Ensure that the following initialization parameter is set to the required value. It is checked by Repository Creation Assistant.

To check the value of the initialization parameter using SQL*Plus, you can use the SHOW PARAMETER command.

As the SYS user, issue the SHOW PARAMETER command as follows:

SQL> SHOW PARAMETER processes

Set the initialization parameter using the following command:

SQL> ALTER SYSTEM SET processes=300 SCOPE=SPFILE

Restart the database.

6.4.1.4 Installing and Configuring an LDAP Provider

For production environments, it is a mandatory requirement for Oracle WebCenter high availability topologies to have an external LDAP policy store. For more information on the supported policy stores, as well as instructions on configuring LDAP, see the Oracle Fusion Middleware Administrator's Guide for Oracle WebCenter.

6.4.1.5 Terminology for Directories and Directory Environment Variables

The follow list describes the directories and variables used in this chapter:

• ORACLE_BASE: This environment variable and related directory path refers to the base directory under which Oracle products are installed.

• MW_HOME: This environment variable and related directory path refers to the location where Fusion Middleware (FMW) resides.

• WL_HOME: This environment variable and related directory path contains installed files necessary to host a WebLogic Server.

• ORACLE_HOME: This environment variable and related directory path refers to the location where Oracle WebCenter is installed.

• ORACLE_COMMON_HOME: This environment variable and related directory path refers to the Oracle home that contains the binary and library files required for the Oracle Enterprise Manager Fusion Middleware Control and Java Required Files (JRF).

• DOMAIN_HOME: This directory path refers to the location where the Oracle WebLogic Domain information (configuration artifacts) is stored.

The values used and recommended for consistency for this directories are:

ORACLE_BASE:

/u01/app/oracle

MW HOME (Apptier):

ORACLE_BASE/product/fmw

WL_HOME:

MW_HOME/wlserver_10.3

ORACLE_HOME:

MW_HOME/WC1

ORACLE_COMMON_HOME:

MW_HOME/oracle_common

6.4.1.6 Using Oracle Fusion Middleware Repository Creation Utility to Load the Fusion Middleware Schemas in the Database

Install the 11g (11.1.1) Oracle Fusion Middleware Metadata Store and WebCenter schemas into a Real Application Cluster database before you install Oracle Fusion Middleware WebCenter components. Oracle Fusion Middleware provides a tool, Oracle Fusion Middleware Repository Creation Utility (RCU), to create the component schemas in an existing database.

Use the latest version of RCU to install the 11g (11.1.1) Oracle Fusion Middleware Repository into a Real Application Clusters database.

See Oracle Fusion Middleware Repository Creation Utility User's Guide for more information about obtaining and running the latest version of RCU.

6.4.2.1 Validating Oracle HTTP Server

To verify that Oracle HTTP Server is set up correctly, access the root URL context of the server by using the following URL a browser:

WebHost1:7777/

If Oracle HTTP Server is set up correctly, the Hello World page appears in the browser.

6.4.3.1 Installing Oracle WebLogic Server

See "Understanding Your Installation Starting Point" in Oracle Fusion Middleware Installation Planning Guide for the version of Oracle WebLogic Server to use with the latest version of Oracle Fusion Middleware.

To install Oracle WebLogic Server on all nodes in the application tier:

1. On UNIX platforms, if the /etc/oraInst.loc or /var/opt/oracle/oraInst.loc file exists, check that its contents are correct. Specifically, check that the inventory directory is correct and that you have write permissions for that directory.

   If the /etc/oraInst.loc file does not exist, skip this step.

2. Start Oracle WebLogic Server Installer.

3. In the Welcome screen, click Next.

4. In the Choose Middleware Home Directory screen:

   • Select Create a New Middleware Home

   • For the Middleware Home Directory field, enter MW_HOME

   • Click Next.

5. In the Register for Security Updates screen, enter your contact information for security update notifications, and click Next.

6. In the Choose Install Type screen, select Custom, and click Next.

7. In the JDK Selection screen, select only Oracle JRockit 1.6.0_14 SDK, and click Next.

8. In the Choose Product Installation Directories screen, accept the following directory:

   WL_HOME

   Click Next.

9. In the Installation Summary screen, click Next.

10. In the Installation Complete screen, deselect Run QuickStart, and click Done.

6.4.3.2 Installing Oracle Fusion Middleware for Oracle WebCenter

To install Oracle Fusion Middleware for Oracle WebCenter on all the nodes in the application tier:

1. Start Oracle Fusion Middleware for Oracle Fusion Middleware 11g WebCenter Installer:

   On UNIX, (Linux used in this example):

   APPHOST1> runInstaller
   

   On Windows:

   APPHOST1> setup.exe
   

   When Oracle Fusion Middleware 11g WebCenter Installer prompts you for a JRE/JDK location enter Oracle SDK location created in the Oracle WebLogic Server installation, for example, MW_HOME/jrockit_160_14_R27.6.4-18.

2. In the Welcome screen, click Next.

3. In the Prerequisite Check screen, verify that the checks complete successfully, and click Next.

4. In the Specify Installation Location screen:

   • For Middleware Home, enter MW_HOME

   • For Oracle Home Directory, enter the directory you want to use, for example, wc

   Click Next.

5. In the Installation Summary screen, click Install.

6. In the Installation Complete screen, click Finish.

6.4.7.1 Starting the Administration Server on APPHOST1

To Start the Administration Server on APPHOST1 run the following commands:

APPHOST1> cd ORACLE_BASE/product/fmw/user_projects/domains/wcdomain/bin

APPHOST1> ./startWebLogic.sh

6.4.7.2 Validating the Administration Server

To verify that the Administration Server is properly configured:

1. In a browser, go to http://VIP1:7001/console.

2. Log in as the administrator.

3. Verify that all the managed servers (WC_Spaces1, WC_Spaces2, and so on) are listed.

4. Verify that all Clusters are listed.

5. Verify that you can access Enterprise Manager at http://VIP1:7001/em.

6.4.7.3 Disabling Host Name Verification for the Administration Server and the Managed Servers for APPHOST1 and APPHOST2

This step is required if you have not set up SSL communication between the Administration Server and the Node Manager. If SSL is not set up, you receive an error message unless you disable host name verification.

You can re-enable host name verification when you have set up SSL communication between the Administration Server and the Node Manager.

To disable host name verification:

1. In Oracle WebLogic Server Administration Console, select Servers, and then AdminServer.

2. Select SSL, and then Advanced.

3. In the Change Center, click Lock & Edit.

4. When prompted, save the changes and activate them.

5. Set Hostname Verification to None.

6. Select WC_Spaces1, SSL, and then Advanced.

7. Set Hostname Verification to None.

8. Repeat Steps 6 and 7 for all the Managed Servers.

9. Restart the AdminServers and all the Managed Servers.

6.4.7.4 Starting Node Manager on APPHOST1

Perform these steps to start Node Manager on APPHOST1:

1. Run the setNMProps.sh script, which is located in the ORACLE_COMMON_HOME/common/bin directory, to set the StartScriptEnabled property to true before starting Node Manager:

   APPHOST1> cd ORACLE_COMMON_HOME/common/bin
   APPHOST1> ./setNMProps.sh
   

   Note:

   You must use the StartScriptEnabled property to avoid class loading failures and other problems.

2. Start Node Manager:

   APPHOST1> cd WL_HOME/server/bin
   APPHOST1> ./startNodeManager.sh
   

6.4.11.1 Configuring a Virtual Host for Oracle Pagelet Producer and Sharepoint

The WebCenter suite includes two applications that use / as the web context root:

• Sharepoint Root - deployed on WC_Spaces

• Pagelet Producer Root - deployed on WC_Portlet

<Location  />
    SetHandler weblogic-handler
      WebLogicHost webcenter.example.com
      WebLogicPort 8889
</Location>

6.4.11.2 Validating Access through Oracle HTTP Server

Verify the URLS to ensure that appropriate routing and failover is working from the HTTP Server to Oracle WebCenter cluster.

1. Start WC_Spaces1, WC_Spaces2, WC_Portlet1 and WC_Portlet2 from the WebLogic Server Administration Console as follows:

   1. Access the Administration Console at the following URL

      http://APHHOST1/console

   2. Click Servers.

   3. Open the Control tab.

   4. Select WC_Spaces1, WC_Spaces2, WC_Portlet1 and WC_Portlet2.

   5. Click Start.

   6. Verify direct access to the managed servers using the following URLs:

      apphost1:8888/webcenter

      apphost2:8888/webcenter

      apphost1:8889/portalTools

      apphost2:8889/portalTools

2. While WC_Spaces2 and WC_Portlet2 are running, stop WC_Spaces1 and WC_Portlet1 from Oracle WebLogic Server Administration Console.

3. Access the following URLs and verify the appropriate functionality:

   • WebHost1:7777/webcenter

   • WebHost1:7777/portalTools

4. Start WC_Spaces1 and WC_Portlet1 from the WebLogic Server Administration Console.

5. Stop WC_Spaces2 and WC_Portlet2.

6. Access the following URLs and verify the appropriate functionality:

   • WebHost1:7777/webcenter

   • WebHost1:7777/portalTools

6.4.16.1 Configure the WebCenter Spaces Servers

1. Open the WLST shell:

   ORACLE_HOME/common/bin/wlst.sh
   

2. Connect to WLS Server:

   connect('weblogic_admin_username', 'weblogic_admin_pwd', 'APPHOST1:8888')
   

   Note that you are connecting to the host and port of the Spaces Server.

3. Create the Analytics Collector connection and make it the default connection:

   createAnalyticsCollectorConnection('webcenter','HAConn1',isUnicast=1,
   collectorHost='localhost',collectorPort=31314,isEnabled=1,timeout=30,default=1)
   

4. List the changes made:

   listDefaultAnalyticsCollectorConnection('webcenter')
   

6.4.19.1 Scaling Up the Topology (Adding Managed Servers to Existing Nodes)

In this case, you already have a node that runs a managed server configured with WebCenter components. The node contains a Middleware home and a WebCenter directory in shared storage.

You can use the existing installations (Middleware home, and domain directories) for creating new Oracle WebCenter managed servers. There is no need to install WebCenter binaries in a new location, or to run pack and unpack. Running multiple managed servers on one node is only supported for the WC_Spaces servers and the WC_Portlet servers.

Follow these steps for scaling up the topology:

1. Using the Administration Console, clone WC_Spaces1 or WC_Portlet1 into a new managed server. The source managed server to clone should be one that already exists on the node where you want to run the new managed server.

   To clone a managed server:

   1. Select Environment -> Servers from the Administration Console.

   2. Select the managed server that you want to clone (for example, WC_Spaces1 or WC_Portlet1).

   3. Select Clone.

   Name the new managed server SERVER_NAMEn, where n is a number to identify the new managed server.

2. For the listen address, assign the host name or IP to use for this new managed server.

   Ensure the port number for this managed server is available on this node.

3. Add the new managed server to the Java Object Cache Cluster (see Section 6.4.13, "Configuring the Java Object Cache").

4. Reconfigure the Oracle HTTP Server module with the new member in the cluster. See Section 6.4.11, "Configuring Oracle HTTP Server for the Administration Server and Oracle WebCenter Managed Servers."

6.4.19.2 Scaling Out the Topology (Adding Managed Servers to New Nodes)

In scaling out your topology, you add new managed servers configured with Oracle WebCenter applications to new nodes.

Before performing the steps in this section, check that you meet these requirements:

• In your topology, there are existing nodes running managed servers configured with WebCenter applications.

• The new node can access the existing home directories for WebLogic Server and Oracle WebCenter. You use the existing installations in shared storage for creating a new managed server. There is no need to install WebLogic Server or WebCenter binaries in a new location, although you need to run pack and unpack to create a managed server domain.

Follow these steps for scaling out the topology:

1. On the new node, mount the existing Middleware home, which should include the WebCenter installation and the domain directory, and ensure that the new node has access to this directory, just like the rest of the nodes in the domain.

2. To attach ORACLE_HOME in shared storage to the local Oracle Inventory, execute the following commands:

   WCHOSTn> cd ORACLE_BASE/product/fmw/wc/
   WCHOSTn> ./attachHome.sh -jreLoc ORACLE_BASE/fmw/jrockit_160_20_D1.0.1-2124
   

   To update the Middleware home list, create (or edit, if another WebLogic installation exists in the node) the MW_HOME/bea/beahomelist file and add ORACLE_BASE/product/fmw to it.

3. Log into the Oracle WebLogic Administration Console.

4. Create a new machine for the new node that will be used, and add the machine to the domain.

5. Update the machine's Node Manager's address to map the IP of the node that is being used for scale out.

6. Use the Oracle WebLogic Server Administration Console to clone either WC_Spaces1 or WC_Portlet1 or WC_Collaboration1 or WC_Utilities1 into a new managed server. Name it WLS_XXXn, where n is a number and assign it to the new machine.

7. For the listen address, assign the host name or IP to use for the new managed server. Perform these steps to set the managed server listen address:

   1. Log into the Oracle WebLogic Server Administration Console.

   2. In the Change Center, click Lock & Edit.

   3. Expand the Environment node in the Domain Structure window.

   4. Click Servers. The Summary of Servers page is displayed.

   5. Select the managed server whose listen address you want to update in the Names column of the table. The Setting page for that managed server is displayed.

   6. Set the Listen Address to WCHOSTn where WCHOSTn is the DNS name of your new machine.

   7. Click Save.

   8. Save and activate the changes.

   The changes will not take effect until the managed server is restarted.

8. Run the pack command on WCHOST1 to create a template pack as follows:

   WCHOST1> cd ORACLE_COMMON_HOME/common/bin
   WCHOST1> ./pack.sh -managed=true -domain=MW_HOME/user_projects/
   domains/wcdomain/ -template=webcenterdomaintemplateScale.jar
   -template_name=webcenter_domain_templateScale
   

   Run the following command on WCHOST1 to copy the template file created to WCHOSTn:

   WCHOST1> scp wcdomaintemplateScale.jar oracle@WCHOSTn:/ ORACLE_BASE/product/
   fmw/wc/common/bin
   

   Run the unpack command on WCHOSTn to unpack the template in the managed server domain directory as follows:

   WCHOSTn> cd ORACLE_BASE/product/fmw/wc/common/bin
   WCHOSTn> ./unpack.sh -domain=ORACLE_BASE/product/fmw/user_projects/domains/
   wcdomain/ -template=wcdomaintemplateScale.jar
   

9. Start the Node Manager on the new node. To start the Node Manager, use the installation in shared storage from the existing nodes, and start Node Manager by passing the host name of the new node as a parameter as follows:

   WCHOSTn> WL_HOME/server/bin/startNodeManager <new_node_ip>
   

10. If this is a new Collaboration managed server, ensure that you have followed the steps in Section 6.4.18, "Configuring Clustering for Discussion Server" to configure clustering for the new Discussion Server. Ensure also that the steps in Section 6.4.21, "Converting Discussions from Multicast to Unicast" are performed, using the hostname of the new host for the coherence.localhost parameter.

11. If this is a new Utilities managed server, ensure that Activity Graph is disabled by following the steps in Section 6.4.17, "Configuring Activity Graph." Ensure also that the steps for configuring a new Analytics Collector in Section 6.4.16, "Configuring the Analytics Collectors" have been followed.

12. Start and test the new managed server from the Oracle WebLogic Server Administration Console:

    1. Shut down all the existing managed servers in the cluster.

    2. Ensure that the newly created managed server is running.

    3. Access the application on the newly created managed server. The application should be functional.

6.4.20.1 Troubleshooting Oracle WebCenter Deployment Issues

WebCenter Applications are deployed when the managed server is first started. Use Oracle WebLogic Server Administration Console first to check that all application deployments were successful:

Click Deployments in the left hand pane. The right hand pane shows the application deployments and their status. The state of all applications, assuming all the servers are running, should be ACTIVE.

If an application deployment has failed, the server logs may provide some indication of why the application was not deployed successfully. The server logs are located in the DOMAIN_HOME/servers/SERVER_NAME/logs directory. Common issues include:

• Unavailability of external resources, such as database resources. Examine the error, fix it, and attempt to redeploy the application.

• The appropriate applications or libraries are not targeted correctly to the right managed server or Cluster.

6.4.20.2 Troubleshooting Oracle WebCenter Replication and Failover Issues

State Replication is most prominent in failover scenarios. A user working on one server may discover that, upon failover:

• Windows may be closed or state might be reset.

• Screens may require a reset.

• The application may be redirecting to the logon screen.

The following steps provide guidance in troubleshooting and diagnosing state replication issues.

1. Confirm that this is not a known replication issue.

   See Section 6.3.2.8, "Expected Behavior for Application Failover" for possible expected behaviors. Before proceeding to further diagnose the issue, first confirm that the failover behavior is not an expected behavior.

2. Check load balancer settings.

   For replication and failover to function correctly, the load balancer must be configured with the appropriate persistence settings. For more details on configuring Hardware Load Balancers for Oracle WebLogic Server, see Oracle Fusion Middleware Using Clusters for Oracle WebLogic Server.

3. Check the cluster status.

   Replication occurs within the context of a cluster. For failover to be successful, there must be at least one other healthy member of the cluster available. You can check luster status in one of two ways:

   • Check the cluster status using Oracle WebLogic Server Administration Console - In the Left-hand pane, click on Servers. Verify the state of all servers in the cluster.

   • Check the cluster status using weblogic.Admin utility The weblogic.Admin command can be used to query the state of all servers in a specific cluster. For example:

     $ java weblogic.Admin -url Adminhost:7001 -username <username> -password
      <password>  CLUSTERSTATE -clustername Spaces_Cluster
     

     This example returns:

     There are 2 server(s) in cluster: Spaces_Cluster
     The alive servers and their respective states are listed below:
     WC_Spaces---RUNNING
     WC_Spaces2---RUNNING
     

4. Check cluster communications.

   Although Cluster members may all be running, there may be communication issues which prevent them from communicating replication information to each other. There are two types of cluster communication configurations. Troubleshooting depends on the cluster type:

   • Checking Unicast cluster communications - For Unicast clusters, managed servers must be able to access each other's hosts and each other's default listening port.

     Ensure that all individual managed servers have their Listen Address set correctly. You can find this setting by selecting Configuration, General for each managed server.

   • Checking Multicast cluster communications - For multicast clusters, servers must be able to intercept the same multicast traffic. Ensure that multicast is configured correctly by running the WebLogic utility utils.MulticastTest on each machine. For example:

     $ java utils.MulticastTest -H
     

5. Confirm application configuration.

   WebCenter applications are configured correctly by default. For user applications, in-memory replication take place only with the proper configuration in weblogic.xml:

   <session-descriptor>
   <persistent-store-type>replicated_if_clustered</persistent-store-type>
   </session-descriptor>
   

   A persistent-store-type of replicated is also acceptable.

6. Enable Debug.

   Check the server logs for any unusual messages on managed server startup. In particular, if the managed server is unable to locate other members of the cluster. The server logs are located in the DOMAIN_HOME/servers/SERVER_NAME/logs directory.

   For further debugging, enable the flags DebugCluster, DebugClusterAnnouncements, DebugFailOver, DebugReplication, and DebugReplicationDetails. Each flag can be enabled with the weblogic.Admin utility:

   $ java weblogic.Admin -url Adminhost:7001 -username <username> -password
    <password> SET -type ServerDebug -property DebugCluster true
   

6.4.20.3 Troubleshooting Lost Changes to Policies

Policies are refreshed at an interval defined in jps-config.xml by the variable oracle.security.jps.ldap.policystore.refresh.interval. By default, this interval is ten minutes. If a server is lost, requests are routed to another server in the cluster where the original policies are cached. Recent policy changes that occurred since the last refresh may appear to be lost. For example, roles created immediately before the failover may be seen as unavailable.

The policies are still in the back-end policy store. Refresh the cache to restore policy information. Re-try after a few minutes, or log in and log out of the application to force a policy cache refresh.

6.4.20.4 Troubleshooting JOC Configuration

If attempts to start a Managed Server that uses the Java Object Cache, such as OWSM or WebCenter Spaces Managed Servers, fail, and the following errors appear in the logs:

J2EE JOC-058 distributed cache initialization failure
J2EE JOC-043 base exception:
J2EE JOC-803 unexpected EOF during read.

Another process is using the same port that JOC is attempting to obtain. To solve this problem, either stop that process, or reconfigure JOC for this cluster to use another port in the recommended port range.