Section - 4 : Horizontal Cluster Node Installation


Note: 1. Make sure the JDK (v181+) and WebLogic Infrastructure, SOA, and OSB are installed for all the nodes that are used to run the scripts.
Note: 2. Make sure same JDK Home & Oracle Home are maintained for all the nodes.
Note: 3. Make sure all the nodes are installed with Chef (v13.3.31).

If you selected ‘3’ in the following screen:

Enter the details as shown in the following sections:

Linux User/Group Section

  1. Linux Username

    Example: oracle (the user you installed Fusion Middleware with)

  2. FMW Products Installed Group

    Example: oinstall or oracle (the user group you installed Fusion Middleware with)

  3. Second Node Host name

    Note: JDK version and JAVA_HOME should be same for all the nodes

Database Section

OIDG and RCU schemas are created in the same database instance.

IS OIDG Schema and RCU Infra Schemas were created on the Same Database/Pluggable Database (y/n): y

Enter the following values to proceed further:

Database hostname: Database hostname
Database service name: DB Instance name
Database port: Listener port (Example: 1521)
Database OIDG User name: ‘OIDXAPPUSER’
RCU Infra Schema Prefix: RCU Schema’s Prefix

OIDG and RCU Schemas are created in Different database instances.

IS OIDG Schema and RCU Infra Schemas were created on the Same Database/Pluggable Database (y/n): n

Enter the DB inputs:

RCU Database hostname: Database hostname
RCU Database service name: DB Instance name
RCU Database port: Listener port (Example: 1521)
RCU Infra Schema Prefix: RCU Schema’s Prefix
OIDG Database User name: ‘OIDXAPPUSER’
OIDG Database hostname: OIDG Database hostname
OIDG Database service name: OIDG DB Instance name
OIDG Database port: OIDG Listener port (Example: 1521)

Middleware Section

OIDG Fusion Middleware HOME Path: Enter the middleware home path including Oracle_Home

OIPA Input Section

Provide the OIPA server inputs

Confirmation Section

Correct? (y/n) : (y/n)

Credentials Section

WebLogic Credentials

  • Set the WebLogic User Password

    • You have to use this password for WebLogic login after domain creation.

Database Credentials

  • Set the RCU Password for all Schemas

    Provide the common password used for the RCU Infra Schemas

  • Set the password for OIDG's schema

    Provide the password for the OIDXAPPUSER.

GnuPGCredentials

  • Set the password for PGP keys

    Set the password for PGP Key creation for Encryption/Decryption

Security Realm user Credentials

Note: Users OIDG User and PII are created for configuring Security Credentials in EM and Encryption of PII Data

  • OIDG user Password:

    Set the password for user: oidguser

  • PII Password:

    Set the password for user: oidgpii

Confirmation

Please verify and resolve any errors that have occurred before running sh install_aml_oidg_cluster_primary.sh..

OIDG Installation Manual Steps

This section includes following topics:

  • Defining JNDI Providers
  • Creating Security Policies

Defining JNDI Providers

JNDI Provider resources are required by OSB components to locate and communicate with Enterprise Java Beans (EJB) components.

  1. Log on to the Service Bus console (http://hostname:port/sbconsole), where “hostname:port” are the host name and port of your administration server.

  2. In the project explorer, navigate to All Projects → System → JNDI Providers.

    Verify that DataServices_JNDIProvider is defined.

    If it is defined, skip steps 3-7. If not, continue with the rest of the steps.

  3. In the upper left corner of the page, click Create to begin a new update session.

  4. Right-click the JNDI Providers folder to display the context menu, then click Create → Create JNDI Provider.

    Image represents creating JNDI provider.
    Figure 3.4.1 - Create JNDI Provider
  5. In the Create JNDI dialog, enter “DataServices_JNDIProvider” for Resource Name. Click Create.

    Figure 3.4.2 - Create JNDI Dialog
  6. On the JNDI Definition page, fill in the following fields:

    1. Provider URL – the host name or IP and port of the managed server to which OIDX_POC_DSL is targeted (Example: “t3://<hostname of node1>,<hostname of node2>:<aml managed server port>")

    2. Initial Context Factory – select “weblogic.jndi.WLInitialContextFactory”

    3. User Name, New Password, and Confirm Password – the login credentials for the server. Usually credentials are the same as administration server login credentials.

      Image represents JNDI definition
      Figure 3.4.3 - JNDI Definition
  7. Click Save located on the upper-right corner of the page.

    Image represents Save icon
    Figure 3.4.4 - Save Icon
  8. Click Activate to reflect the changes after saving the details.

Creating Security Policies

  1. Login to the WebLogic Enterprise Manager (<hostname>:< Admin Server Port Number>/em) with WebLogic credentials.

  2. WebLogic Domain → Security → System Policies.

    Image represents web logic enterprise options
    Figure 3.4.5 - WebLogic Enterprise
  3. Under Search section, select Name as Includes.

    Image represents search section
    Figure 3.4.6 - Search Section
  4. Type “OIDX_POC_DSL” in the search box, and click the arrow icon.

    Image represents search box
    Figure 3.4.7 - Search Box
  5. Check to see if the search results include a row as shown below.

    Image represents search result area
    Figure 3.4.8 - Search Result Area
  6. If this policy does not exist, create it by following these steps:

    Image represents create like
    Figure 3.4.9 - Create Like
    1. Select the row that includes the text oracle.wsm.console.core.view, and then click Create Like.

    2. On the Create System Grant Like page, change the text in the Codebase box to the following:

      file:${oracle.deployed.app.dir}/OIDX_POC_DSL${oracle.deployed.app.ext}

    3. Select the CredentialAccessPermission row in the Permission Class table and click Edit.

    4. In the Edit Permission dialog change Permission Actions from read,write,delete,update to just read. Click Ok.

    5. Click Ok again.

Deploying ACORD_AML Libraries

In August 2017, ACORD stopped making the AML specification available to non-members. As a consequence of this and the associated licensing changes, OIDX cannot include the AML XSD as part of its deployments. The user must independently acquire a copy of the AML XSD distribution (ACORD Asia Pacific GI XML - 1.2.0 - Members.zip) and install the schema resources into the OIDX service virtualization deployment.

XSD Installation

Follow these steps:

  1. Log into the Service Bus Console (<hostname>:<Admin Server port number>/sbconsole).
  2. Create a new update session by clicking Create..
    Image represents importing ZIP file.
    Figure 3.4.10 - Oracle Service Bus Console
  3. Click Import.
    Image represents secelting zip file from your local drive.
    Figure 3.4.11 - Import
  4. Click Choose File and upload the file.
    Image represents imported heirarchy.
    Figure 3.4.12 - Choosing and uploading file
  5. Click Next (right arrow).
    Image represents select all except highlighted files.
    Figure 3.4.13 - Next button
  6. Click Import.
    Image represents the root aml folder structure.
    Figure 3.4.14 - Import button
  7. Click Close.

    Image represents the root aml folder structure.
    Figure 3.4.15 - Close button

OIDG Post-Script Installation

After completing the steps in 'OIDG Installation Manual Steps' topic run the following command from root to complete the installation.

Command: sh install_alm_oidg_cluster_primary_post.sh (from root)

Please check the status of Chef Script: Finished (or) Failed

Note: Script run usually takes about 20-30 minutes to complete all the deployments.

Note: After Post Script Installation follow Extension Deployment steps. Please refer to “OIDG Extension Deployment” and “OIDG Extension installation using Chef scripts” sections.

Coherence Cache Configuration

  1. Place ‘oidx_cache_distributed_config.xml' to ‘Oracle/Middleware/oidg/config’ to the both master and slave machines copying it from build’s ‘DataSrv’ folder.
  2. Login to WebLogic Console (“<hostname>:<AdminServerport>/console”).
  3. Click lock & Edit.
  4. Expand environments and select Coherence Clusters.
  5. Click New.
  6. Environment Name as defaultCoherenceCluster.
  7. Click Next.
  8. Select cluster node as Unicast.
  9. Select cluster listen Port as 8085.
  10. Select Multi cast Listen Address as 224.3.6.0.
  11. Click Save.
  12. Go to Cache Configurations > click New.
  13. Enter name as oidx_cache.
  14. Enter JNDI Name as oidx_cache.
  15. Enter Cache Configuration File location (Distributed Cache file Location).
  16. Ex : oidx_cache_distributed_config.xml

  17. Click Next.
  18. Select aml_cluster,soa_cluster and osb_cluster as Target and click Save.
  19. Click Activate changes button.
  20. Select CoherenceAdapter jar.
  21. Select AML_Server1 managed server and navigate to 'Server Start' tab.
  22. Add the below Arguments to the server start arguments:
    1. -Dtangosol.coherence.distributed.localstorage=true -Dtangosol.coherence.cluster=defaultCoherenceCluster -Dtangosol.coherence.clusteraddress=< Multicast Listen Address >
    2. -Dtangosol.coherence.clusterport=< Cluster listen Address>
  23. Select second node AML_Managed_Server and navigate to the 'Server Start' tab.
  24. Add the below Arguments to the server start arguments:
    1. -Dtangosol.coherence.distributed.localstorage=false-Dtangosol.coherence.cluster=defaultCoherenceCluster -Dtangosol.coherence.clusteraddress=< Multicast Listen Address >
    2. -Dtangosol.coherence.clusterport=< Cluster listen Address>
  25. Note: Repeat the steps 22 and 23 if more than two nodes are being installed.

  26. Click Save and Activate the changes.

Update Coherence Cluster Well Known Addresses

  1. Log on to the Service Bus console (http://hostname:port/console), where “hostname:port” are the host name and port of your administration server.
  2. Click Lock & Edit.
  3. Expand Environments → Go to coherence cluster.
  4. Click defaultCoherenceCluster.
  5. Click Wellknown Addresses tab.
  6. Click New.
  7. Enter Name As WKA.
  8. Enter Listen Address as IP Address of Second Node machine.
  9. Note: These addresses depend on the number of node(s) selected.

    Figure 3.4.16 - Creating Well Known addresses
  10. Click Activate changes button.

Note:

Note: 1. Copy entire Chef Directory located in /scratch/Chef from primary node to all the secondary nodes and maintain same location.
Note: 2. Copy cluster_managed_servers.txt, and oidg_domain.jar located in /tmp/ from primary node to all the secondary nodes of /tmp/.
Note: 3. Copy Plan.xml located in /scratch/idxAdmin/Oracle/Middleware/Oracle_Home/soa/soa/ from primary node to all the secondary nodes of /tmp /.

Run install_aml_oidg_cluster_secondary.sh in all the secondary nodes one by one.

Run install_aml_oidg_cluster_primary_post_final.sh in the primary node.

WebLogic Performance tuning settings

This topic describes deployment settings that may need to be adjust depending on environmental factors. This application integrates with external services that may have requirements where these settings should be adjust for optimal performance. In addition, hardware, co-location, and networking may require adjustments to these settings.

Follow these steps:

  1. Update the ‘Stuck Thread Timer Interval’ value to 36000 AML server.

    Home Page → Environment → Servers → aml_server →(Update to available server1,server2,server3,…...,servern)→ Configuration → Tuning → Update ‘Stuck Thread Max Time’ and click Save.

    Image represents thread max time.

  2. Update the ‘Stuck Thread Max Time’ value to 1800 for OSB and SOA servers:

    Home Page → Environment → Servers→ OSB_server→(Update to available server1,server2,server3,…...,servern) → Configuration → Tuning → Update ‘Stuck Thread Max Time’ and click Save.

    Image represents Stuck thread max time

    Note: Repeat the above steps for all SOA Server (Update to available server1,server2, server3,…...,servern).

  3. Navigate to Home Page → Services → Data Sources → SOA Datasource and click ‘Transactions’ tab and select ‘Set XA Transaction Timeout’ check box. Make sure XA Transaction Timeout value is set to ‘0’.

    Image represents Settings for SOADataSources.
    Figure 3.4.17 - Settings for SOADataSources
  4. Click ‘Save’ and click Activate Changes.

Note: Click the 'View changes and restarts' link in Change Center section (located at the top left corner of the page) of WebLogic admin console and click 'Restart Checklist' tab. Select and click Restart if any resource types are listed in the 'Server Restart Checklist' table.

Update DI properties

  1. Click Home Page → Environment → Servers → Soa_server(Update to available server1,server2,server3,…...,servern) → Configuration → Server Start → Update ‘Arguments’ to ‘-Dcom.ibm.CORBA.UseSchemaIDR=false’
    Image represents updating DI properties.
    Figure 3.4.18 - Updating DI Properties
  2. Click 'Save'.

Cluster Performance Configuration

Database

Follow these steps to increase database process limit:

  1. Connect to the OIDG DB or PDB with SYSDBA privilege.

  2. Increase database process limit to 8000.(Depends on Database sources Values of Max Capacity. If the Total value is more than 8000 then increase value here and vice-versa).

    Ex: alter system set processes=8000 scope=spfile;
    alter system reset sessions scope=spfile sid='*';
    shutdown immediate;
    startup;
    show parameter sessions;
    show parameter processes;

  3. Set distributed lock timeout (distributed_lock_timeout) 60 sec to 300 sec.

    Ex : Use SYS User. ..(Depends on

    “Set SOA Bean Timeout in SOA-INFRA Application” TimeOut Value and vice-versa)


    SELECT name,value FROM v$parameter where
    NAME='distributed_lock_timeout' ;
    ALTER SYSTEM SET distributed_lock_timeout=300 scope=spfile; COMMIT;
    ALTER SYSTEM DISABLE DISTRIBUTED RECOVERY; COMMIT;
    ALTER SYSTEM ENABLE DISTRIBUTED RECOVERY; COMMIT;

  4. Give following grant to SOAINFRA Data Source

    grant select on sys.dba_pending_transactions to SLC13NND_SOAINFRA;
    grant force any TRANSACTION to SLC13NND_SOAINFRA;

Update the following table Data Source Connection Pool Sizes in Weblogic Console:

  1. Home Page → Services → Data Sources.

    Image represents Domain Structure.
    Figure 3.4.19 - Domain Structure
  2. Click EDNDataSource under 'Summary of JDBC Data Sources' section.

  3. Go to EDNDataSource > Connection Pool >

  4. Enter 10 in ‘Initial Capacity’, 40 in ‘Maximum Capacity’, 10 in ‘Minimum Capacity’ and 10 in ‘Statement Cache Size’ values and Click Save.

    Image represents Connection Pool Capacity updates.
    Figure 3.4.20 - Connection Pool Capacity update
  5. Click 'Activate changes' button.

Note: Repeat the same above steps for below all data Sources also.

Database sources

SR. No

JNDI Name

DB schema name

Initial Capacity

Maximum Capacity

Minimum Capacity

Statement Cache Size

1

EDNDataSource

RCU NAME_SOAINFRA

10

40

10

10

2

EDNLocalTxDataSource

RCU NAME_SOAINFRA

10

40

10

10

3

EssDS

RCU NAME_ESS

50

200

10

10

4

EssInternalDS

RCU NAME_ESS

50

200

10

10

5

EssXADS

RCU NAME_ESS

30

125

10

10

6

LocalSvcTblDataSource

RCU NAME_STB

50

200

10

10

7

mds-ESS_MDS_DS

RCU NAME_MDS

30

100

10

10

8

mds-owsm

RCU NAME_MDS

100

400

50

10

9

mds-soa

RCU NAME_MDS

100

400

50

10

10

OIDX

OIDX

150

600

100

100

11

opss-audit-DBDS

RCU NAME_IAU_APPEND

10

50

10

10

12

opss-audit-viewDS

RCU NAME_IAU_VIEWER

10

50

10

10

13

opss-data-source

RCU NAME_OPSS

50

200

10

10

14

OraSDPMDataSource

RCU NAME_UMS

50

300

10

10

15

SOADataSource

RCU NAME_SOAINFRA

100

500

50

50

16

SOALocalTxDataSource

RCU NAME_SOAINFRA

100

400

50

50

17

wlsbjmsrpDataSource

RCU NAME_SOAINFRA

100

400

50

10

18

WLSSchemaDataSource

RCU NAME_WLS_RUNTIME

50

200

10

10

Set SOA Bean Timeout in SOA-INFRA Application

  1. Go to Deployment page and click extend soa-infra application, edit the below ejb beans has Transaction Timeout to set 300 to 600 sec. (Note: This value should be more than the value given in DataBase section Step 3).

    SOA Bean Timeout in SOA-INFRA Application

    Transaction Timeout

    BPELActivityManagerBean

    600

    BPELDeliveryBean

    600

    BPELDispatcherBean

    600

    BPELEngineBean

    600

    BPELFinderBean

    600

    BPELInstanceManagerBean

    600

    BPELProcessManagerBean

    600

    BPELSensorValuesBean

    600

    BPELServerManagerBean

    600

    SOADirectInvokerBean

    600

  2. Go to each EJB Bean and click it.

  3. Go to Configuration tab, enter Transaction timeout 600 Sec, and Save it.

    Image represents setting transaction timeout.
    Figure 3.4.21 - Configuring memory arguments
  4. After saving this, save this Deployment Plan in New Plan XML and it creates a NewPlan.XML in the location

    (/scratch/idxAdmin/Oracle/Middleware/Oracle_Home/soa/soa).

  5. Copy NewPlan.XML file to other cluster machines in the same location.

  6. Go to Control tab in Deployments, check soa-infra application and start, servicing all Requests.

Set Work Managers

  1. Go to Environment > Work Managers.

  2. Click SOAIncomingRequests_maxThreads and edit the value to 40.

    Image represents setting count.
    Figure 3.4.22 - Setting count
  3. Click SOAInternalProcessing_maxThreads, and edit the value to 40 and save it.

    Image represents setting count.
    Figure 3.4.23 - Setting count

These values depend on the server machine OP configurations.

Set JVM options

Configure min and max memory arguments as 4GB for aml_server1, aml_server2, soa_server1, soa_server2, osb_server1 and osb_server2 managed servers.

  1. Click Home Page → Environment → Servers → Aml_server1 → Configuration → Server Start → Update ‘Arguments’

    -Xms4096m -Xmx4096m -XX:+UseG1GC -XX:ParallelGCThreads=10 -XX:ConcGCThreads=4 -XX:InitiatingHeapOccupancyPercent=50

    Image represents Configuring memory arguments.
    Figure 3.4.24 - Configuring memory arguments
  2. Click ‘Save’

Note: Repeat the above step for Aml_server2, osb_server1, osb_server2, soa_server1 and soa_server2.

OSB Performance configurations

Update BusinessValidation12_BS

  1. Login to http://<HostName>:<Admin Port /sbconsole with weblogic credentials.

  2. Click Create.

  3. Go to SVProducer → BusinessValidation12_BS.

  4. Click Transport button.

  5. Change load balancing Algorithm to random Weighted.

  6. Update Endpoint URL of Node1 and Node 2.

  7. Update weight for as 100 for the Server1 and 50 for server.

  8. Click Save and Activate the changes.

Image represents Updating BusinessValidation12_BS
Figure 3.4.25 - Updating BusinessValidation12_BS

Update IDXCS_JMSProducer_BS

  1. Click Create.

  2. Go to SVProducer → IDXCS_JMSProducer_BS.

  3. Click Transport button.

  4. Change load balancing Algorithm to random Weighted.

  5. Update Endpoint URL of Node1 and Node 2.

  6. Update weight for as 100 for the Server1 and 50 for server

  7. Click Save and Activate changes.

Image represents Updating IDXCS_JMSProducer_BS
Figure 3.4.26 - Updating IDXCS_JMSProducer_BS

Update BusinessValidation12_BS

  1. Click Create.

  2. Go to SVShared → BusinessValidation12_BS.

  3. Click Transport button.

  4. Change load balancing Algorithm to random Weighted.

  5. Update Endpoint URL of Node 1 and Node 2.

  6. Update weight as 100 for the Server 1 and 50 for Server 2.

  7. Click Save and Activate the changes.

Image represents Updating BusinessValidation12_BS
Figure 3.4.27 - Updating BusinessValidation12_BS

Update the targets of opss-data-source

  1. Login to weblogic console. http://<hostname>:<Admin port>/console with weblogic credentials.
  2. Click Lock and Edit.
  3. Figure 3.4.28 - Domain Structure
  4. Go to Home Page > Services > Data Sources.
  5. Click opss-data-source under 'Summary of JDBC Data Sources' section.
  6. Go to opss-data-source > Targets.
  7. Add ess_server1 as a target
  8. Click Save and Activate the changes.