JMS Data Intake set up - Uniform Distributed Queue - Cluster
Configuration Steps
Pre-requisites for running Data Intake:
- JMS module will be pointed to a Cycle cluster and every instance in the cluster should be up and running.
- The property dataIntake.enabled should be set to true on server side of Cycle.
- Each cycle instance should have its own cycle-coherence-config.xml file and each instance should have a unique port number.
- Need to provide Customer Name, Intake Profile name , and Class Name (In case of Enrollment) in the loading file, which should be created on OIPA side prior to Data Intake. The steps to be followed on OIPA side can be found in the below Page. Data Intake Guide.
- Two new properties are introduced in Cycle.properties files on server side.
- datatIntake.reProcessingLimit = 10 and datatIntake.yieldTimePeriod = 5000
- If they are not mentioned/configured - system will use their default values 10 and 5000 respectively.
The following steps are done in the Web Logic Server Console, beginning with the left-hand navigation menu.
Create Cluster:
Environment -> Clusters
- Click on New tab and provide the name of the cluster and click on OK.
- Click on Cluster which is created in previous step.
- Click on Severs tab.
- Scroll down on the page to see the servers table.
- Click on Add button and add the Cycle instances ( Note: Add more than one cycle instance to the cluster ).
Adding Members to the Cluster :
Create a JMS Server:
Services > Messaging > JMS Servers
- Select New
- Name: DataIntakeJMSServer
- Persistent Store: (none)
- Target: CycleCluster
- Finish. The JMS server should now be visible in the list with Health OK.
Create a JMS Module:
Services > Messaging > JMS Modules
-
Select New.
-
Name: DataIntakeJMSModule
-
Leave the other options empty
-
Targets: CycleCluster
-
-
Press Next.
-
Leave “Would you like to add resources to this JMS system module” unchecked and press Finish.
Create a Sub Deployment:
A subdeployment is not necessary for the JMS queue to work, but it allows you to easily target subcomponents of the JMS module to a single target or group of targets. We will use the subdeployment in this to target the following connection factory and JMS queue to the JMS server we created earlier.
Services > Messaging > JMS Modules
- Select DataIntakeJMSModule
- Select the Subdeployments tab and New Subdeployment Name: DataIntakeSubdeployment
- Press Next
Here you can select the target(s) for the subdeployment. You can choose either Servers (i.e. WebLogic managed servers, such as the DataIntake) or JMS Servers (DataIntakeJMSServer) such as the JMS Server created earlier. As the purpose of our subdeployment in this is to target a specific JMS server, we will choose the JMS Server option.
- Select the DataIntakeJMSServer created earlier.
- Press Finish.
Create a Connection Factory:
Services > Messaging > JMS Modules
- Select DataIntakeJMSModule and press New
- Select Connection Factory and Next
- Name: IntakeConnectionFactory
- JNDI Name: IntakeConnectionFactory
- Leave the other values at default
- Targets: CycleCluster
- Press Finish
- Click on IntakeConnectionFactory - Make sure that Default targeting is enabled.
Note: Uncheck "Server Affinity enabled" checkbox in Intake Connection factory to maintain uniform distribution of JMS messages across the cluster.
Create a JMS Distributed Queue:
Services > Messaging > JMS Modules
- Select DataIntakeJMSModule and press New
- Select DistributedQueue and Next
- Name: DIQueue
- JNDI Name: DIQueue
- Template: None
- Target: Click on Advanced Targeting ->Select DataIntakeSubdeployment
- Press Next
- Finish. The JMS queue is now complete and can be accessed using the JNDI names IntakeConnectionFactory and DIQueue.
Create a WorkManager
Environments > Work Managers
- Click on New
- Select Work Manager radio button
- Provide name (wm/default)
- Select target server(CycleCluster )
Note: After the above steps are completed, set dataIntake.enabled property to true in cycle Properties file and a server restart is required to pick up data intake configuration.
Change in Data Intake client utility (pas.di.testclient)
- Add the below two jar files to pas.di.testclient utility lib folder
- wlclient.jar
- wljmsclient.jar
- Modify conf/CycleClient.properties file in pas.di.testclient with following changes
- Uncomment weblogic context factory and comment other context factories
- contextFactory=weblogic.jndi.WLInitialContextFactory
- Modify provider url
- provider.url=t3://localhost:7003(provide appropriate url ex: Cycle url)
Each cycle instance should have its own Cycle-coherence-cache-config.xml with unique port numbers and it should be registered as well-known address list of other.
Performance Recommendations
WebLogic Settings
S.No | Settings | Description |
---|---|---|
1 | Login Timeout =25000 | <ManagedServer> -- Configuration -- tuning -- Login Timeout should be 25000 |
2 | HTTP Duration = 60 |
<ManagedServer> -- Configuration -- Protocols -- HTTP --Duration should be 60 |
3 | Max Capacity=250 Initial Capacity = 100 |
Services --<DataSource> --Configuration -- ConnectionPool -- Maximum Capacity should be 250 and Initial Capacity should be 100 |
4 | GC Algorithm “-XX:+UseG1GC” | G1GC to be part of jvm arguments. |
5 | -Xms4g –Xmx4g | Heap memory settings. |
6 | Uncheck - Server Affinity Enabled | Services--Messaging–JMS Modules--DataIntakeJMSModule--IntakeConnectionFactory–Load Balance tab. |