dev@genericjmsra.java.net

Re: GenericJMSRA -- Issue with dequeueing messages from OracleAQ

From: Ramesh <Ramesh.Parthasarathy_at_Sun.COM>
Date: Tue, 02 Dec 2008 21:45:42 +0530

Hi Ashish,
    What is the value MaxPoolSize you have set in the Activation spec
(deployment descriptor). If you have not set it, could you set it to "2"
and see if there is reduction in cpu utilization (and how it affects
your throughput requirements)..
Default value (if you do not set anything) is 8.

<activation-config-property>
         
<activation-config-property-name>MaxPoolSize</activation-config-property-name>
         
<activation-config-property-value>2</activation-config-property-value>
</activation-config-property>

Thanks
-Ramesh

ashish.banidvadekar_at_bt.com wrote:
>
> Hi All,
> We are facing a issue with GenericJMSRA.
>
> Background:
>
> We have a J2EE component (EAR) deployed on Sun appserver 8.1
> The component is basically a MDB that listens to a queue hosted by
> ORACLE AQ .
> The component consumes the messages off the queue(MIC_OUTBOUND_QUEUE)
> and then does a http to another component.
> The connector module to consume messages from ORACLE AQ is
> GenericJMSRA deployed on Sun appserver 8.1.
> The end to end flow works perfectly fine.
> However , there seems to be a problem when we are running a non
> functional capacity test to find out the throughput of this
> component(The name of the component is MIC Outbound).
>
> Test Description:
>
> The flow is : MIC_OUTBOUND_QUEUE (OracleAQ on DB) ------>[J2EE
> listener] MIC_OUBOUND_COMPONENT [http to Stub]----------> Stub
> returning 202
>
> To run this test approx 60,000 messages are piled up on the
> MIC_OUTBOUND_QUEUE .
> Then the MIC_OUBOUND_COMPONENT (listener) is enabled on the appserver
> so that the messages get dequeued from MIC_OUTBOUND_QUEUE.
>
> The throughput is then calculated based on time taken to consume
> 60,000 messages off the queue.
> It is observed that while the test is running, the database CPU
> reaches 100% and all 60,000 messages get consumed successfully.
>
> So far so good.
>
> The problem actually begins when the test has finished execution.
>
>
> Problem Summary:
> The problem starts after the execution of the test is over.
> Though there is not test running on the appserver and database, the
> database CPU continuously fluctuates between 60% to 100%.
>
> There are only three ways of getting the database CPU back to normal
> ie: 0%.
>
> 1)The CPU on database becomes 0% after the SUN appserver 8.1 instance
> is stopped.
> However it again goes up and fluctuates betweeb 60% and 100% after
> the appserver instance is started.
>
> 2)The CPU on database goes down after MIC_OUTBOUND_COMPONENT on Sun
> appserver 8.1 is disabled .
> Sometimes it is observed that though the component is disabled
> manually from the console, the appserver though shows it in disabled
> state it does not disable it internally and hence the CPU on the
> database still fluctuates between 60% and 100%.
>
> Then we have to go for the 3rd option.
>
> 3)Undeploy the MIC_OUTBOUND component from the SUN appserver 8.1.
> Sometimes it is observed that though the component gets undeployed
> from the console it does not get undeployed internally and component
> fliles exist under nodeagent/serverinstance/application/j2ee-apps
>
> and the database CPU still fluctuates between 60% and 100%.
> Then the files under nodeagent/serverinstance/application/j2ee-apps
> need to be manually deleted and then the database CPU goes to 0%.
>
> The obvious question is what are the processes that run on the
> Database server and are responsible for 60 to 100% of CPU utilization
> on the database when actually it is supposed to remain idle as there
> are no messages on the queue?
>
> The below process continuously runs and multiplies in n umbers on the
> database and hogs the database CPU :
> begin dbms_aqin.aq$_dequeue_in( :1, :2, :3, :4, :5, :6, :7, :8, :9,
> :10, :11, :12, :13, :14, :15, :16, :17, :18, :19, :20, :21, :22, :23,
> :24, :25, :26, :27, :28, :29); end;
>
> There are 65 connections from Appserver to database which are
> continuously in Established state.
> As we have a MDB (MIC_OUTBOUND_COMPONENT ) running continuously
> listening to the queue , the Established state of the connections is
> justified.
>
> However, the problem is that genericJMSRA is triggerring dequeue
> process on the DB though there are no messages on the queue.
>
> These processes multiply in numbers and CPU on the database goes up
> though there is no test running and there are no messages on the queue.
>
> The probable cause for this problem is the way in which genericJMSRA
> works.
> GenericJMSRA is suppose to trigger a dequeue process on the Oracle AQ
> only when there are messages on the queue.
> After we are through with running the MIC outbound test and dequeueing
> approx. 60,000 messages off the Oracle AQ, genericJMSRA is not
> stabalizing .
>
> It tends to still trigger the dequeue process on the database when it
> is not required.
>
> Note:
> The same Sun Appserver 8.1 after the execution of the test when
> pointed to a different database server by changing the IP address in
> the OID,
>
> it seems to start the begin dbms_aqin.aq$_dequeue_in( :1, :2, :3, :4,
> : process on the different database and gets the CPU of the
> database to fluctuate between 60 to 100%.
>
> This suggests that the issue is with GenericJMSRA and not OracleAQ
> /Database server.
>
> Please let me know if you need any furtehr information.
>
> Regards,
> Ashish
>