users@grizzly.java.net

Re: Grizzly Thread Pool Monitoring

From: Oleksiy Stashok <oleksiy.stashok_at_oracle.com>
Date: Mon, 30 Mar 2015 10:10:43 -0700

Hi,

sorry, I've just come back from vacation.
After quick check it looks like there is no attribute showing the number
of active threads in the pool (the ones, which are currently processing
tasks).
Can you pls. file an RFE - I'll add this feature today and let you know.

Thanks.

WBR,
Alexey.

On 30.03.15 09:43, Broide Uri wrote:
>
> Alexey?
>
> *From:*Broide Uri
> *Sent:* Wednesday, March 25, 2015 7:41 AM
> *To:* 'users_at_grizzly.java.net'
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Hi Alexey,
>
> Kind reminder
>
> Thanks
>
> Uri
>
> *From:*Broide Uri
> *Sent:* Sunday, March 22, 2015 10:10 AM
> *To:* 'users_at_grizzly.java.net'
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Thanks Alexey,
>
> We have looked for this attribute and didn't find.
>
> Can share the exact attribute name with us?
>
> Thanks
>
> Uri
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Thursday, March 19, 2015 6:08 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Hi Uri,
> allocated thread count represents the number of threads in the thread
> pool, which could be either idle or active. There should be another
> value (i'm writing from the phone, so cant check the exact name), that
> represents the number of threads, which are busy processing tasks.
>
> On Mar 19, 2015 3:49 AM, Broide Uri <Uri.Broide_at_comverse.com
> <mailto:Uri.Broide_at_comverse.com>> wrote:
>
> Hi Alexey,
>
> It looks that your 2.3.20-SNAPSHOTfixed the problem (by mistake I took
> only grizzly-framework.jar without the grizzly-framework-monitoring.jar).
>
> One more clarification, is the *thread-pool-allocated-thread-count* is
> indeed the current active threads count ?
>
> Alexey, thanks again your quick response
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Monday, March 16, 2015 6:27 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Ok, let's do this, there is a monitoring sample in our docs [1]
> Can you pls. run it with the snapshot and tell me if the values are
> correct for you?
>
> Thank you.
>
> WBR,
> Alexey.
>
> public static void main(String[] args) {
>
> final GrizzlyJmxManager manager = GrizzlyJmxManager.instance();
> final TCPNIOTransport transport1 = TCPNIOTransportBuilder.newInstance().build();
> final TCPNIOTransport transport2 = TCPNIOTransportBuilder.newInstance().build();
> try {
> Object jmxTransportObject1 =
> transport1.getMonitoringConfig().createManagementObject();
>
> Object jmxTransportObject2 =
> transport2.getMonitoringConfig().createManagementObject();
>
> manager.registerAtRoot(jmxTransportObject1, "Transport1");
> manager.registerAtRoot(jmxTransportObject2, "Transport2");
> transport1.start();
> transport1.bind(9999);
> System.out.println("Press any key to stop the example...");
> System.in.read();
> } catch (IOException ioe) {
> ioe.printStackTrace();
> System.exit(1);
> } finally {
> try {
> transport1.shutdownNow();
> } catch (IOException ignored) {}
> }
> }
>
>
>
> [1] https://grizzly.java.net/monitoring.html
> <http://cp.mcafee.com/d/1jWVIp4x0SyMCU-COCeKrKrjhpKCOyyCYrjhpKCOy-CYrjhpKOyYeupdETK-UCMCC-rrzby4qOgY5bCvzj-ndHCvzj-ndKzCbzHLLZvAnHCzCVEVjWZOWqpEVvVdOWpEVd7bfbnjIyCHss_OEuvkzaT0QSyrpdTV55YsYqem1P8UTsSjDdqymohz8k53OT2eCtU6UHm4FRza17wsrrFYq5WC8v454oO51gYJMzFDu1FJBMszsQsCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMr4DMJ>
>
> On 16.03.15 07:35, Broide Uri wrote:
>
> Hi Alexey,
>
> Unfortunately it would be very difficult to supply a test case due
> to the complexity of our application. Is there anything else we
> can open (Grizzly loggers, etc ..)?
>
> Can you enter some System.out in order for us to make sure that we
> are working with the right version (although I am pretty sure we
> do, according to the grizzly-framework.jar Manifest).
>
> Anyhow, note that :
>
> 1.the *thread-pool-allocated-thread-count* value is _always_
> stabilized with *thread-pool-max-num-threads* minus
> *thread-pool-core-pool-size*
>
> 2.*thread-pool-core-pool-size*is always set to 12
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Monday, March 16, 2015 9:33 AM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hmm,
>
> can you pls. provide the testcase, because I can't reproduce the
> problem after the change I made.
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 16.03.15 00:30, Broide Uri wrote:
>
> Hi Alexey,
>
> Right, even started is reported wrong as you can see from the
> screenshot below (that was taken NOT under load).
>
> And yes we verified that it is the right version (2.3.20-SNAPSHOT)
>
> Thanks
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Sunday, March 15, 2015 5:21 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
> you mean even "started" is reported wrong?
> It should be something wrong with the version you picked up.
>
> Did you use this snapshots repository:
> https://maven.java.net/content/repositories/snapshots/
> <http://cp.mcafee.com/d/5fHCNEg6gUqdEI9KccFL6zB4TsSCyPtdB55dUSCyPtdB5ZdUSCyPtB5UsYOrhLtZNdxddYST6n48RAxUanc_6DYKrnc_6DYKrs7cW_tZ_HYy-OqekNPfnKnjjupVdYttVMQsYJt6OaaJS6ul3PWApmU6CQjqpK_9IInojvvpjdTdAVPmEBCaWp-72eCtU6VqI9_2uhZqJ9jH6nQOAdmEjHeC6SWv6xuFy7N1h6cxgkfbs8WpTwqrhu79FLCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMrnVhOYlH0>
>
> Thank you.
>
> WBR,
> Alexey.
>
> On 15.03.15 01:47, Broide Uri wrote:
>
> Hi Alexey,
>
> Unfortunately the fix doesn’t work. Can we gather some
> more information on this?
>
> Please note that we have a protection mechanism that
> monitors the usage of the thread pools, and currently it
> blocks traffic based on wrong reporting in the thread pool
> JMX objects.
>
> Thanks
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Wednesday, March 11, 2015 9:22 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
> On 11.03.15 07:54, Broide Uri wrote:
>
> 1.We opened the requested BUG (GRIZZLY-1748
> <http://cp.mcafee.com/d/avndzgQrhojshKye7c6XCQQmrFIEEFL6QQmrFIELFL6QQmrIEL3DCjqdXLK9I9FLCSUOUx6IAf1iVDUQ_BPqVDUQ_BPrZT3hPuXX_nV5wsMzvHTbFTovVdVBNV7BHFShhlhKPOEuvkzaT0QSyrhdTV5MQsFIT7e6zBcTsSjDdqymo8WpTwrwzaeUgzkNjY2gb3g90AawsKyeoudJQ-d2Zj4fy2ycp2wEumUhQPL0QSyZtB5MS2_id40iz8k53ONEw19efWwq80JmFVEw6WMBeIpg8CSm3qCJWcYkr>).
>
> 2.Can you answer my Questions below
>
> The questions are as follows:
>
> 1.Are we looking on the right MBean attributes?
>
> Yes.
>
>
> 2.Assuming 1 is true, then why for the same load model the
> thread pool is reaching this high number (max = core +
> allocated)?
>
> It's because the probe values reported based on events
> coming from the thread pool and at the time of probe
> registration some events are already missed.
>
>
> 3.Why the *thread-pool-started* attribute is false?
>
> same as above.
>
> I think I've fixed the issue, can you pls. check the
> latest 2.3.20-SNAPSHOT?
>
> Thanks.
>
> WBR,
> Alexey.
>
>
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Tuesday, March 10, 2015 8:15 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
> looks like a bug to me, 12 core threads are not counted as
> allocated.
> Can you pls. file an issue?
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 10.03.15 10:26, Broide Uri wrote:
>
> Hi
>
> We are performing Stability load run against our
> Grizzly based application.
>
> We are trying to determine how to monitor the Thread
> Pool usage via JMX.
>
> We are looking at the ThreadPool MBean
> (org.glassfish.grizzly:pp=/gmbal-root/TCPNIOTransport[EndPoint.IMAP.NON_SECURED],type=ThreadPool,name=ThreadPool),
> on the following attributes:
>
> ·thread-pool-max-num-threads
>
> ·thread-pool-allocated-thread-count
>
> ·thread-pool-core-pool-size
>
> We have conducted 2 load runs (with *_exactly_* the
> same load model):
>
> 1.max threads was configured to be *200* (verified
> against thread-pool-max-num-threads)
>
> ofrom some point of the load run the
> *thread-pool-allocated-thread-count* was stabilized on 188
>
> o*thread-pool-core-pool-size* was fixed on 12
>
> 2.max threads was configured to be *300* (verified
> against thread-pool-max-num-threads)
>
> ofrom some point of the load run the
> *thread-pool-allocated-thread-count* was stabilized on 288
>
> o*thread-pool-core-pool-size* was fixed on 12
>
> The questions are as follows:
>
> 1.Are we looking on the right MBean attributes?
>
> 2.Assuming 1 is true, then why for the same load model
> the thread pool is reaching this high number (max =
> core + allocated)?
>
> 3.Why the *thread-pool-started* attribute is false?
>
> Attaching the JConsole print screen of the second run
>
> cid:image001.png_at_01D05B52.9D7A53B0
>
> Thanks
>
> Uri
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that constitutes
> proprietary information of Comverse Inc. or its
> subsidiaries. If you are not the intended recipient of
> this message, you are hereby notified that any review,
> use or distribution of this information is absolutely
> prohibited and we request that you delete all copies
> and contact us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial
> or privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you
> are not the intended recipient of this message, you are
> hereby notified that any review, use or distribution of
> this information is absolutely prohibited and we request
> that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>.
> Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial
> or privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you
> are not the intended recipient of this message, you are
> hereby notified that any review, use or distribution of
> this information is absolutely prohibited and we request
> that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>.
> Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you are
> not the intended recipient of this message, you are hereby
> notified that any review, use or distribution of this
> information is absolutely prohibited and we request that you
> delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any
> review, use or distribution of this information is absolutely
> prohibited and we request that you delete all copies and contact
> us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any review,
> use or distribution of this information is absolutely prohibited and
> we request that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any review,
> use or distribution of this information is absolutely prohibited and
> we request that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com. Thank You.”