users@grizzly.java.net

Re: Grizzly Thread Pool Monitoring

From: Oleksiy Stashok <oleksiy.stashok_at_oracle.com>
Date: Wed, 01 Apr 2015 09:33:39 -0700

Hi Uri,

it's done, please check the latest 2.3.20-SNAPSHOT

WBR,
Alexey.

On 31.03.15 01:32, Broide Uri wrote:
>
> Hi Alexey,
>
> Thanks allot, I opened *GRIZZLY-1756* in Jira as you requested.
>
> *GRIZZLY-1755*was opened by mistake by me, please discard it (I didn't
> find out how to do it myself)
>
> Thanks
>
> Uri
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Monday, March 30, 2015 8:11 PM
> *To:* users_at_grizzly.java.net
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi,
>
> sorry, I've just come back from vacation.
> After quick check it looks like there is no attribute showing the
> number of active threads in the pool (the ones, which are currently
> processing tasks).
> Can you pls. file an RFE - I'll add this feature today and let you know.
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 30.03.15 09:43, Broide Uri wrote:
>
> Alexey?
>
> *From:*Broide Uri
> *Sent:* Wednesday, March 25, 2015 7:41 AM
> *To:* 'users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>'
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Hi Alexey,
>
> Kind reminder
>
> Thanks
>
> Uri
>
> *From:*Broide Uri
> *Sent:* Sunday, March 22, 2015 10:10 AM
> *To:* 'users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>'
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Thanks Alexey,
>
> We have looked for this attribute and didn't find.
>
> Can share the exact attribute name with us?
>
> Thanks
>
> Uri
>
> *From:*Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Thursday, March 19, 2015 6:08 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* RE: Grizzly Thread Pool Monitoring
>
> Hi Uri,
> allocated thread count represents the number of threads in the
> thread pool, which could be either idle or active. There should be
> another value (i'm writing from the phone, so cant check the exact
> name), that represents the number of threads, which are busy
> processing tasks.
>
> On Mar 19, 2015 3:49 AM, Broide Uri <Uri.Broide_at_comverse.com
> <mailto:Uri.Broide_at_comverse.com>> wrote:
>
> Hi Alexey,
>
> It looks that your 2.3.20-SNAPSHOTfixed the problem (by mistake I
> took only grizzly-framework.jar without the
> grizzly-framework-monitoring.jar).
>
> One more clarification, is the
> *thread-pool-allocated-thread-count* is indeed the current active
> threads count ?
>
> Alexey, thanks again your quick response
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Monday, March 16, 2015 6:27 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Ok, let's do this, there is a monitoring sample in our docs [1]
> Can you pls. run it with the snapshot and tell me if the values
> are correct for you?
>
> Thank you.
>
> WBR,
> Alexey.
>
> public static void main(String[] args) {
>
>
>
> final GrizzlyJmxManager manager = GrizzlyJmxManager.instance();
>
> final TCPNIOTransport transport1 = TCPNIOTransportBuilder.newInstance().build();
>
> final TCPNIOTransport transport2 = TCPNIOTransportBuilder.newInstance().build();
>
> try {
>
> Object jmxTransportObject1 =
>
> transport1.getMonitoringConfig().createManagementObject();
>
>
>
> Object jmxTransportObject2 =
>
> transport2.getMonitoringConfig().createManagementObject();
>
>
>
> manager.registerAtRoot(jmxTransportObject1, "Transport1");
>
> manager.registerAtRoot(jmxTransportObject2, "Transport2");
>
> transport1.start();
>
> transport1.bind(9999);
>
> System.out.println("Press any key to stop the example...");
>
> System.in.read();
>
> } catch (IOException ioe) {
>
> ioe.printStackTrace();
>
> System.exit(1);
>
> } finally {
>
> try {
>
> transport1.shutdownNow();
>
> } catch (IOException ignored) {}
>
> }
>
> }
>
>
>
>
> [1] https://grizzly.java.net/monitoring.html
> <http://cp.mcafee.com/d/1jWVIp4x0SyMCU-COCeKrKrjhpKCOyyCYrjhpKCOy-CYrjhpKOyYeupdETK-UCMCC-rrzby4qOgY5bCvzj-ndHCvzj-ndKzCbzHLLZvAnHCzCVEVjWZOWqpEVvVdOWpEVd7bfbnjIyCHss_OEuvkzaT0QSyrpdTV55YsYqem1P8UTsSjDdqymohz8k53OT2eCtU6UHm4FRza17wsrrFYq5WC8v454oO51gYJMzFDu1FJBMszsQsCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMr4DMJ>
>
> On 16.03.15 07:35, Broide Uri wrote:
>
> Hi Alexey,
>
> Unfortunately it would be very difficult to supply a test case
> due to the complexity of our application. Is there anything
> else we can open (Grizzly loggers, etc ..)?
>
> Can you enter some System.out in order for us to make sure
> that we are working with the right version (although I am
> pretty sure we do, according to the grizzly-framework.jar
> Manifest).
>
> Anyhow, note that :
>
> 1.the *thread-pool-allocated-thread-count* value is _always_
> stabilized with *thread-pool-max-num-threads* minus
> *thread-pool-core-pool-size*
>
> 2.*thread-pool-core-pool-size*is always set to 12
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Monday, March 16, 2015 9:33 AM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hmm,
>
> can you pls. provide the testcase, because I can't reproduce
> the problem after the change I made.
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 16.03.15 00:30, Broide Uri wrote:
>
> Hi Alexey,
>
> Right, even started is reported wrong as you can see from
> the screenshot below (that was taken NOT under load).
>
> And yes we verified that it is the right version
> (2.3.20-SNAPSHOT)
>
> Thanks
>
> Uri
>
> *From:* Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Sunday, March 15, 2015 5:21 PM
> *To:* users_at_grizzly.java.net <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
> you mean even "started" is reported wrong?
> It should be something wrong with the version you picked up.
>
> Did you use this snapshots repository:
> https://maven.java.net/content/repositories/snapshots/
> <http://cp.mcafee.com/d/5fHCNEg6gUqdEI9KccFL6zB4TsSCyPtdB55dUSCyPtdB5ZdUSCyPtB5UsYOrhLtZNdxddYST6n48RAxUanc_6DYKrnc_6DYKrs7cW_tZ_HYy-OqekNPfnKnjjupVdYttVMQsYJt6OaaJS6ul3PWApmU6CQjqpK_9IInojvvpjdTdAVPmEBCaWp-72eCtU6VqI9_2uhZqJ9jH6nQOAdmEjHeC6SWv6xuFy7N1h6cxgkfbs8WpTwqrhu79FLCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMrnVhOYlH0>
>
> Thank you.
>
> WBR,
> Alexey.
>
> On 15.03.15 01:47, Broide Uri wrote:
>
> Hi Alexey,
>
> Unfortunately the fix doesn’t work. Can we gather some
> more information on this?
>
> Please note that we have a protection mechanism that
> monitors the usage of the thread pools, and currently
> it blocks traffic based on wrong reporting in the
> thread pool JMX objects.
>
> Thanks
>
> Uri
>
> *From:* Oleksiy Stashok
> [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Wednesday, March 11, 2015 9:22 PM
> *To:* users_at_grizzly.java.net
> <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
>
> On 11.03.15 07:54, Broide Uri wrote:
>
> 1.We opened the requested BUG (GRIZZLY-1748
> <http://cp.mcafee.com/d/avndzgQrhojshKye7c6XCQQmrFIEEFL6QQmrFIELFL6QQmrIEL3DCjqdXLK9I9FLCSUOUx6IAf1iVDUQ_BPqVDUQ_BPrZT3hPuXX_nV5wsMzvHTbFTovVdVBNV7BHFShhlhKPOEuvkzaT0QSyrhdTV5MQsFIT7e6zBcTsSjDdqymo8WpTwrwzaeUgzkNjY2gb3g90AawsKyeoudJQ-d2Zj4fy2ycp2wEumUhQPL0QSyZtB5MS2_id40iz8k53ONEw19efWwq80JmFVEw6WMBeIpg8CSm3qCJWcYkr>).
>
> 2.Can you answer my Questions below
>
> The questions are as follows:
>
> 1.Are we looking on the right MBean attributes?
>
> Yes.
>
>
>
> 2.Assuming 1 is true, then why for the same load model
> the thread pool is reaching this high number (max =
> core + allocated)?
>
> It's because the probe values reported based on events
> coming from the thread pool and at the time of probe
> registration some events are already missed.
>
>
>
> 3.Why the *thread-pool-started* attribute is false?
>
> same as above.
>
> I think I've fixed the issue, can you pls. check the
> latest 2.3.20-SNAPSHOT?
>
> Thanks.
>
> WBR,
> Alexey.
>
>
>
>
> *From:* Oleksiy Stashok
> [mailto:oleksiy.stashok_at_oracle.com]
> *Sent:* Tuesday, March 10, 2015 8:15 PM
> *To:* users_at_grizzly.java.net
> <mailto:users_at_grizzly.java.net>
> *Subject:* Re: Grizzly Thread Pool Monitoring
>
> Hi Uri,
>
> looks like a bug to me, 12 core threads are not
> counted as allocated.
> Can you pls. file an issue?
>
> Thanks.
>
> WBR,
> Alexey.
>
> On 10.03.15 10:26, Broide Uri wrote:
>
> Hi
>
> We are performing Stability load run against our
> Grizzly based application.
>
> We are trying to determine how to monitor the
> Thread Pool usage via JMX.
>
> We are looking at the ThreadPool MBean
> (org.glassfish.grizzly:pp=/gmbal-root/TCPNIOTransport[EndPoint.IMAP.NON_SECURED],type=ThreadPool,name=ThreadPool),
> on the following attributes:
>
> ·thread-pool-max-num-threads
>
> ·thread-pool-allocated-thread-count
>
> ·thread-pool-core-pool-size
>
> We have conducted 2 load runs (with *_exactly_*
> the same load model):
>
> 1.max threads was configured to be *200* (verified
> against thread-pool-max-num-threads)
>
> ofrom some point of the load run the
> *thread-pool-allocated-thread-count* was
> stabilized on 188
>
> o*thread-pool-core-pool-size* was fixed on 12
>
> 2.max threads was configured to be *300* (verified
> against thread-pool-max-num-threads)
>
> ofrom some point of the load run the
> *thread-pool-allocated-thread-count* was
> stabilized on 288
>
> o*thread-pool-core-pool-size* was fixed on 12
>
> The questions are as follows:
>
> 1.Are we looking on the right MBean attributes?
>
> 2.Assuming 1 is true, then why for the same load
> model the thread pool is reaching this high number
> (max = core + allocated)?
>
> 3.Why the *thread-pool-started* attribute is false?
>
> Attaching the JConsole print screen of the second run
>
> cid:image001.png_at_01D05B52.9D7A53B0
>
> Thanks
>
> Uri
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that
> constitutes proprietary information of Comverse
> Inc. or its subsidiaries. If you are not the
> intended recipient of this message, you are hereby
> notified that any review, use or distribution of
> this information is absolutely prohibited and we
> request that you delete all copies and contact us
> by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that constitutes
> proprietary information of Comverse Inc. or its
> subsidiaries. If you are not the intended recipient of
> this message, you are hereby notified that any review,
> use or distribution of this information is absolutely
> prohibited and we request that you delete all copies
> and contact us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential,
> commercial or privileged information that constitutes
> proprietary information of Comverse Inc. or its
> subsidiaries. If you are not the intended recipient of
> this message, you are hereby notified that any review,
> use or distribution of this information is absolutely
> prohibited and we request that you delete all copies
> and contact us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial
> or privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you
> are not the intended recipient of this message, you are
> hereby notified that any review, use or distribution of
> this information is absolutely prohibited and we request
> that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>.
> Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary
> information of Comverse Inc. or its subsidiaries. If you are
> not the intended recipient of this message, you are hereby
> notified that any review, use or distribution of this
> information is absolutely prohibited and we request that you
> delete all copies and contact us by e-mailing to:
> security_at_comverse.com <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any
> review, use or distribution of this information is absolutely
> prohibited and we request that you delete all copies and contact
> us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
>
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any
> review, use or distribution of this information is absolutely
> prohibited and we request that you delete all copies and contact
> us by e-mailing to: security_at_comverse.com
> <mailto:security_at_comverse.com>. Thank You.”
>
> ------------------------------------------------------------------------
> “This e-mail message may contain confidential, commercial or
> privileged information that constitutes proprietary information of
> Comverse Inc. or its subsidiaries. If you are not the intended
> recipient of this message, you are hereby notified that any review,
> use or distribution of this information is absolutely prohibited and
> we request that you delete all copies and contact us by e-mailing to:
> security_at_comverse.com. Thank You.”