Date: Mon, 13 Apr 2015 15:32:34 +0000
Hi Alexey,
1. Can you please share with us when the latest formal Grizzly release (2.3.20) is planned?
2. We have done a load run to see the behavior of the new attribute you have (thread-pool-busy-thread-count) added and have noticed that is very low (in average) comparing to the thread-pool-allocated-thread-count.
Have you ever considered implementing some kind of pool reduction mechanism (attached a screenshot of last 8 hours of our 31 hours load run on our server), unless there is such a mechanism we are not aware of?
3. why is the tasks queue (thread-pool-queued-task-count) is active despite of the fact the pool never reached its maximum limit (according to the allocated threads count)?
Thanks Uri.
[cid:image003.jpg_at_01D07606.00A48E10]
From: Broide Uri
Sent: Thursday, April 02, 2015 10:16 AM
To: users_at_grizzly.java.net
Cc: Tsukerman Dana
Subject: RE: Grizzly Thread Pool Monitoring
Thanks allot Alexey,
I will and keep you updated
Uri
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Wednesday, April 01, 2015 7:34 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hi Uri,
it's done, please check the latest 2.3.20-SNAPSHOT
WBR,
Alexey.
On 31.03.15 01:32, Broide Uri wrote:
Hi Alexey,
Thanks allot, I opened GRIZZLY-1756 in Jira as you requested.
GRIZZLY-1755 was opened by mistake by me, please discard it (I didn't find out how to do it myself)
Thanks
Uri
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Monday, March 30, 2015 8:11 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hi,
sorry, I've just come back from vacation.
After quick check it looks like there is no attribute showing the number of active threads in the pool (the ones, which are currently processing tasks).
Can you pls. file an RFE - I'll add this feature today and let you know.
Thanks.
WBR,
Alexey.
On 30.03.15 09:43, Broide Uri wrote:
Alexey?
From: Broide Uri
Sent: Wednesday, March 25, 2015 7:41 AM
To: 'users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>'
Subject: RE: Grizzly Thread Pool Monitoring
Hi Alexey,
Kind reminder
Thanks
Uri
From: Broide Uri
Sent: Sunday, March 22, 2015 10:10 AM
To: 'users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>'
Subject: RE: Grizzly Thread Pool Monitoring
Thanks Alexey,
We have looked for this attribute and didn't find.
Can share the exact attribute name with us?
Thanks
Uri
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Thursday, March 19, 2015 6:08 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: RE: Grizzly Thread Pool Monitoring
Hi Uri,
allocated thread count represents the number of threads in the thread pool, which could be either idle or active. There should be another value (i'm writing from the phone, so cant check the exact name), that represents the number of threads, which are busy processing tasks.
On Mar 19, 2015 3:49 AM, Broide Uri <Uri.Broide_at_comverse.com<mailto:Uri.Broide_at_comverse.com>> wrote:
Hi Alexey,
It looks that your 2.3.20-SNAPSHOT fixed the problem (by mistake I took only grizzly-framework.jar without the grizzly-framework-monitoring.jar).
One more clarification, is the thread-pool-allocated-thread-count is indeed the current active threads count ?
Alexey, thanks again your quick response
Uri
[cid:image006.jpg_at_01D06229.E7BFEB00]
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Monday, March 16, 2015 6:27 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Ok, let's do this, there is a monitoring sample in our docs [1]
Can you pls. run it with the snapshot and tell me if the values are correct for you?
Thank you.
WBR,
Alexey.
public static void main(String[] args) {
final GrizzlyJmxManager manager = GrizzlyJmxManager.instance();
final TCPNIOTransport transport1 = TCPNIOTransportBuilder.newInstance().build();
final TCPNIOTransport transport2 = TCPNIOTransportBuilder.newInstance().build();
try {
Object jmxTransportObject1 =
transport1.getMonitoringConfig().createManagementObject();
Object jmxTransportObject2 =
transport2.getMonitoringConfig().createManagementObject();
manager.registerAtRoot(jmxTransportObject1, "Transport1");
manager.registerAtRoot(jmxTransportObject2, "Transport2");
transport1.start();
transport1.bind(9999);
System.out.println("Press any key to stop the example...");
System.in.read();
} catch (IOException ioe) {
ioe.printStackTrace();
System.exit(1);
} finally {
try {
transport1.shutdownNow();
} catch (IOException ignored) {}
}
}
[1] https://grizzly.java.net/monitoring.html<http://cp.mcafee.com/d/1jWVIp4x0SyMCU-COCeKrKrjhpKCOyyCYrjhpKCOy-CYrjhpKOyYeupdETK-UCMCC-rrzby4qOgY5bCvzj-ndHCvzj-ndKzCbzHLLZvAnHCzCVEVjWZOWqpEVvVdOWpEVd7bfbnjIyCHss_OEuvkzaT0QSyrpdTV55YsYqem1P8UTsSjDdqymohz8k53OT2eCtU6UHm4FRza17wsrrFYq5WC8v454oO51gYJMzFDu1FJBMszsQsCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMr4DMJ>
On 16.03.15 07:35, Broide Uri wrote:
Hi Alexey,
Unfortunately it would be very difficult to supply a test case due to the complexity of our application. Is there anything else we can open (Grizzly loggers, etc ..)?
Can you enter some System.out in order for us to make sure that we are working with the right version (although I am pretty sure we do, according to the grizzly-framework.jar Manifest).
Anyhow, note that :
1. the thread-pool-allocated-thread-count value is always stabilized with thread-pool-max-num-threads minus thread-pool-core-pool-size
2. thread-pool-core-pool-size is always set to 12
Uri
[cid:image003.jpg_at_01D06229.26AAE6E0]
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Monday, March 16, 2015 9:33 AM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hmm,
can you pls. provide the testcase, because I can't reproduce the problem after the change I made.
Thanks.
WBR,
Alexey.
On 16.03.15 00:30, Broide Uri wrote:
Hi Alexey,
Right, even started is reported wrong as you can see from the screenshot below (that was taken NOT under load).
And yes we verified that it is the right version (2.3.20-SNAPSHOT)
Thanks
Uri
[cid:image004.jpg_at_01D06229.26AAE6E0]
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Sunday, March 15, 2015 5:21 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hi Uri,
you mean even "started" is reported wrong?
It should be something wrong with the version you picked up.
Did you use this snapshots repository:
https://maven.java.net/content/repositories/snapshots/<http://cp.mcafee.com/d/5fHCNEg6gUqdEI9KccFL6zB4TsSCyPtdB55dUSCyPtdB5ZdUSCyPtB5UsYOrhLtZNdxddYST6n48RAxUanc_6DYKrnc_6DYKrs7cW_tZ_HYy-OqekNPfnKnjjupVdYttVMQsYJt6OaaJS6ul3PWApmU6CQjqpK_9IInojvvpjdTdAVPmEBCaWp-72eCtU6VqI9_2uhZqJ9jH6nQOAdmEjHeC6SWv6xuFy7N1h6cxgkfbs8WpTwqrhu79FLCMnWhEw2kp2wEumd4099N_k3h05GRfd40Tm4FRza14SOMrnVhOYlH0>
Thank you.
WBR,
Alexey.
On 15.03.15 01:47, Broide Uri wrote:
Hi Alexey,
Unfortunately the fix doesn’t work. Can we gather some more information on this?
Please note that we have a protection mechanism that monitors the usage of the thread pools, and currently it blocks traffic based on wrong reporting in the thread pool JMX objects.
Thanks
Uri
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Wednesday, March 11, 2015 9:22 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hi Uri,
On 11.03.15 07:54, Broide Uri wrote:
1. We opened the requested BUG (GRIZZLY-1748<http://cp.mcafee.com/d/avndzgQrhojshKye7c6XCQQmrFIEEFL6QQmrFIELFL6QQmrIEL3DCjqdXLK9I9FLCSUOUx6IAf1iVDUQ_BPqVDUQ_BPrZT3hPuXX_nV5wsMzvHTbFTovVdVBNV7BHFShhlhKPOEuvkzaT0QSyrhdTV5MQsFIT7e6zBcTsSjDdqymo8WpTwrwzaeUgzkNjY2gb3g90AawsKyeoudJQ-d2Zj4fy2ycp2wEumUhQPL0QSyZtB5MS2_id40iz8k53ONEw19efWwq80JmFVEw6WMBeIpg8CSm3qCJWcYkr>).
2. Can you answer my Questions below
The questions are as follows:
1. Are we looking on the right MBean attributes?
Yes.
2. Assuming 1 is true, then why for the same load model the thread pool is reaching this high number (max = core + allocated)?
It's because the probe values reported based on events coming from the thread pool and at the time of probe registration some events are already missed.
3. Why the thread-pool-started attribute is false?
same as above.
I think I've fixed the issue, can you pls. check the latest 2.3.20-SNAPSHOT?
Thanks.
WBR,
Alexey.
From: Oleksiy Stashok [mailto:oleksiy.stashok_at_oracle.com]
Sent: Tuesday, March 10, 2015 8:15 PM
To: users_at_grizzly.java.net<mailto:users_at_grizzly.java.net>
Subject: Re: Grizzly Thread Pool Monitoring
Hi Uri,
looks like a bug to me, 12 core threads are not counted as allocated.
Can you pls. file an issue?
Thanks.
WBR,
Alexey.
On 10.03.15 10:26, Broide Uri wrote:
Hi
We are performing Stability load run against our Grizzly based application.
We are trying to determine how to monitor the Thread Pool usage via JMX.
We are looking at the ThreadPool MBean (org.glassfish.grizzly:pp=/gmbal-root/TCPNIOTransport[EndPoint.IMAP.NON_SECURED],type=ThreadPool,name=ThreadPool), on the following attributes:
• thread-pool-max-num-threads
• thread-pool-allocated-thread-count
• thread-pool-core-pool-size
We have conducted 2 load runs (with exactly the same load model):
1. max threads was configured to be 200 (verified against thread-pool-max-num-threads)
o from some point of the load run the thread-pool-allocated-thread-count was stabilized on 188
o thread-pool-core-pool-size was fixed on 12
2. max threads was configured to be 300 (verified against thread-pool-max-num-threads)
o from some point of the load run the thread-pool-allocated-thread-count was stabilized on 288
o thread-pool-core-pool-size was fixed on 12
The questions are as follows:
1. Are we looking on the right MBean attributes?
2. Assuming 1 is true, then why for the same load model the thread pool is reaching this high number (max = core + allocated)?
3. Why the thread-pool-started attribute is false?
Attaching the JConsole print screen of the second run
[cid:image001.png_at_01D05B52.9D7A53B0]
Thanks
Uri
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com<mailto:security_at_comverse.com>. Thank You.”
________________________________
“This e-mail message may contain confidential, commercial or privileged information that constitutes proprietary information of Comverse Inc. or its subsidiaries. If you are not the intended recipient of this message, you are hereby notified that any review, use or distribution of this information is absolutely prohibited and we request that you delete all copies and contact us by e-mailing to: security_at_comverse.com. Thank You.”
(image/jpeg attachment: image003.jpg)