Hi,
We're performance tuning an application running on Glassfish, and are
trying to make sense of the values we get from the monitoring. Our
application is a Java EE 6 application, running a front end part with
JSF on one server and a back end part with EJBs on a different server,
and we're getting very high response times on the front end server.
When we measure the performance, we see that the load is high. This
should indicate that something is being blocked, but the CPU
utilization is low there is hardly any disk I/O. The database on a
third server is also doing fine. Somehow, something is causing the
servers not to be utilized fully, and we're trying to find out what
buttons we have to push to make it actually use our servers. We've
ruled out incoming HTTP connections, since we turned up the number of
max incoming connections, and got worse performance. The problem seems
to be that the work doesn't get executed fast enough, even though we
have plenty of free CPU and network/disk I/O.
So, we've turned on monitoring and are trying to find out where the problem is.
In particular, we're wondering about the relationship between
server.thread-pool.orb.threadpool.thread-pool-1.currentbusythreads-count
and server.thread-pool.orb.threadpool.thread-pool-1.numberofworkitemsinqueue-current.
On both the front end and the back end servers,
currentbusythreads-count is consistently low (around 15 - 17), and
numberofworkitemsinqueue-current consistently high (400 - 500) - this
despite the number of available threads is very high on the back end
server (server.thread-pool.orb.threadpool.thread-pool-1.numberofavailablethreads-count
is around 900).
The way we're reading this, is that work gets queued, even though
there are plently of threads available.
Are we reading this the right way? Why is
numberofworkitemsinqueue-current so high?
Are there other values we should be monitoring?
Any other ideas?
We're using Glassfish 3.1.1, JDK 1.6 and Linux.
Regards,
Vetle