users@glassfish.java.net

Re: Glassfish stops responding.

From: Ryan de Laplante <ryan_at_ijws.com>
Date: Wed, 04 Mar 2009 09:21:29 -0500

All Grizzly threads are definitely locked. I'm surprised to see you have
almost 1000 grizzly threads for your http listener since the default is 5:

Thread "httpWorkerThread-8080-999" thread-id 9,063
thread-stateWAITINGWaiting on lock:
com.sun.enterprise.web.connector.grizzly.LinkedListPipeline_at_ef0860e
     at: java.lang.Object.wait(Native Method)
     at: java.lang.Object.wait(Object.java:485)
     at:
com.sun.enterprise.web.connector.grizzly.LinkedListPipeline.getTask(LinkedListPipeline.java:291)
     at:
com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:114)

Thread "httpSSLWorkerThread-8181-9" thread-id 97
thread-stateWAITINGWaiting on lock:
com.sun.enterprise.web.connector.grizzly.ssl.SSLPipeline_at_69c97a6b
     at: java.lang.Object.wait(Native Method)
     at: java.lang.Object.wait(Object.java:485)
     at:
com.sun.enterprise.web.connector.grizzly.LinkedListPipeline.getTask(LinkedListPipeline.java:291)
     at:
com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:104)

Thread "httpWorkerThread-4848-9" thread-id 111
thread-stateWAITINGWaiting on lock:
com.sun.enterprise.web.connector.grizzly.LinkedListPipeline_at_5e5f60f2
     at: java.lang.Object.wait(Native Method)
     at: java.lang.Object.wait(Object.java:485)
     at:
com.sun.enterprise.web.connector.grizzly.LinkedListPipeline.getTask(LinkedListPipeline.java:291)
     at:
com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:114)

... those seem to be waiting on something like this:

Thread "GrizzlyPipelineStatWorkerThread-8080-0" thread-id 84
thread-stateTIMED_WAITINGWaiting on lock:
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject_at_581154
     at: sun.misc.Unsafe.park(Native Method)
     at:
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
     at:
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
     at: java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
     at:
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
     at:
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
     at:
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:946)
     at:
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:906)
     at:
com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:107)


and I don't know what that means... it looks like something Sun will
have to help you with. It is not immediately obvious to me. When I
used to have this problem the grizzly threads were blocked on a
ResourceManager inside of GlassFish. The ResourceManager was blocked on
a JDBC driver trying to establish a connection forever.

Also beware that passwords are stored in plain text inside domain.xml
(such as jdbc connection pools). The one you attached seems to be for a
development or test computer, but remember this before emailing
production domain.xml files to peope -- even Sun. I always scrub
passwords and other sensitive information before sending to Sun Support.


Thanks,
Ryan



glassfish_at_javadesktop.org wrote:
> Hi.
>
> We're running a 100k.daily pageview website on glassfish v2ur2.
> Some times glassfish stops responding once in a few days although there is not too much traffic.
>
> Could you please check our thread dump.
>
> Regards.
> [Message sent by forum member 'osmanizbat' (osmanizbat)]
>
> http://forums.java.net/jive/thread.jspa?messageID=334999
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>
>
>