users@glassfish.java.net

Re: All request processing threads are busy and connections count increasing

From: Jagadish Prasath Ramu <Jagadish.Ramu_at_Sun.COM>
Date: Mon, 23 Mar 2009 19:49:08 +0530

On Mon, 2009-03-23 at 15:01 +0100, Jeanfrancois Arcand wrote:
> Salut,
>
> Jagadish Prasath Ramu wrote:
> > Hi Jean-Francois,
> >
> > I do not suspect this as related to connection pool. This is an after
> > effect.
>
> I'm not following you. Have you looked at the current thread dump? 100%
> of the time they all lock on the code.
Yes, I did.
There are 10 httpSSL threads
httpSSLWorkerThread-8080- (0-9)
of which 3 are w.r.t pool and 7 are not related to pool and are not
dependent on pool. I do not see that as an issue.

@osmanizbat :
However, you can try the steps as indicated in the thread below (end of
the thread)
http://forums.java.net/jive/thread.jspa?threadID=58379&start=10&tstart=0
and let us know the results.

Also, if you are using v2.1, you can enable the diagnostic flag as
indicated the following issue
https://glassfish.dev.java.net/issues/show_bug.cgi?id=6624

More responses inline,

>
> > As I have posted previouly, please refer the other two thread dumps on
> > different occasions (different dates) where it was hung that does not
> > have connection pool.
> >
> > 1)
> > "All Grizzly threads are definitely locked. I'm surprised to see you
> > have
> > almost 1000 grizzly threads for your http listener since the default is
> > 5:"
> > http://forums.java.net/jive/thread.jspa?messageID=334999
>
> This is the normal behavior? Threads are just waiting for work :)
The user is stating that it is hung. Also, please refer the other stack
trace (2) below.

@osmanizbat : can you confirm ?

Thanks,
-Jagadish

>
> Thanks
>
> -- Jeanfrancois
>
>
> >
> > 2)
> > http://forums.java.net/jive/servlet/JiveServlet/download/56-59168-338326-8122/glassfish-20032009.tdump
> > http://forums.java.net/jive/servlet/JiveServlet/download/56-58413-334999-7799/thread_dump_04032009.txt.tar.gz
> >
> > that does not have connection pool involved.
> >
> > Thanks,
> > -Jagadish
> >
> >
> >
> >
> >
> > On Mon, 2009-03-23 at 14:20 +0100, Jeanfrancois Arcand wrote:
> >> Salut,
> >>
> >> yes :
> >>
> >>> "httpSSLWorkerThread-8080-9" daemon prio=10 tid=0x00000000477aec00 nid=0x1870 waiting for monitor entry [0x0000000042327000..0x0000000042329b10]
> >>> java.lang.Thread.State: BLOCKED (on object monitor)
> >>> at com.sun.enterprise.resource.AbstractResourcePool.startConnectionLeakTracing(AbstractResourcePool.java:309)
> >>> - locked <0x00002aaabc809058> (a java.lang.Object)
> >>> at com.sun.enterprise.resource.AbstractResourcePool.setResourceStateToBusy(AbstractResourcePool.java:302)
> >>> at com.sun.enterprise.resource.AbstractResourcePool.getUnenlistedResource(AbstractResourcePool.java:688)
> >>> at com.sun.enterprise.resource.AbstractResourcePool.internalGetResource(AbstractResourcePool.java:606)
> >>> at com.sun.enterprise.resource.AbstractResourcePool.getResource(AbstractResourcePool.java:455)
> >>> at com.sun.enterprise.resource.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:248)
> >>> at com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:176)
> >>> at com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:327)
> >>> at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
> >>> at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
> >>> at com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
> >>> at com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:108)
> >>> at oracle.toplink.essentials.jndi.JNDIConnector.connect(JNDIConnector.java:145)
> >>> at oracle.toplink.essentials.sessions.DatasourceLogin.connectToDatasource(DatasourceLogin.java:184)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatasourceAccessor.connect(DatasourceAccessor.java:233)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.connect(DatabaseAccessor.java:242)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatasourceAccessor.reconnect(DatasourceAccessor.java:436)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.reconnect(DatabaseAccessor.java:1177)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatasourceAccessor.incrementCallCount(DatasourceAccessor.java:220)
> >>> - locked <0x00002aaae8dfa168> (a oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:479)
> >>> at oracle.toplink.essentials.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:452)
> >>> at oracle.toplink.essentials.threetier.ServerSession.executeCall(ServerSession.java:473)
> >> all thread dead end there :-) Binod/Jagadish can you help? It i possibly
> >> a configuration issue.
> >>
> >> A+
> >>
> >> -- Jeanfrancois
> >>
> >> glassfish_at_javadesktop.org wrote:
> >>> JeanFrancois : I guess, osmanizbat posted another thread-dump which does not have anything to do with jdbc-connection-pool some time back.
> >>> Can you please refer this ?
> >>> http://forums.java.net/jive/thread.jspa?messageID=334999
> >>> [Message sent by forum member 'jr158900' (jr158900)]
> >>>
> >>> http://forums.java.net/jive/thread.jspa?messageID=338297
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> >>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> >> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
> >>
> >