users@glassfish.java.net

Re: GlassFish v3 jdbc In-use connections equal max-pool-size

From: Jagadish Prasath Ramu <Jagadish.Ramu_at_Sun.COM>
Date: Fri, 03 Dec 2010 21:39:15 +0530

Hi,

The common stack trace that you are seeing is application server's call
trace. There will be application's call trace that acquired the
connection will be printed in server.log
It indicates the caller that has acquired the connection but did not
return the connection back to the pool in specified timeout period
(connection-leak-timeout-in-seconds attribute of jdbc-connection-pool).

For more information about leak-tracing and recliam :
http://blogs.sun.com/kshitiz/entry/connection_leak_tracing

If connection leak-reclaim is enabled, connection will be returned to
the pool (it may be killed to avoid the leaked client from misusing it
and will be replaced by a new connection).

Note : It may happen that the application is caching the connection in
which case it will not be returned to the pool within specified period.
You may have to tune the pool settings. The maximum load on your system
is more than what the pool could offer. (refer max-pool-size)

Also, you have set max-connection-usage to 1 which indicates that the
connection will be destroyed (and a new one will be replaced with) after
re-using it only once which will prove to be a costlier operation.

You can try the following :

1) disable max-connection-usage
2) disable associate-with-thread
3) disable connection-leak-reclaim
4) enable connection-leak-timeout.
Find the callers that did not return the connection. If you see those
callers are not closing the connection, application need to be fixed. If
not, check the load on your system and tune the pool settings
(max-pool-size, pool-resize-quantity, max-wait-time-in-milliseconds)
accordingly.

Thanks,
-Jagadish








On Tue, 2010-11-23 at 12:44 -0800, forums_at_java.net wrote:
> I'm using GlassFish v3.0.1 in our production environment. We have several
> applications deployed, each using their own datasource. We are encountering
> an issue where we seem to run out of database connections. We constantly seem
> to hit the following error:
>
> /*java.sql.SQLException: Error in allocating a connection. Cause: In-use
> connections equal max-pool-size and expired max-wait-time. Cannot allocate
> more connections.*/
>
> To try and identify the issue I enabled leak detection by setting the Leak
> Timeout to 180 seconds and enabled Leak Reclaim. This resulted in stack
> traces being written out to the log file stating:
>
> /*A potential connection leak detected for connection pool */
>
> While this would tell me about the leaked connection it did not help identify
> the offender because the leaks appear in different datasources / applications
> each time. Each application uses a its own datasource and in some cases a
> different persistence technology: (one uses JPA with EclipseLink, others use
> Hibernate with Spring) This leads me to believe that there may be an issue
> with GlassFish and the way it is managing the connections.
>
>
>
> I have configured each connection pool with the following characteristics:
>
> Wrap JDBC Objects: enabled
>
> Pooling: enabled
>
> Leak Timeout: 180
>
> Leak Reclaim: enabled
>
> Creation Retry Attempts: 6
>
> Retry Interval: 10
>
> Associate With Thread: enabled
>
> Max Connection Usage: 1
>
> Connection Validation: enabled
>
> Validation Method: auto-commit
>
> Transaction Isolation: read-committed
>
> Isolation Level: guaranteed
>
>
>
> Each time a stack trace is printed pertaining to a leaked connection, the one
> thing that seems constant is the method from which the exception was thrown
> in the stack trace:
>
>
>
> /The stack trace of the thread is provided below :/
> /com.sun.enterprise.resource.pool.ConnectionPool.setResourceStateToBusy(ConnectionPool.java:319)/
> /com.sun.enterprise.resource.pool.ConnectionPool.getResourceFromPool(ConnectionPool.java:694)/
> /com.sun.enterprise.resource.pool.ConnectionPool.getUnenlistedResource(ConnectionPool.java:572)/
> /com.sun.enterprise.resource.pool.AssocWithThreadResourcePool.getUnenlistedResource(AssocWithThreadResourcePool.java:164)/
> /com.sun.enterprise.resource.pool.ConnectionPool.internalGetResource(ConnectionPool.java:467)/
> /com.sun.enterprise.resource.pool.ConnectionPool.getResource(ConnectionPool.java:369)/
> /com.sun.enterprise.resource.pool.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:226)/
> /com.sun.enterprise.resource.pool.PoolManagerImpl.getResource(PoolManagerImpl.java:150)/
> /com.sun.enterprise.connectors.ConnectionManagerImpl.getResource(ConnectionManagerImpl.java:327)/
> /com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:290)/
> /com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:227)/
> /com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:159)/
> /com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:154)/
> /com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:105)/ It
> appears that there may be a deadlock situation occurring
> in /ConnectionPool.setResourceStateToBusy./ Currently the only ay to
> resolve this issue is to bounce the glassfish instance as I eventually run
> out of connection, but this is not a viable work-around in a production
> environment. Are there any configuration changes that I can make that will
> prevent this from occurring? Is this a know issue that will be addressed in a
> patch or an up-coming release?
>
>
>
> --
>
> [Message sent by forum member 'fericit_bostan']
>
> View Post: http://forums.java.net/node/722732
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>