Please disable connection reclaim and this deadlock will not happen. So
it will display all connection leaks, however will not reclaim any
leaking connection. In such a case, connection may get exhausted due to
connection leaks. Fix for bug 3960 is not available right now.
Meanwhile you can fix your application to avoid connection leaks.
Thanks,
Kshitiz
Shalini Muthukrishnan wrote:
> Looks like this is related to
> https://glassfish.dev.java.net/issues/show_bug.cgi?id=3960
> This will be fixed in 9.1.1.
>
> glassfish_at_javadesktop.org wrote:
>> We are facing problems with [b]deadlocks[/b] in glassfish server 9.1
>> (build b58g-fcs).
>>
>> Our application leaks connections, so we enabled connection leak
>> detection. But a timer thread of the leak detection collides with a
>> work thread. Clean shutdown was impossible.
>>
>> Our parameters:
>>
>> pool size 64 or 80
>> Leak Timeout = 300s
>> Leak Reclaim enabled
>> Lazy Association enabled
>> Lazy Connection Enlistment enabled
>> Associate with Thread enabled
>> Match Connections enabled
>>
>> <quotation from log>
>>
>> Found one Java-level deadlock:
>> =============================
>> "RMI TCP Connection(37)-127.0.0.1":
>> waiting to lock monitor 0x0830c1e4 (object 0x62895970, a
>> com.sun.enterprise.resource.AssocWithThreadResourcePool),
>> which is held by "httpSSLWorkerThread-39080-1"
>> "httpSSLWorkerThread-39080-1":
>> waiting to lock monitor 0x08ec27f0 (object 0x6289e308, a
>> java.lang.Object),
>> which is held by "Timer-6"
>> "Timer-6":
>> waiting to lock monitor 0x0830c1e4 (object 0x62895970, a
>> com.sun.enterprise.resource.AssocWithThreadResourcePool),
>> which is held by "httpSSLWorkerThread-39080-1"
>>
>> Java stack information for the threads listed above:
>> ===================================================
>> "RMI TCP Connection(37)-127.0.0.1":
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.cancelResizerTask(AbstractResourcePool.java:1782)
>>
>> - waiting to lock <0x62895970> (a
>> com.sun.enterprise.resource.AssocWithThreadResourcePool)
>> at
>> com.sun.enterprise.resource.PoolManagerImpl.killPool(PoolManagerImpl.java:573)
>>
>> at
>> com.sun.enterprise.resource.PoolManagerImpl.killAllPools(PoolManagerImpl.java:867)
>>
>> at
>> com.sun.enterprise.server.ApplicationServer.onTermination(ApplicationServer.java:574)
>>
>> at
>> com.sun.enterprise.server.Shutdown.startShutdown(Shutdown.java:76)
>> at
>> com.sun.enterprise.admin.event.AdminEventMulticaster.invokeShutdownEventListener(AdminEventMulticaster.java:1501)
>>
>> at
>> com.sun.enterprise.admin.event.AdminEventMulticaster.handleShutdownEvent(AdminEventMulticaster.java:1490)
>>
>> at
>> com.sun.enterprise.admin.event.AdminEventMulticaster.processEvent(AdminEventMulticaster.java:480)
>>
>> at
>> com.sun.enterprise.admin.event.AdminEventMulticaster.multicastEvent(AdminEventMulticaster.java:176)
>>
>> at
>> com.sun.enterprise.admin.server.core.channel.AdminChannelServer.sendNotification(AdminChannelServer.java:121)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
>> at sun.rmi.transport.Transport$1.run(Transport.java:159)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at sun.rmi.transport.Transport.serviceCall(Transport.java:155)
>> at
>> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
>> at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
>>
>> at
>> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
>>
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
>>
>> at java.lang.Thread.run(Thread.java:619)
>> "httpSSLWorkerThread-39080-1":
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.startConnectionLeakTracing(AbstractResourcePool.java:309)
>>
>> - waiting to lock <0x6289e308> (a java.lang.Object)
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.setResourceStateToBusy(AbstractResourcePool.java:301)
>>
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.getResourceFromPool(AbstractResourcePool.java:778)
>>
>> - locked <0x62895970> (a
>> com.sun.enterprise.resource.AssocWithThreadResourcePool)
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.getUnenlistedResource(AbstractResourcePool.java:652)
>>
>> at
>> com.sun.enterprise.resource.AssocWithThreadResourcePool.getUnenlistedResource(AssocWithThreadResourcePool.java:136)
>>
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.internalGetResource(AbstractResourcePool.java:594)
>>
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.getResource(AbstractResourcePool.java:443)
>>
>> at
>> com.sun.enterprise.resource.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:248)
>>
>> at
>> com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:176)
>>
>> at
>> com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:327)
>>
>> at
>> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
>>
>> at
>> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
>>
>> at
>> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
>>
>> at
>> com.sun.gjc.spi.base.DataSource.getConnection(DataSource.java:108)
>> at
>> de.denic.guiservices.ejb.AuthServiceManagerBean.getRoles(AuthServiceManagerBean.java:60)
>>
>> at sun.reflect.GeneratedMethodAccessor323.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> com.sun.enterprise.security.SecurityUtil$2.run(SecurityUtil.java:182)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at
>> com.sun.enterprise.security.application.EJBSecurityManager.doAsPrivileged(EJBSecurityManager.java:1041)
>>
>> at
>> com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:187)
>> at
>> com.sun.ejb.containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:2895)
>>
>> at
>> com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:3986)
>> at
>> com.sun.ejb.containers.EJBObjectInvocationHandler.invoke(EJBObjectInvocationHandler.java:203)
>>
>> at
>> com.sun.ejb.containers.EJBObjectInvocationHandlerDelegate.invoke(EJBObjectInvocationHandlerDelegate.java:77)
>>
>> at $Proxy45.getRoles(Unknown Source)
>> at sun.reflect.GeneratedMethodAccessor322.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.privateInvoke(StubInvocationHandlerImpl.java:233)
>>
>> at
>> com.sun.corba.ee.impl.presentation.rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:152)
>>
>> at
>> com.sun.corba.ee.impl.presentation.rmi.bcel.BCELStubBase.invoke(BCELStubBase.java:225)
>>
>> at
>> de.denic.guiservices.__AuthServiceManager_Remote_DynamicStub.getRoles(de/denic/guiservices/__AuthServiceManager_Remote_DynamicStub.java)
>>
>> at
>> de.denic.guiservices._AuthServiceManager_Wrapper.getRoles(de/denic/guiservices/_AuthServiceManager_Wrapper.java)
>>
>> at
>> de.denic.mir.controller.Authorisation.getRegAccIdsWhereMemberIsDenied(Authorisation.java:165)
>>
>> - locked <0x63f4eb38> (a de.denic.mir.controller.Authorisation)
>> at
>> de.denic.mir.controller.MirRegAccController.loadRegAccListTest(MirRegAccController.java:484)
>>
>> at
>> de.denic.mir.controller.MirRegAccController.loadRegAccList(MirRegAccController.java:417)
>>
>> at
>> de.denic.mir.controller.MirRegAccController.init(MirRegAccController.java:153)
>>
>> at
>> org.apache.jsp.de.jsf.mirRegAcc.Init_jsp._jspService(Init_jsp.java
>> from :86)
>> at
>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:93)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
>> at
>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:373)
>>
>> at
>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:470)
>> at
>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:364)
>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
>> at sun.reflect.GeneratedMethodAccessor325.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:276)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
>> at
>> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
>> at
>> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:192)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:404)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.doInvoke(ApplicationDispatcher.java:855)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:703)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:542)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:474)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher.access$000(ApplicationDispatcher.java:107)
>>
>> at
>> org.apache.catalina.core.ApplicationDispatcher$PrivilegedForward.run(ApplicationDispatcher.java:122)
>>
>> at java.security.AccessController.doPrivileged(Native Method)
>> at
>> org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:352)
>>
>> at
>> com.sun.faces.context.ExternalContextImpl.dispatch(ExternalContextImpl.java:408)
>>
>> at
>> com.sun.faces.application.ViewHandlerImpl.executePageToBuildView(ViewHandlerImpl.java:442)
>>
>> at
>> com.sun.faces.application.ViewHandlerImpl.renderView(ViewHandlerImpl.java:115)
>>
>> at
>> com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:106)
>>
>> at
>> com.sun.faces.lifecycle.LifecycleImpl.phase(LifecycleImpl.java:251)
>> at
>> com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:144)
>> at
>> javax.faces.webapp.FacesServlet.service(FacesServlet.java:245)
>> at sun.reflect.GeneratedMethodAccessor358.invoke(Unknown Source)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at
>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:276)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
>> at
>> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:309)
>> at
>> org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:192)
>>
>> at
>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:404)
>>
>> at
>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:290)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:271)
>>
>> at
>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:202)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
>>
>> at
>> com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:94)
>> at
>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:206)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
>>
>> at
>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
>> at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:150)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:632)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:577)
>>
>> at
>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:571)
>>
>> at
>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:1080)
>> at
>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:270)
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:637)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.doProcess(DefaultProcessorTask.java:568)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.process(DefaultProcessorTask.java:813)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultReadTask.executeProcessorTask(DefaultReadTask.java:339)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:261)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.DefaultReadTask.doTask(DefaultReadTask.java:212)
>>
>> at
>> com.sun.enterprise.web.portunif.PortUnificationPipeline$PUTask.doTask(PortUnificationPipeline.java:361)
>>
>> at
>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:265)
>> at
>> com.sun.enterprise.web.connector.grizzly.ssl.SSLWorkerThread.run(SSLWorkerThread.java:106)
>>
>> "Timer-6":
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.freeResource(AbstractResourcePool.java:1060)
>>
>> - waiting to lock <0x62895970> (a
>> com.sun.enterprise.resource.AssocWithThreadResourcePool)
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.potentialConnectionLeakFound(AbstractResourcePool.java:344)
>>
>> - locked <0x6289e308> (a java.lang.Object)
>> at
>> com.sun.enterprise.resource.AbstractResourcePool.access$000(AbstractResourcePool.java:62)
>>
>> at
>> com.sun.enterprise.resource.AbstractResourcePool$ConnectionLeakTask.run(AbstractResourcePool.java:1543)
>>
>> at java.util.TimerThread.mainLoop(Timer.java:512)
>> at java.util.TimerThread.run(Timer.java:462)
>>
>> Found 1 deadlock.
>> [Message sent by forum member 'wjp' (wjp)]
>>
>> http://forums.java.net/jive/thread.jspa?messageID=267464
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>