Can someone in the glassfish help here?
- Binod.
On 02-Aug-09, at 11:27 AM, daniel_leo wrote:
>
> Hi,
>
> We got one problem, which looks hanging of some EJB threads.
>
> - 2 nodes are configured as a sailfin cluster, e.g, instance1, and
> instance2
> - run traffic towards both nodes, i.e., instance1 and instance2
> - reboot one node, e.g., instance2
> - instance suddenly does NOT work
>
> Part of the thread dump as below.
> It looks like quite a lot of threads are in hanging state.
> All of these threads are locked at <0x00002aaab87a7ef8>
> All of these threads are waiting for some monitor.
> Any possible reasons for this problem?
> Thanks!
>
>
> "p: thread-pool-1; w: 8" daemon prio=10 tid=0x00002aab78279800
> nid=0x1e5d in
> Object.wait() [0x0000000040ffc000..0x0000000040ffdca0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.countedWait(POAManagerImpl.java:
> 172)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.checkState(POAManagerImpl.java:
> 537)
> at
> com
> .sun.corba.ee.impl.oa.poa.POAManagerImpl.enter(POAManagerImpl.java:
> 571)
> - locked <0x00002aaab87a7ef8> (a
> com.sun.corba.ee.impl.oa.poa.POAManagerImpl)
> at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1576)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .FullServantCacheLocalCRDImpl
> .internalPreinvoke(FullServantCacheLocalCRDImpl.java:80)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .LocalClientRequestDispatcherBase
> .servant_preinvoke(LocalClientRequestDispatcherBase.java:217)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaClientDelegateImpl
> .servant_preinvoke(CorbaClientDelegateImpl.java:516)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi
> .StubInvocationHandlerImpl
> .privateInvoke(StubInvocationHandlerImpl.java:210)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:
> 152)
> at
> com
> .sun
> .corba
> .ee.impl.presentation.rmi.bcel.BCELStubBase.invoke(BCELStubBase.java:
> 225)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api
> .__SessionCorrelationService_Remote_DynamicStub.claimSessionID(com/
> ericsson/jee/tcap/fw/sessioncorrelation/api/
> __SessionCorrelationService_Remote_DynamicStub.java)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api._SessionCorrelationService_Wrapper.claimSessionID(com/ericsson/
> jee/tcap/fw/sessioncorrelation/api/
> _SessionCorrelationService_Wrapper.java)
> at
> com.ericsson.jee.tcap.fw.util.client.ClientSipState
> $StateConnecting.claimSessionId(ClientSipState.java:235)
> at
> com.ericsson.jee.tcap.fw.util.client.ClientSipState
> $StateConnecting.onEntry(ClientSipState.java:215)
> at com.ericsson.util.state.State.enter(State.java:40)
> - locked <0x00002aaac022b5d0> (a
> com.ericsson.jee.tcap.fw.util.client.SipMessageProcessor)
> - locked <0x00002aaabb2ab638> (a
> com.ericsson.jee.tcap.fw.util.client.ClientSipState$StateConnecting)
> at com.ericsson.util.state.State.gotoState(State.java:86)
> - locked <0x00002aaac10ff850> (a
> com.ericsson.jee.tcap.fw.util.client.ClientSipState$StateInitial)
> at
> com.ericsson.jee.tcap.fw.util.client.ClientSipState
> $StateInitial.sendTcapMessage(ClientSipState.java:162)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .client.SipMessageProcessor.sendTcapMessage(SipMessageProcessor.java:
> 152)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .client.TcapMessageProcessor.onTcapMessage(TcapMessageProcessor.java:
> 64)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .beans
> .TcapMessageReceiverMDB.onTcapMessage(TcapMessageReceiverMDB.java:69)
> at sun.reflect.GeneratedMethodAccessor183.invoke(Unknown Source)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> com
> .sun
> .enterprise
> .security
> .application.EJBSecurityManager.runMethod(EJBSecurityManager.java:
> 1011)
> at
> com.sun.enterprise.security.SecurityUtil.invoke(SecurityUtil.java:175)
> at
> com
> .sun
> .ejb
> .containers.BaseContainer.invokeTargetBeanMethod(BaseContainer.java:
> 2928)
> at
> com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:
> 4019)
> at
> com
> .sun
> .ejb
> .containers
> .MessageBeanContainer.deliverMessage(MessageBeanContainer.java:1111)
> at
> com
> .sun
> .ejb
> .containers
> .MessageBeanListenerImpl.deliverMessage(MessageBeanListenerImpl.java:
> 74)
> at
> com
> .sun
> .enterprise
> .connectors
> .inflow
> .MessageEndpointInvocationHandler
> .invoke(MessageEndpointInvocationHandler.java:179)
> at $Proxy128.onTcapMessage(Unknown Source)
> at
> com
> .ericsson
> .jee
> .ss7
> .ra.inbound.MessageDistributor.sendMessage(MessageDistributor.java:
> 119)
> at
> com
> .ericsson
> .jee.ss7.ra.inbound.MessageDistributor.run(MessageDistributor.java:98)
> at com.sun.enterprise.connectors.work.OneWork.doWork(OneWork.java:76)
> at
> com.sun.corba.ee.impl.orbutil.threadpool.ThreadPoolImpl
> $WorkerThread.run(ThreadPoolImpl.java:555)
>
> "SipContainer-serversWorkerThread-5060-14" daemon prio=10
> tid=0x00002aab808b3800 nid=0x1d36 in Object.wait()
> [0x0000000042165000..0x0000000042167e20]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.countedWait(POAManagerImpl.java:
> 172)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.checkState(POAManagerImpl.java:
> 537)
> at
> com
> .sun.corba.ee.impl.oa.poa.POAManagerImpl.enter(POAManagerImpl.java:
> 571)
> - locked <0x00002aaab87a7ef8> (a
> com.sun.corba.ee.impl.oa.poa.POAManagerImpl)
> at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1576)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .getServantWithPI(CorbaServerRequestDispatcherImpl.java:378)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .dispatch(CorbaServerRequestDispatcherImpl.java:224)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaMessageMediatorImpl
> .handleRequestRequest(CorbaMessageMediatorImpl.java:1846)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .SharedCDRClientRequestDispatcherImpl
> .marshalingComplete(SharedCDRClientRequestDispatcherImpl.java:183)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:219)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi
> .StubInvocationHandlerImpl
> .privateInvoke(StubInvocationHandlerImpl.java:192)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:
> 152)
> at
> com
> .sun
> .corba
> .ee.impl.presentation.rmi.bcel.BCELStubBase.invoke(BCELStubBase.java:
> 225)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api
> .__SessionCorrelationService_Remote_DynamicStub.releaseSessionID(com/
> ericsson/jee/tcap/fw/sessioncorrelation/api/
> __SessionCorrelationService_Remote_DynamicStub.java)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api._SessionCorrelationService_Wrapper.releaseSessionID(com/
> ericsson/jee/tcap/fw/sessioncorrelation/api/
> _SessionCorrelationService_Wrapper.java)
> at
> com
> .ericsson.jee.tcap.fw.util.SasDataWrapper.remove(SasDataWrapper.java:
> 87)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .TerminatingServiceDomainSelection
> .onInitialDp(TerminatingServiceDomainSelection.java:95)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2.EventTypeBCSMSelector.onInitialDp(EventTypeBCSMSelector.java:
> 129)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInitialDp(Cap2Processor.java:231)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInvokeEvent(Cap2Processor.java:182)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase.onComponentEvent(TcapEventProcessorBase.java:
> 74)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase
> .onIncomingTcapMessage(TcapEventProcessorBase.java:55)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $Processing.onEntry(TcapServletState.java:254)
> at com.ericsson.util.state.State.enter(State.java:40)
> - locked <0x00002aaabefc3208> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletContext)
> - locked <0x00002aaac3679460> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$Processing)
> at com.ericsson.util.state.State.gotoState(State.java:86)
> - locked <0x00002aaac3679800> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$WaitingForAck)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $WaitingForAck.doAck(TcapServletState.java:201)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw.util.server.TcapServletContext.doAck(TcapServletContext.java:149)
> at
> com
> .ericsson.jee.tcap.fw.util.server.TcapServlet.doAck(TcapServlet.java:
> 156)
> at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:59)
> at
> com.ericsson.jee.sip.SipServletBase.doRequest(SipServletBase.java:54)
> at
> com
> .ericsson
> .jee.tcap.fw.util.server.TcapServlet.doRequest(TcapServlet.java:90)
> at javax.servlet.sip.SipServlet.service(SipServlet.java:46)
> at
> com
> .ericsson
> .ssa.container.sim.SipServletFacade.service(SipServletFacade.java:91)
> at com.ericsson.ssa.sip.INVITESession.dispatch(INVITESession.java:
> 1330)
> at com.ericsson.ssa.sip.UA.dispatch(UA.java:655)
> at
> com
> .ericsson
> .ssa
> .container
> .sim
> .ApplicationDispatcher
> .dispatchViaStatelessProxy(ApplicationDispatcher.java:802)
> at
> com
> .ericsson
> .ssa
> .container
> .sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:348)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:52)
> at
> com
> .ericsson
> .ssa
> .container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:
> 878)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:562)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:
> 134)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:
> 102)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.transaction.TransactionManager.next(TransactionManager.java:
> 270)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core
> .sip
> .SipLoadBalancerManagerBackEnd
> .next(SipLoadBalancerManagerBackEnd.java:182)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipRequestGroup.handleIncomingRequest(SipRequestGroup.java:
> 121)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipLoadBalancerManager.next(SipLoadBalancerManager.java:147)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .OutboundFlowManager.processOutboundRequest(OutboundFlowManager.java:
> 183)
> at
> com
> .ericsson
> .ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:
> 1256)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.processMessage(MessageProcessorFilter.java:
> 406)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.execute(MessageProcessorFilter.java:312)
> at
> com
> .sun
> .grizzly
> .DefaultProtocolChain
> .executeProtocolFilter(DefaultProtocolChain.java:136)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 103)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 89)
> at
> com
> .sun
> .grizzly
> .ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
> at
> com
> .sun
> .grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
> at
> com
> .sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:
> 325)
> at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:
> 184)
>
> "SipContainer-serversWorkerThread-5060-13" daemon prio=10
> tid=0x00002aab79e36400 nid=0x1d35 in Object.wait()
> [0x0000000040fda000..0x0000000040fdcda0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.countedWait(POAManagerImpl.java:
> 172)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.checkState(POAManagerImpl.java:
> 537)
> at
> com
> .sun.corba.ee.impl.oa.poa.POAManagerImpl.enter(POAManagerImpl.java:
> 571)
> - locked <0x00002aaab87a7ef8> (a
> com.sun.corba.ee.impl.oa.poa.POAManagerImpl)
> at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1576)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .getServantWithPI(CorbaServerRequestDispatcherImpl.java:378)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .dispatch(CorbaServerRequestDispatcherImpl.java:224)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaMessageMediatorImpl
> .handleRequestRequest(CorbaMessageMediatorImpl.java:1846)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .SharedCDRClientRequestDispatcherImpl
> .marshalingComplete(SharedCDRClientRequestDispatcherImpl.java:183)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:219)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi
> .StubInvocationHandlerImpl
> .privateInvoke(StubInvocationHandlerImpl.java:192)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .presentation
> .rmi.StubInvocationHandlerImpl.invoke(StubInvocationHandlerImpl.java:
> 152)
> at
> com
> .sun
> .corba
> .ee.impl.presentation.rmi.bcel.BCELStubBase.invoke(BCELStubBase.java:
> 225)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api
> .__SessionCorrelationService_Remote_DynamicStub.releaseSessionID(com/
> ericsson/jee/tcap/fw/sessioncorrelation/api/
> __SessionCorrelationService_Remote_DynamicStub.java)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .sessioncorrelation
> .api._SessionCorrelationService_Wrapper.releaseSessionID(com/
> ericsson/jee/tcap/fw/sessioncorrelation/api/
> _SessionCorrelationService_Wrapper.java)
> at
> com
> .ericsson.jee.tcap.fw.util.SasDataWrapper.remove(SasDataWrapper.java:
> 87)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .TerminatingServiceDomainSelection
> .onInitialDp(TerminatingServiceDomainSelection.java:95)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2.EventTypeBCSMSelector.onInitialDp(EventTypeBCSMSelector.java:
> 129)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInitialDp(Cap2Processor.java:231)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInvokeEvent(Cap2Processor.java:182)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase.onComponentEvent(TcapEventProcessorBase.java:
> 74)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase
> .onIncomingTcapMessage(TcapEventProcessorBase.java:55)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $Processing.onEntry(TcapServletState.java:254)
> at com.ericsson.util.state.State.enter(State.java:40)
> - locked <0x00002aaabe73bc40> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletContext)
> - locked <0x00002aaac196b4f0> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$Processing)
> at com.ericsson.util.state.State.gotoState(State.java:86)
> - locked <0x00002aaac196b890> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$WaitingForAck)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $WaitingForAck.doAck(TcapServletState.java:201)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw.util.server.TcapServletContext.doAck(TcapServletContext.java:149)
> at
> com
> .ericsson.jee.tcap.fw.util.server.TcapServlet.doAck(TcapServlet.java:
> 156)
> at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:59)
> at
> com.ericsson.jee.sip.SipServletBase.doRequest(SipServletBase.java:54)
> at
> com
> .ericsson
> .jee.tcap.fw.util.server.TcapServlet.doRequest(TcapServlet.java:90)
> at javax.servlet.sip.SipServlet.service(SipServlet.java:46)
> at
> com
> .ericsson
> .ssa.container.sim.SipServletFacade.service(SipServletFacade.java:91)
> at com.ericsson.ssa.sip.INVITESession.dispatch(INVITESession.java:
> 1330)
> at com.ericsson.ssa.sip.UA.dispatch(UA.java:655)
> at
> com
> .ericsson
> .ssa
> .container
> .sim
> .ApplicationDispatcher
> .dispatchViaStatelessProxy(ApplicationDispatcher.java:802)
> at
> com
> .ericsson
> .ssa
> .container
> .sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:348)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:52)
> at
> com
> .ericsson
> .ssa
> .container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:
> 878)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:562)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:
> 134)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:
> 102)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.transaction.TransactionManager.next(TransactionManager.java:
> 270)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core
> .sip
> .SipLoadBalancerManagerBackEnd
> .next(SipLoadBalancerManagerBackEnd.java:182)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipRequestGroup.handleIncomingRequest(SipRequestGroup.java:
> 121)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipLoadBalancerManager.next(SipLoadBalancerManager.java:147)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .OutboundFlowManager.processOutboundRequest(OutboundFlowManager.java:
> 183)
> at
> com
> .ericsson
> .ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:
> 1256)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.processMessage(MessageProcessorFilter.java:
> 406)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.execute(MessageProcessorFilter.java:312)
> at
> com
> .sun
> .grizzly
> .DefaultProtocolChain
> .executeProtocolFilter(DefaultProtocolChain.java:136)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 103)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 89)
> at
> com
> .sun
> .grizzly
> .ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
> at
> com
> .sun
> .grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
> at
> com
> .sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:
> 325)
> at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:
> 184)
>
> "SipContainer-serversWorkerThread-5060-12" daemon prio=10
> tid=0x00002aab798d2400 nid=0x1d34 in Object.wait()
> [0x0000000040f56000..0x0000000040f58b20]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> at java.lang.Object.wait(Object.java:485)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.countedWait(POAManagerImpl.java:
> 172)
> at
> com
> .sun
> .corba.ee.impl.oa.poa.POAManagerImpl.checkState(POAManagerImpl.java:
> 537)
> at
> com
> .sun.corba.ee.impl.oa.poa.POAManagerImpl.enter(POAManagerImpl.java:
> 571)
> - locked <0x00002aaab87a7ef8> (a
> com.sun.corba.ee.impl.oa.poa.POAManagerImpl)
> at com.sun.corba.ee.impl.oa.poa.POAImpl.enter(POAImpl.java:1576)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .getServantWithPI(CorbaServerRequestDispatcherImpl.java:378)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaServerRequestDispatcherImpl
> .dispatch(CorbaServerRequestDispatcherImpl.java:224)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaMessageMediatorImpl
> .handleRequestRequest(CorbaMessageMediatorImpl.java:1846)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .SharedCDRClientRequestDispatcherImpl
> .marshalingComplete(SharedCDRClientRequestDispatcherImpl.java:183)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol
> .CorbaClientDelegateImpl.invoke(CorbaClientDelegateImpl.java:219)
> at
> com
> .sun
> .corba
> .ee
> .impl
> .protocol.CorbaClientDelegateImpl.is_a(CorbaClientDelegateImpl.java:
> 327)
> at org.omg.CORBA.portable.ObjectImpl._is_a(ObjectImpl.java:112)
> at
> org
> .omg.CosNaming.NamingContextHelper.narrow(NamingContextHelper.java:69)
> at com.sun.jndi.cosnaming.CNCtx.callResolve(CNCtx.java:472)
> at com.sun.jndi.cosnaming.CNCtx.lookup(CNCtx.java:523)
> at com.sun.jndi.cosnaming.CNCtx.lookup(CNCtx.java:501)
> at javax.naming.InitialContext.lookup(InitialContext.java:392)
> at
> com
> .sun
> .enterprise
> .naming
> .factory.IIOPObjectFactory.getObjectInstance(IIOPObjectFactory.java:
> 88)
> at
> javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:
> 304)
> at
> com
> .sun
> .enterprise
> .naming
> .LocalSerialContextProviderImpl
> .lookup(LocalSerialContextProviderImpl.java:123)
> at
> com.sun.enterprise.naming.SerialContext.lookup(SerialContext.java:409)
> at javax.naming.InitialContext.lookup(InitialContext.java:392)
> at
> com
> .sun
> .ejb
> .containers
> .RemoteBusinessObjectFactory
> .getObjectInstance(RemoteBusinessObjectFactory.java:70)
> at
> javax.naming.spi.NamingManager.getObjectInstance(NamingManager.java:
> 304)
> at
> com.sun.enterprise.naming.SerialContext.lookup(SerialContext.java:414)
> at javax.naming.InitialContext.lookup(InitialContext.java:392)
> at
> com
> .ericsson
> .jee
> .mae
> .sessioncorrelation
> .api
> .MaeSessionCorrelationUtil
> .lookupService(MaeSessionCorrelationUtil.java:47)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .OriginatingServiceDomainSelection
> .getMaeCorrelationService(OriginatingServiceDomainSelection.java:458)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .OriginatingServiceDomainSelection
> .generateCorrelationId(OriginatingServiceDomainSelection.java:551)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .OriginatingServiceDomainSelection
> .generateCorrelationAndAttachToSession
> (OriginatingServiceDomainSelection.java:521)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2
> .OriginatingServiceDomainSelection
> .onInitialDp(OriginatingServiceDomainSelection.java:270)
> at
> com
> .ericsson
> .jee
> .mae
> .impl
> .cap2.EventTypeBCSMSelector.onInitialDp(EventTypeBCSMSelector.java:
> 129)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInitialDp(Cap2Processor.java:231)
> at
> com
> .ericsson
> .jee.mae.impl.cap2.Cap2Processor.onInvokeEvent(Cap2Processor.java:182)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase.onComponentEvent(TcapEventProcessorBase.java:
> 74)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw
> .util
> .TcapEventProcessorBase
> .onIncomingTcapMessage(TcapEventProcessorBase.java:55)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $Processing.onEntry(TcapServletState.java:254)
> at com.ericsson.util.state.State.enter(State.java:40)
> - locked <0x00002aaac0f6fe10> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletContext)
> - locked <0x00002aaabc555118> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$Processing)
> at com.ericsson.util.state.State.gotoState(State.java:86)
> - locked <0x00002aaabd3c0108> (a
> com.ericsson.jee.tcap.fw.util.server.TcapServletState$WaitingForAck)
> at
> com.ericsson.jee.tcap.fw.util.server.TcapServletState
> $WaitingForAck.doAck(TcapServletState.java:201)
> at
> com
> .ericsson
> .jee
> .tcap
> .fw.util.server.TcapServletContext.doAck(TcapServletContext.java:149)
> at
> com
> .ericsson.jee.tcap.fw.util.server.TcapServlet.doAck(TcapServlet.java:
> 156)
> at javax.servlet.sip.SipServlet.doRequest(SipServlet.java:59)
> at
> com.ericsson.jee.sip.SipServletBase.doRequest(SipServletBase.java:54)
> at
> com
> .ericsson
> .jee.tcap.fw.util.server.TcapServlet.doRequest(TcapServlet.java:90)
> at javax.servlet.sip.SipServlet.service(SipServlet.java:46)
> at
> com
> .ericsson
> .ssa.container.sim.SipServletFacade.service(SipServletFacade.java:91)
> at com.ericsson.ssa.sip.INVITESession.dispatch(INVITESession.java:
> 1330)
> at com.ericsson.ssa.sip.UA.dispatch(UA.java:655)
> at
> com
> .ericsson
> .ssa
> .container
> .sim
> .ApplicationDispatcher
> .dispatchViaStatelessProxy(ApplicationDispatcher.java:802)
> at
> com
> .ericsson
> .ssa
> .container
> .sim.ApplicationDispatcher.dispatch(ApplicationDispatcher.java:348)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:52)
> at
> com
> .ericsson
> .ssa
> .container.sim.ApplicationDispatcher.next(ApplicationDispatcher.java:
> 878)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.LocalRouteManager.next(LocalRouteManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at com.ericsson.ssa.sip.DialogManager.next(DialogManager.java:562)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com.ericsson.ssa.sip.dns.ResolverManager.next(ResolverManager.java:
> 134)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.persistence.ReplicationManager.next(ReplicationManager.java:
> 102)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.sip.transaction.TransactionManager.next(TransactionManager.java:
> 270)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core
> .sip
> .SipLoadBalancerManagerBackEnd
> .next(SipLoadBalancerManagerBackEnd.java:182)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipRequestGroup.handleIncomingRequest(SipRequestGroup.java:
> 121)
> at
> org
> .jvnet
> .glassfish
> .comms
> .clb
> .core.sip.SipLoadBalancerManager.next(SipLoadBalancerManager.java:147)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .OutboundFlowManager.processOutboundRequest(OutboundFlowManager.java:
> 183)
> at
> com
> .ericsson
> .ssa.container.OutboundFlowManager.next(OutboundFlowManager.java:98)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa.container.GrizzlyNetworkManager.next(GrizzlyNetworkManager.java:
> 1256)
> at com.ericsson.ssa.sip.LayerHelper.next(LayerHelper.java:59)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.processMessage(MessageProcessorFilter.java:
> 406)
> at
> com
> .ericsson
> .ssa
> .container
> .MessageProcessorFilter.execute(MessageProcessorFilter.java:312)
> at
> com
> .sun
> .grizzly
> .DefaultProtocolChain
> .executeProtocolFilter(DefaultProtocolChain.java:136)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 103)
> at
> com
> .sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:
> 89)
> at
> com
> .sun
> .grizzly
> .ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:67)
> at
> com
> .sun
> .grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:56)
> at
> com
> .sun.grizzly.util.WorkerThreadImpl.processTask(WorkerThreadImpl.java:
> 325)
> at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:
> 184)
>
> --
> View this message in context: http://www.nabble.com/hanging-in-sailfin-when-rebooting-one-node-in-cluster-tp24775217p24775217.html
> Sent from the SailFin - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_sailfin.dev.java.net
> For additional commands, e-mail: users-help_at_sailfin.dev.java.net
>