users@glassfish.java.net

Glassfish as a service --- spurious shut-downs

From: <glassfish_at_javadesktop.org>
Date: Thu, 14 Jan 2010 05:58:28 PST

Hi,

I've got a few glassfish domains running as Windows services on the same machine. Every so often, these domains just suddenly decide to shut down for no apparent reason. I haven't figured out any regularity in this, yet, except that sometimes two or more of them die at the same time, interestingly without Windows ever noticing: the Services manager shows them as running and I actually have to restart them to get them back up.

From the logs, I can see that they shut down in good order: the stack trace from when the ServletContextListener is called is:
--- begin stack trace ---
my.package.ApplicationContextListener.contextDestroyed(ApplicationContextListener.java:66)
        org.apache.catalina.core.StandardContext.listenerStop(StandardContext.java:4567)
        org.apache.catalina.core.StandardContext.stop(StandardContext.java:5355)
        com.sun.enterprise.web.WebModule.stop(WebModule.java:357)
        org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1286)
        com.sun.enterprise.web.VirtualServer.stop(VirtualServer.java:424)
        org.apache.catalina.core.ContainerBase.stop(ContainerBase.java:1286)
        org.apache.catalina.core.StandardEngine.stop(StandardEngine.java:538)
        org.apache.catalina.startup.Embedded.stop(Embedded.java:970)
        com.sun.enterprise.web.WebContainer.stop(WebContainer.java:931)
        com.sun.enterprise.web.PEWebContainer.stopInstance(PEWebContainer.java:814)
        com.sun.enterprise.web.PEWebContainerLifecycle.onShutdown(PEWebContainerLifecycle.java:95)
        com.sun.enterprise.server.ApplicationServer.onShutdown(ApplicationServer.java:549)
        com.sun.enterprise.server.ondemand.OnDemandServer.onShutdown(OnDemandServer.java:131)
        com.sun.enterprise.server.Shutdown.startShutdown(Shutdown.java:75)
        com.sun.enterprise.admin.event.AdminEventMulticaster.invokeShutdownEventListener(AdminEventMulticaster.java:1501)
        com.sun.enterprise.admin.event.AdminEventMulticaster.handleShutdownEvent(AdminEventMulticaster.java:1490)
        com.sun.enterprise.admin.event.AdminEventMulticaster.processEvent(AdminEventMulticaster.java:480)
        com.sun.enterprise.admin.event.AdminEventMulticaster.multicastEvent(AdminEventMulticaster.java:176)
        com.sun.enterprise.admin.server.core.channel.AdminChannelServer.sendNotification(AdminChannelServer.java:121)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        java.lang.reflect.Method.invoke(Method.java:597)
        sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:305)
        sun.rmi.transport.Transport$1.run(Transport.java:159)
        java.security.AccessController.doPrivileged(Native Method)
        sun.rmi.transport.Transport.serviceCall(Transport.java:155)
        sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:535)
        sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:790)
        sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:649)
        java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:885)
        java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        java.lang.Thread.run(Thread.java:619)
--- end stack trace ---

which looks a lot like my servers actually get a shut-down signal.

JConsole doesn't show any peaks or drops in memory consumption or the number of active threads just before the shut-down.

I've looked at Windows' event log but couldn't make out anything that might be connected.

What I'm currently looking into is the -Xrs flag that is to be set, according to the wiki, whenever glassfish is run as a service. It wasn't set in my config, but that doesn't seem to have mattered, since the services run with the System uid so their user never logs in and out. Or does it?

I've tried setting it now, but as the shut-downs were quite sporadic, I can't tell if that fixed it even if my domains stay up for a few days.

I'd be very grateful for any help, thanks a lot in advance!

Cheers,
Johannes
[Message sent by forum member 'tatome' (tenavaj_at_tatome.de)]

http://forums.java.net/jive/thread.jspa?messageID=380932