users@glassfish.java.net

Re: Glassfish Hang - Daily

From: Alexis Moussine-Pouchkine <alexis.mp_at_sun.com>
Date: Tue, 12 Jan 2010 09:59:42 +0100

That's a lot of JVM tuning that you have here (some of which I've never seen before)...

What would be interesting is to get a heap-dump (kill -3, jstack) of the hung process if possible.
The other important data would be the reason for the outofmemory error (there could be multiple reasons for it).

Also when moving from a 2GB to an 8GB heap did you start using a 64-bit JVM or were you using it from the beginning?
If you don't require that you may want to consider going back to 2GB and a 32-bit JVM (the reason for the OOM error would help here).
64-bit JVMs are usually needed when you have large sets of data in memory.

Also I would recommend trying GlassFish 2.1.1 if possible.
Memory "leaks" can usually be observed when an explicit call to the GC doesn't reclaim all the expected memory.

-Alexis

On 12 janv. 2010, at 09:28, glassfish_at_javadesktop.org wrote:

> Guys,
>
> We develop a Hospital Information System that manage patient registration, appoinment and billing. Currently we have deploy our system in 3 hospital and we got problem where we have to restart glassfish server on daily basis to avoid glassfish from getting hang.
>
> Below is some information about our configuration and server :
>
> OS : Red Hat Enterprise Linux 5.3 kernel 2.6.18.8.el5
> Apps Server : Sun Glassfish Enterprise Server v2.1
> RAM : 16GB
>
> JVM Setting :
>
> <jvm-options>-XX:+UseCMSInitiatingOccupancyOnly</jvm-options>
> <jvm-options>-XX:+UseCMSCompactAtFullCollection</jvm-options>
> <jvm-options>-XX:CMSFullGCsBeforeCompaction=50</jvm-options>
> <jvm-options>-XX:+PrintGCTimeStamps</jvm-options>
> <jvm-options>-XX:+PrintGCDetails</jvm-options>
> <jvm-options>-verbose:gc</jvm-options>
> <jvm-options>-XX:CMSInitiatingOccupancyFraction=1</jvm-options>
> <jvm-options>-XX:LargePageSizeInBytes=256m</jvm-options>
> <jvm-options>-XX:ParallelGCThreads=2</jvm-options>
> <jvm-options>-Xmn2g</jvm-options>
> <jvm-options>-Xms8G</jvm-options>
> <jvm-options>-Xmx8G</jvm-options>
> <jvm-options>-d64</jvm-options>
> <jvm-options>-XX:-UseParallelOldGC</jvm-options>
> <jvm-options>-XX:+UseConcMarkSweepGC</jvm-options>
> <jvm-options>-XX:MaxPermSize=192m</jvm-options>
> <jvm-options>-server</jvm-options>
> <jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
> <jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
> <jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
> <jvm-options>-Dsun.rmi.dgc.server.gcInterval=3600000</jvm-options>
> <jvm-options>-Dsun.rmi.dgc.client.gcInterval=3600000</jvm-options>
> <jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
> <jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
> <jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext${path.separator}${com.sun.aas.derbyRoot}/lib</jvm-options>
> <jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
> <jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder</jvm-options>
> <jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
> <jvm-options>-Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar</jvm-options>
> <jvm-options>-Dcom.sun.enterprise.taglisteners=jsf-impl.jar</jvm-options>
> <jvm-options>-XX:NewRatio=2</jvm-options>
>
> I also attach some snapshot from the server itself during hang happen. In the early stage of system going live we already come across this problem plus outofmemory error and already report it to local sun support and I also asked opinion from some java expert in my office. What they suggest just to increase the -Xms and -Xmx from 2G to 8G plus some other changes to jvm setting.
>
> For me this is not a good suggestion and it will help us for temporary only. As I told you this thing already happen again. What I suspect is some serious memory leaks from the application itself (you can see in the attach files). I think the -Xms and -Xmx already high enough.
>
> Additional info :
> Our system run on office hour basis 9am to 6pm only. The screenshot is from 2 different sites.
>
> My question :
> 1. Is the 8GB setting for jvm is normal for java application? (8GB for 1 site, another 2 site is using 12GB ; other setting would be the same)
> 2. What your opinion/suggestion for our problem.
> 3. If you think it is memory leaks, how can I proof it to the development team?
> [Message sent by forum member 'badul' (dzafarul_at_gmail.com)]
>
> http://forums.java.net/jive/thread.jspa?messageID=380228
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>