Guys,
We develop a Hospital Information System that manage patient registration, appoinment and billing. Currently we have deploy our system in 3 hospital and we got problem where we have to restart glassfish server on daily basis to avoid glassfish from getting hang.
Below is some information about our configuration and server :
OS : Red Hat Enterprise Linux 5.3 kernel 2.6.18.8.el5
Apps Server : Sun Glassfish Enterprise Server v2.1
RAM : 16GB
JVM Setting :
<jvm-options>-XX:+UseCMSInitiatingOccupancyOnly</jvm-options>
<jvm-options>-XX:+UseCMSCompactAtFullCollection</jvm-options>
<jvm-options>-XX:CMSFullGCsBeforeCompaction=50</jvm-options>
<jvm-options>-XX:+PrintGCTimeStamps</jvm-options>
<jvm-options>-XX:+PrintGCDetails</jvm-options>
<jvm-options>-verbose:gc</jvm-options>
<jvm-options>-XX:CMSInitiatingOccupancyFraction=1</jvm-options>
<jvm-options>-XX:LargePageSizeInBytes=256m</jvm-options>
<jvm-options>-XX:ParallelGCThreads=2</jvm-options>
<jvm-options>-Xmn2g</jvm-options>
<jvm-options>-Xms8G</jvm-options>
<jvm-options>-Xmx8G</jvm-options>
<jvm-options>-d64</jvm-options>
<jvm-options>-XX:-UseParallelOldGC</jvm-options>
<jvm-options>-XX:+UseConcMarkSweepGC</jvm-options>
<jvm-options>-XX:MaxPermSize=192m</jvm-options>
<jvm-options>-server</jvm-options>
<jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
<jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
<jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
<jvm-options>-Dsun.rmi.dgc.server.gcInterval=3600000</jvm-options>
<jvm-options>-Dsun.rmi.dgc.client.gcInterval=3600000</jvm-options>
<jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
<jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
<jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext${path.separator}${com.sun.aas.derbyRoot}/lib</jvm-options>
<jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.admin.server.core.jmx.AppServerMBeanServerBuilder</jvm-options>
<jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
<jvm-options>-Dcom.sun.enterprise.taglibs=appserv-jstl.jar,jsf-impl.jar</jvm-options>
<jvm-options>-Dcom.sun.enterprise.taglisteners=jsf-impl.jar</jvm-options>
<jvm-options>-XX:NewRatio=2</jvm-options>
I also attach some snapshot from the server itself during hang happen. In the early stage of system going live we already come across this problem plus outofmemory error and already report it to local sun support and I also asked opinion from some java expert in my office. What they suggest just to increase the -Xms and -Xmx from 2G to 8G plus some other changes to jvm setting.
For me this is not a good suggestion and it will help us for temporary only. As I told you this thing already happen again. What I suspect is some serious memory leaks from the application itself (you can see in the attach files). I think the -Xms and -Xmx already high enough.
Additional info :
Our system run on office hour basis 9am to 6pm only. The screenshot is from 2 different sites.
My question :
1. Is the 8GB setting for jvm is normal for java application? (8GB for 1 site, another 2 site is using 12GB ; other setting would be the same)
2. What your opinion/suggestion for our problem.
3. If you think it is memory leaks, how can I proof it to the development team?
[Message sent by forum member 'badul' (dzafarul_at_gmail.com)]
http://forums.java.net/jive/thread.jspa?messageID=380228