users@glassfish.java.net

Re: Glassfish Memory Issues

From: Kedar Mhaswade <Kedar.Mhaswade_at_Sun.COM>
Date: Thu, 12 Jul 2007 11:26:58 -0700

One can always use asadmin generate-jvm-report to get the JConsole
like data, in most cases.

Kedar

Sivakumar Thyagarajan wrote:
> Hi
>
> ITVGuy2000 wrote:
>> Please be patient, I will try to get the information you ask for, but
>> I have
>> some other configuration issue I need to deal with to use the
>> jconsole. If
> Thanks for providing these.
>
>> you could tell me how to change the jmx host to a different domain I
>> would
>> be grateful. Currently this is misconfigured to an old domain I used
>> to use
>
> In the server log, you would see a INFO statement [similar to the one
> below] providing the JMXServiceURL to connect to. You could then open
> jconsole, choose the remote tab, provide the JMXServiceURL,
> username/password and log in.
>
> [#|2007-07-11T16:57:11.072+0530|INFO|sun-appserver9.1|javax.enterprise.system.tools.admin|_ThreadID=12;_ThreadName=Thread-17;service:jmx:rmi:///jndi/rmi://localhost:8686/jmxrmi;|ADM1504:
> Here is the JMXServiceURL for the Standard JMXConnectorServer:
> [service:jmx:rmi:///jndi/rmi://localhost:8686/jmxrmi]. This is where
> the remote administrative clients should connect using the standard JMX
> connectors|#]
>
>
>> (my hosting provider can't seem to forget). I found this in both of the
>> following files, but it looks autogenerated.
>> ./jbi/system/private/config/com/sun/jbi/framework/JBIFramework.properties
>> ./jbi/system/esb-registry/registry.xml
>> Is there someplace else the JMX host is configured?
>>
>> [itvguy2000]/home/hhartley>ulimit -a
>> time(seconds) unlimited
>> file(blocks) unlimited
>> data(kbytes) unlimited
>> stack(kbytes) 8192
>> coredump(blocks) unlimited
>> nofiles(descriptors) 256
>> vmemory(kbytes) unlimited
>>
>
> As a workaround, does it help if you increase the number of fds [nofiles]?
> ulimit -n 1024.
>
>> I will work through the links you forwarded and try to provide some info.
>
> Thanks. Much appreciated.
>
> Thanks
> --Siva.
>
>>
>> ITVGuy2000
>>
>>
>> Sivakumar Thyagarajan wrote:
>>> Hi
>>>
>>> Is your issue also a "permgen space increasing unbound" type? Could you
>>> use jconsole to monitor permgen space and see if it is increasing?
>>> Could you
>>> use Tim Quinn's ZipFileMonitor
>>> [http://blogs.sun.com/quinn/entry/tool_for_diagnosing_failed_glassfish]
>>> and provide us a list of all open jars?
>>>
>>> > Caused by: java.io.IOException: error=12, Not enough space
>>> > at java.lang.UNIXProcess.forkAndExec(Native Method)
>>> Could you post your "ulimit -a"? Have you tried increasing number of
>>> file-descriptors
>>> [http://docs.sun.com/app/docs/doc/819-3681/6n5srlhk7?a=view#abeja],
>>> increasing swap space
>>> [http://docs.sun.com/app/docs/doc/819-3681/6n5srlhk6?a=view#abeiv],
>>> if it solves the problem temproarily?
>>>
>>> As I had responded in the other thread, I couldn't reproduce it locally
>>> and it would be great if you could provide a reproducible testcase or
>>> more
>>> information.
>>>
>>> Thanks
>>> --Siva.
>>>
>>> ITVGuy2000 wrote:
>>>> My SJAS server is apparently running out of memory. The memory consumed
>>>> by
>>>> the server continues to increase with every deploy and compile until my
>>>> server no longer responds to incoming requests. If I am lucky I get an
>>>> exception in the log file indicating out of space (see at bottom of
>>>> post).
>>>>
>>>> This problem occurs on both my development laptop and server (see
>>>> configuration below).
>>>>
>>>> In searching the net and newsgroups I have found the following bug and
>>>> post:
>>>> I believe could be part of my issues are due to glassfish not freeing
>>>> memory
>>>> after JSP compiles.
>>>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=2375
>>>> There is a very long thread about that discusses this unresolved
>>>> bug. http://forums.java.net/jive/thread.jspa?messageID=202780
>>>>
>>>> I am kind of surprised that this bug is not getting more attention.
>>>> I am
>>>> surprised its not being reported more often in newsgroups. To me
>>>> this is
>>>> a
>>>> show stopper (P1) bug. During heavy development/deployment I end up
>>>> having
>>>> to restart my glassfish server at least once a day.
>>>>
>>>> Is anyone else experiencing this?
>>>> Why isn’t this bug a P1 level bug?
>>>> What can I do to avoid my system hanging daily?
>>>>
>>>> As it stands I am kind if in a pinch. I have migrated all of my
>>>> applications
>>>> off of Tomcat onto the glassfish platform. I have gone from Rock Solid
>>>> forget about it installation, to unstable needing to reboot every
>>>> day. Please help!
>>>>
>>>> Thanks,
>>>>
>>>> ITVGuy2000
>>>>
>>>> -------------------------------- My Configuration
>>>> ------------------------------------------------
>>>>
>>>> My Development configuration:
>>>> Sony Laptop Vaio (1.7Ghz, 1.5G Ram, 88G drive (23G free)) Windows XP
>>>> Service Pack 2 JDK 1.5.0_11 SJAS Version 9 update 1 patch 1 NB 5.5.1
>>>> MySQL 5.0.15
>>>>
>>>> My Server configuration:
>>>> Sun 220R (1G Ram, 18G Drive)
>>>> Solaris 9 (SunOS 5.9)
>>>> JDK 1.5.0_2
>>>> MySQL 5.0.15
>>>> SJAS Version 9 update 1 patch 1
>>>> ------------------------------------ Error from server.log file
>>>> -----------------------------------------
>>>> [#|2007-07-09T09:35:36.191-0400|SEVERE|sun-appserver-pe9.0|javax.enterprise.system.container.web|_ThreadID=25;_ThreadName=httpWorkerThread-80-2;_RequestID=d2d27139-7fa5-4da4-902f-584dc55171e4;|StandardWrapperValve[jsp]:
>>>>
>>>> Servlet.service() for servlet jsp threw exception
>>>> Error running /opt/SDK/jdk/bin/javac compiler
>>>> at
>>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:473)
>>>>
>>>> at
>>>> org.apache.tools.ant.taskdefs.compilers.JavacExternal.execute(JavacExternal.java:47)
>>>>
>>>> at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:931)
>>>> at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:757)
>>>> at
>>>> org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:428)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:528)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:507)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:495)
>>>> at
>>>> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:530)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:412)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:318)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>> at
>>>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:397)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:278)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:240)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:179)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:73)
>>>> at
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:182)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> com.sun.enterprise.web.VirtualServerPipeline.invoke(VirtualServerPipeline.java:120)
>>>>
>>>> at
>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>>> at
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:137)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>>>
>>>> at
>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>>> at
>>>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:239)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.invokeAdapter(ProcessorTask.java:667)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.processNonBlocked(ProcessorTask.java:574)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.process(ProcessorTask.java:844)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ReadTask.executeProcessorTask(ReadTask.java:287)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ReadTask.doTask(ReadTask.java:212)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:252)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.WorkerThread.run(WorkerThread.java:75)
>>>>
>>>> Caused by: java.io.IOException: Cannot run program
>>>> "/opt/SDK/jdk/bin/javac"
>>>> (in directory "/opt/SDK/domains/domain1"): error=12, Not enough space
>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>> at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:834)
>>>>
>>>> at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:435)
>>>> at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:449)
>>>> at
>>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:470)
>>>>
>>>> ... 36 more
>>>> Caused by: java.io.IOException: error=12, Not enough space
>>>> at java.lang.UNIXProcess.forkAndExec(Native Method)
>>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
>>>> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
>>>> ... 44 more
>>>> --- Nested Exception ---
>>>> java.io.IOException: Cannot run program "/opt/SDK/jdk/bin/javac" (in
>>>> directory "/opt/SDK/domains/domain1"): error=12, Not enough space
>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>>> at java.lang.Runtime.exec(Runtime.java:593)
>>>> at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
>>>> at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>
>>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>>> at
>>>> org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:834)
>>>>
>>>> at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:435)
>>>> at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:449)
>>>> at
>>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:470)
>>>>
>>>> at
>>>> org.apache.tools.ant.taskdefs.compilers.JavacExternal.execute(JavacExternal.java:47)
>>>>
>>>> at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:931)
>>>> at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:757)
>>>> at
>>>> org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:428)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:528)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:507)
>>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:495)
>>>> at
>>>> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:530)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:412)
>>>>
>>>> at
>>>> org.apache.jasper.servlet.JspServlet.service(JspServlet.java:318)
>>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>>> at
>>>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:397)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:278)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:240)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:179)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:73)
>>>> at
>>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:182)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> com.sun.enterprise.web.VirtualServerPipeline.invoke(VirtualServerPipeline.java:120)
>>>>
>>>> at
>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>>> at
>>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:137)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>>>
>>>> at
>>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>>>
>>>> at
>>>> org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>>> at
>>>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:239)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.invokeAdapter(ProcessorTask.java:667)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.processNonBlocked(ProcessorTask.java:574)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.process(ProcessorTask.java:844)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ReadTask.executeProcessorTask(ReadTask.java:287)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.ReadTask.doTask(ReadTask.java:212)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:252)
>>>>
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.WorkerThread.run(WorkerThread.java:75)
>>>>
>>>> Caused by: java.io.IOException: error=12, Not enough space
>>>> at java.lang.UNIXProcess.forkAndExec(Native Method)
>>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
>>>> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
>>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
>>>> ... 44 more
>>>>
>>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>