users@glassfish.java.net

Re: Glassfish Memory Issues

From: Sivakumar Thyagarajan <Sivakumar.Thyagarajan_at_Sun.COM>
Date: Thu, 12 Jul 2007 22:43:42 +0530

Hi

ITVGuy2000 wrote:
> Please be patient, I will try to get the information you ask for, but I have
> some other configuration issue I need to deal with to use the jconsole. If
Thanks for providing these.

> you could tell me how to change the jmx host to a different domain I would
> be grateful. Currently this is misconfigured to an old domain I used to use

In the server log, you would see a INFO statement [similar to the one below]
providing the JMXServiceURL to connect to. You could then open jconsole, choose
the remote tab, provide the JMXServiceURL, username/password and log in.

[#|2007-07-11T16:57:11.072+0530|INFO|sun-appserver9.1|javax.enterprise.system.tools.admin|_ThreadID=12;_ThreadName=Thread-17;service:jmx:rmi:///jndi/rmi://localhost:8686/jmxrmi;|ADM1504:
Here is the JMXServiceURL for the Standard JMXConnectorServer:
[service:jmx:rmi:///jndi/rmi://localhost:8686/jmxrmi]. This is where the remote
administrative clients should connect using the standard JMX connectors|#]


> (my hosting provider can't seem to forget). I found this in both of the
> following files, but it looks autogenerated.
> ./jbi/system/private/config/com/sun/jbi/framework/JBIFramework.properties
> ./jbi/system/esb-registry/registry.xml
> Is there someplace else the JMX host is configured?
>
> [itvguy2000]/home/hhartley>ulimit -a
> time(seconds) unlimited
> file(blocks) unlimited
> data(kbytes) unlimited
> stack(kbytes) 8192
> coredump(blocks) unlimited
> nofiles(descriptors) 256
> vmemory(kbytes) unlimited
>

As a workaround, does it help if you increase the number of fds [nofiles]?
ulimit -n 1024.

> I will work through the links you forwarded and try to provide some info.

Thanks. Much appreciated.

Thanks
--Siva.

>
> ITVGuy2000
>
>
> Sivakumar Thyagarajan wrote:
>> Hi
>>
>> Is your issue also a "permgen space increasing unbound" type? Could you
>> use
>> jconsole to monitor permgen space and see if it is increasing? Could you
>> use Tim
>> Quinn's ZipFileMonitor
>> [http://blogs.sun.com/quinn/entry/tool_for_diagnosing_failed_glassfish]
>> and
>> provide us a list of all open jars?
>>
>> > Caused by: java.io.IOException: error=12, Not enough space
>> > at java.lang.UNIXProcess.forkAndExec(Native Method)
>> Could you post your "ulimit -a"? Have you tried increasing number of
>> file-descriptors
>> [http://docs.sun.com/app/docs/doc/819-3681/6n5srlhk7?a=view#abeja],
>> increasing
>> swap space
>> [http://docs.sun.com/app/docs/doc/819-3681/6n5srlhk6?a=view#abeiv],
>> if it solves the problem temproarily?
>>
>> As I had responded in the other thread, I couldn't reproduce it locally
>> and it
>> would be great if you could provide a reproducible testcase or more
>> information.
>>
>> Thanks
>> --Siva.
>>
>> ITVGuy2000 wrote:
>>> My SJAS server is apparently running out of memory. The memory consumed
>>> by
>>> the server continues to increase with every deploy and compile until my
>>> server no longer responds to incoming requests. If I am lucky I get an
>>> exception in the log file indicating out of space (see at bottom of
>>> post).
>>>
>>> This problem occurs on both my development laptop and server (see
>>> configuration below).
>>>
>>> In searching the net and newsgroups I have found the following bug and
>>> post:
>>> I believe could be part of my issues are due to glassfish not freeing
>>> memory
>>> after JSP compiles.
>>> https://glassfish.dev.java.net/issues/show_bug.cgi?id=2375
>>>
>>> There is a very long thread about that discusses this unresolved bug.
>>> http://forums.java.net/jive/thread.jspa?messageID=202780
>>>
>>> I am kind of surprised that this bug is not getting more attention. I am
>>> surprised its not being reported more often in newsgroups. To me this is
>>> a
>>> show stopper (P1) bug. During heavy development/deployment I end up
>>> having
>>> to restart my glassfish server at least once a day.
>>>
>>> Is anyone else experiencing this?
>>> Why isn’t this bug a P1 level bug?
>>> What can I do to avoid my system hanging daily?
>>>
>>> As it stands I am kind if in a pinch. I have migrated all of my
>>> applications
>>> off of Tomcat onto the glassfish platform. I have gone from Rock Solid
>>> forget about it installation, to unstable needing to reboot every day.
>>> Please help!
>>>
>>> Thanks,
>>>
>>> ITVGuy2000
>>>
>>> -------------------------------- My Configuration
>>> ------------------------------------------------
>>>
>>> My Development configuration:
>>> Sony Laptop Vaio (1.7Ghz, 1.5G Ram, 88G drive (23G free))
>>> Windows XP Service Pack 2
>>> JDK 1.5.0_11
>>> SJAS Version 9 update 1 patch 1
>>> NB 5.5.1
>>> MySQL 5.0.15
>>>
>>> My Server configuration:
>>> Sun 220R (1G Ram, 18G Drive)
>>> Solaris 9 (SunOS 5.9)
>>> JDK 1.5.0_2
>>> MySQL 5.0.15
>>> SJAS Version 9 update 1 patch 1
>>>
>>> ------------------------------------ Error from server.log file
>>> -----------------------------------------
>>> [#|2007-07-09T09:35:36.191-0400|SEVERE|sun-appserver-pe9.0|javax.enterprise.system.container.web|_ThreadID=25;_ThreadName=httpWorkerThread-80-2;_RequestID=d2d27139-7fa5-4da4-902f-584dc55171e4;|StandardWrapperValve[jsp]:
>>> Servlet.service() for servlet jsp threw exception
>>> Error running /opt/SDK/jdk/bin/javac compiler
>>> at
>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:473)
>>> at
>>> org.apache.tools.ant.taskdefs.compilers.JavacExternal.execute(JavacExternal.java:47)
>>> at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:931)
>>> at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:757)
>>> at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:428)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:528)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:507)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:495)
>>> at
>>> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:530)
>>> at
>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324)
>>> at
>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:412)
>>> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:318)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:397)
>>> at
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:278)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>> at
>>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:240)
>>> at
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:179)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:73)
>>> at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:182)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> com.sun.enterprise.web.VirtualServerPipeline.invoke(VirtualServerPipeline.java:120)
>>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>> at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:137)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>> at
>>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:239)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.invokeAdapter(ProcessorTask.java:667)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.processNonBlocked(ProcessorTask.java:574)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.process(ProcessorTask.java:844)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ReadTask.executeProcessorTask(ReadTask.java:287)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ReadTask.doTask(ReadTask.java:212)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:252)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.WorkerThread.run(WorkerThread.java:75)
>>> Caused by: java.io.IOException: Cannot run program
>>> "/opt/SDK/jdk/bin/javac"
>>> (in directory "/opt/SDK/domains/domain1"): error=12, Not enough space
>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>> at java.lang.Runtime.exec(Runtime.java:593)
>>> at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:834)
>>> at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:435)
>>> at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:449)
>>> at
>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:470)
>>> ... 36 more
>>> Caused by: java.io.IOException: error=12, Not enough space
>>> at java.lang.UNIXProcess.forkAndExec(Native Method)
>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
>>> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
>>> ... 44 more
>>> --- Nested Exception ---
>>> java.io.IOException: Cannot run program "/opt/SDK/jdk/bin/javac" (in
>>> directory "/opt/SDK/domains/domain1"): error=12, Not enough space
>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:459)
>>> at java.lang.Runtime.exec(Runtime.java:593)
>>> at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>> at java.lang.reflect.Method.invoke(Method.java:597)
>>> at
>>> org.apache.tools.ant.taskdefs.Execute$Java13CommandLauncher.exec(Execute.java:834)
>>> at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:435)
>>> at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:449)
>>> at
>>> org.apache.tools.ant.taskdefs.compilers.DefaultCompilerAdapter.executeExternalCompile(DefaultCompilerAdapter.java:470)
>>> at
>>> org.apache.tools.ant.taskdefs.compilers.JavacExternal.execute(JavacExternal.java:47)
>>> at org.apache.tools.ant.taskdefs.Javac.compile(Javac.java:931)
>>> at org.apache.tools.ant.taskdefs.Javac.execute(Javac.java:757)
>>> at org.apache.jasper.compiler.Compiler.generateClass(Compiler.java:428)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:528)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:507)
>>> at org.apache.jasper.compiler.Compiler.compile(Compiler.java:495)
>>> at
>>> org.apache.jasper.JspCompilationContext.compile(JspCompilationContext.java:530)
>>> at
>>> org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:324)
>>> at
>>> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:412)
>>> at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:318)
>>> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>>> at
>>> org.apache.catalina.core.ApplicationFilterChain.servletService(ApplicationFilterChain.java:397)
>>> at
>>> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:278)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>> at
>>> org.apache.catalina.core.StandardContextValve.invokeInternal(StandardContextValve.java:240)
>>> at
>>> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:179)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:73)
>>> at
>>> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:182)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> com.sun.enterprise.web.VirtualServerPipeline.invoke(VirtualServerPipeline.java:120)
>>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>> at
>>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:137)
>>> at
>>> org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:566)
>>> at
>>> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:536)
>>> at org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:939)
>>> at
>>> org.apache.coyote.tomcat5.CoyoteAdapter.service(CoyoteAdapter.java:239)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.invokeAdapter(ProcessorTask.java:667)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.processNonBlocked(ProcessorTask.java:574)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ProcessorTask.process(ProcessorTask.java:844)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ReadTask.executeProcessorTask(ReadTask.java:287)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.ReadTask.doTask(ReadTask.java:212)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:252)
>>> at
>>> com.sun.enterprise.web.connector.grizzly.WorkerThread.run(WorkerThread.java:75)
>>> Caused by: java.io.IOException: error=12, Not enough space
>>> at java.lang.UNIXProcess.forkAndExec(Native Method)
>>> at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
>>> at java.lang.ProcessImpl.start(ProcessImpl.java:65)
>>> at java.lang.ProcessBuilder.start(ProcessBuilder.java:452)
>>> ... 44 more
>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
>> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>>
>>
>>
>