users@glassfish.java.net

Re: GFv2_b45: OpenFileDescriptorCount is growing, growing and eventually server crashes...

From: Witold Szczerba <pljosh.mail_at_gmail.com>
Date: Thu, 28 Jun 2007 00:16:59 +0200

Hi there,
in my first post in this thread I wrote about the environment:

Ubuntu 7.04,
Linux sop 2.6.20-15-server #2 SMP Sun Apr 15 07:41:34 UTC 2007 i686 GNU/Linux
Java(TM) SE Runtime Environment (build 1.6.0-b105)
Glassfish v2 beta3 b45

Unfortunately, server crashed few minutes after that post, I restarted
it, but since then no one in the company of our customer was there, so
I will wait till tomorrow; let them use the application for a while to
make the bug visible, then I will supply the information using netstat
as you mentioned.

-Witold Szczerba

2007/6/27, Jeanfrancois Arcand <Jeanfrancois.Arcand_at_sun.com>:
> I agree :-)...that make little sense to increase the fd limit on
> Solaris...looks like we have a fd leak somewhere. Can you file an issue
> here:
>
> https://glassfish.dev.java.net/servlets/ProjectIssues
>
> Do you think you can come with a test case that demonstrate the problem?
> One thing to add to the bug description is the output of 'netstat -an
> | grep <<http|iiop_port>>' to see the state of the fd opened. From you
> lsof dump, the socket state cannot be found:
>
> > java 25533 sop 265r FIFO 0,6 1246867 pipe
> > java 25533 sop 266w FIFO 0,6 1246867 pipe
>
>
> Do you have an idea about which protocol is producing the leak (IIOP or
> HTTP)? Hopefully IIOP (just kidding :-))...netstat -an will tell which
> one is leaking. Which JDK version are you using (because such leak was
> present in JDK 1.5_01 to _09.
>
> Thanks for reporting this!
>
> -- Jeanfrancois
>
>
> >
> > 2007/6/27, Binod <Binod.Pg_at_sun.com>:
> >> http://forums.java.net/jive/thread.jspa?messageID=216741
> >>
> >> Your issue seem to be as described in the above thread. Look like
> >> you will need to set the limit to 2048 to escape the JDK bug
> >>
> >> http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6533291
> >>
> >> thanks,
> >> Binod
> >>
> >> Witold Szczerba wrote:
> >>
> >> > Hi there,
> >> > I have a major issue with Glassfish v2 beta3 b45. My project is going
> >> > into last state, right now our customer is testing it on his own
> >> > server and we are encountering problems with
> >> > java.io.IOException: Too many open files.
> >> >
> >> > I really do not know what to do about this, I am monitoring server
> >> > process using jconsole. When server starts, its
> >> > OpenFileDescriptorCount < 270, and then it is growing constantly.
> >> > After less than 24 hours the 1024 limit is overrun and server stops
> >> > responding, it is filling logs with stacktrace full of IOExceptions
> >> > and CPU is 100% on some single thread.
> >> >
> >> > Today, I have registered opened files using lsof 3 times like this:
> >> > lsof |grep 25533 > lsof-2007-06-27-.....
> >> >
> >> > Here are the files:
> >> > http://josh.neostrada.pl/lsof/lsof-2007-06-27-09h20m.txt
> >> > http://josh.neostrada.pl/lsof/lsof-2007-06-27-12h32m.txt
> >> > http://josh.neostrada.pl/lsof/lsof-2007-06-27-14h00m.txt
> >> >
> >> > The first file was made at 9:20am, it has 411 lines,
> >> > second one (12:32pm) has 645 and
> >> > last (2:00pm) one has 888 lines.
> >> >
> >> > What is wrong?
> >> >
> >> > About the enviroment:
> >> > Ubuntu 7.04,
> >> > Linux sop 2.6.20-15-server #2 SMP Sun Apr 15 07:41:34 UTC 2007 i686
> >> > GNU/Linux
> >> > Java(TM) SE Runtime Environment (build 1.6.0-b105)
> >> > Glassfish v2 beta3 b45
> >> >
> >> > The application is using EJB3 + JPA, client application (application
> >> > client container) is a desktop Swing, downloaded by Java WebStart.
> >> >
> >> > If there is anything more I should provide, please tell me
> >> >
> >> > Thanks,
> >> > Witold Szczerba