users@grizzly.java.net

Re: Old problem still persists ...

From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
Date: Tue, 26 Aug 2008 10:27:56 -0400

Salut Alan,

looking at the thread dump:

> "httpWorkerThread-80-32" daemon prio=10 tid=0x0a005000 nid=0x6483 in Object.wait() [0xb40c8000..0xb40c8db0]
> 78 java.lang.Thread.State: WAITING (on object monitor)
> 79 at java.lang.Object.wait(Native Method)
> 80 - waiting on <0x3995c2f0> (a com.sun.grizzly.http.LinkedListPipeline)
> 81 at java.lang.Object.wait(Object.java:485)
> 82 at com.sun.grizzly.DefaultPipeline.waitForIoTask(DefaultPipeline.java:328)
> 83 - locked <0x3995c2f0> (a com.sun.grizzly.http.LinkedListPipeline)
> 84 at com.sun.grizzly.DefaultPipeline.waitForIoTask(DefaultPipeline.java:53)
> 85 at com.sun.grizzly.util.WorkerThreadImpl.run(WorkerThreadImpl.java:167)

All threads seems to be idle, waiting for works. Now just to recall the
original problem:

> 643 Caused by: java.util.zip.ZipException: error in opening zip file
> 644 at java.util.zip.ZipFile.open(Native Method)
> 645 at java.util.zip.ZipFile.<init>(ZipFile.java:114)
> 646 at java.util.jar.JarFile.<init>(JarFile.java:133)
> 647 at java.util.jar.JarFile.<init>(JarFile.java:70)
> 648

Which is really caused because of a file descriptor leaks. Can you grab a

% netstat -an | grep 80 | wc -l

count when this happens. On our side , I suspect the exception is
swallowed when the OP_ACCEPT (accepting the connection) fail because the
  OS run out of file descriptor. Let me send you a patch that will add
some debugging information (so we can get more information about who is
leaking file descriptor).

We did a lot of tests using Grizzly in GlassFish v3 and so far I haven't
see any file descriptor leaks, but we never know.

Give me 1 hour and I will send you a patch.

Thanks

-- Jeanfrancois



Oleksiy Stashok wrote:
> Hi Alan,
>
> it looks like some file handle leak either in Grizzly or your custom code.
>
> It could be very useful to see your code, if it's possible. If you can
> propose the testcase, which reproduces your issue - it could be even
> better :)
>
> Thanks.
>
> WBR,
> Alexey.
>
> On Aug 26, 2008, at 13:57 , Alan Williamson wrote:
>
>> Guys,
>>
>> We are having major problems keeping Grizzly up under heavy load. Its
>> just not hacking it. I've reported this problem before, but it wasn't
>> resolved.
>>
>> What happens is that Grizzly simply stops accepting requests on Port#80.
>>
>> It won't do a thing more when it receives the error @
>>
>> http://clients.aw20.net/whiteboard/?b=92181#610
>>
>> The stack trace of the Java program yeilds the following:
>>
>> http://clients.aw20.net/whiteboard/?b=92181
>>
>> As you can see, its all stuck in Grizzly threads, with the engine
>> doing absolutely nothing!!!! Grizzly has just given up.
>>
>> This is the latest 1.8.5.3 JAR files for Grizzly, and we are simply
>> using the framework and http-webserver JAR files.
>>
>> Running on JDK1.6 / Redhat
>>
>> Please advise. This is a major show stopper.
>>
>> --
>> Alan Williamson
>> Registrationless email/sms reminders: http://yourli.st/
>> blog: http://alan.blog-city.com/
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>