users@glassfish.java.net

Re: GF v2.1: TCP conn hanging with CLOSE_WAIT

From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
Date: Mon, 06 Apr 2009 17:31:19 -0400

Salut,

glassfish_at_javadesktop.org wrote:
> We've recently migrated our business app from WLS to GF v2.1 and just experienced the following:
> - GF started to slow down at a point where most of the new incoming requests hang.
> - netstat showed about 350 tcp connections with CLOSE_WAIT status.
> - admin console was not accessible (hang).
> - in attempt to shutdown the domain, asadmin stop-domain returned with success but it didn't actually terminate the process listening the http port of the given domain (had to manually kill the process).
> - Web application deployed on the given domain have relatively light traffic over all (150-200 users a day, 5-10 concurrent users).
> - happened on the 6th day after we went live.
> - no errors/exceptions/nothing abnormal in the server log.
>

Have you added:

<jvm-options>-Dcom.sun.enterprise.server.ss.ASQuickStartup=false</jvm-options>


> We did:
> - look at "Thread: https (8181) not responding - v2_rc1_b58" and reviewed the recommendation of configuration described in "Jean-Francois Arcand's Blog" but we didn't see much missing piece.
> - try to recreate the issue by running stress tests in the mirror environment with 100 users but so far no luck.

Is your application opening connection or reading file? Can you take
snapshot of the file descriptor like every day?

As far as I can tell we never leaked socket when ASQuickStartup is
false. Are you enabling Comet by any chance?

Thanks

-- Jeanfrancois


>
> Environment:
> Sun GlassFish Enterprise Server v2.1
> M3000 SPARC64 VII 16GB
> Solaris 10
>
> If you have any insights for what may caused GF to not call 'close' on open sockets, we'd appreciate it!!!!
> [Message sent by forum member 'ur_afroinu' (ur_afroinu)]
>
> http://forums.java.net/jive/thread.jspa?messageID=340555
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>