users@glassfish.java.net

Re: HTTP-service thread pool exhaustion / web service invocation freeze

From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
Date: Mon, 08 Sep 2008 11:17:40 -0400

Salut,

glassfish_at_javadesktop.org wrote:
> I've just conducted an experiment that reproduced a problem I experienced in a PoC that used GlassFish v2 ur2.
>
> The experiment involved web services hosted on the same GlassFish instance that call each other synchronously (nested invocation, if you will). My experiment had two web services that called each other N times, back and forth, hitting a depth of N invocations (plus the initial invocation by a client to start things off). Using the default configuration of GlassFish, when N reached a depth of 5, the final attempt to invoke a web service "freezes", until eventually on of the currently open HTTP connections times out.
>
> The default maximum value for the HTTP service request processor's thread pool is 5. When I raised this to 10, the freeze wouldn't occur until my experiment's call depth reached 10.
>
> It appears that GlassFish dedicates one thread from the HTTP service's request processing pool for each new web service invocation, even if that service invocation is from within the same GF instance. As the chain of synchronous service invocations grows, it consumes more and more threads from the pool.
>
> If this is all true (and my experiment seems to bear this out),

Yes, this is true. Grizzly (the component that allocate the thread) has
no information about from where the request is coming from, e.g. from
the same instance or not.

then I have some major concerns, both in terms of security
(vulnerability to a rather simple DoS attack) and in terms of system
scalability & reliability/availability.

Hum...I not sure I agree with you. In my opinion, deploying an
application that does what you describe is the problem, e.g. an admin
should never deploy an application that call itself like you have
described. At least in GlassFish such use case cannot be supported
because your application is really locking threads.

I suspect the same application will lock in Jetty or Tomcat. There is no
way IMO to detect your use case.

Thanks

-- Jeanfrancois

>
> I know that modern NIO-based protocol stacks avoid the "dedicated, often blocked thread" per protocol invocation problems of the past, as do asynchronous protocols (or at least protocol implementations) at a higher level in the stack. Have I just found a way to bypass all those modern conveniences in GlassFish?
> [Message sent by forum member 'rtenhove' (rtenhove)]
>
> http://forums.java.net/jive/thread.jspa?messageID=297830
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: users-help_at_glassfish.dev.java.net
>