users@glassfish.java.net

Cisco Load Balancer fills up connection pool

From: <glassfish_at_javadesktop.org>
Date: Wed, 09 Apr 2008 09:19:21 PDT

We have 2 glassfish cluster environments (preprod and production), both fronted by a Cisco load balancer. In the preprod environment, everything works perfectly with the Cisco LB.

In the production environment, the instances of the cluster seem to fill their connection pools without ever reclaiming closed connections, and every morning this week the instances of the cluster have had the "Maximum connections reached: 4096" error message.

This is currently a lightly used application (we haven't cut over to production yet, that's scheduled for [b]tomorrow[/b]!). This is to be an extremely mission critical app, but it will never see more than a few hundred short-lived connections at a time (users changing their password).

This is Glassfish v2ur1 on Solaris 10 Sparc. Both the servers and Glassfish are all using standard, out of the box settings for networking (no entries in /etc/system, no tweaks to the http-listener settings). The only difference is that the preprod cluster all lives on zones on one physical system, and the prod cluster is spread out on zones on 3 physical systems.

I've analyzed the keepalives that the load balancer is sending. It is doing a plain HTTP 1.1 GET on "/" and looking for a 200 response.
[Message sent by forum member 'mcking' (mcking)]

http://forums.java.net/jive/thread.jspa?messageID=268377