users@glassfish.java.net

Re: glassfish 3.1.2.2 (build 5) - default install - problem with keeping too many connections in the queue, while system usage is low

From: Mladen Adamovic <mladen.adamovic_at_gmail.com>
Date: Tue, 24 Jul 2012 00:10:35 +0200

Hi Mo,

thanks for your message. It looks like that when I turn keep alive to 1,
that many connections remain in TIME_WAIT mode.
It might be beneficial for investigation if you can write how many hits do
you have per second and average processing time of request in ms (both
value from monitoring).

Thanks and cheers



On Fri, Jul 20, 2012 at 10:24 PM, Mo Maison <momaison_at_yahoo.fr> wrote:

>
> Hello,
>
> From our experiments, I confirm that 1024 files is not sufficient
> if your server is under load, probably because of keep-alive
> connections.
> We got a valuable error message in server log though : something
> like 'too many open files' and the server was freezed (did not
> answer to requests ; this is not what you observe however).
>
> In case, you may watch the currently opened file descriptors with
> something like lsof | grep | wc
> or ls /proc/{pid}/fd | wc -l (under linux)
>
> Increasing the ulimit -n to 4096 did not solve our problem.
> We had to set the (badly named) parameter "Maximum number of
> connections in keep-alive mode" to -1 (or 0 or 1 I don't remember
> exactly), so that connections are always closed server-side. This
> is not a big concern for our application.
>
> Regards,
>
> M. Maison
>
>
>
> On 19/07/2012 20:34, Mladen Adamovic wrote:
>
>> Hi guys,
>>
>> I migrated some of my websites from glassfish 2.1 to glassfish 3.1.2.2
>> and I'm having problems.
>>
>> The problem I'm encountering is that after a while, many connections
>> wait in connection queue and it even starts to refuse connections, but
>> system usage is low.
>> #uptime
>> 20:28:58 up 51 min, 1 user, load average: 0.09, 0.14, 0.24
>>
>> If I test the website with #ab -n 1000 -c 5
>> http://www.numbeo.com/cost-of-**living/<http://www.numbeo.com/cost-of-living/>
>> it gets around 980 failures out of 1000.
>> I'm playing with config files, but couldn't figure out where is the
>> problem.
>> ulimit -n
>> is 1024,cannot rise it, since running in Virtuozzo enviroment.
>>
>>


-- 
Mladen Adamovic
Numbeo
Drziceva 9, 11120 Belgrade-Zvezdara, Serbia
(Business Registration Number 62612240)
Tel. +381-66-058-595
email: mladen.adamovic_at_gmail.com
web: http://www.numbeo.com