users@glassfish.java.net

"GRIZZLY0023: Interrupting idle Thread" filling log, web apps not responding

From: Bobby Bissett <bbissett_at_gmail.com>
Date: Mon, 2 Apr 2012 10:13:06 -0400

Hi all,

Using GF 3.1.2, we've started seeing that the server stops responding
to web requests, and the logs are filled with these:

[#|2012-03-31T10:50:38.589-0400|WARNING|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=12;_ThreadName=Thread-2;|GRIZZLY0023:
Interrupting idle Thread: http-thread-pool-443(2).|#]

[#|2012-03-31T10:50:38.589-0400|WARNING|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=12;_ThreadName=Thread-2;|GRIZZLY0023:
Interrupting idle Thread: http-thread-pool-443(5).|#]

[#|2012-03-31T10:50:38.590-0400|WARNING|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=12;_ThreadName=Thread-2;|GRIZZLY0023:
Interrupting idle Thread: http-thread-pool-443(3).|#]

[#|2012-03-31T10:50:38.590-0400|WARNING|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=12;_ThreadName=Thread-2;|GRIZZLY0023:
Interrupting idle Thread: http-thread-pool-443(1).|#]

[#|2012-03-31T10:50:38.590-0400|WARNING|glassfish3.1.2|com.sun.grizzly.config.GrizzlyServiceListener|_ThreadID=12;_ThreadName=Thread-2;|GRIZZLY0023:
Interrupting idle Thread: http-thread-pool-443(4).|#]

As you can see, we get about 5 of them/second, and we have to restart
the app server to get things working again. Hunting around, I gather
that this happens when a web request is running for more than 15
minutes -- is that correct? We don't have anything in the web app that
does this directly, but we are using an ICEPush feature to push
changes to the user, and maybe that's somehow causing this. We've only
started seeing the problem recently, though, which makes this more
curious (maybe we just have more users now).

Can someone give me a pointer about how I can debug the issue to see
what's really going on? I can try a thread dump on one of the servers
the next time it gets in this state, but is there any other suggestion
in the meantime? Is there any way to make this situation *not* make
the server unresponsive?

Thanks,
Bobby