users@grizzly.java.net

Re: Comet - Selector spinning - 100% CPU

From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
Date: Mon, 20 Apr 2009 16:45:31 -0400

Salut,

Rodrigo Ramirez wrote:
> Hi,
>
> We are having this high CPU issue with Comet: After few minutes or hours
> some times, we get a 100% CPU usage for one of the server’s CPUs. The
> situation is pretty similar to the one described at:
> http://www.nabble.com/Comet-handler-starts-terminating-TCP-connections-with-RST--td20337445.html
> and http://www.nabble.com/do-select-spin-on-linux-td22974389.html
>
> We used jconsole / jtop to find the thread using high CPU and got this:
>
> Name: SelectorThread-443
> State: RUNNABLE
> Total blocked: 327 Total waited: 0
>
> Stack trace:
> sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
> sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:184)
> sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
> sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
> - locked sun.nio.ch.Util$1_at_3c304a
> - locked java.util.Collections$UnmodifiableSet_at_18682d0
> - locked sun.nio.ch.EPollSelectorImpl_at_1a7f9da
> sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
> com.sun.enterprise.web.connector.grizzly.SelectorThread.doSelect(SelectorThread.java:1348)
> com.sun.enterprise.web.connector.grizzly.SelectorThread.startListener(SelectorThread.java:1295)
> - locked [Ljava.lang.Object;@104959c
> com.sun.enterprise.web.connector.grizzly.SelectorThread.startEndpoint(SelectorThread.java:1258)
> com.sun.enterprise.web.connector.grizzly.SelectorThread.run(SelectorThread.java:1234)
>
>
> Some points to note:
>
> We see the issue in different environments, including:
> - Glassfish v2 ur2 and Glassfish v2.1
> - Ubuntu Linux 2.6.15-26-server and 2.6.27-7-server
> - JVM 1.6.0 and 1.6.10
>
>
> This is what we have tried since we upgraded to Glassfish V2:
>
> 1. Applied the patch grizzly-comet-selector-patch2.jar
>
> 2. Removed the patch, set CometContext.setExpirationDelay(-1) and used a
> ScheduledExecutorService to expire the Comet Handlers, but even that is not
> working.
>
>
> Since all my comet implementation is encapsulated in a single class, I am
> attaching it. May be I am doing something silly. The commented out code is
> for my failed workaround using set CometContext.setExpirationDelay(-1) and a
> ScheduledExecutorService to expire the Comet Handlers.
>
> Does any one can suggest a workaround different from using java 1.7? Should
> the latest Grizzly help?

I suspect the latest Grizzly will not help. I need to find a way to
reproduce the issue and then I will try to apply the patch. Let me try
to find something...jdk 6ur16 will have the fix. Meanwhile, can you try
with JDK 1.7 b50 and makes sure the problem never happens?

Thanks!

-- Jeanfrancois



>
> Thanks :-)
>
> http://www.nabble.com/file/p23142529/PortalCometQueue.java
> PortalCometQueue.java