Hi,
the problem seems to happen inside CXF, so it may be not (I'd even say
just "not" :) ) GF related.
One of the links you referenced have a stacktrace, which suggests that
spinning may occur when more than 1 thread tries to deal with the same
GZip stream... may be CXF has the same problem?
What I can suggest you to try is to disable gzip on the CXF level
(remove @GZIP annotation) but enable gzip on GF level via admin console.
Thanks.
WBR,
Alexey.
On 10/13/2011 11:48 PM, forums_at_java.net wrote:
> We are in a situation where we find our Glassfish 3.0.1 instance
> permanently
> running at 100% CPU, with heap memory use normal. A thread dump
> (successive
> times) shows that two of our Glassfish threads are spinning
> indefinitely in
> the GZIP compression phase. There appear to be others who are seeing
> similar behaviors (but with no answers) and it is starting to look like a
> Glassfish GZIP bug. We are simply using the @GZIP annotation on our web
> service, and have enabled gzip,deflate in the network/http listener in
> the
> glassfish admin console.
>
> Normally all is well (we see gzipped payloads), so this is NOT the issue
> mentioned here:
>
> http://www.java.net/forum/topic/glassfish/glassfish/gzip-compression-glassfish-3-doesnt-work?force=899
>
> [1] Others are seeing similar (maybe exactly the same) problems:
> https://forums.oracle.com/forums/thread.jspa?messageID=4627097 [2] and
> there
> are several "clues" in various other stack dumps in a google search on
> (for
> example "Deflater.java:306") that suggest that either this is a
> "normal" situation (like socket reads being marked as runnable - I think
> not in this case), or that there is some systematic problem with GZIP
> support
> in the JVM. I have found no answers that seem to address this specific
> problem.
> We have two threads that are in basically the same situation, with
> different
> locks but otherwise stuck at the same place:
>
> "http-thread-pool-8080-(18)" - Thread t_at_950
>
> java.lang.Thread.State: RUNNABLE at
> java.util.zip.Deflater.deflateBytes(Native Method) at
> java.util.zip.Deflater.deflate(Deflater.java:306) - locked <21b0c5> (a
> java.util.zip.ZStreamRef) at
> java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:159)
> at
> java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:118) at
>
> java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72) - locked
> <132ba84> (a java.util.zip.GZIPOutputStream) at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
>
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:46)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.unBuffer(AbstractThresholdOutputStream.java:89)
>
> at
> org.apache.cxf.io.AbstractThresholdOutputStream.close(AbstractThresholdOutputStream.java:100)
>
> at
> org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) at
>
> org.apache.cxf.transport.http.AbstractHTTPDestination$BackChannelConduit.close(AbstractHTTPDestination.java:619)
>
> at
> org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:62)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
> - locked <1ab9b7e> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.interceptor.AbstractFaultChainInitiatorObserver.onMessage(AbstractFaultChainInitiatorObserver.java:99)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:315)
>
> - locked <16129e9> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.interceptor.OutgoingChainInterceptor.handleMessage(OutgoingChainInterceptor.java:77)
>
> at
> org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)
>
> - locked <1649a52> (a org.apache.cxf.phase.PhaseInterceptorChain) at
> org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:113)
>
> at
> org.apache.cxf.transport.servlet.ServletDestination.invoke(ServletDestination.java:97)
>
> at
> org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:461)
>
> at
> org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:149)
>
> at
> org.apache.cxf.transport.servlet.AbstractCXFServlet.invoke(AbstractCXFServlet.java:148)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:179)
>
> at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:108)
>
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:734) at
> org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:159)
>
> at
> org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1523)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:343)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
> at
> com.cablelabs.ipr.context.filter.AcceptServletFilter.doFilter(AcceptServletFilter.java:40)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
>
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:215)
>
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:277)
>
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:188)
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
> at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:97) at
> com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:85)
>
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:185)
>
> at
> org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:641)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:322)
>
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:226)
>
> at
> com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:165)
>
> at
> com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:791)
> at
> com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:693) at
> com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:954) at
> com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:170)
>
> at
> com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:135)
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:102)
>
> at
> com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:88)
> at
> com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:76)
> at
> com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:53)
>
> at
> com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:57)
>
> at com.sun.grizzly.ContextTask.run(ContextTask.java:69) at
> com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:330)
>
> at
> com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:309)
>
> at java.lang.Thread.run(Thread.java:662) Locked ownable synchronizers: -
> locked <1cfb6a0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
> I would love to have someone smarter and more experienced than me to shed
> some light on this.
>
>
> [1]
> http://www.java.net/forum/topic/glassfish/glassfish/gzip-compression-glassfish-3-doesnt-work?force=899
> [2] https://forums.oracle.com/forums/thread.jspa?messageID=4627097
>
> --
>
> [Message sent by forum member 'steveglennon']
>
> View Post: http://forums.java.net/node/853023
>
>