users@grizzly.java.net

Re: Several weird errors under high load with grizzly in glassfish

From: Oleksiy Stashok <oleksiy.stashok_at_oracle.com>
Date: Wed, 26 Jan 2011 11:16:51 +0100

Hi William,

according to the stacktrace, you're using pretty old Glassfish 2.1,
can you at least update it to the latest 2.1.1?

Thanks.


WBR,
Alexey.

On Jan 24, 2011, at 21:16 , Guillaume wrote:

> Hi Alexey,
>
> Unfortunately, we are stuck with glassfish 2.1 for a long time
> before we can change grizzly or glassfish version.
>
> I can't provide a test case, it is the result of a load test. I
> don't know what can produce such "stack traces".
>
> We had a lot of exceptions like these during 10 minutes and it
> stops. Sometimes, some of them appear again but not that much.
>
> I just wanted to know if these errors are known issues or can be the
> result of an overload.
>
> If we can't find out what the problem is, I think our best chance is
> to upgrade glassfish but without certainty. If we can't identify the
> source of the error, then we can't be sure it won't happen again.
>
> If anyone has an idea, don't hesitate to contact me.
>
> Thanks.
>
> William.
>
> On Mon, Jan 24, 2011 at 7:41 PM, Oleksiy Stashok <oleksiy.stashok_at_oracle.com
> > wrote:
> Hi William,
>
> did you try this on Glassfish 3.0.1 or 3.1 (latest promoted build)?
>
> Can you pls. provide the simple testcase for it?
>
> Thanks.
>
> WBR,
> Alexey.
>
> On Jan 24, 2011, at 15:55 , william.d wrote:
>
>
> Hello,
>
> On glassfish 2.1, I have the following errors the server.log files :
>
> 1./ java.lang.StackOverflowError
>
> Very strange and scary error but it is printed as a warning in log
> files.
> Does anyone have already seen this error ?
>
> WARNING|sun-appserver2.1|javax.enterprise.system.stream.err|
> _ThreadID=172;_ThreadName=httpWorkerThread-8280-466;_Reque
> stID=355fd080-200e-4c78-b7c2-9186137e84fb;|
> java.lang.StackOverflowError
> at
> org.apache.tomcat.util.http.MimeHeaders.setValue(MimeHeaders.java:304)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1394)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush
> (AsynchronousOutputBuffer.java:103)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
> (SocketChannelOutputBuffer.java:145)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
> (AsynchronousOutputBuffer.java:70)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 378)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:
> 815)
> at
> org
> .apache
> .coyote
> .http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
> at
> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse
> (DefaultProcessorTask.java:1457)
> at
> com
> .sun
> .enterprise
> .web
> .connector
> .grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
> at org.apache.coyote.Response.action(Response.java:235)
> at
> org
> .apache
> .coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:
> 360)
> at
> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush
> (SocketChannelOutputBuffer.java:180)
>
> 2./ NPE on DefaultAsyncExecutor.postExecute :
>
> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|
> _ThreadID=171;_ThreadName=httpWorkerThread-8280-86;_Requ
> estID=73f0a3ee-5cfa-4bca-a70e-c3938863bc03;|The log message is null.
> java.lang.NullPointerException
> at
> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.postExecute
> (DefaultAsyncExecutor.java:189)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsyncProcessorTask.doTask
> (AsyncProcessorTask.java:99)
> at
> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:
> 265)
> at
> com
> .sun
> .enterprise
> .web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:116)
>
>
> 3./ NPE in CometEngine.handle
>
> For this one, it seems that the selection key is null (by looking at
> the
> code). Can it be reproduced by a use case ?
>
> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|
> _ThreadID=170;_ThreadName=httpWorkerThread-8280-43;_Requ
> estID=1d902f71-86f0-46f7-9d66-7e9cb47f729f;|The log message is null.
> java.lang.NullPointerException
> at
> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
> at
> com
> .sun
> .enterprise
> .web.connector.grizzly.comet.CometEngine.handle(CometEngine.java:346)
> at
> com.sun.enterprise.web.connector.grizzly.comet.CometAsyncFilter.doFilter
> (CometAsyncFilter.java:84)
> at
> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.invokeFilters
> (DefaultAsyncExecutor.java:175)
> at
> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.interrupt
> (DefaultAsyncExecutor.java:153)
> at
> com.sun.enterprise.web.connector.grizzly.async.AsyncProcessorTask.doTask
> (AsyncProcessorTask.java:92)
> at
> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:
> 265)
> at
> com
> .sun
> .enterprise
> .web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:116)
> |#]
>
>
> Errors 2. and 3. seems to be related to async processing.... Do you
> have any
> idea why a selection key or a processor task could be null ?
>
> 4. NPE in a ressource bundle.
>
> For the last one it seems that a bundle is missing but I don't know
> how to
> finf it in order to fix it.
>
> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|
> _ThreadID=120;_ThreadName=httpWorkerThread-8280-234;_Req
> uestID=84dd1cd9-036c-43fc-930a-af79dc664a6a;|
> StandardWrapperValve[update]:
> Cannot find message associated with key
> standardWrapper.acknowledgeExceptio
> n
> java.lang.NullPointerException
> |#]
>
>
> Thanks.
>
> William.
>
>
> --
> View this message in context: http://old.nabble.com/Several-weird-errors-under-high-load-with-grizzly-in-glassfish-tp30748059p30748059.html
> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>
>
>
>