users@grizzly.java.net

Re: Several weird errors under high load with grizzly in glassfish

From: Guillaume <itchy75_at_hotmail.fr>
Date: Mon, 24 Jan 2011 21:16:13 +0100

Hi Alexey,

Unfortunately, we are stuck with glassfish 2.1 for a long time before we can
change grizzly or glassfish version.

I can't provide a test case, it is the result of a load test. I don't know
what can produce such "stack traces".

We had a lot of exceptions like these during 10 minutes and it stops.
Sometimes, some of them appear again but not that much.

I just wanted to know if these errors are known issues or can be the result
of an overload.

If we can't find out what the problem is, I think our best chance is to
upgrade glassfish but without certainty. If we can't identify the source of
the error, then we can't be sure it won't happen again.

If anyone has an idea, don't hesitate to contact me.

Thanks.

William.

On Mon, Jan 24, 2011 at 7:41 PM, Oleksiy Stashok <oleksiy.stashok_at_oracle.com
> wrote:

> Hi William,
>
> did you try this on Glassfish 3.0.1 or 3.1 (latest promoted build)?
>
> Can you pls. provide the simple testcase for it?
>
> Thanks.
>
> WBR,
> Alexey.
>
> On Jan 24, 2011, at 15:55 , william.d wrote:
>
>
>> Hello,
>>
>> On glassfish 2.1, I have the following errors the server.log files :
>>
>> 1./ java.lang.StackOverflowError
>>
>> Very strange and scary error but it is printed as a warning in log files.
>> Does anyone have already seen this error ?
>>
>>
>> WARNING|sun-appserver2.1|javax.enterprise.system.stream.err|_ThreadID=172;_ThreadName=httpWorkerThread-8280-466;_Reque
>> stID=355fd080-200e-4c78-b7c2-9186137e84fb;|
>> java.lang.StackOverflowError
>> at
>> org.apache.tomcat.util.http.MimeHeaders.setValue(MimeHeaders.java:304)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1394)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.flush(AsynchronousOutputBuffer.java:103)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes(SocketChannelOutputBuffer.java:145)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes(AsynchronousOutputBuffer.java:70)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.prepareResponse(DefaultProcessorTask.java:1457)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action(DefaultProcessorTask.java:863)
>> at org.apache.coyote.Response.action(Response.java:235)
>> at
>>
>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.SocketChannelOutputBuffer.flush(SocketChannelOutputBuffer.java:180)
>>
>> 2./ NPE on DefaultAsyncExecutor.postExecute :
>>
>>
>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=171;_ThreadName=httpWorkerThread-8280-86;_Requ
>> estID=73f0a3ee-5cfa-4bca-a70e-c3938863bc03;|The log message is null.
>> java.lang.NullPointerException
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.postExecute(DefaultAsyncExecutor.java:189)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsyncProcessorTask.doTask(AsyncProcessorTask.java:99)
>> at
>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:265)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:116)
>>
>>
>> 3./ NPE in CometEngine.handle
>>
>> For this one, it seems that the selection key is null (by looking at the
>> code). Can it be reproduced by a use case ?
>>
>>
>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=170;_ThreadName=httpWorkerThread-8280-43;_Requ
>> estID=1d902f71-86f0-46f7-9d66-7e9cb47f729f;|The log message is null.
>> java.lang.NullPointerException
>> at
>> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.comet.CometEngine.handle(CometEngine.java:346)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.comet.CometAsyncFilter.doFilter(CometAsyncFilter.java:84)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.invokeFilters(DefaultAsyncExecutor.java:175)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.interrupt(DefaultAsyncExecutor.java:153)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.async.AsyncProcessorTask.doTask(AsyncProcessorTask.java:92)
>> at
>> com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:265)
>> at
>>
>> com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:116)
>> |#]
>>
>>
>> Errors 2. and 3. seems to be related to async processing.... Do you have
>> any
>> idea why a selection key or a processor task could be null ?
>>
>> 4. NPE in a ressource bundle.
>>
>> For the last one it seems that a bundle is missing but I don't know how to
>> finf it in order to fix it.
>>
>>
>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=120;_ThreadName=httpWorkerThread-8280-234;_Req
>> uestID=84dd1cd9-036c-43fc-930a-af79dc664a6a;|StandardWrapperValve[update]:
>> Cannot find message associated with key
>> standardWrapper.acknowledgeExceptio
>> n
>> java.lang.NullPointerException
>> |#]
>>
>>
>> Thanks.
>>
>> William.
>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/Several-weird-errors-under-high-load-with-grizzly-in-glassfish-tp30748059p30748059.html
>> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>>
>>
>
>