users@grizzly.java.net

Re: Several weird errors under high load with grizzly in glassfish

From: Guillaume <itchy75_at_hotmail.fr>
Date: Thu, 27 Jan 2011 13:44:20 +0100

Thank you for your answers,

Our goal is to use grizzly as a standalone server and stop using glassfish
server. Glassfish provide a lot of stuff that we don't need so we think we
only need grizzly. Do you think it is a good solution ?

I have tried to follow the documentation link about embeddable server
(Embeddable
HTTP WebServer<http://download.java.net/maven/glassfish/com/sun/grizzly/grizzly-http-webserver/1.9.30/>
 *Getting Started!<http://weblogs.java.net/blog/jfarcand/archive/2008/07/extending_the_g.html>
) *on grizzly web page the link is broken. I found some information on JF
Arcand's blog. Do you have a document that explain how to configure girzzly
as a standalone web server (ie available properties, parameters, etc) ?

Thanks,

William.

On Thu, Jan 27, 2011 at 1:02 PM, Oleksiy Stashok <oleksiy.stashok_at_oracle.com
> wrote:

> Hi William,
>
> That's true we do not use the 2.1.1 version of glassfish, only a 2.1 with
> patches. We tried to switch to 2.1.1 last year but we had a problem with
> memory (it was always growing).
>
> I remember there was an issue with some JDK Collection (if I'm not
> mistaken), so we started to use different one.
>
> We could try with a newer version but as we only use comet in glassfish
> server, we think that it can be easier to use a standalone grizzly http
> server. This will allow us to use the lastest grizzly 1.x and upgrade with
> the lastest version easely.
>
> Agree, if you use latest Grizzly 1.9.30 release - you'd be easily able to
> integrate your app into coming Glassfish 3.1.
>
> According to you, is it easy to switch from glassfish to a grizzly
> standalone server ? For what I have seen, only configuration is more
> difficult.
>
> The comet part was changed a bit between 1.0.x and 1.9.x version, but not
> so much... mostly it's package name change com.sun.enterprise.* to
> com.sun.grizzly.*
>
> Another question; during your last load test with grizzly, how many
> concurrent users can be handle by a single node when streaming is used (not
> long polling)?
>
> Unfortunately we don't do such a tests, but Glassfish performance team
> does. I can check with them and let you know.
>
> Thanks.
>
> WBR,
> Alexey.
>
>
> Thanks.
>
> William.
>
> On Wed, Jan 26, 2011 at 11:16 AM, Oleksiy Stashok <
> oleksiy.stashok_at_oracle.com> wrote:
>
>> Hi William,
>>
>> according to the stacktrace, you're using pretty old Glassfish 2.1, can
>> you at least update it to the latest 2.1.1?
>>
>> Thanks.
>>
>>
>> WBR,
>> Alexey.
>>
>> On Jan 24, 2011, at 21:16 , Guillaume wrote:
>>
>> Hi Alexey,
>>
>> Unfortunately, we are stuck with glassfish 2.1 for a long time before we
>> can change grizzly or glassfish version.
>>
>> I can't provide a test case, it is the result of a load test. I don't know
>> what can produce such "stack traces".
>>
>> We had a lot of exceptions like these during 10 minutes and it stops.
>> Sometimes, some of them appear again but not that much.
>>
>> I just wanted to know if these errors are known issues or can be the
>> result of an overload.
>>
>> If we can't find out what the problem is, I think our best chance is to
>> upgrade glassfish but without certainty. If we can't identify the source of
>> the error, then we can't be sure it won't happen again.
>>
>> If anyone has an idea, don't hesitate to contact me.
>>
>> Thanks.
>>
>> William.
>>
>> On Mon, Jan 24, 2011 at 7:41 PM, Oleksiy Stashok <
>> oleksiy.stashok_at_oracle.com> wrote:
>>
>>> Hi William,
>>>
>>> did you try this on Glassfish 3.0.1 or 3.1 (latest promoted build)?
>>>
>>> Can you pls. provide the simple testcase for it?
>>>
>>> Thanks.
>>>
>>> WBR,
>>> Alexey.
>>>
>>> On Jan 24, 2011, at 15:55 , william.d wrote:
>>>
>>>
>>>> Hello,
>>>>
>>>> On glassfish 2.1, I have the following errors the server.log files :
>>>>
>>>> 1./ java.lang.StackOverflowError
>>>>
>>>> Very strange and scary error but it is printed as a warning in log
>>>> files.
>>>> Does anyone have already seen this error ?
>>>>
>>>>
>>>> WARNING|sun-appserver2.1|javax.enterprise.system.stream.err|_ThreadID=172;_ThreadName=httpWorkerThread-8280-466;_Reque
>>>> stID=355fd080-200e-4c78-b7c2-9186137e84fb;|
>>>> java.lang.StackOverflowError
>>>> at
>>>> org.apache.tomcat.util.http.MimeHeaders.setValue(MimeHeaders.java:304)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1394)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.flush
>>>> (AsynchronousOutputBuffer.java:103)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.realWriteBytes
>>>> (SocketChannelOutputBuffer.java:145)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsynchronousOutputBuffer.realWriteBytes
>>>> (AsynchronousOutputBuffer.java:70)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:378)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.write(InternalOutputBuffer.java:815)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.sendStatus(InternalOutputBuffer.java:502)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.DefaultProcessorTask.prepareResponse
>>>> (DefaultProcessorTask.java:1457)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.action
>>>> (DefaultProcessorTask.java:863)
>>>> at org.apache.coyote.Response.action(Response.java:235)
>>>> at
>>>>
>>>> org.apache.coyote.http11.InternalOutputBuffer.flush(InternalOutputBuffer.java:360)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.SocketChannelOutputBuffer.flush
>>>> (SocketChannelOutputBuffer.java:180)
>>>>
>>>> 2./ NPE on DefaultAsyncExecutor.postExecute :
>>>>
>>>>
>>>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=171;_ThreadName=httpWorkerThread-8280-86;_Requ
>>>> estID=73f0a3ee-5cfa-4bca-a70e-c3938863bc03;|The log message is null.
>>>> java.lang.NullPointerException
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.DefaultAsyncExecutor.postExecute
>>>> (DefaultAsyncExecutor.java:189)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsyncProcessorTask.doTask
>>>> (AsyncProcessorTask.java:99)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run
>>>> (TaskBase.java:265)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run
>>>> (WorkerThreadImpl.java:116)
>>>>
>>>>
>>>> 3./ NPE in CometEngine.handle
>>>>
>>>> For this one, it seems that the selection key is null (by looking at the
>>>> code). Can it be reproduced by a use case ?
>>>>
>>>>
>>>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=170;_ThreadName=httpWorkerThread-8280-43;_Requ
>>>> estID=1d902f71-86f0-46f7-9d66-7e9cb47f729f;|The log message is null.
>>>> java.lang.NullPointerException
>>>> at
>>>> java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:881)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.comet.CometEngine.handle
>>>> (CometEngine.java:346)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.comet.CometAsyncFilter.doFilter
>>>> (CometAsyncFilter.java:84)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.DefaultAsyncExecutor.invokeFilters
>>>> (DefaultAsyncExecutor.java:175)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.DefaultAsyncExecutor.interrupt
>>>> (DefaultAsyncExecutor.java:153)
>>>> at
>>>> com.sun.enterprise.
>>>> web.connector.grizzly.async.AsyncProcessorTask.doTask
>>>> (AsyncProcessorTask.java:92)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.TaskBase.run
>>>> (TaskBase.java:265)
>>>> at
>>>> com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run
>>>> (WorkerThreadImpl.java:116)
>>>> |#]
>>>>
>>>>
>>>> Errors 2. and 3. seems to be related to async processing.... Do you have
>>>> any
>>>> idea why a selection key or a processor task could be null ?
>>>>
>>>> 4. NPE in a ressource bundle.
>>>>
>>>> For the last one it seems that a bundle is missing but I don't know how
>>>> to
>>>> finf it in order to fix it.
>>>>
>>>>
>>>> SEVERE|sun-appserver2.1|javax.enterprise.system.container.web|_ThreadID=120;_ThreadName=httpWorkerThread-8280-234;_Req
>>>>
>>>> uestID=84dd1cd9-036c-43fc-930a-af79dc664a6a;|StandardWrapperValve[update]:
>>>> Cannot find message associated with key
>>>> standardWrapper.acknowledgeExceptio
>>>> n
>>>> java.lang.NullPointerException
>>>> |#]
>>>>
>>>>
>>>> Thanks.
>>>>
>>>> William.
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/Several-weird-errors-under-high-load-with-grizzly-in-glassfish-tp30748059p30748059.html
>>>> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>
>>
>
>