users@grizzly.java.net

Re: stress test with bear2.0

From: Oleksiy Stashok <Oleksiy.Stashok_at_Sun.COM>
Date: Thu, 01 Apr 2010 15:12:26 +0200

Hi Eric,

that what I've suspected, that's why wanted to see the log why it
happens :)
I'll check more, but would like to ask you to try to reproduce issue
too...

Thank you!

WBR,
Alexey.

On Apr 1, 2010, at 9:13 , Eric Fong wrote:

>
> Hi Alexey
>
> Thanks for your reply. The problem still occur even I using the
> latest code.
> Btw, if I set the log level to FINEST, the msg rate dropped
> significantly
> thus the problem cannot be replicated... I've tried another way to
> locate
> the problem and found that:
>
> - the AsyncWriteQueueRecord item has been recycled by
> AbstractNIOAsyncQueueWriter.onWriteCompleted() -> record.recycle()
>
> - however, the recycled item has been processed by
> AbstractNIOAsyncQueueWriter.processAsync() again, thus
> nullpointerexception
> throw out.
>
> Regards
> Eric
>
>
> Oleksiy Stashok wrote:
>>
>> Hi Eric,
>>
>> can I ask you to checkout the latest Grizzly 2.0 sources and try them
>> out with FINEST log level enabled for class:
>> "com.sun.grizzly.nio.AbstractNIOAsyncQueueWriter.level=FINEST"
>>
>> If you'll see the exception - please send the output you've got from
>> the logger.
>>
>> Thanks.
>>
>> WBR,
>> Alexey.
>>
>> On Mar 31, 2010, at 11:56 , Eric Fong wrote:
>>
>>>
>>> Hi Alexey,
>>>
>>> May I explain more on the stress test program here:
>>> - This is a 1server <-> 1client program,
>>> - Once stress client connects to stress server, both server and
>>> client will
>>> create a thread (StressCommon.startSendMsg()) to send message out.
>>> The rate
>>> is now set to 10,000 msg/sec (config in StressCommon, l_iRate). i.e.
>>> For
>>> both side, there're 10,000 msg in / out in the same time
>>> - nullporinterexception will be thrown after a few sec:
>>> 2010年3月31日 下午05:31:33
>>> com.sun.grizzly.nio.transport.TCPNIOTransport fireIOE
>>> vent
>>> 警告: Unexpected exception occurred
>>> fireIOEvent().connection=com.sun.grizzly.nio
>>> .transport.TCPNIOConnection_at_1010058 event=WRITE
>>> java.lang.NullPointerException
>>> at
>>> com
>>> .sun.grizzly.nio.transport.TCPNIOTransport.write(TCPNIOTransport.j
>>> ava:891)
>>> at
>>> com
>>> .sun.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsy
>>> ncQueueWriter.java:69)
>>> at
>>> com
>>> .sun.grizzly.nio.AbstractNIOAsyncQueueWriter.doWrite(AbstractNIOAs
>>> yncQueueWriter.java:324)
>>> at
>>> com
>>> .sun.grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(Abstract
>>> NIOAsyncQueueWriter.java:239)
>>> at
>>> com
>>> .sun.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterC
>>> hain.java:173)
>>> at
>>> com.sun.grizzly.ProcessorExecutor.resume(ProcessorExecutor.java:73)
>>> at
>>> com.sun.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:61)
>>> at
>>> com
>>> .sun.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTrans
>>> port.java:634)
>>> at
>>> com.sun.grizzly.strategies.WorkerThreadStrategy
>>> $1.run(WorkerThreadStr
>>> ategy.java:115)
>>> at
>>> com.sun.grizzly.threadpool.AbstractThreadPool
>>> $Worker.doWork(AbstractT
>>> hreadPool.java:417)
>>> at
>>> com.sun.grizzly.threadpool.AbstractThreadPool
>>> $Worker.run(AbstractThre
>>> adPool.java:398)
>>> at java.lang.Thread.run(Thread.java:619)
>>> - I do the same test with M3 before while M3 doesnt throw any
>>> exception and
>>> runs smoothly for more than a hour (
>>> http://old.nabble.com/file/p28093714/New%2BNIO%2BStress%2BTest.doc
>>> New NIO
>>> Stress Test Result.doc )
>>>
>>> Attached please find the updated stress test source for your
>>> reference.
>>> http://old.nabble.com/file/p28093714/src.zip src.zip
>>>
>>>
>>>
>>>
>>> Ken--_at_newsgroupstats.hk wrote:
>>>>
>>>> It happen during normal 1server <-> 1client and not the 'stop
>>>> client'
>>>> case.
>>>>
>>>>
>>>> Oleksiy Stashok wrote:
>>>>>
>>>>> Ken, does it happen during the normal run 1server <-> 1client, or
>>>>> when
>>>>> you stop the client for example or something else?
>>>>>
>>>>> Thanks.
>>>>>
>>>>> WBR,
>>>>> Alexey.
>>>>>
>>>>> On Mar 30, 2010, at 13:04 , Ken--_at_newsgroupstats.hk wrote:
>>>>>
>>>>>>
>>>>>> attached
>>>>>> http://old.nabble.com/file/p28080913/src.zip src.zip
>>>>>>
>>>>>> It's a very rough code create by our teammate.
>>>>>>
>>>>>> Regards,
>>>>>> Ken
>>>>>>
>>>>>>
>>>>>>
>>>>>> Oleksiy Stashok wrote:
>>>>>>>
>>>>>>> Hi Ken,
>>>>>>>
>>>>>>> agree with Ming Qin, it could be interesting to take a look at
>>>>>>> the
>>>>>>> code, or its part, which actually causes this.
>>>>>>> From the exception it looks like you pass null buffer for the
>>>>>>> write
>>>>>>> operation, can you pls. check this?
>>>>>>>
>>>>>>> Thanks.
>>>>>>>
>>>>>>> WBR,
>>>>>>> Alexey.
>>>>>>>
>>>>>>> On Mar 30, 2010, at 6:51 , Ming Qin wrote:
>>>>>>>
>>>>>>>> Ken:
>>>>>>>> Can you share with community of your testing environment
>>>>>>>> settings
>>>>>>>> and testing case which generated those nullpointException?
>>>>>>>>
>>>>>>>>
>>>>>>>> Ming Qin
>>>>>>>> Cell Phone 858-353-2839
>>>>>>>>
>>>>>>>> --- On Mon, 3/29/10, Ken--_at_newsgroupstats.hk
>>>>>>>> <dragonken_at_gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> From: Ken--_at_newsgroupstats.hk <dragonken_at_gmail.com>
>>>>>>>> Subject: stress test with bear2.0
>>>>>>>> To: users_at_grizzly.dev.java.net
>>>>>>>> Date: Monday, March 29, 2010, 8:19 PM
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I download your latest code from trunk and recompile it for
>>>>>>>> stress
>>>>>>>> test. The
>>>>>>>> result is much better than the snapshot last week. However, the
>>>>>>>> server
>>>>>>>> sometime will throw NullPointerException.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Ken
>>>>>>>>
>>>>>>>>
>>>>>>>> 2010年3月29日 下午12:42:34
>>>>>>>> com.sun.grizzly.nio.transport.TCPNIOTransport fireIOE
>>>>>>>> vent
>>>>>>>> 警告: Unexpected exception occurred
>>>>>>>> fireIOEvent().connection=com.sun.grizzly.nio
>>>>>>>> .transport.TCPNIOConnection_at_1e2afb2 event=WRITE
>>>>>>>> java.lang.NullPointerException
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.transport.TCPNIOTransport.write(TCPNIOTransport.j
>>>>>>>> ava:891)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsy
>>>>>>>> ncQueueWriter.java:69)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.AbstractNIOAsyncQueueWriter.doWrite(AbstractNIOAs
>>>>>>>> yncQueueWriter.java:324)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.AbstractNIOAsyncQueueWriter.processAsync(Abstract
>>>>>>>> NIOAsyncQueueWriter.java:239)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.filterchain.DefaultFilterChain.process(DefaultFilterC
>>>>>>>> hain.java:173)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun.grizzly.ProcessorExecutor.resume(ProcessorExecutor.java:
>>>>>>>> 73)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:
>>>>>>>> 61)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTrans
>>>>>>>> port.java:634)
>>>>>>>> at
>>>>>>>> com.sun.grizzly.strategies.WorkerThreadStrategy
>>>>>>>> $1.run(WorkerThreadStr
>>>>>>>> ategy.java:115)
>>>>>>>> at
>>>>>>>> com.sun.grizzly.threadpool.AbstractThreadPool
>>>>>>>> $Worker.doWork(AbstractT
>>>>>>>> hreadPool.java:417)
>>>>>>>> at
>>>>>>>> com.sun.grizzly.threadpool.AbstractThreadPool
>>>>>>>> $Worker.run(AbstractThre
>>>>>>>> adPool.java:398)
>>>>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>>>> java.io.IOException: Connection is closed
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyn
>>>>>>>> cQueueWriter.java:112)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyn
>>>>>>>> cQueueWriter.java:93)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .sun
>>>>>>>> .grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyn
>>>>>>>> cQueueWriter.java:67)
>>>>>>>> at
>>>>>>>> com.sun.grizzly.AbstractWriter.write(AbstractWriter.java:
>>>>>>>> 68)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .mycompany
>>>>>>>> .net.GrizzlyMessageHandler.write(GrizzlyMessageHandler.java:
>>>>>>>> 61)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .mycompany
>>>>>>>> .net.GrizzlyMessageHandler.publishToSubscribers(GrizzlyMessa
>>>>>>>> geHandler.java:40)
>>>>>>>> at
>>>>>>>> com
>>>>>>>> .mycompany
>>>>>>>> .queue.test.QTestASCIIServOutItem.run(QTestASCIIServOutItem.
>>>>>>>> java:26)
>>>>>>>> at
>>>>>>>> com.mycompany.executor.SerialExecutor
>>>>>>>> $1.run(SerialExecutor.java:
>>>>>>>> 76)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor
>>>>>>>> $Worker.runTask(ThreadPoolExec
>>>>>>>> utor.java:886)
>>>>>>>> at
>>>>>>>> java.util.concurrent.ThreadPoolExecutor
>>>>>>>> $Worker.run(ThreadPoolExecutor
>>>>>>>> .java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>>>> --
>>>>>>>> View this message in context:
>>>>>>>> http://old.nabble.com/stress-test-with-bear2.0-tp28077647p28077647.html
>>>>>>>> Sent from the Grizzly - Users mailing list archive at
>>>>>>>> Nabble.com.
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>>>> For additional commands, e-mail: users-
>>>>>>>> help_at_grizzly.dev.java.net
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/stress-test-with-bear2.0-tp28077647p28080913.html
>>>>>> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>> --
>>> View this message in context:
>>> http://old.nabble.com/stress-test-with-bear2.0-tp28077647p28093714.html
>>> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>
>>
>>
>
> --
> View this message in context: http://old.nabble.com/stress-test-with-bear2.0-tp28077647p28105277.html
> Sent from the Grizzly - Users mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>