Alexey helped me out on this one offline and I just want to share the
resolution.
In the end we figured out that the problem was not caused by grizzly,
but by bugs in my code and in the benchmarking code, combination of
which made it pretty difficult to spot the real problem.
When it comes to issue in my code, the delay was caused by calling
SocketChannel#register before Selector#wakeup, which always resulted
in a nasty blocking == delay. Alexey recommended to rewrite the code
so that the registration is executed by the selector thread, which
processes a registration queue (similarly to how grizzly does this
already). I did that and the delay is now gone.
Thanks Alexey!
/i
On Oct 23, 2008, at 2:10 AM, Oleksiy Stashok wrote:
>>>
>>> Hello Igor,
>>>
>>>> I'm building an ARP filter for grizzly that I use in glassfish-
>>>> v2ur2 and I'm seeing huge request latency under moderate load (20
>>>> users). When I run tests with a single user, the delay is usually
>>>> insignificant.
>>>>
>>>> I wrote a faban benchmark for my filter and also added some
>>>> debugging code that tracks processing time in the filter and I
>>>> came to the conclusion that the delay occurs before the request
>>>> hits the doFilter method of my filter, which means that the delay
>>>> is occurring in grizzly or glassfish and not in my filter.
>>>>
>>>> I see the problem repeatedly. The 90th percentile of the delay is
>>>> around 10 seconds (the average is 4-5 seconds, max is 30+ seconds).
>>>>
>>>> Are there any known issues with grizzly used in glassfish that
>>>> could cause this behavior?
>>> No, it's not known, at least for me. It could be related to
>>> situation, when some thread (other than Selector thread) tries to
>>> operate with a selector directly.
>>> Please make sure your Filter doesn't do something like that.
>>>
>>> For sure it could be good if you can provide some testcase, which
>>> reproduces the problem, so it's possible to run it locally and
>>> debug.
>>>
>>> Thank you.
>>>
>>> WBR,
>>> Alexey.
>>
>> Since I work for Sun, I could set up a box on SWAN and give you
>> access to it. Another alternative is that I'll send you all the
>> code and instructions on how to set it up. The code is open source
>> so either way will work for me.
> I prefer 1st option :) Please let me know the coordinates of the
> machine.
>
> Thank you.
>
> WBR,
> Alexey.
>
>
>>
>>
>> thanks,
>> Igor
>>
>>
>>>
>>>
>>>>
>>>>
>>>> cheers,
>>>> Igor
>>>>
>>>>
>>>>
>>>> Here are relevant parts of the domain.xml:
>>>>
>>>> <jvm-options>-Dcom.sun.enterprise.web.connector.grizzly.asyncHandlerClass
>>>> =
>>>> com
>>>> .sun.enterprise.web.connector.grizzly.async.DefaultAsyncHandler</
>>>> jvm-options>
>>>> <jvm-options>-Dcom.sun.enterprise.web.connector.grizzly.asyncHandler.ports
>>>> =8080</jvm-options>
>>>> <jvm-options>-Dcom.sun.enterprise.web.connector.grizzly.asyncFilters
>>>> =com.igorminar.grizzlysendfile.SendfileFilter</jvm-options>
>>>>
>>>> <http-listener acceptor-threads="5" address="0.0.0.0" blocking-
>>>> enabled="false" default-virtual-server="server" enabled="true"
>>>> family="inet" id="http-listener-1" port="8080" security-
>>>> enabled="false" server-name="" xpowered-by="true">
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
>> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>