On Sep 16, 2009, at 12:59 PM, Jeanfrancois Arcand wrote:
> Salut,
>
> Igor Minar wrote:
>> Hi there,
>> I'd like to give you an update on the grizzly-sendfile progress.
>> Due to a combination of reasons I wasn't able to dedicate much time
>> to the project until recently, but the progress I made in the last
>> few weeks should offset the delay.
>> Some notable changes:
>> - http range support for all single range combination defined in
>> the HTTP1.1 spec (this means that partial/resumable downloads are
>> now fully supported)
>> - support for ssl transfers (YAY!)
>> - data source abstraction (a file on the filesystem is just one of
>> many supported data sources, the source of data can be local
>> memory, database, hadoop fs, memcached, you name it)
>> - new plugin hook that should make it trivial to add gzip
>> compression or other kind of data manipulation before the data is
>> written to the socket channel
>> - lots of refactoring to clean things up and optimize code
>> execution paths
>> - as the result of the refactoring, I'm now considering supporting
>> grizzly 1.0.x, 1.9.x and 2.0 all from the same grizzly-sendfile
>> core code base
>
> This is great. BTW since you have commit privileges, you can
> backport the fix you need in 1.0.31 :-)
sure, can you point me to the source code? I know that it was
originally in the glassfish repo and then it was moved to the grizzly
repo, but I can't find it there :-/
>
>
>> that's about it for the good news, no is the time for some bad
>> news :)
>> The last version of grizzly that worked well with grizzly-sendfile
>> was 1.9.15, I tried using 1.9.16, 1.9.17, 1.9.18, 1.9.18a but no
>> release after 1.9.15 works well.
>> Here are issues I know about:
>> 1) Under stress, grizzly starts spewing these exceptions after
>> several hundreds or thousands of requests:
>> Sep 15, 2009 3:42:22 PM com.sun.grizzly.SelectorHandlerRunner
>> handleSelectException
>> WARNING: Channel was unexpectedly closed
>> Sep 15, 2009 3:42:22 PM com.sun.grizzly.http.SelectorThread$3
>> onException
>> SEVERE: Exception during controller processing
>> java.nio.channels.ClosedChannelException
>> at
>> java
>> .nio
>> .channels
>> .spi
>> .AbstractSelectableChannel.register(AbstractSelectableChannel.java:
>> 167) at
>> com
>> .sun
>> .grizzly
>> .DefaultSelectionKeyHandler
>> .register(DefaultSelectionKeyHandler.java:162) at
>> com
>> .sun
>> .grizzly
>> .TCPSelectorHandler
>> .processPendingOperations(TCPSelectorHandler.java:464) at
>> com
>> .sun.grizzly.TCPSelectorHandler.preSelect(TCPSelectorHandler.java:
>> 395)
>> at
>> com
>> .sun
>> .grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:
>> 183) at
>> com
>> .sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:
>> 130)
>> at java.util.concurrent.ThreadPoolExecutor
>> $Worker.runTask(ThreadPoolExecutor.java:886) at
>> java.util.concurrent.ThreadPoolExecutor
>> $Worker.run(ThreadPoolExecutor.java:908) at
>> java.lang.Thread.run(Thread.java:637)
>> at java.lang.Thread.run(Thread.java:637)
>> this issue was introduced in 1.9.17 and is still present in 1.9.18a
>
> OK file an issue so we don't forget it.
https://grizzly.dev.java.net/issues/show_bug.cgi?id=736
>
>
>
>> 2) an older issue that was reintroduced in 1.9.17 and is still
>> present in 1.9.18a - https://grizzly.dev.java.net/issues/show_bug.cgi?id=548
>> it completely prevents grizzly-sendfile from supporting blocking
>> IO, so I had to revert the default to 100% non-blocking IO and get
>> used to the performance penalty.
>> Exception in thread "GrizzlySendfileWorker-0"
>> java.nio.channels.IllegalBlockingModeException
>> at
>> java
>> .nio
>> .channels
>> .spi
>> .AbstractSelectableChannel
>> .configureBlocking(AbstractSelectableChannel.java:257) at
>> com.igorminar.grizzlysendfile.SendfileTask
>> $DefaultSendfileTask.run(SendfileTask.java:131) at
>> java.util.concurrent.ThreadPoolExecutor
>> $Worker.runTask(ThreadPoolExecutor.java:886) at
>> java.util.concurrent.ThreadPoolExecutor
>> $Worker.run(ThreadPoolExecutor.java:908) at
>> java.lang.Thread.run(Thread.java:637)
>> or occasionally:
>> Sep 15, 2009 9:23:26 AM com.sun.grizzly.SelectorHandlerRunner
>> handleSelectException
>> SEVERE: doSelect exception
>> java.nio.channels.IllegalBlockingModeException
>> at
>> java
>> .nio
>> .channels
>> .spi
>> .AbstractSelectableChannel.register(AbstractSelectableChannel.java:
>> 172) at
>> com
>> .sun
>> .grizzly
>> .DefaultSelectionKeyHandler
>> .register(DefaultSelectionKeyHandler.java:162) at
>> com
>> .sun
>> .grizzly
>> .TCPSelectorHandler
>> .processPendingOperations(TCPSelectorHandler.java:464) at
>> com
>> .sun.grizzly.TCPSelectorHandler.preSelect(TCPSelectorHandler.java:
>> 395)
>> at
>> com
>> .sun
>> .grizzly.SelectorHandlerRunner.doSelect(SelectorHandlerRunner.java:
>> 183) at
>> com
>> .sun.grizzly.SelectorHandlerRunner.run(SelectorHandlerRunner.java:
>> 130)
>> at java.util.concurrent.ThreadPoolExecutor
>> $Worker.runTask(ThreadPoolExecutor.java:886) at
>> java.util.concurrent.ThreadPoolExecutor
>> $Worker.run(ThreadPoolExecutor.java:908) at
>> java.lang.Thread.run(Thread.java:637)
>> relevant code:
>> //in an AsyncFilter
>> final ProcessorTask task = asyncExecutor.getProcessorTask();
>>
>> task.getSelectionKey().attach(SelectionKeyAttachment.DEREGISTERED);
>> task.getSelectionKey().cancel();
>> //elsewhere later
>> selectionKey.cancel();
>> socketChannel.configureBlocking(true); //>>> Exception: at
>> com.igorminar.grizzlysendfile.SendfileTask
>> $DefaultSendfileTask.run(SendfileTask.java:131)
>
> OK that one I don't get it. The code above looks good. So it's
> between 1.9.16 and 17 that we have re-introduced the regression.
> Looking at it.
thanks
>
>
>> There might be more issues, but they are not visible due the issues
>> described above.
>
>
>> If you want to try grizzly-sendfile, you can just grab the latest
>> jar from https://kenai.com/svn/grizzly-sendfile~maven/com/igorminar/
>> grizzly-sendfile/grizzly-sendfile-server/0.3.2-SNAPSHOT/grizzly-
>> sendfile-server-0.3.2-SNAPSHOT-jar-with-dependencies.jar You can
>> run the server as:
>> java -jar grizzly-sendfile-server-0.3.2-SNAPSHOT-jar-with-
>> dependencies.jar /path/to/some/files
>> If you want to test the bug #2:
>> java -jar grizzly-sendfile-server-0.3.2-SNAPSHOT-jar-with-
>> dependencies.jar -a
>> com.igorminar.grizzlysendfile.algorithm.EqualBlockingAlgorithm /
>> path/to/some/files
>> once these final issues are resolved, grizzly-sendfile code will be
>> ready to be used by grizzly.
>
> Great work! I would really like to make it work/merge and allow
> users of GlassFish v3 to use it.
Almost there! Once these issues are resolved, I'm going to do a round
of thorough load testing and then grizzly-sendfile will be ready for
the integration.
/i