users@grizzly.java.net

Grizzly no more responding when clients are doing many big files upload and download

From: Vrignaud Etienne <evrignaud_at_axway.com>
Date: Thu, 19 Aug 2010 06:37:56 -0700 (PDT)

Hi folks,

I am using Grizzly 1.9.19-beta7.
My use case is that the clients of my application needs to upload or
download very big files (more than 50 Go).
Every time a client do an upload or a download of that kind of big file, one
thread of the Grizzly http server is busy during many times, and only
dedicated to this task.
The consequence is that, if many clients are doing upload or download at the
same time, on the server part all the Grizzly threads dedicated to manage
requests are busy in receiving or sending the files. And the server does not
answer any more to the simple http client requests. They have to wait until
some upload or download are finished.

I have seen that there is by default 5 threads that manage http requests.
Using selectorThread.setMaxThreads() I succeed to increase this value to
256.
Unfortunately the 256 value does not seem to fit my needs, because it
consumes too many resources on my server (mainly memory, TCP ports), and
also I can't imagine to limit the number of simultaneous uploads/downloads
to 256.
I cannot set this kind of limit in my application.

Could you tell me what are the best ways to manage this kind of edge case
with Grizzly.
Are there some dedicated thread pools that can manage file upload/download
in parallel?
Are there some internal mechanisms in Grizzly that could be used to enable
NonBlocking IO to receive or send the files?

At this time my implementation for the upload is based on
commons-fileupload, and for download I simply return an OutputStream as
request response.

My big requirement is that on the client part, a browser must be enough to
download the file. This implies that it's not possible to use my own
algorithm to split the file in blocks.

Thanks for your help,
/Etienne VRIGNAUD

-- 
View this message in context: http://old.nabble.com/Grizzly-no-more-responding-when-clients-are-doing-many-big-files-upload-and-download-tp29482040p29482040.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.