users@grizzly.java.net

Help with slow file uploads

From: Thomas Gideon <tgideon_at_learningobjects.com>
Date: Mon, 27 Apr 2009 13:48:12 -0400

I am new to the list and have already tried to search the mailing list
for an answer without success.

Our application runs in Glassfish v2.1 with whichever version of
Grizzly is bundled with it. We've followed the tuning
recommendations, specifically to use a number of request threads for
our http-listeners that are a small multiple of the physical cores on
our production servers.

We have an issue from our customers, though, that has me looking
through the Grizzly sources trying to figure out how to better handle
file uploads that may potentially be quite large and may come in from
slow connections. If those are both the case, we see problems with
the available threads to serve http requests being consumed and
affecting other users of the system.

This is not an uncommon situation. Groups of users can be on the
application at the same time and responding to a course instructor
asking each of them to upload some rich media like audio or video.

We have already solved the download problem for large files on slow
connections by installing Ignor Minar's grizzly-sendfile. I haven't
found anything similar for file uploads and it is unclear to me in
reviewing Grizzly's architecture where would be the best place for me
to intercede to push file uploads off for async processing outside of
the request processing thread pool.

I'd prefer not to have to set up a separate port on the server just
for file upload. I suspect it is possible for me to do enough parsing
of a request to figure out when to push a file upload off to a
separate thread pool/executor vs. allowing Grizzly's default logic to
run as usual.


TIA,

Thomas Gideon