users@jersey.java.net

Re: [Jersey] regarding issue 238 (FormDataMultiPart exceeds heap space for large files)

From: Paul Sandoz <Paul.Sandoz_at_Sun.COM>
Date: Mon, 01 Jun 2009 08:55:35 -0700

Hi Keith,

The use of @FormParam with multipart/* is deprecated.

Originally i tried to overload @FormParam but it really does not work
because the "application/x-www-form-urlencoded" and "multipart/*"
media types are quite different in the way one might consume the data.

Instead use @FormDataParam:

     @Path("/candidate-data")
     @POST
     @Consumes("multipart/form-data")
     @Produces("text/html")
     public Response uploadDataFile(
             @FormDataParam("fileFormat") String dataFileFormat,
             @FormDataParam("fileName") String dataFileName,
             @FormDataParam("file") InputStream dataFile)
     {

I would link to the JavaDoc but java.net is down.

Paul.

On Jun 1, 2009, at 7:02 AM, Keith Sheppard wrote:

> Hello again, I have some follow up information for you about this...
>
> So after downloading the jars I had initially neglected to I tried
> again and was surprised to see that I was still getting the out of
> mem error. I then compared by code to the example shown in the issue
> page and found that the most obvious difference is that I'm using
> FormParam annotations and the example was not. I switched my code to
> use a FormDataMultiPart like the example and was relieved to see the
> out of memory error dissapear. Here is my function before and after:
>
> BEFORE:
> @Path("/candidate-data")
> @POST
> @Consumes("multipart/form-data")
> @Produces("text/html")
> public Response uploadDataFile(
> @FormParam("fileFormat") String dataFileFormat,
> @FormParam("fileName") String dataFileName,
> @FormParam("file") InputStream dataFile)
> {
> try
> {
> FileFormat ff = FileFormat.valueOf(dataFileFormat);
> final FlatFileFormat flatFileFormat;
> switch(ff)
> {
> case CSV_FORMAT:
> {
> flatFileFormat =
> CommonFlatFileFormat.CSV_RFC_4180;
> }
> break;
>
> case TAB_DELIMITED_FORMAT:
> {
> flatFileFormat =
> CommonFlatFileFormat.TAB_DELIMITED_UNIX;
> }
> break;
>
> default: throw new IllegalArgumentException(
> "Bad file format string: " +
> dataFileFormat);
> }
>
> FlatFileReader ffr = new FlatFileReader(
> new InputStreamReader(dataFile),
> flatFileFormat);
> this.getCandidateDatabase().uploadDataFile(dataFileName,
> ffr);
>
> return Response.ok(dataFileName).build();
> }
> catch(Exception ex)
> {
> LOG.log(Level.SEVERE,
> "failed to upload data file",
> ex);
> return Response.status(Status.BAD_REQUEST).build();
> }
> }
>
>
> AFTER:
> @Path("/candidate-data")
> @POST
> @Consumes("multipart/form-data")
> @Produces("text/html")
> public Response uploadDataFile(
> FormDataMultiPart formData)
> {
> try
> {
> String dataFileFormat =
> formData.getField("fileFormat").getValue();
> String dataFileName =
> formData.getField("fileName").getValue();
> InputStream dataFile =
> formData.getField("file").getValueAs(
> InputStream.class);
>
> FileFormat ff = FileFormat.valueOf(dataFileFormat);
> final FlatFileFormat flatFileFormat;
> switch(ff)
> {
> case CSV_FORMAT:
> {
> flatFileFormat =
> CommonFlatFileFormat.CSV_RFC_4180;
> }
> break;
>
> case TAB_DELIMITED_FORMAT:
> {
> flatFileFormat =
> CommonFlatFileFormat.TAB_DELIMITED_UNIX;
> }
> break;
>
> default: throw new IllegalArgumentException(
> "Bad file format string: " +
> dataFileFormat);
> }
>
> FlatFileReader ffr = new FlatFileReader(
> new InputStreamReader(dataFile),
> flatFileFormat);
> this.getCandidateDatabase().uploadDataFile(dataFileName,
> ffr);
>
> return Response.ok(dataFileName).build();
> }
> catch(Exception ex)
> {
> LOG.log(Level.SEVERE,
> "failed to upload data file",
> ex);
> return Response.status(Status.BAD_REQUEST).build();
> }
> }
>
> So, I am guessing that if you change the test code to use @FormParam
> you will find that you are still buffering to RAM in that case.
>
> The FormDataMultiPart approach is working great for me, but should
> we reopen the issue so that it works the same for @FormParam
> annotations?
>
> Thanks
> Keith
>
> On Thu, May 28, 2009 at 4:23 PM, Keith Sheppard
> <keithshep_at_gmail.com> wrote:
> Oh, sorry I misunderstood the dependencies doc
> (https://jersey.dev.java.net/source/browse/*checkout*/jersey/tags/jersey-1.0.3/jersey/dependencies.html
> )
> and I left out mimepull.jar and jersey-multipart.jar (surprisingly it
> works for small file uploads without them). Thank you for your helpful
> response.
>
> Keith
>
> On Thu, May 28, 2009 at 3:30 PM, Paul Sandoz <Paul.Sandoz_at_sun.com>
> wrote:
> > Hi Keith,
> >
> > Issue 238 was fixed in 1.0.3.
> >
> > Are you using 1.0.3 for all modules, including the jersey-multpart
> module?
> >
> > Paul.
> >
> > On May 28, 2009, at 11:31 AM, Keith Sheppard wrote:
> >
> >> Hello,
> >>
> >> I'm using 1.0.3 and I'm bumping up against this issue
> >> (https://jersey.dev.java.net/issues/show_bug.cgi?id=238), but I
> >> probably don't have time to wait for the update release since I
> need
> >> to release my webapp pretty soon. Could you recommend the best
> course
> >> of action for me to fix this in the near term? ie. which do you
> think
> >> would be the most "stable" solution:
> >> - get the head of the repo or some tagged version and work with
> that
> >> - patch 1.0.3 with some minimal set of changes (and if so which
> patches?)
> >> - use jersey-1.1.0-ea instead
> >> - something else
> >>
> >> Thanks!
> >> Keith
> >>
> >>
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
> >> For additional commands, e-mail: users-help_at_jersey.dev.java.net
> >>
> >
> >
> >
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe_at_jersey.dev.java.net
> > For additional commands, e-mail: users-help_at_jersey.dev.java.net
> >
> >
>
>
>
> --
> keithsheppard.name