users@jersey.java.net

Re: [Jersey] Very Large files causing java Out of Memory error

From: Charles Brooking <public+java_at_charlie.brooking.id.au>
Date: Wed, 11 Mar 2009 12:26:19 +1000 (EST)

> On Mar 9, 2009, at 9:00 PM, jmiller wrote:
> We do have another API that supports caching stuff to disk. It uses
> the JavaMail API but processes body parts linearly and so may not
> buffer (i have not verified if this is the case).
>
> Could you try with that and see if it resolves your issue?
>
> @POST
> @Path("postbigfile")
> @Consumes(MediaType.MULTIPART_FORM_DATA)
> @Produces("text/plain")
> public XXX post(FormDataMultiPart f) {
> try {
> } finally {
> f.cleanup();
> }
> }

I'm not the original poster, but have tried this:

    @POST
    @Path("files")
    @Consumes("multipart/form-data")
    public void post(FormDataMultiPart formData) {
        try {
            FormDataBodyPart n = formData.getField("name");
            String fileName = n.getValue();

            FormDataBodyPart p = formData.getField("file");
            InputStream fileStream = p.getValueAs(InputStream.class);
            String mimeType = p.getMediaType().toString();

            saveFile(fileName, mimeType, fileStream);
        }
        finally {
            formData.cleanup();
        }
    }

and there's a call to ByteArrayOutputStream.toByteArray(...) that causes
the heap to explode:

java.lang.OutOfMemoryError: Java heap space
java.util.Arrays.copyOf(Arrays.java:2786)
java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:133)
javax.mail.internet.MimeMultipart.parsebm(MimeMultipart.java:1019)
javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:466)
javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:242)
com.sun.jersey.multipart.impl.MultiPartReader.readFrom(MultiPartReader.java:163)
com.sun.jersey.multipart.impl.MultiPartReader.readFrom(MultiPartReader.java:74)
com.sun.jersey.spi.container.ContainerRequest.getEntity(ContainerRequest.java:393)
com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityInjectable.getValue(EntityParamDispatchProvider.java:81)
com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$EntityParamInInvoker.getParams(EntityParamDispatchProvider.java:99)
com.sun.jersey.server.impl.model.method.dispatch.EntityParamDispatchProvider$ResponseOutInvoker._dispatch(EntityParamDispatchProvider.java:155)
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:67)
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:154)
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:71)
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:111)
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:63)
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:556)
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:515)
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:506)
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:307)
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:424)
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:589)
javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
sun.reflect.GeneratedMethodAccessor109.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
java.lang.reflect.Method.invoke(Method.java:597)
org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:244)
java.security.AccessController.doPrivileged(Native Method)
javax.security.auth.Subject.doAsPrivileged(Subject.java:517)
org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:276)
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:162)

Later
Charlie