Hi Raphael,
I think you need to set the chunked encoding size.
https://jersey.dev.java.net/nonav/apidocs/latest/jersey/com/sun/jersey/api/client/Client.html
#setChunkedEncodingSize%28java.lang.Integer%29
By default HttpURLConnection will buffer the request entity to
determine the Content-Length.
Paul.
On Jun 7, 2010, at 10:51 AM, Raphael Jolivet wrote:
> Hi,
>
> In my Web service, I need to POST large amount of data from client
> to server.
> I have been careful to do streaming everywhere and to avoid any
> "full load" of data at both end.
>
> However, the "post" method of the Jersey client side throws a OOM
> exception when trying to send large amount of data.
>
> Here is the code of the client side.
> I have created a Pipe between a output stream and an input stream.
> The post method is created in a separate thread and consumes data
> from the PipedInputStream. In the main thread, a "writer" produces
> some data to the PipedOutStream.
>
>
> // Create output pipe
> PipedOutputStream outputPipe = new PipedOutputStream();
>
> // Create input pipe connected to the output pipe
> final PipedInputStream inputPipe = new PipedInputStream(
> outputPipe);
>
>
> // Create a CSV writer, ready to write to the output pipe
> final CSVWriter writer =
> new WindParkDatasetCSVWriter(
> outputPipe,
> myData);
>
> // Init and starts the HTTP request within a new thread
> // Its POST content is fed with the output of the CSV
> writer, via the pipe
> new Thread() {
>
> // Main method of the thread
> @Override public void run() { try {
>
> // Prepare query params
> MultivaluedMap<String, String> params = new
> MultivaluedMapImpl();
> params.add(PARAM_OVERWRITE,
> String.valueOf(overwrite));
> params.add(HttpHeaders.USER_AGENT, info.client);
> if (info.comment != null)
> params.add(PARAM_COMMENT, info.comment);
>
> // Init the HTTP request and read data from the
> pipe
> ClientRepository.this.
> webResource.
> path(METHOD_PUT_DATA).
> queryParams(params).
> type(CSV_MIME_TYPE).
> post(inputPipe);
>
> } catch (Exception e) {throw new Error(e);}}
>
> }.start(); // Starts the thread now!
>
> // Write lines to CSV writer
> writer.open();
> writer.writeLines(
> lines,
> overwrite,
> info);
> writer.close();
>
> And here is the stack trace of the OOM exception :
>
> Exception in thread "Thread-1" java.lang.OutOfMemoryError: Java heap
> space
> at java.util.Arrays.copyOf(Unknown Source)
> at java.io.ByteArrayOutputStream.write(Unknown Source)
> at sun.net.www.http.PosterOutputStream.write(Unknown Source)
> at
> com
> .sun
> .jersey
> .api.client.CommittingOutputStream.write(CommittingOutputStream.java:
> 87)
> at
> com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:112)
> at
> com
> .sun
> .jersey
> .core
> .provider
> .AbstractMessageReaderWriterProvider
> .writeTo(AbstractMessageReaderWriterProvider.java:73)
> at
> com
> .sun
> .jersey
> .core
> .impl
> .provider
> .entity.InputStreamProvider.writeTo(InputStreamProvider.java:95)
> at
> com
> .sun
> .jersey
> .core
> .impl
> .provider
> .entity.InputStreamProvider.writeTo(InputStreamProvider.java:58)
> at
> com
> .sun
> .jersey
> .api
> .client
> .TerminatingClientHandler
> .writeRequestEntity(TerminatingClientHandler.java:291)
> at
> com
> .sun
> .jersey
> .client
> .urlconnection
> .URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:
> 179)
> at
> com
> .sun
> .jersey
> .client
> .urlconnection
> .URLConnectionClientHandler.handle(URLConnectionClientHandler.java:
> 126)
> at
> com
> .sun
> .jersey
> .api
> .client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:78)
> at com.sun.jersey.api.client.Client.handle(Client.java:551)
> at
> com.sun.jersey.api.client.WebResource.voidHandle(WebResource.java:578)
> at com.sun.jersey.api.client.WebResource.access
> $400(WebResource.java:69)
> at com.sun.jersey.api.client.WebResource
> $Builder.post(WebResource.java:487)
> at fr.armines.anemos.repository.ClientRepository
> $ClientDatasetWriter$1.run(ClientRepository.java:246)
>
>
>
> Any help would be appreciated,
> Thanks,
>
> Raphael
>