Hi,
In my Web service, I need to POST large amount of data from client to
server.
I have been careful to do streaming everywhere and to avoid any "full load"
of data at both end.
However, the "post" method of the Jersey client side throws a OOM exception
when trying to send large amount of data.
Here is the code of the client side.
I have created a Pipe between a output stream and an input stream.
The post method is created in a separate thread and consumes data from the
PipedInputStream. In the main thread, a "writer" produces some data to the
PipedOutStream.
// Create output pipe
PipedOutputStream outputPipe = new PipedOutputStream();
// Create input pipe connected to the output pipe
final PipedInputStream inputPipe = new PipedInputStream(
outputPipe);
// Create a CSV writer, ready to write to the output pipe
final CSVWriter writer =
new WindParkDatasetCSVWriter(
outputPipe,
myData);
// Init and starts the HTTP request within a new thread
// Its POST content is fed with the output of the CSV writer,
via the pipe
new Thread() {
// Main method of the thread
@Override public void run() { try {
// Prepare query params
MultivaluedMap<String, String> params = new
MultivaluedMapImpl();
params.add(PARAM_OVERWRITE, String.valueOf(overwrite));
params.add(HttpHeaders.USER_AGENT, info.client);
if (info.comment != null) params.add(PARAM_COMMENT,
info.comment);
// Init the HTTP request and read data from the pipe
ClientRepository.this.
webResource.
path(METHOD_PUT_DATA).
queryParams(params).
type(CSV_MIME_TYPE).
post(inputPipe);
} catch (Exception e) {throw new Error(e);}}
}.start(); // Starts the thread now!
// Write lines to CSV writer
writer.open();
writer.writeLines(
lines,
overwrite,
info);
writer.close();
And here is the stack trace of the OOM exception :
Exception in thread "Thread-1" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at sun.net.www.http.PosterOutputStream.write(Unknown Source)
at
com.sun.jersey.api.client.CommittingOutputStream.write(CommittingOutputStream.java:87)
at
com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:112)
at
com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:73)
at
com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:95)
at
com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:58)
at
com.sun.jersey.api.client.TerminatingClientHandler.writeRequestEntity(TerminatingClientHandler.java:291)
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:179)
at
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:126)
at
com.sun.jersey.api.client.filter.HTTPBasicAuthFilter.handle(HTTPBasicAuthFilter.java:78)
at com.sun.jersey.api.client.Client.handle(Client.java:551)
at
com.sun.jersey.api.client.WebResource.voidHandle(WebResource.java:578)
at
com.sun.jersey.api.client.WebResource.access$400(WebResource.java:69)
at
com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:487)
at
fr.armines.anemos.repository.ClientRepository$ClientDatasetWriter$1.run(ClientRepository.java:246)
Any help would be appreciated,
Thanks,
Raphael