users@jersey.java.net

[Jersey] Re: Jersey 2 Client that would write the stream of data to POST call chunk by chunk

From: Naseer Sargi <naseer.sargi_at_gmail.com>
Date: Wed, 24 Aug 2016 18:16:26 +0530

Hi Gili ,

No , it is not RAM :-)

Let me explain in detail :

I was having an requirement where in I should use limited hard disk memory.

Now coming to my use case is that I was using Jersey Client to send a
request xml of size 1GB .

Earlier I was generating the 1GB of request xml data as file i.e. .xml, so
I was reading the file as InputStream and passing to

*post *method as mentioned below :

            File requestTemplateXml = new File("file directory");
 FileInputStream fis = FileUtils.openInputStream(requestTemplateXml);

            WebTarget target = client.target("http://localhost:8080");
            response = target.path("/somepath").
                    request().post(Entity.entity(fis,
MediaType.APPLICATION_OCTET_STREAM_TYPE));
Now above approach has serious memory consumption , if I have different
scenario with different request xml data size , say 1GB , 500MB , 10MB etc.
so all above file will consume the hard disk space , in order to avoid
storing the request xml file , I thought to generate the request data as
bytes
on-fly and write to OutputStream chunk by chunk .
Hope you got the solution , Let me know if you need more info .

On Tue, Aug 23, 2016 at 8:35 PM, cowwoc <cowwoc_at_bbs.darktech.org> wrote:

> By "hard disk space", I assume you mean RAM ;) because this solution won't
> do anything for hard disk space.
>
> Gili
>
>
> On 2016-08-23 8:37 AM, Naseer Sargi wrote:
>
> Hi All ,
>
> After doing much research I found the solution and here it goes :
>
> *First *we need to implements javax.ws.rs.core.StreamingOutput as
> mentioned below:
>
> public class JerseyStreamingOutput implements StreamingOutput {
>
>
> /**
> * Overriding the write method to write request data directly to
> Jersey outputStream .
> * @param outputStream
> * @throws IOException
> * @throws WebApplicationException
> */
> @Override
> public void write(OutputStream outputStream) throws IOException,
> WebApplicationException {
>
> // Write to Jersey OutputStream chunk by chunk here
> }
> }
>
>
> *Jersey Cient code :*
>
> JerseyStreamingOutput jerseyStreamingOutput =
> new JerseyStreamingOutput(10240,1024);
> WebTarget target = client.target("http://localhost:8080");
> response = target.path("/somepath").
> request().post(Entity.entity(jerseyStreamingOutput,
> MediaType.APPLICATION_OCTET_STREAM_TYPE));
>
> So the above solution helps in saving hard disk memory by writing the
> request chunk by chunk to outputStream , without this approach we would end
> up keeping the request xml file say 1GB in hard disk.
>
>
>
> On Mon, Aug 15, 2016 at 5:07 PM, Pavel Bucek <pavel.bucek_at_oracle.com>
> wrote:
>
>> Hi Naseer,
>>
>> have you seen this test?
>>
>> https://github.com/jersey/jersey/blob/2.x/tests/e2e/src/test
>> /java/org/glassfish/jersey/tests/e2e/client/ClientBufferi
>> ngDisabledTest.java
>>
>> Regards,
>> Pavel
>>
>> On 15/08/16 12:53, Naseer Sargi wrote:
>>
>> I wanted to write a Jersey 2 Client that would write the stream of data
>> to POST call chunk by chunk.
>>
>> Why? , this will help me to avoid to keep the whole inputstream request
>> data to store in disk memory before sending via a POST call.
>>
>> I have searched over the net and looked into Jersey2 API as well but did
>> not find any solution , however there are solution for server side which
>> sends the huge response in stream and reads the same in Jersey Client by
>> doing a GET call at Client , but I wanted send the huge payload say 1 GB of
>> XML data as stream vis POST call.
>>
>> I tried using solution given in here
>> <http://stackoverflow.com/questions/10326460/how-to-avoid-outofmemoryerror-when-uploading-a-large-file-using-jersey-client> ,
>> but this solution again uses the system memory.
>>
>> I do not want to store 1GB of data in disk , instead create on-fly the
>> 1GB request stream of data / write the 1GB data to POST call directly chunk
>> by chunk.
>>
>> Any help highly appreciated. Thanks in Advance.
>>
>>
>>
>
>