Sébastien Lorber wrote:
> Hello,
>
> I would like to know if it's possible to upload a file with AHC /
> Grizzly in streaming, I mean without loading the whole file bytes in
> memory.
>
> The default behavior seems to allocate a byte[] which contans the
> whole file, so it means that my server can be OOM if too many users
> upload a large file in the same time.
>
>
> I've tryied with a Heap and ByteBuffer memory managers, with
> reallocate=true/false but no more success.
>
> It seems the whole file content is appended wto the
> BufferOutputStream, and then the underlying buffer is written.
>
> At least this seems to be the case with AHC integration:
> https://github.com/AsyncHttpClient/async-http-client/blob/6faf1f316e5546110b0779a5a42fd9d03ba6bc15/providers/grizzly/src/main/java/org/asynchttpclient/providers/grizzly/bodyhandler/PartsBodyHandler.java
>
>
> So, is there a way to patch AHC to stream the file so that I could
> eventually consume only 20mo of heap while uploading a 500mo file?
> Or is this simply impossible with Grizzly?
> I didn't notice anything related to that in the documentation.
It's possible with the FeedableBodyGenerator. But if you're tied to
using Multipart uploads, you'd have to convert the multipart data to
Buffers manually and send using the FeedableBodyGenerator.
I'll take a closer look to see if this area can be improved.
>
> Btw in my case it is a file upload. I receive a file with CXF and have
> to transmit it to a storage server (like S3). CXF doesn't consume
> memory bevause it is streaming the large fle uploads to the file
> system, and then provides an input stream on that file.
>
> Thanks
>
>