Sébastien Lorber wrote:
> So in the end I've end up with an implementation that's working for me.
>
>
> I think there are 2 bugs:
>
> 1) The bytes can accumulate in the FeedableBodyGenerator queue if the
> initialize(ctx) method is not called fast enough.
> This can be solved by using a BlockingQueue of size 1 and the put()
> method.
>
> 2) Once the context is injected, the FeedableBodyGenerator flushes the
> queue.
> The matter is that if the connection is new, not warmed up by a
> previous request, then the SSL handshake is not done yet, and it seems
> that the bytes are accumulated in some part of the SSL filter which
> doesn't deliver them to the connection until the handshake has
> completed, so c.canWrite() continues to return true.
>
>
>
>
> I have replaced some part of the FeedableBodyGenerator to test this
> and it works pretty fine. See what I have changed:
>
> 1)
> private final BlockingQueue<BodyPart> queue = new
> LinkedBlockingQueue<BodyPart>(1);
>
>
> 2)
> public void feed(final Buffer buffer, final boolean last) throws
> IOException {
> try {
> queue.put(new BodyPart(buffer, last));
> } catch (InterruptedException e) {
> throw new RuntimeException(e);
> }
> queueSize.incrementAndGet();
> if (context != null) {
> blockUntilConnectionIsReadyToWrite(context);
> flushQueue(true);
> }
> }
>
> private void blockUntilConnectionIsReadyToWrite(FilterChainContext
> fcc) {
> if ( !connectionIsReadyToWrite(fcc) ) {
> while ( !connectionIsReadyToWrite(fcc) ) {
> try { Thread.sleep(10); } catch ( Exception e ) { throw new
> RuntimeException(e); }
> }
> }
> }
>
> private boolean connectionIsReadyToWrite(FilterChainContext fcc) {
> Connection connection = fcc.getConnection();
> SSLEngine sslEngine = SSLUtils.getSSLEngine(connection);
> return sslEngine != null && !SSLUtils.isHandshaking(sslEngine);
> }
>
>
>
> What do you think?
We had come to similar conclusions on this end. I'm still working
through testing the idea I mentioned previously (took longer than I
expected - sorry).
I hope to have something for you to test very soon.
Note that it will be taking the above into account as well.
>
>
>
>
> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com
> <mailto:lorber.sebastien_at_gmail.com>>
>
>
> I have tried to put a while ( context == null ) Thread.sleep but
> it doesn't seem to work, when the context gets injected, after the
> sleeps, there's an OOM
>
> So I hope you'll have more success with your alternative :)
>
>
>
> I have done another test, remember my code that worked, which
> previously "warmed" the Thread with an useless request.
>
> private Runnable uploadSingleDocumentRunnable = new Runnable() {
> @Override
> public void run() {
> try {
> getUselessSessionCode();
> Thread.sleep(X);
> uploadSingleDocument();
> } catch ( Exception e ) {
> throw new RuntimeException("file upload failed",e);
> }
> }
> };
>
> I have put a sleep of X between the useless warmup request, and
> the real upload request
>
>
> What I noticed is that there is a very different behavior
> according to the value of X
>
>
> Under 10 seconds, it seems the stuff is still warm, I can upload
> the documents.
> Around 10 seconds I get a stack which seems to be "connection
> closed" or something
> Above 10 seconds, I get OOM like if the stuff wasn't warm.
>
>
> The stacks I get for 10 seconds looks like
>
> Caused by: javax.net.ssl.SSLException: SSLEngine is CLOSED
> at
> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:295)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143) ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
> at
> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
> ~[grizzly-framework-2.3.5.jar:2.3.5]
> at
> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:133)
> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
> at
> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:95)
> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>
> I think I got some other different stacks saying Connection Closed
> Remotely or something like that.
>
>
>
>
> So it seems that something is bound to my thread, and it stays
> bound to it for about 10 seconds, do you have any idea what it
> could be?
> (My connection timeout setting seems to have no effect on this 10s
> threshold)
>
>
>
>
>
>
> 2013/9/5 Ryan Lubke <ryan.lubke_at_oracle.com
> <mailto:ryan.lubke_at_oracle.com>>
>
> That is one solution. I'm working out an alternative right
> now. Stay tuned!
>
>>
>> Anyway it's not a problem, I think the
>> FeedableBodyGenerator.feed() method just has to block until a
>> context has been (and ThreadCache initialized) to avoid OOM
>> errors
>>
>>
>>
>>
>>
>>
>>
>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com
>> <mailto:lorber.sebastien_at_gmail.com>>
>>
>>
>> What is very strange is that I tested with/without the
>> same sessionCode with our previous code, the one not
>> using FeedableBodyGenerator, which has a high memory
>> consumption.
>> Despites the fact it had high memory consumption, it
>> seems work fine to upload multiple documents if allocated
>> with a large heap, and the sessionCode seems to have no
>> effect.
>>
>> On the new impl using the FeedableBodyGenerator, the
>> sessionCode sent as a multipart bodypart seems to have an
>> effect.
>>
>> I have tried to feed the queue before sending the request
>> to AHC, but this leads to this exception (with/without
>> sessionCode switching)
>> Caused by: java.util.concurrent.TimeoutException: Timeout
>> exceeded
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.timeout(GrizzlyAsyncHttpProvider.java:528)
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$3.onTimeout(GrizzlyAsyncHttpProvider.java:361)
>> at
>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:383)
>> at
>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:362)
>> at
>> org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:158)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>
>>
>>
>>
>>
>>
>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com
>> <mailto:lorber.sebastien_at_gmail.com>>
>>
>> By the way, by using a low timeout with the same
>> sessioncode, I got the following NPE:
>>
>> Caused by: java.lang.NullPointerException
>> at
>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:184)
>> at
>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>> at
>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>> at
>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>
>>
>> GrizzlyAsyncHttpProvider.HttpTransactionContext httpCtx =
>> getHttpTransactionContext(c);
>> httpCtx.abort(e.getCause());
>>
>> I guess the httpCtx is not already available to be
>> aborted
>>
>>
>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com
>> <mailto:lorber.sebastien_at_gmail.com>>
>>
>>
>>
>> This is right, here's a log I have when I use the
>> same session code, ie the remote host is blocking
>> the data or something.
>> This is obtained by running 5 parallel uploads.
>>
>> *Flushing queue of size 0 with allowBlocking = false*
>> Flushing queue of size 1 with allowBlocking = true
>> *Flushing queue of size 97 with allowBlocking =
>> false*
>> *Flushing queue of size 100 with allowBlocking =
>> false*
>> *Flushing queue of size 160 with allowBlocking =
>> true*
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 0 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> java.lang.OutOfMemoryError: GC overhead limit
>> exceeded
>> Dumping heap to /ome/lorber/ureau/om ...
>> Unable to create /ome/lorber/ureau/om: Le fichier
>> existe
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Disconnected from the target VM, address:
>> '127.0.0.1:49268 <http://127.0.0.1:49268>',
>> transport: 'socket'
>>
>>
>>
>>
>>
>> Otherwise, with different session codes, I get
>> the following:
>>
>> *Flushing queue of size 0 with allowBlocking = false*
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> *Flushing queue of size 0 with allowBlocking = false*
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> *Flushing queue of size 0 with allowBlocking = false*
>> *Flushing queue of size 0 with allowBlocking = false*
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> Flushing queue of size 1 with allowBlocking = true
>> ... and this continues without OOM
>>
>>
>> So, this seems to be the problem.
>>
>>
>>
>> I think it would be great to be able to be able
>> to choose the queue impl behind
>> that FeedableBodyGenerator, like I suggested in
>> my pull request.
>>
>> See here:
>> https://github.com/slorber/async-http-client/blob/79b0c3b28a61b0aa4c4b055bca8f6be11d9ed1e6/src/main/java/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.java
>>
>> Using a LinkedBlockingQueue seems to be a nice
>> idea in this context, and in my case I would
>> probably use a queue of size 1
>>
>>
>> This would handle the blocking of the feed
>> method, without having to use this:
>> if (context != null) {
>> flushQueue(true);
>> }
>>
>>
>> Or perhaps the feed() method have to wait until a
>> context is set in the BodyGenerator ?
>>
>>
>> I think it would be more clear if
>> the initializeAsynchronousTransfer simply didn't
>> flush the queue but just setup the context.
>> Then the feed method would block until there's a
>> context set, and then flush the queue with
>> blocking behavior.
>>
>> This is probably the next step, but as we are
>> using AHC for async, it would probably be great
>> if that blocking feed() method was called in a
>> worker thread instead of our main thread.
>> I won't use this but someone who really wants a
>> non-blocking impl of performant multipart
>> fileupload would probably need this, or will use
>> an ExecutorService for the feeding operations as
>> a workaround.
>>
>>
>>
>> Thanks again for your reactivity
>>
>>
>>
>>
>>
>> 2013/9/4 Ryan Lubke <ryan.lubke_at_oracle.com
>> <mailto:ryan.lubke_at_oracle.com>>
>>
>>
>>
>> Sébastien Lorber wrote:
>>> I've integrated this change and it works
>>> fine except a little detail.
>>>
>>>
>>>
>>>
>>> I'm uploading files to a third party API (a
>>> bit like S3).
>>> This API requires a "sessionCode" in each
>>> request. So there is a multipart StringPart
>>> with that SessionCode.
>>>
>>> We used to have a cache which holds the
>>> sessionCode 30min per user so that we do not
>>> need to init a new session each time.
>>>
>>> I had troubles in this very specific case:
>>> when I upload 5 docs with the same session code.
>>> When I remove the cache and use 5 different
>>> session codes, it works fine.
>>>
>>> I guess the remote service is blocking
>>> concurrent uploads with the same session
>>> code. I don't know at all.
>>>
>>>
>>>
>>> Where I want to go is that I wouldn't have
>>> expected Grizzly to OOM
>>>
>>>
>>> Avertissement: Exception during FilterChain
>>> execution
>>> java.lang.OutOfMemoryError: Java heap space
>>> at
>>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>> at
>>> java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>> at
>>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>> at
>>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>> at
>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>> at
>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>>> at
>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:263)
>>> at
>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>>> at
>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>> at
>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>> at
>>> org.glassfish.grizzly.filterchain.FilterChainContext$1.run(FilterChainContext.java:196)
>>> at
>>> org.glassfish.grizzly.filterchain.FilterChainContext.resume(FilterChainContext.java:220)
>>> at
>>> org.glassfish.grizzly.ssl.SSLFilter$SSLHandshakeContext.completed(SSLFilter.java:383)
>>> at
>>> org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:278)
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>>> at
>>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>> at
>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>> at
>>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>> at
>>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>>
>>>
>>>
>>>
>>>
>>> Caused by:
>>> java.util.concurrent.TimeoutException: null
>>> at
>>> org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:367)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:274)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:177)
>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>
>>>
>>>
>>> multipart.body.generator.feeder.buffer=100000 ->
>>> size of each Buffer sent to the
>>> FeedableBodyGenerator
>>> transport.max.pending.bytes=1000000
>>> (I tried other settings, including AUTO_SIZE)
>>>
>>>
>>>
>>> Do you have any idea why is there an OOM
>>> with these settings?
>>>
>>>
>>>
>>>
>>> Perhaps it is because the feed() method of
>>> FeedableBodyGenerator doesn't block until
>>> the context is initialized.
>>> I guess the initializeAsynchronousTransfer
>>> is called only once the connection is
>>> established, and perhaps a lot of Buffer are
>>> added to the queue...
>> Yes, it's invoked once the request has been
>> dispatched, so if the generator is fed a lot
>> before the request, I could see this happening.
>> I'll see what I can do to alleviate that case.
>>
>>>
>>> But I'm not sure at all because the session
>>> code is transmitted as a BodyPart and I get
>>> the same problem if i put it as the first or
>>> last multipart.
>>>
>>> It's not a big deal, perhaps I should always
>>> use a different session code for concurrent
>>> operations but I'd like to be sure that we
>>> won't have this issue in production...
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/9/3 Ryan Lubke <ryan.lubke_at_oracle.com
>>> <mailto:ryan.lubke_at_oracle.com>>
>>>
>>> Good catch. Fixed.
>>>
>>>
>>> Sébastien Lorber wrote:
>>>>
>>>>
>>>> Hello,
>>>>
>>>>
>>>> There's a little mistake in the grizzly
>>>> ahc provider relative to the write
>>>> queue size.
>>>>
>>>>
>>>> https://github.com/AsyncHttpClient/async-http-client/blob/b5d97efe9fe14113ea92fb1f7db192a2d090fad7/src/main/java/com/ning/http/client/providers/grizzly/GrizzlyAsyncHttpProvider.java#L419
>>>>
>>>> As you can see, the TransportCustomizer
>>>> is called, and then the write queue
>>>> size (among other things) is set to
>>>> AUTO_SIZE (instead of previously UNLIMITED)
>>>>
>>>> clientTransport.getAsyncQueueIO().getWriter()
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> .setMaxPendingBytesPerConnection(AsyncQueueWriter.AUTO_SIZE);
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> I think the default settings like this
>>>> AUTO_SIZE attribute should be set
>>>> before the customization of the
>>>> transport, or they would override the
>>>> value we customized.
>>>>
>>>>
>>>> This is actually my case, since I can't
>>>> reproduce my "bug" which is "high
>>>> memory consumption", even when using -1
>>>> / UNLIMITED in the TransportCustomizer.
>>>>
>>>>
>>>> This could work fine for me with
>>>> AUTO_SIZE, but I'd rather be able to
>>>> tune this parameter during load tests
>>>> to see the effect.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> 2013/8/31 Sebastien Lorber
>>>> <lorber.sebastien_at_gmail.com
>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>
>>>> Thanks i will ckeck that on monday.
>>>> I can now upload a 500m file with
>>>> 40m heap size ;)
>>>>
>>>> Envoyé de mon iPhone
>>>>
>>>> Le 30 août 2013 à 20:49, Ryan Lubke
>>>> <ryan.lubke_at_oracle.com
>>>> <mailto:ryan.lubke_at_oracle.com>> a
>>>> écrit :
>>>>
>>>>> I'm going to be updating the
>>>>> Grizzly provider such that
>>>>> AUTO_SIZE (not AUTO_TUNE) is the
>>>>> default, so you can avoid the use
>>>>> of the TransportCustomizer.
>>>>>
>>>>> Ryan Lubke wrote:
>>>>>> Regarding your tuning question, I
>>>>>> would probably set the value to
>>>>>> AsyncQueueWriter.AUTO_TUNE (this
>>>>>> will be four times the socket
>>>>>> write buffer) and see how that works.
>>>>>>
>>>>>> Ryan Lubke wrote:
>>>>>>> A question first. With these
>>>>>>> changes, your memory usage is
>>>>>>> more inline with what you were
>>>>>>> looking for?
>>>>>>>
>>>>>>> Sébastien Lorber wrote:
>>>>>>>> By the way, do you have any
>>>>>>>> idea when the 1.7.20 will be
>>>>>>>> released (with these new
>>>>>>>> improvements?)
>>>>>>>>
>>>>>>>> We would like to know if we
>>>>>>>> wait for a release or if we
>>>>>>>> install our own temp release on
>>>>>>>> Nexus :)
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/8/30 Sébastien Lorber
>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>
>>>>>>>> Thank you, it works fine!
>>>>>>>>
>>>>>>>>
>>>>>>>> I just had to modify a
>>>>>>>> single line after your commit.
>>>>>>>>
>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider#initializeTransport
>>>>>>>> ->
>>>>>>>> clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(10000);
>>>>>>>>
>>>>>>>>
>>>>>>>> If I let the initial value
>>>>>>>> (-1) it won't block,
>>>>>>>> canWrite always returns true
>>>>>>>>
>>>>>>>>
>>>>>>>> Btw, on AHC I didn't find
>>>>>>>> any way to pass this value
>>>>>>>> as a config attribute,
>>>>>>>> neither the size of the
>>>>>>>> write buffer you talked about.
>>>>>>>>
>>>>>>>> So in the end, is there a
>>>>>>>> way with current AHC code
>>>>>>>> to use this "canWrite =
>>>>>>>> false" behavior?
>>>>>>>> If not, can you please
>>>>>>>> provide a way to set this
>>>>>>>> behavior on v1.7.20 ? thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> PS: does it make sens to
>>>>>>>> use the same number of
>>>>>>>> bytes un the feed(Buffer)
>>>>>>>> method and in the
>>>>>>>> setMaxPendingBytesPerConnection(10000);
>>>>>>>> ? do you have any tuning
>>>>>>>> recommandation?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/8/29 Ryan Lubke
>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>
>>>>>>>> Please disregard.
>>>>>>>>
>>>>>>>>
>>>>>>>> Ryan Lubke wrote:
>>>>>>>>> Sébastien,
>>>>>>>>>
>>>>>>>>> Could you also provide
>>>>>>>>> a sample of how you're
>>>>>>>>> performing your feed?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> -rl
>>>>>>>>>
>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>> Sébastien,
>>>>>>>>>>
>>>>>>>>>> I'd recommend looking
>>>>>>>>>> at
>>>>>>>>>> Connection.canWrite()
>>>>>>>>>> [1] and
>>>>>>>>>> Connection.notifyCanWrite(WriteListener)
>>>>>>>>>> [1]
>>>>>>>>>>
>>>>>>>>>> By default, Grizzly
>>>>>>>>>> will configure the
>>>>>>>>>> async write queue
>>>>>>>>>> length to be four
>>>>>>>>>> times the write
>>>>>>>>>> buffer size (which is
>>>>>>>>>> based off the socket
>>>>>>>>>> write buffer).
>>>>>>>>>> When this queue
>>>>>>>>>> exceeds this value,
>>>>>>>>>> canWrite() will
>>>>>>>>>> return false.
>>>>>>>>>>
>>>>>>>>>> When this occurs, you
>>>>>>>>>> can register a
>>>>>>>>>> WriteListener to be
>>>>>>>>>> notified when the
>>>>>>>>>> queue length is below
>>>>>>>>>> the configured max
>>>>>>>>>> and then simulate
>>>>>>>>>> blocking
>>>>>>>>>> until the
>>>>>>>>>> onWritePossible()
>>>>>>>>>> callback has been
>>>>>>>>>> invoked.
>>>>>>>>>>
>>>>>>>>>> ----------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> final
>>>>>>>>>> FutureImpl<Boolean>
>>>>>>>>>> future =
>>>>>>>>>> Futures.createSafeFuture();
>>>>>>>>>>
>>>>>>>>>> // Connection
>>>>>>>>>> may be obtained by
>>>>>>>>>> calling
>>>>>>>>>> FilterChainContext.getConnection().
>>>>>>>>>>
>>>>>>>>>> connection.notifyCanWrite(new
>>>>>>>>>> WriteHandler() {
>>>>>>>>>>
>>>>>>>>>> @Override
>>>>>>>>>> public
>>>>>>>>>> void
>>>>>>>>>> onWritePossible()
>>>>>>>>>> throws Exception {
>>>>>>>>>>
>>>>>>>>>> future.result(Boolean.TRUE);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> @Override
>>>>>>>>>> public
>>>>>>>>>> void
>>>>>>>>>> onError(Throwable t) {
>>>>>>>>>>
>>>>>>>>>> future.failure(Exceptions.makeIOException(t));
>>>>>>>>>> }
>>>>>>>>>> });
>>>>>>>>>>
>>>>>>>>>> try {
>>>>>>>>>> final
>>>>>>>>>> long writeTimeout = 30;
>>>>>>>>>>
>>>>>>>>>> future.get(writeTimeout,
>>>>>>>>>> TimeUnit.MILLISECONDS);
>>>>>>>>>> } catch
>>>>>>>>>> (ExecutionException e) {
>>>>>>>>>>
>>>>>>>>>> HttpTransactionContext httpCtx
>>>>>>>>>> =
>>>>>>>>>> HttpTransactionContext.get(connection);
>>>>>>>>>>
>>>>>>>>>> httpCtx.abort(e.getCause());
>>>>>>>>>> } catch
>>>>>>>>>> (Exception e) {
>>>>>>>>>> HttpTransactionContext httpCtx
>>>>>>>>>> =
>>>>>>>>>> HttpTransactionContext.get(connection);
>>>>>>>>>>
>>>>>>>>>> httpCtx.abort(e);
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> http://grizzly.java.net/docs/2.3/apidocs/org/glassfish/grizzly/OutputSink.html.
>>>>>>>>>>
>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>> Ryan, I've did some
>>>>>>>>>>> other tests.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> It seems that using
>>>>>>>>>>> a blocking queue in
>>>>>>>>>>> the
>>>>>>>>>>> FeedableBodyGenerator is
>>>>>>>>>>> totally useless
>>>>>>>>>>> because the thread
>>>>>>>>>>> consuming it is not
>>>>>>>>>>> blocking and the
>>>>>>>>>>> queue never blocks
>>>>>>>>>>> the feeding, which
>>>>>>>>>>> was my intention in
>>>>>>>>>>> the beginning. Maybe
>>>>>>>>>>> it depends on the IO
>>>>>>>>>>> strategy used?
>>>>>>>>>>> I use AHC default
>>>>>>>>>>> which seems to
>>>>>>>>>>> use SameThreadIOStrategy
>>>>>>>>>>> so I don't think
>>>>>>>>>>> it's related to the
>>>>>>>>>>> IO strategy.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> So in the end I can
>>>>>>>>>>> upload a 70m file
>>>>>>>>>>> with a heap of 50m,
>>>>>>>>>>> but I have to put a
>>>>>>>>>>> Thread.sleep(30)
>>>>>>>>>>> between each 100k
>>>>>>>>>>> Buffer send to the
>>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>>>
>>>>>>>>>>> The connection with
>>>>>>>>>>> the server is not
>>>>>>>>>>> good here, but in
>>>>>>>>>>> production it is
>>>>>>>>>>> normally a lot
>>>>>>>>>>> better as far as I know.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I've tried things
>>>>>>>>>>> like clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(100000);
>>>>>>>>>>> but it doesn't seem
>>>>>>>>>>> to work for me.
>>>>>>>>>>>
>>>>>>>>>>> I'd like the Grizzly
>>>>>>>>>>> internals to block
>>>>>>>>>>> when there are too
>>>>>>>>>>> much pending bytes
>>>>>>>>>>> to send. Is it possible?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> PS: I've just been
>>>>>>>>>>> able to send a 500mo
>>>>>>>>>>> file with 100mo
>>>>>>>>>>> heap, but it needed
>>>>>>>>>>> a sleep of 100ms
>>>>>>>>>>> between each 100k
>>>>>>>>>>> Buffer sent to the
>>>>>>>>>>> bodygenerator
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/8/29 Sébastien
>>>>>>>>>>> Lorber
>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> By chance do you
>>>>>>>>>>> if I can remove
>>>>>>>>>>> the
>>>>>>>>>>> MessageCloner
>>>>>>>>>>> used in the SSL
>>>>>>>>>>> filter?
>>>>>>>>>>> SSLBaseFilter$OnWriteCopyCloner
>>>>>>>>>>>
>>>>>>>>>>> It seems to
>>>>>>>>>>> allocate a lot
>>>>>>>>>>> of memory.
>>>>>>>>>>> I don't really
>>>>>>>>>>> understand why
>>>>>>>>>>> messages have to
>>>>>>>>>>> be cloned, can I
>>>>>>>>>>> remove this? How?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/8/29
>>>>>>>>>>> Sébastien Lorber
>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I'm trying
>>>>>>>>>>> to send a
>>>>>>>>>>> 500m file
>>>>>>>>>>> for my tests
>>>>>>>>>>> with a heap
>>>>>>>>>>> of 400m.
>>>>>>>>>>>
>>>>>>>>>>> In our real
>>>>>>>>>>> use cases we
>>>>>>>>>>> would
>>>>>>>>>>> probably
>>>>>>>>>>> have files
>>>>>>>>>>> under 20mo
>>>>>>>>>>> but we want
>>>>>>>>>>> to reduce
>>>>>>>>>>> the memory
>>>>>>>>>>> consumption
>>>>>>>>>>> because we
>>>>>>>>>>> can have x
>>>>>>>>>>> parallel
>>>>>>>>>>> uploads on
>>>>>>>>>>> the same
>>>>>>>>>>> server
>>>>>>>>>>> according to
>>>>>>>>>>> the user
>>>>>>>>>>> activity.
>>>>>>>>>>>
>>>>>>>>>>> I'll try to
>>>>>>>>>>> check if
>>>>>>>>>>> using this
>>>>>>>>>>> BodyGenerator reduced
>>>>>>>>>>> the memory
>>>>>>>>>>> footprint or
>>>>>>>>>>> if it's
>>>>>>>>>>> almost like
>>>>>>>>>>> before.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/8/28
>>>>>>>>>>> Ryan Lubke
>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>
>>>>>>>>>>> At this
>>>>>>>>>>> point in
>>>>>>>>>>> time, as
>>>>>>>>>>> far as
>>>>>>>>>>> the SSL
>>>>>>>>>>> buffer
>>>>>>>>>>> allocation
>>>>>>>>>>> is
>>>>>>>>>>> concerned,
>>>>>>>>>>> it's
>>>>>>>>>>> untunable.
>>>>>>>>>>>
>>>>>>>>>>> That
>>>>>>>>>>> said,
>>>>>>>>>>> feel
>>>>>>>>>>> free to
>>>>>>>>>>> open a
>>>>>>>>>>> feature
>>>>>>>>>>> request.
>>>>>>>>>>>
>>>>>>>>>>> As to
>>>>>>>>>>> your
>>>>>>>>>>> second
>>>>>>>>>>> question, there
>>>>>>>>>>> is no
>>>>>>>>>>> suggested size.
>>>>>>>>>>> This is
>>>>>>>>>>> all very
>>>>>>>>>>> application
>>>>>>>>>>> specific.
>>>>>>>>>>>
>>>>>>>>>>> I'm
>>>>>>>>>>> curious,
>>>>>>>>>>> how
>>>>>>>>>>> large of
>>>>>>>>>>> a file
>>>>>>>>>>> are you
>>>>>>>>>>> sending?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sébastien Lorber
>>>>>>>>>>> wrote:
>>>>>>>>>>>> I have
>>>>>>>>>>>> seen a
>>>>>>>>>>>> lot of
>>>>>>>>>>>> buffers
>>>>>>>>>>>> which
>>>>>>>>>>>> have a
>>>>>>>>>>>> size of
>>>>>>>>>>>> 33842
>>>>>>>>>>>> and it
>>>>>>>>>>>> seems
>>>>>>>>>>>> the
>>>>>>>>>>>> limit
>>>>>>>>>>>> is near
>>>>>>>>>>>> half
>>>>>>>>>>>> the
>>>>>>>>>>>> capacity.
>>>>>>>>>>>>
>>>>>>>>>>>> Perhaps
>>>>>>>>>>>> there's
>>>>>>>>>>>> a way
>>>>>>>>>>>> to tune
>>>>>>>>>>>> that
>>>>>>>>>>>> buffer
>>>>>>>>>>>> size so
>>>>>>>>>>>> that it
>>>>>>>>>>>> consumes less
>>>>>>>>>>>> memory?
>>>>>>>>>>>> Is
>>>>>>>>>>>> there
>>>>>>>>>>>> an
>>>>>>>>>>>> ideal
>>>>>>>>>>>> Buffer
>>>>>>>>>>>> size to
>>>>>>>>>>>> send to
>>>>>>>>>>>> the
>>>>>>>>>>>> feed
>>>>>>>>>>>> method?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/8/28
>>>>>>>>>>>> Ryan
>>>>>>>>>>>> Lubke
>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>
>>>>>>>>>>>> I'll be
>>>>>>>>>>>> reviewing
>>>>>>>>>>>> the
>>>>>>>>>>>> PR
>>>>>>>>>>>> today,
>>>>>>>>>>>> thanks
>>>>>>>>>>>> again!
>>>>>>>>>>>>
>>>>>>>>>>>> Regarding
>>>>>>>>>>>> the
>>>>>>>>>>>> OOM: as
>>>>>>>>>>>> it
>>>>>>>>>>>> stands
>>>>>>>>>>>> now, for
>>>>>>>>>>>> each new
>>>>>>>>>>>> buffer
>>>>>>>>>>>> that is
>>>>>>>>>>>> passed
>>>>>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>> SSLFilter,
>>>>>>>>>>>> we
>>>>>>>>>>>> allocate
>>>>>>>>>>>> a
>>>>>>>>>>>> buffer
>>>>>>>>>>>> twice
>>>>>>>>>>>> the
>>>>>>>>>>>> size in
>>>>>>>>>>>> order
>>>>>>>>>>>> to
>>>>>>>>>>>> accommodate
>>>>>>>>>>>> the
>>>>>>>>>>>> encrypted
>>>>>>>>>>>> result.
>>>>>>>>>>>> So
>>>>>>>>>>>> there's
>>>>>>>>>>>> an
>>>>>>>>>>>> increase.
>>>>>>>>>>>>
>>>>>>>>>>>> Depending
>>>>>>>>>>>> on
>>>>>>>>>>>> the
>>>>>>>>>>>> socket
>>>>>>>>>>>> configurations
>>>>>>>>>>>> of
>>>>>>>>>>>> both endpoints,
>>>>>>>>>>>> and
>>>>>>>>>>>> how
>>>>>>>>>>>> fast the
>>>>>>>>>>>> remote
>>>>>>>>>>>> is
>>>>>>>>>>>> reading
>>>>>>>>>>>> data,
>>>>>>>>>>>> it
>>>>>>>>>>>> could
>>>>>>>>>>>> be
>>>>>>>>>>>> the
>>>>>>>>>>>> write
>>>>>>>>>>>> queue
>>>>>>>>>>>> is
>>>>>>>>>>>> becoming
>>>>>>>>>>>> too
>>>>>>>>>>>> large.
>>>>>>>>>>>> We
>>>>>>>>>>>> do
>>>>>>>>>>>> have a
>>>>>>>>>>>> way
>>>>>>>>>>>> to
>>>>>>>>>>>> detect
>>>>>>>>>>>> this situation,
>>>>>>>>>>>> but
>>>>>>>>>>>> I'm
>>>>>>>>>>>> pretty
>>>>>>>>>>>> sure
>>>>>>>>>>>> the
>>>>>>>>>>>> Grizzly
>>>>>>>>>>>> internals
>>>>>>>>>>>> are
>>>>>>>>>>>> currently
>>>>>>>>>>>> shielded
>>>>>>>>>>>> here.
>>>>>>>>>>>> I
>>>>>>>>>>>> will see
>>>>>>>>>>>> what I
>>>>>>>>>>>> can
>>>>>>>>>>>> do
>>>>>>>>>>>> to
>>>>>>>>>>>> allow
>>>>>>>>>>>> users
>>>>>>>>>>>> to
>>>>>>>>>>>> leverage
>>>>>>>>>>>> this.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Sébastien
>>>>>>>>>>>> Lorber
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've
>>>>>>>>>>>>> made
>>>>>>>>>>>>> my
>>>>>>>>>>>>> pull
>>>>>>>>>>>>> request.
>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/pull/367
>>>>>>>>>>>>>
>>>>>>>>>>>>> With
>>>>>>>>>>>>> my
>>>>>>>>>>>>> usecase
>>>>>>>>>>>>> it
>>>>>>>>>>>>> works,
>>>>>>>>>>>>> the file
>>>>>>>>>>>>> is
>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>> like
>>>>>>>>>>>>> before.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> But I
>>>>>>>>>>>>> didn't
>>>>>>>>>>>>> notice
>>>>>>>>>>>>> a
>>>>>>>>>>>>> big memory
>>>>>>>>>>>>> improvement.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Is
>>>>>>>>>>>>> it
>>>>>>>>>>>>> possible
>>>>>>>>>>>>> that
>>>>>>>>>>>>> SSL doesn't
>>>>>>>>>>>>> allow
>>>>>>>>>>>>> to
>>>>>>>>>>>>> stream
>>>>>>>>>>>>> the body
>>>>>>>>>>>>> or
>>>>>>>>>>>>> something
>>>>>>>>>>>>> like
>>>>>>>>>>>>> that?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In
>>>>>>>>>>>>> memory,
>>>>>>>>>>>>> I
>>>>>>>>>>>>> have
>>>>>>>>>>>>> a
>>>>>>>>>>>>> lot of:
>>>>>>>>>>>>> -
>>>>>>>>>>>>> HeapByteBuffer
>>>>>>>>>>>>> Which
>>>>>>>>>>>>> are hold
>>>>>>>>>>>>> by
>>>>>>>>>>>>> SSLUtils$3
>>>>>>>>>>>>> Which
>>>>>>>>>>>>> are hold
>>>>>>>>>>>>> by
>>>>>>>>>>>>> BufferBuffers
>>>>>>>>>>>>> Which
>>>>>>>>>>>>> are hold
>>>>>>>>>>>>> by
>>>>>>>>>>>>> WriteResult
>>>>>>>>>>>>> Which
>>>>>>>>>>>>> are hold
>>>>>>>>>>>>> by
>>>>>>>>>>>>> AsyncWriteQueueRecord
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here
>>>>>>>>>>>>> is
>>>>>>>>>>>>> an
>>>>>>>>>>>>> exemple
>>>>>>>>>>>>> of
>>>>>>>>>>>>> the OOM
>>>>>>>>>>>>> stacktrace:
>>>>>>>>>>>>>
>>>>>>>>>>>>> java.lang.OutOfMemoryError:
>>>>>>>>>>>>> Java
>>>>>>>>>>>>> heap
>>>>>>>>>>>>> space
>>>>>>>>>>>>> at
>>>>>>>>>>>>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:227)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:404)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:319)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2503)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:132)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:101)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder$FeedBodyGeneratorOutputStream.write(MultipartBodyGeneratorFeeder.java:222)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.multipart.FilePart.sendData(FilePart.java:179)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.multipart.Part.send(Part.java:331)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.multipart.Part.sendParts(Part.java:397)
>>>>>>>>>>>>> at
>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder.feed(MultipartBodyGeneratorFeeder.java:144)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/8/27
>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Excellent!
>>>>>>>>>>>>> Looking
>>>>>>>>>>>>> forward
>>>>>>>>>>>>> to
>>>>>>>>>>>>> the
>>>>>>>>>>>>> pull
>>>>>>>>>>>>> request!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> works
>>>>>>>>>>>>>> fine,
>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>> a pull
>>>>>>>>>>>>>> request
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>> tomorrow
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> a better
>>>>>>>>>>>>>> code
>>>>>>>>>>>>>> using
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> same
>>>>>>>>>>>>>> Part
>>>>>>>>>>>>>> classes
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> already
>>>>>>>>>>>>>> exist.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I created
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>> OutputStream
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> redirects
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> BodyGenerator
>>>>>>>>>>>>>> feeder.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The
>>>>>>>>>>>>>> problem
>>>>>>>>>>>>>> I currently
>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> feeder
>>>>>>>>>>>>>> feeds
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>> faster
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> async
>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>> I need
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> expose
>>>>>>>>>>>>>> a limit
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>> size
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> something,
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> work
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> that,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>> better
>>>>>>>>>>>>>> than
>>>>>>>>>>>>>> a thread
>>>>>>>>>>>>>> sleep
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>> down
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> filepart
>>>>>>>>>>>>>> reading
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/8/27
>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes,
>>>>>>>>>>>>>> something
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>> I was
>>>>>>>>>>>>>> going
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> tackle
>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>> something
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>> follow
>>>>>>>>>>>>>> up
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> something
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>> test
>>>>>>>>>>>>>> out.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok
>>>>>>>>>>>>>>> thanks!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>> I see
>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>> I could
>>>>>>>>>>>>>>> do,
>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>> that:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>>>>>>> bodyGenerator
>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>> FeedableBodyGenerator();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder
>>>>>>>>>>>>>>> bodyGeneratorFeeder
>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder(bodyGenerator);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Request
>>>>>>>>>>>>>>> uploadRequest1
>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>> RequestBuilder("POST")
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .setUrl("url")
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .setBody(bodyGenerator)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .build();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ListenableFuture<Response>
>>>>>>>>>>>>>>> asyncRes
>>>>>>>>>>>>>>> = asyncHttpClient
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .prepareRequest(uploadRequest1)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> .execute(new
>>>>>>>>>>>>>>> AsyncCompletionHandlerBase());
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param1","value1");
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param2","value2");
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("fileToUpload",fileInputStream);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> bodyGeneratorFeeder.end();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Response
>>>>>>>>>>>>>>> uploadResponse
>>>>>>>>>>>>>>> = asyncRes.get();
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Does
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> seem
>>>>>>>>>>>>>>> ok
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> you?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I guess
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> interesting
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder
>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> Grizzly
>>>>>>>>>>>>>>> since
>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>> people
>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> achieve
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>> thing
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/8/26
>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> upload
>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> / Grizzly
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> streaming,
>>>>>>>>>>>>>>> I mean
>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>> loading
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> bytes
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> memory.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>> a byte[]
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> contans
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>> file,
>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> means
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> OOM
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>> upload
>>>>>>>>>>>>>>> a large
>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>> tryied
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> a Heap
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> ByteBuffer
>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>> managers,
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> reallocate=true/false
>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>> success.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> content
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> appended
>>>>>>>>>>>>>>> wto
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> BufferOutputStream,
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> underlying
>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> written.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> At
>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> integration:
>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/6faf1f316e5546110b0779a5a42fd9d03ba6bc15/providers/grizzly/src/main/java/org/asynchttpclient/providers/grizzly/bodyhandler/PartsBodyHandler.java
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So,
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> stream
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> I could
>>>>>>>>>>>>>>> eventually
>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>> 20mo
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> heap
>>>>>>>>>>>>>>> while
>>>>>>>>>>>>>>> uploading
>>>>>>>>>>>>>>> a 500mo
>>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>> Or
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>> impossible
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> Grizzly?
>>>>>>>>>>>>>>> I didn't
>>>>>>>>>>>>>>> notice
>>>>>>>>>>>>>>> anything
>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> documentation.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> FeedableBodyGenerator.
>>>>>>>>>>>>>>> But
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>> tied
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>> Multipart
>>>>>>>>>>>>>>> uploads,
>>>>>>>>>>>>>>> you'd
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> convert
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> multipart
>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> Buffers
>>>>>>>>>>>>>>> manually
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> FeedableBodyGenerator.
>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>> take
>>>>>>>>>>>>>>> a closer
>>>>>>>>>>>>>>> look
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> area
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> improved.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Btw
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>> upload.
>>>>>>>>>>>>>>> I receive
>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> CXF
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> transmit
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> a storage
>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>> S3).
>>>>>>>>>>>>>>> CXF
>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>> bevause
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> streaming
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> large
>>>>>>>>>>>>>>> fle
>>>>>>>>>>>>>>> uploads
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>> provides
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>> input
>>>>>>>>>>>>>>> stream
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>
>>>
>>
>>
>>
>>
>
>