Okay no problem.
Sébastien Lorber wrote:
> This won't be possible because these are not unit tests but
> integration tests.
>
> We test this multipart upload against a real external API which
> require certificates and is hosted by our customer. We do not have any
> mock.
> As you can see, I don't even publish my app package name in the stacks
> because I was told to not do so...
Yes, I saw that- Assumed it was just a simple testing framework because
of it.
We'll roll our own and get back with you asap.
>
>
> 2013/9/12 Ryan Lubke <ryan.lubke_at_oracle.com
> <mailto:ryan.lubke_at_oracle.com>>
>
> Would it be possible to share you testing framework?
>
> Sébastien Lorber wrote:
>> Thanks,
>>
>>
>>
>> So in the end it seems I can run concurrent uploads. But it is
>> kind of limited, with 10 threads it works fine, and with 20
>> threads it seems the flush call is sometimes done too early
>> because the feed method is not blocking, which produces OOM in
>> some cases. The threshold seems to be around 15.
>>
>>
>> I found a solution which seems to work in every situation:
>> - Use a worker thread pool > number of concurrent requests
>> - Wait a bit before starting the feeding.
>>
>>
>>
>> My waiting code is:
>>
>> private volatile boolean alreadyFlushed = false;
>> @Override
>> public synchronized void flush() {
>> if ( alreadyFlushed ) {
>> return;
>> }
>> try { Thread.sleep(1000); } catch ( Exception e ) {
>> e.printStackTrace(); }
>> startFeeding();
>> alreadyFlushed = true;
>> }
>>
>>
>>
>> With a 35 worker thread pool and this waiting code, I can
>> run**30 concurrent uploads of large files, without having
>> performed any request before.
>>
>>
>>
>> If I remove any of these 2 tricks, I get problems.
>>
>>
>>
>> *1) If I put a low worker thread number:*
>> I mean, for 30 concurrent uploads, it starts to fail even for
>> something like 31/32 worker threads
>>
>> Sometime I get timeout stacks:
>>
>> Caused by: java.util.concurrent.TimeoutException: Timeout exceeded
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.timeout(GrizzlyAsyncHttpProvider.java:528)
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$3.onTimeout(GrizzlyAsyncHttpProvider.java:361)
>> at
>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:383)
>> at
>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:362)
>> at
>> org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:158)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>
>>
>> Sometime I get OOM stacks:
>>
>> Caused by: java.lang.OutOfMemoryError: Java heap space
>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>> at
>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>> at
>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>> at
>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>> at
>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>> at
>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:227)
>> at
>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>> at
>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>> at
>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:263)
>> at
>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>> at
>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>> at
>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>> at
>> org.glassfish.grizzly.filterchain.FilterChainContext$1.run(FilterChainContext.java:196)
>> at
>> org.glassfish.grizzly.filterchain.FilterChainContext.resume(FilterChainContext.java:220)
>> at
>> org.glassfish.grizzly.ssl.SSLFilter$SSLHandshakeContext.completed(SSLFilter.java:383)
>> at
>> org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:278)
>> at
>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>> at
>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>> at
>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>> at
>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>> at
>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>> at
>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>> at
>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>
>>
>> Another very strange behavior I have is that some FileInputStream
>> that are used for the requests are read prematurely for a reason
>> I can't understand.
>> Not all, but some of the 30 concurrent uploads. I can see just
>> after the firing of the 30 upload requests in my log:
>>
>> [Grizzly(1)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_61187ada after having read
>> 10632755 bytes
>> [Grizzly(15)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_1ba83dc after having read 10632755
>> bytes
>> [Grizzly(13)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_7c26e166 after having read
>> 10632755 bytes
>> [Grizzly(10)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_5c982f37 after having read
>> 10632755 bytes
>> [Grizzly(6)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_b43f35f after having read 10632755
>> bytes
>> [Grizzly(4)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_1a1ee7c0 after having read
>> 10632755 bytes
>> [Grizzly(9)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_6300fba5 after having read
>> 10632755 bytes
>> [Grizzly(16)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_49d88b78 after having read
>> 10632755 bytes
>> [Grizzly(4)] INFO com....DocumentService$1:276 - Will close file
>> stream java.io.FileInputStream_at_3b7b65f8 after having read 598016
>> bytes
>> --asept. 12, 2013 5:54:14 PM
>> org.glassfish.grizzly.filterchain.DefaultFilterChain execute
>> Avertissement: Exception during FilterChain execution
>> java.lang.OutOfMemoryError: GC overhead limit exceeded
>>
>> This close operation on the file stream is supposed to happen
>> after more than 30 seconds because the connection is slow and the
>> file is 10M
>>
>> This generally leads to OOM errors, it seems the whole file is
>> mounted in memory for unknown reason in some cases, but only when
>> there are not enough worker threads.
>>
>>
>> It's not a big deal, I guess it is "normal" to use at least 30
>> threads to run 30 concurrent uploads :) It's just that when this
>> happens it's not easy to understand why we have OOMs and how it
>> relates to the AHC grizzly thread pool.
>>
>>
>>
>> *2) If I remove that sleep(1000) before feeding or use a lower
>> number*
>> This leads to the same kind of behavior, with the InputStream
>> being read prematurely.
>>
>>
>>
>>
>> So I guess, the problem is that when the flush() method is called
>> it seems the connection always return canWrite=true, so the
>> blocking never happen until something happen and the connection
>> starts to block the feeding
>>
>> I tried to debug that and here's the connection queue state:
>>
>> * connectionQueue =
>> {org.glassfish.grizzly.asyncqueue.TaskQueue_at_4804}
>> * isClosed = true
>> * queue = {java.util.concurrent.ConcurrentLinkedQueue_at_5106}
>> size = 0
>> * currentElement =
>> {java.util.concurrent.atomic.AtomicReference_at_5107}"null"
>> * spaceInBytes =
>> {java.util.concurrent.atomic.AtomicInteger_at_5108}"291331"
>> * maxQueueSizeHolder =
>> {org.glassfish.grizzly.nio.NIOConnection$1_at_5109}
>> * writeHandlersCounter =
>> {java.util.concurrent.atomic.AtomicInteger_at_5110}"0"
>> * writeHandlersQueue =
>> {java.util.concurrent.ConcurrentLinkedQueue_at_5111} size = 0
>>
>>
>>
>> Do you have any idea what could be the problem?
>>
>> This happens when flush() is called from both init() and
>> handshakeComplete() methods
>>
>>
>> 2013/9/12 Ryan Lubke <ryan.lubke_at_oracle.com
>> <mailto:ryan.lubke_at_oracle.com>>
>>
>>
>>
>> Sébastien Lorber wrote:
>>>
>>>
>>>
>>> I noticed something strange.
>>> On the FileInputStream I have, I've added a log on the
>>> close() of the stream which is called once the whole file
>>> has been read to be sent to the feeder.
>>>
>>> 1) If I perform a request (ie my session init request in the
>>> previous discussions) before doing my multipart upload, the
>>> thread that does execute the feeding is the thread that
>>> fires the request, and not a Grizzly worker.
>>> [Thread-30] INFO Will close file stream
>>> java.io.FileInputStream_at_27fe4315 after having read 1440304 bytes
>>>
>>>
>>> 2) If I don't do any request before firing the multipart
>>> upload, the thread that does the feeding is a Grizzly
>>> threadpool worker thread:
>>> [Grizzly(22)] INFO Will close file stream
>>> java.io.FileInputStream_at_59ac4002 after having read 1440304 bytes
>>>
>>>
>>> Is this normal? I would expect a worker thread to always be
>>> used, and the main applicative thread that performs the
>>> request to never be blocking. (but it's not such an issue
>>> for me, we don't have a reactive non-blocking app anyway)
>> It's currently expected behavior. Will need to re-evaluate
>> this based on the new semantics of the FeedableBodyGenerator.
>>>
>>>
>>> On the 1st case, here's the stacktrace when the feed method
>>> is called:
>>>
>>> *at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.initializeAsynchronousTransfer(FeedableBodyGenerator.java:178)*
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$BodyGeneratorBodyHandler.doHandle(GrizzlyAsyncHttpProvider.java:2210)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.sendRequest(GrizzlyAsyncHttpProvider.java:564)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientFilter.sendAsGrizzlyRequest(GrizzlyAsyncHttpProvider.java:913)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientFilter.handleWrite(GrizzlyAsyncHttpProvider.java:795)
>>> at
>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>> at
>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.write(DefaultFilterChain.java:437)
>>> at
>>> org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:387)
>>> at
>>> org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:361)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.execute(GrizzlyAsyncHttpProvider.java:307)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$1.completed(GrizzlyAsyncHttpProvider.java:224)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$1.completed(GrizzlyAsyncHttpProvider.java:210)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$ConnectionManager.doAsyncTrackedConnection(GrizzlyAsyncHttpProvider.java:2289)
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.execute(GrizzlyAsyncHttpProvider.java:244)
>>> *at
>>> com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:534)*
>>>
>>> So it seems this happen when the handshake has already been
>>> done when *initializeAsynchronousTransfer* is called, so
>>> that we do not go through the HandshakeListener
>>>
>>>
>>> Notice that this case seems to also happen in a few threads
>>> on the 2nd case, but most of the threads are Grizzly workers.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/9/12 Sébastien Lorber <lorber.sebastien_at_gmail.com
>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>
>>>
>>>
>>> Hi.
>>>
>>>
>>> Thanks, it seems to work.
>>>
>>> I would suggest to throw IOException on the flush method
>>> since the feed method is supposed to be called here
>>>
>>>
>>> My implementation of flush is:
>>>
>>> @Override
>>> public void flush() {
>>> Part[] partsArray = parts.toArray(new
>>> Part[parts.size()]);
>>> try ( OutputStream outputStream =
>>> createFeedingOutputStream() ) {
>>>
>>> Part.sendParts(outputStream,partsArray,multipartBoundary);
>>> } catch (Exception e) {
>>> throw new IllegalStateException("Unable to feed
>>> the FeedableBodyGenerator",e);
>>> }
>>> }
>>>
>>> Is this correct? The OutputStream redirects the bytes
>>> written to the feed(Buffer) method
>>>
>>>
>>>
>>>
>>>
>>>
>>> There seems to be some concurrency issue. Because the
>>> upload of 1 file seems fine, but when using multiple
>>> threads, I often get the following stack:
>>> Caused by: java.io.IOException: Stream Closed
>>> at java.io.FileInputStream.readBytes(Native Method)
>>> at java.io.FileInputStream.read(FileInputStream.java:242)
>>> at
>>> com.google.common.io.CountingInputStream.read(CountingInputStream.java:62)
>>> at
>>> java.io.FilterInputStream.read(FilterInputStream.java:133)
>>> at
>>> java.io.FilterInputStream.read(FilterInputStream.java:107)
>>> at
>>> com.ning.http.multipart.FilePart.sendData(FilePart.java:178)
>>> at com.ning.http.multipart.Part.send(Part.java:331)
>>> at com.ning.http.multipart.Part.sendParts(Part.java:397)
>>>
>>>
>>> This is because the flush() method is called multiple
>>> times for the same request on some cases.
>>> I guess this is not supposed to happen.
>>> What I understand is that the flush() method is supposed
>>> to be called only once.
>>>
>>>
>>> Using debug logging breakpoints I get the following:
>>>
>>> myapp--api-test 12/09/2013-12:20:46.042 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 0 started
>>> myapp--api-test 12/09/2013-12:20:46.042 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 1 started
>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 2 started
>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 3 started
>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 4 started
>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 5 started
>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 6 started
>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 7 started
>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 8 started
>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 9 started
>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 10 started
>>> myapp--api-test 12/09/2013-12:20:46.046 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 11 started
>>> myapp--api-test 12/09/2013-12:20:46.047 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 12 started
>>> myapp--api-test 12/09/2013-12:20:46.048 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 13 started
>>> myapp--api-test 12/09/2013-12:20:46.049 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 14 started
>>> myapp--api-test 12/09/2013-12:20:46.049 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 15 started
>>> myapp--api-test 12/09/2013-12:20:46.050 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 16 started
>>> myapp--api-test 12/09/2013-12:20:46.050 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 17 started
>>> myapp--api-test 12/09/2013-12:20:46.051 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 18 started
>>> myapp--api-test 12/09/2013-12:20:46.051 [] [] [main]
>>> INFO com.myapp.perf.DocumentUploadPerfIntegrationTest:77
>>> - Thread 19 started
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_417de6ff
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6c0d6ef7
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_7799b411
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_1c940409
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_480f9510
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_3e888183
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_17840db8
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_2cbad94b
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_3d5d9ee8
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_51557949
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6f95de2f
>>> Adding handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_346d784c
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_7799b411
>>> *Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>> *Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>> *Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>> *Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6*
>>> *Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6*
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_346d784c
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_674735a8
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_480f9510
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6c0d6ef7
>>> Completing and removing handshake listener for
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_1daf8fd8
>>>
>>>
>>> As you can see the same HandshakeListener seems to be
>>> called multiple times for the same FeedableBodyGenerator
>>>
>>> When this happens, the stack is:
>>>
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$1.onComplete(FeedableBodyGenerator.java:198)
>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter.notifyHandshakeComplete(SSLBaseFilter.java:880)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:282)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>> at
>>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>> at
>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>
>>>
>>>
>>>
>>> So I tried with the following code:
>>>
>>> private boolean alreadyFlushed = false;
>>> @Override
>>> public synchronized void flush() {
>>> if ( alreadyFlushed ) {
>>> return;
>>> }
>>> startFeeding();
>>> alreadyFlushed = true;
>>> }
>>>
>>> It works fine when upload small files.
>>>
>>> But with larger files, I often get TimeoutException
>>> stacks for some of the threads:
>>>
>>> Caused by: java.util.concurrent.TimeoutException
>>> at
>>> org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:367)
>>> at
>>> org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:274)
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.block(FeedableBodyGenerator.java:349)
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.blockUntilQueueFree(FeedableBodyGenerator.java:339)
>>> at
>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.feed(FeedableBodyGenerator.java:306)
>>>
>>>
>>>
>>>
>>> Did I miss something?
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/9/12 Ryan Lubke <ryan.lubke_at_oracle.com
>>> <mailto:ryan.lubke_at_oracle.com>>
>>>
>>> Committed another small change. Please make sure
>>> you're at the latest when you build.
>>>
>>> -rl
>>>
>>>
>>> Ryan Lubke wrote:
>>>> Okay, I've committed another set of refactorings to
>>>> the FeedableBodyGenerator.
>>>>
>>>> For your use case, you should extend
>>>> FeedableBodyGenerator.SimpleFeeder.
>>>>
>>>> Let me know if you run into issues.
>>>>
>>>>
>>>>
>>>> Sébastien Lorber wrote:
>>>>> Yes I think it would, so that I can feed the queue
>>>>> at once
>>>>>
>>>>> One thread will be locked during the feeding for
>>>>> nothing but it's not a real problem in my usecase.
>>>>>
>>>>>
>>>>> 2013/9/10 Ryan Lubke <ryan.lubke_at_oracle.com
>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>
>>>>> Would having a different listener that will be
>>>>> notified once async transferring has been
>>>>> started work better for you?
>>>>>
>>>>> Something like:
>>>>>
>>>>> onAsyncTransferInitiated() {
>>>>> // invoke your feed method
>>>>> }
>>>>>
>>>>> ?
>>>>>
>>>>>
>>>>>
>>>>> Sébastien Lorber wrote:
>>>>>> Unfortunatly I won't be able to use the
>>>>>> Feeder non-blocking stuff for now, because of
>>>>>> how the multipart request in handled in AHC
>>>>>>
>>>>>>
>>>>>> Here's my feeding method:
>>>>>>
>>>>>> public void feed() throws IOException {
>>>>>> Part[] partsArray = parts.toArray(new
>>>>>> Part[parts.size()]);
>>>>>> try ( OutputStream outputStream =
>>>>>> createFeedingOutputStream() ) {
>>>>>>
>>>>>> Part.sendParts(outputStream,partsArray,multipartBoundary);
>>>>>> } catch (Exception e) {
>>>>>> throw new IllegalStateException("Unable
>>>>>> to feed the FeedableBodyGenerator",e);
>>>>>> }
>>>>>> }
>>>>>>
>>>>>>
>>>>>> As you can see, the multipart Parts array can
>>>>>> only be pushed to the OutputStream, I don't
>>>>>> have any way to "pull" the data when the
>>>>>> canFeed() method is triggered.
>>>>>>
>>>>>>
>>>>>> But I've seen that there's
>>>>>> a com.ning.http.multipart.MultipartBody#read
>>>>>> that seems to provide a memory efficient way
>>>>>> to pull data from a Multipart body...
>>>>>>
>>>>>> Should see what I come up with this
>>>>>>
>>>>>>
>>>>>> 2013/9/10 Sébastien Lorber
>>>>>> <lorber.sebastien_at_gmail.com
>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>
>>>>>> It seems the Feeder is highly recommended
>>>>>> but not mandatory so I tried without.
>>>>>>
>>>>>> With my existing code it seems there is a
>>>>>> synchronization problem.
>>>>>>
>>>>>>
>>>>>> The feeding threads get locked to
>>>>>> the prematureFeed.get();
>>>>>>
>>>>>> So the Grizzly kernel threads are unable
>>>>>> to acquire the lock required to enter
>>>>>> the initializeAsynchronousTransfer method
>>>>>>
>>>>>>
>>>>>>
>>>>>> Will try with an implementation of Feeder
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/9/10 Sébastien Lorber
>>>>>> <lorber.sebastien_at_gmail.com
>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>
>>>>>> Hmmm it seems I have a problem with
>>>>>> one of your maven plugins. I'll try
>>>>>> to bypass it, but for info:
>>>>>>
>>>>>> ➜ ahc2 git:(ahc-1.7.x) mvn clean install
>>>>>> [WARNING]
>>>>>> [WARNING] Some problems were
>>>>>> encountered while building the
>>>>>> effective settings
>>>>>> [WARNING]
>>>>>> 'profiles.profile[default].repositories.repository.id
>>>>>> <http://repositories.repository.id>'
>>>>>> must be unique but found duplicate
>>>>>> repository with id
>>>>>> fullsix-maven-repository @
>>>>>> /home/slorber/.m2/settings.xml
>>>>>> [WARNING]
>>>>>> [INFO] Scanning for projects...
>>>>>> [INFO]
>>>>>> [INFO]
>>>>>> ------------------------------------------------------------------------
>>>>>> [INFO] Building Asynchronous Http
>>>>>> Client 1.7.20-SNAPSHOT
>>>>>> [INFO]
>>>>>> ------------------------------------------------------------------------
>>>>>> [INFO]
>>>>>> [INFO] ---
>>>>>> maven-clean-plugin:2.4.1:clean
>>>>>> (default-clean) @ async-http-client ---
>>>>>> [INFO]
>>>>>> [INFO] ---
>>>>>> maven-enforcer-plugin:1.0-beta-1:enforce
>>>>>> (enforce-maven) @ async-http-client ---
>>>>>> [INFO]
>>>>>> [INFO] ---
>>>>>> maven-enforcer-plugin:1.0-beta-1:enforce
>>>>>> (enforce-versions) @
>>>>>> async-http-client ---
>>>>>> [INFO]
>>>>>> [INFO] ---
>>>>>> maven-resources-plugin:2.4.3:resources (default-resources)
>>>>>> @ async-http-client ---
>>>>>> [INFO] Using 'UTF-8' encoding to copy
>>>>>> filtered resources.
>>>>>> [INFO] skip non existing
>>>>>> resourceDirectory
>>>>>> /home/slorber/Bureau/ahc2/src/main/resources
>>>>>> [INFO]
>>>>>> [INFO] ---
>>>>>> maven-compiler-plugin:2.3.2:compile
>>>>>> (default-compile) @ async-http-client ---
>>>>>> [INFO] Compiling 158 source files to
>>>>>> /home/slorber/Bureau/ahc2/target/classes
>>>>>> [INFO]
>>>>>> *[INFO] ---
>>>>>> animal-sniffer-maven-plugin:1.6:check
>>>>>> (check-java-1.5-compat) @
>>>>>> async-http-client ---*
>>>>>> *[INFO] Checking unresolved
>>>>>> references to
>>>>>> org.codehaus.mojo.signature:java15:1.0*
>>>>>> *[ERROR] Undefined reference:
>>>>>> java/io/IOException.<init>(Ljava/lang/Throwable;)V
>>>>>> in
>>>>>> /home/slorber/Bureau/ahc2/target/classes/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.class*
>>>>>> [INFO]
>>>>>> ------------------------------------------------------------------------
>>>>>> [INFO] BUILD FAILURE
>>>>>> [INFO]
>>>>>> ------------------------------------------------------------------------
>>>>>> [INFO] Total time: 8.747s
>>>>>> [INFO] Finished at: Tue Sep 10
>>>>>> 11:25:41 CEST 2013
>>>>>> [INFO] Final Memory: 30M/453M
>>>>>> [INFO]
>>>>>> ------------------------------------------------------------------------
>>>>>> *[ERROR] Failed to execute goal
>>>>>> org.codehaus.mojo:animal-sniffer-maven-plugin:1.6:check
>>>>>> (check-java-1.5-compat) on project
>>>>>> async-http-client: Signature errors
>>>>>> found. Verify them and put
>>>>>> @IgnoreJRERequirement on them. ->
>>>>>> [Help 1]*
>>>>>> [ERROR]
>>>>>> [ERROR] To see the full stack trace
>>>>>> of the errors, re-run Maven with the
>>>>>> -e switch.
>>>>>> [ERROR] Re-run Maven using the -X
>>>>>> switch to enable full debug logging.
>>>>>> [ERROR]
>>>>>> [ERROR] For more information about
>>>>>> the errors and possible solutions,
>>>>>> please read the following articles:
>>>>>> [ERROR] [Help 1]
>>>>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/9/10 Sébastien Lorber
>>>>>> <lorber.sebastien_at_gmail.com
>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>
>>>>>> Ok thank you, I'll try to
>>>>>> implement that today and will
>>>>>> give you my feedback :)
>>>>>>
>>>>>>
>>>>>> 2013/9/10 Ryan Lubke
>>>>>> <ryan.lubke_at_oracle.com
>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>
>>>>>> Okay,
>>>>>>
>>>>>> I've committed my initial
>>>>>> changes to the AHC
>>>>>> repository. Here's a summary
>>>>>> of the changes:
>>>>>>
>>>>>>
>>>>>> *Improvements to the
>>>>>> FeedableBodyGenerator
>>>>>> (Grizzly's).
>>>>>> - Don't allow queueing of
>>>>>> data before
>>>>>> initiateAsyncTransfer has
>>>>>> been invoked. In low memory
>>>>>> heaps, this could lead to an
>>>>>> OOM if the source is feeding
>>>>>> too fast. The new behavior is to
>>>>>> block until
>>>>>> initiateAsyncTransfer is
>>>>>> called, at which time the
>>>>>> blocked thread may proceed with
>>>>>> the feed operation.
>>>>>> - Introduce the concept of a
>>>>>> Feeder. Implementations are
>>>>>> responsible, at a high level,
>>>>>> for:
>>>>>> + letting the provider know
>>>>>> that data is available to be
>>>>>> fed without blocking
>>>>>> + allowing the registration
>>>>>> of a callback that the Feeder
>>>>>> implementation may invoke
>>>>>> to signal that more data is
>>>>>> available, if it wasn't
>>>>>> available at a previous point
>>>>>> in time.
>>>>>> - When using a Feeder with a
>>>>>> secure request, the SSL
>>>>>> handshake will be kicked off
>>>>>> by the
>>>>>> initiateAsyncTransfer call,
>>>>>> but feeding of data will not
>>>>>> occur until the handshake is
>>>>>> complete.
>>>>>> This is necessary as the
>>>>>> SSLFilter will queue up all
>>>>>> writes until the handshake is
>>>>>> complete,
>>>>>> and currently, the buffer
>>>>>> isn't tied in with the
>>>>>> transport flow control mechanism.
>>>>>> NOTE: This new SSL behavior
>>>>>> is not currently applied when
>>>>>> invoking the feed() method
>>>>>> outside the context of a
>>>>>> Feeder. Still need to address
>>>>>> that.
>>>>>> - Exposed configuration of
>>>>>> the async write queue limit
>>>>>> through the
>>>>>> FeedableBodyGenerator.
>>>>>> This is an improvement on
>>>>>> using a TransportCustomizer
>>>>>> as any configuration there is
>>>>>> transport-wide, and therefor
>>>>>> applied to all Connections.
>>>>>> By exposing it here, each feeder
>>>>>> may have a different byte limit.
>>>>>> - Improved documentation for
>>>>>> this class*
>>>>>>
>>>>>>
>>>>>>
>>>>>> I recommend reading through
>>>>>> the javadoc comments in the
>>>>>> source [1] for
>>>>>> FeedableBodyGenerator
>>>>>> (comments welcome).
>>>>>> Additionally, I would re-work
>>>>>> your code to leverage the
>>>>>> Feeder instead of calling
>>>>>> feed() directly.
>>>>>>
>>>>>> If you have issues
>>>>>> implementing Feeder, do let
>>>>>> us know.
>>>>>>
>>>>>> If you have additional
>>>>>> questions, again, let us know.
>>>>>>
>>>>>> Thanks,
>>>>>> -rl
>>>>>>
>>>>>> [1]
>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/ahc-1.7.x/src/main/java/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.java
>>>>>>
>>>>>>
>>>>>>
>>>>>> Ryan Lubke wrote:
>>>>>>>
>>>>>>>
>>>>>>> Sébastien Lorber wrote:
>>>>>>>> So in the end I've end up
>>>>>>>> with an implementation
>>>>>>>> that's working for me.
>>>>>>>>
>>>>>>>>
>>>>>>>> I think there are 2 bugs:
>>>>>>>>
>>>>>>>> 1) The bytes can accumulate
>>>>>>>> in the
>>>>>>>> FeedableBodyGenerator queue
>>>>>>>> if the initialize(ctx)
>>>>>>>> method is not called fast
>>>>>>>> enough.
>>>>>>>> This can be solved by using
>>>>>>>> a BlockingQueue of size 1
>>>>>>>> and the put() method.
>>>>>>>>
>>>>>>>> 2) Once the context is
>>>>>>>> injected, the
>>>>>>>> FeedableBodyGenerator
>>>>>>>> flushes the queue.
>>>>>>>> The matter is that if the
>>>>>>>> connection is new, not
>>>>>>>> warmed up by a previous
>>>>>>>> request, then the SSL
>>>>>>>> handshake is not done yet,
>>>>>>>> and it seems that the bytes
>>>>>>>> are accumulated in some
>>>>>>>> part of the SSL filter
>>>>>>>> which doesn't deliver them
>>>>>>>> to the connection until the
>>>>>>>> handshake has completed,
>>>>>>>> so c.canWrite() continues
>>>>>>>> to return true.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I have replaced some part
>>>>>>>> of the
>>>>>>>> FeedableBodyGenerator to
>>>>>>>> test this and it works
>>>>>>>> pretty fine. See what I
>>>>>>>> have changed:
>>>>>>>>
>>>>>>>> 1)
>>>>>>>> private final
>>>>>>>> BlockingQueue<BodyPart>
>>>>>>>> queue = new
>>>>>>>> LinkedBlockingQueue<BodyPart>(1);
>>>>>>>>
>>>>>>>>
>>>>>>>> 2)
>>>>>>>> public void feed(final
>>>>>>>> Buffer buffer, final
>>>>>>>> boolean last) throws
>>>>>>>> IOException {
>>>>>>>> try {
>>>>>>>> queue.put(new
>>>>>>>> BodyPart(buffer, last));
>>>>>>>> } catch
>>>>>>>> (InterruptedException e) {
>>>>>>>> throw new
>>>>>>>> RuntimeException(e);
>>>>>>>> }
>>>>>>>>
>>>>>>>> queueSize.incrementAndGet();
>>>>>>>> if (context != null) {
>>>>>>>>
>>>>>>>> blockUntilConnectionIsReadyToWrite(context);
>>>>>>>> flushQueue(true);
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> private void
>>>>>>>> blockUntilConnectionIsReadyToWrite(FilterChainContext
>>>>>>>> fcc) {
>>>>>>>> if (
>>>>>>>> !connectionIsReadyToWrite(fcc)
>>>>>>>> ) {
>>>>>>>> while (
>>>>>>>> !connectionIsReadyToWrite(fcc)
>>>>>>>> ) {
>>>>>>>> try {
>>>>>>>> Thread.sleep(10); } catch (
>>>>>>>> Exception e ) { throw new
>>>>>>>> RuntimeException(e); }
>>>>>>>> }
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> private boolean
>>>>>>>> connectionIsReadyToWrite(FilterChainContext
>>>>>>>> fcc) {
>>>>>>>> Connection
>>>>>>>> connection =
>>>>>>>> fcc.getConnection();
>>>>>>>> SSLEngine sslEngine
>>>>>>>> =
>>>>>>>> SSLUtils.getSSLEngine(connection);
>>>>>>>> return sslEngine !=
>>>>>>>> null &&
>>>>>>>> !SSLUtils.isHandshaking(sslEngine);
>>>>>>>> }
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> What do you think?
>>>>>>>
>>>>>>> We had come to similar
>>>>>>> conclusions on this end.
>>>>>>> I'm still working through
>>>>>>> testing the idea I mentioned
>>>>>>> previously (took longer than
>>>>>>> I expected - sorry).
>>>>>>> I hope to have something for
>>>>>>> you to test very soon.
>>>>>>>
>>>>>>> Note that it will be taking
>>>>>>> the above into account as well.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/9/5 Sébastien Lorber
>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I have tried to put a
>>>>>>>> while ( context == null
>>>>>>>> ) Thread.sleep but it
>>>>>>>> doesn't seem to work,
>>>>>>>> when the context gets
>>>>>>>> injected, after the
>>>>>>>> sleeps, there's an OOM
>>>>>>>>
>>>>>>>> So I hope you'll have
>>>>>>>> more success with your
>>>>>>>> alternative :)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> I have done another
>>>>>>>> test, remember my code
>>>>>>>> that worked, which
>>>>>>>> previously "warmed" the
>>>>>>>> Thread with an useless
>>>>>>>> request.
>>>>>>>>
>>>>>>>> private Runnable
>>>>>>>> uploadSingleDocumentRunnable
>>>>>>>> = new Runnable() {
>>>>>>>> @Override
>>>>>>>> public void run() {
>>>>>>>> try {
>>>>>>>>
>>>>>>>> getUselessSessionCode();
>>>>>>>> Thread.sleep(X);
>>>>>>>>
>>>>>>>> uploadSingleDocument();
>>>>>>>> } catch (
>>>>>>>> Exception e ) {
>>>>>>>> throw new
>>>>>>>> RuntimeException("file
>>>>>>>> upload failed",e);
>>>>>>>> }
>>>>>>>> }
>>>>>>>> };
>>>>>>>>
>>>>>>>> I have put a sleep of X
>>>>>>>> between the useless
>>>>>>>> warmup request, and the
>>>>>>>> real upload request
>>>>>>>>
>>>>>>>>
>>>>>>>> What I noticed is that
>>>>>>>> there is a very
>>>>>>>> different behavior
>>>>>>>> according to the value of X
>>>>>>>>
>>>>>>>>
>>>>>>>> Under 10 seconds, it
>>>>>>>> seems the stuff is
>>>>>>>> still warm, I can
>>>>>>>> upload the documents.
>>>>>>>> Around 10 seconds I get
>>>>>>>> a stack which seems to
>>>>>>>> be "connection closed"
>>>>>>>> or something
>>>>>>>> Above 10 seconds, I get
>>>>>>>> OOM like if the stuff
>>>>>>>> wasn't warm.
>>>>>>>>
>>>>>>>>
>>>>>>>> The stacks I get for 10
>>>>>>>> seconds looks like
>>>>>>>>
>>>>>>>> Caused by:
>>>>>>>> javax.net.ssl.SSLException:
>>>>>>>> SSLEngine is CLOSED
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:295)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>> at
>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:133)
>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>> at
>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:95)
>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>
>>>>>>>> I think I got some
>>>>>>>> other different stacks
>>>>>>>> saying Connection
>>>>>>>> Closed Remotely or
>>>>>>>> something like that.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> So it seems that
>>>>>>>> something is bound to
>>>>>>>> my thread, and it stays
>>>>>>>> bound to it for about
>>>>>>>> 10 seconds, do you have
>>>>>>>> any idea what it could be?
>>>>>>>> (My connection timeout
>>>>>>>> setting seems to have
>>>>>>>> no effect on this 10s
>>>>>>>> threshold)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/9/5 Ryan Lubke
>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>
>>>>>>>> That is one
>>>>>>>> solution. I'm
>>>>>>>> working out an
>>>>>>>> alternative right
>>>>>>>> now. Stay tuned!
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Anyway it's not a
>>>>>>>>> problem, I think
>>>>>>>>> the
>>>>>>>>> FeedableBodyGenerator.feed()
>>>>>>>>> method just has to
>>>>>>>>> block until a
>>>>>>>>> context has been
>>>>>>>>> (and ThreadCache
>>>>>>>>> initialized) to
>>>>>>>>> avoid OOM errors
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/9/5 Sébastien
>>>>>>>>> Lorber
>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> What is very
>>>>>>>>> strange is
>>>>>>>>> that I tested
>>>>>>>>> with/without
>>>>>>>>> the same
>>>>>>>>> sessionCode
>>>>>>>>> with our
>>>>>>>>> previous code,
>>>>>>>>> the one not
>>>>>>>>> using
>>>>>>>>> FeedableBodyGenerator,
>>>>>>>>> which has a
>>>>>>>>> high memory
>>>>>>>>> consumption.
>>>>>>>>> Despites the
>>>>>>>>> fact it had
>>>>>>>>> high memory
>>>>>>>>> consumption,
>>>>>>>>> it seems work
>>>>>>>>> fine to upload
>>>>>>>>> multiple
>>>>>>>>> documents if
>>>>>>>>> allocated with
>>>>>>>>> a large heap,
>>>>>>>>> and the
>>>>>>>>> sessionCode
>>>>>>>>> seems to have
>>>>>>>>> no effect.
>>>>>>>>>
>>>>>>>>> On the new
>>>>>>>>> impl using the
>>>>>>>>> FeedableBodyGenerator,
>>>>>>>>> the
>>>>>>>>> sessionCode
>>>>>>>>> sent as a
>>>>>>>>> multipart
>>>>>>>>> bodypart seems
>>>>>>>>> to have an effect.
>>>>>>>>>
>>>>>>>>> I have tried
>>>>>>>>> to feed the
>>>>>>>>> queue before
>>>>>>>>> sending the
>>>>>>>>> request to
>>>>>>>>> AHC, but this
>>>>>>>>> leads to this
>>>>>>>>> exception
>>>>>>>>> (with/without
>>>>>>>>> sessionCode
>>>>>>>>> switching)
>>>>>>>>> Caused by:
>>>>>>>>> java.util.concurrent.TimeoutException:
>>>>>>>>> Timeout exceeded
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.timeout(GrizzlyAsyncHttpProvider.java:528)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$3.onTimeout(GrizzlyAsyncHttpProvider.java:361)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:383)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:362)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:158)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>>>>> at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/9/5
>>>>>>>>> Sébastien
>>>>>>>>> Lorber
>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>
>>>>>>>>> By the
>>>>>>>>> way, by
>>>>>>>>> using a
>>>>>>>>> low
>>>>>>>>> timeout
>>>>>>>>> with the
>>>>>>>>> same
>>>>>>>>> sessioncode,
>>>>>>>>> I got the
>>>>>>>>> following NPE:
>>>>>>>>>
>>>>>>>>> Caused by:
>>>>>>>>> java.lang.NullPointerException
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:184)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> GrizzlyAsyncHttpProvider.HttpTransactionContext
>>>>>>>>> httpCtx =
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> getHttpTransactionContext(c);
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> httpCtx.abort(e.getCause());
>>>>>>>>>
>>>>>>>>> I guess
>>>>>>>>> the
>>>>>>>>> httpCtx is
>>>>>>>>> not
>>>>>>>>> already
>>>>>>>>> available
>>>>>>>>> to be aborted
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/9/5
>>>>>>>>> Sébastien
>>>>>>>>> Lorber
>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This
>>>>>>>>> is
>>>>>>>>> right,
>>>>>>>>> here's
>>>>>>>>> a log
>>>>>>>>> I have
>>>>>>>>> when I
>>>>>>>>> use
>>>>>>>>> the
>>>>>>>>> same
>>>>>>>>> session code,
>>>>>>>>> ie the
>>>>>>>>> remote
>>>>>>>>> host
>>>>>>>>> is
>>>>>>>>> blocking
>>>>>>>>> the
>>>>>>>>> data
>>>>>>>>> or
>>>>>>>>> something.
>>>>>>>>> This
>>>>>>>>> is
>>>>>>>>> obtained
>>>>>>>>> by
>>>>>>>>> running 5
>>>>>>>>> parallel
>>>>>>>>> uploads.
>>>>>>>>>
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size
>>>>>>>>> 97
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size
>>>>>>>>> 100
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size
>>>>>>>>> 160
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true*
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> java.lang.OutOfMemoryError:
>>>>>>>>> GC
>>>>>>>>> overhead
>>>>>>>>> limit
>>>>>>>>> exceeded
>>>>>>>>> Dumping heap
>>>>>>>>> to
>>>>>>>>> /ome/lorber/ureau/om
>>>>>>>>> ...
>>>>>>>>> Unable
>>>>>>>>> to
>>>>>>>>> create
>>>>>>>>> /ome/lorber/ureau/om:
>>>>>>>>> Le
>>>>>>>>> fichier existe
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Disconnected
>>>>>>>>> from
>>>>>>>>> the
>>>>>>>>> target
>>>>>>>>> VM,
>>>>>>>>> address:
>>>>>>>>> '127.0.0.1:49268
>>>>>>>>> <http://127.0.0.1:49268>',
>>>>>>>>> transport:
>>>>>>>>> 'socket'
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Otherwise,
>>>>>>>>> with
>>>>>>>>> different
>>>>>>>>> session codes,
>>>>>>>>> I get
>>>>>>>>> the
>>>>>>>>> following:
>>>>>>>>>
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> *Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 0
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = false*
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> Flushing
>>>>>>>>> queue
>>>>>>>>> of
>>>>>>>>> size 1
>>>>>>>>> with
>>>>>>>>> allowBlocking
>>>>>>>>> = true
>>>>>>>>> ...
>>>>>>>>> and
>>>>>>>>> this
>>>>>>>>> continues
>>>>>>>>> without OOM
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So,
>>>>>>>>> this
>>>>>>>>> seems
>>>>>>>>> to be
>>>>>>>>> the
>>>>>>>>> problem.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I
>>>>>>>>> think
>>>>>>>>> it
>>>>>>>>> would
>>>>>>>>> be
>>>>>>>>> great
>>>>>>>>> to be
>>>>>>>>> able
>>>>>>>>> to be
>>>>>>>>> able
>>>>>>>>> to
>>>>>>>>> choose
>>>>>>>>> the
>>>>>>>>> queue
>>>>>>>>> impl
>>>>>>>>> behind
>>>>>>>>> that FeedableBodyGenerator,
>>>>>>>>> like I
>>>>>>>>> suggested
>>>>>>>>> in my
>>>>>>>>> pull
>>>>>>>>> request.
>>>>>>>>>
>>>>>>>>> See here:
>>>>>>>>> https://github.com/slorber/async-http-client/blob/79b0c3b28a61b0aa4c4b055bca8f6be11d9ed1e6/src/main/java/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.java
>>>>>>>>>
>>>>>>>>> Using
>>>>>>>>> a
>>>>>>>>> LinkedBlockingQueue
>>>>>>>>> seems
>>>>>>>>> to be
>>>>>>>>> a nice
>>>>>>>>> idea
>>>>>>>>> in
>>>>>>>>> this
>>>>>>>>> context,
>>>>>>>>> and in
>>>>>>>>> my
>>>>>>>>> case I
>>>>>>>>> would
>>>>>>>>> probably
>>>>>>>>> use a
>>>>>>>>> queue
>>>>>>>>> of size 1
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This
>>>>>>>>> would
>>>>>>>>> handle
>>>>>>>>> the
>>>>>>>>> blocking
>>>>>>>>> of the
>>>>>>>>> feed
>>>>>>>>> method, without
>>>>>>>>> having
>>>>>>>>> to use
>>>>>>>>> this:
>>>>>>>>>
>>>>>>>>> if
>>>>>>>>> (context
>>>>>>>>> != null) {
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> flushQueue(true);
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Or
>>>>>>>>> perhaps the
>>>>>>>>> feed()
>>>>>>>>> method
>>>>>>>>> have
>>>>>>>>> to
>>>>>>>>> wait
>>>>>>>>> until
>>>>>>>>> a
>>>>>>>>> context is
>>>>>>>>> set in
>>>>>>>>> the
>>>>>>>>> BodyGenerator
>>>>>>>>> ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I
>>>>>>>>> think
>>>>>>>>> it
>>>>>>>>> would
>>>>>>>>> be
>>>>>>>>> more
>>>>>>>>> clear
>>>>>>>>> if
>>>>>>>>> the initializeAsynchronousTransfer
>>>>>>>>> simply
>>>>>>>>> didn't
>>>>>>>>> flush
>>>>>>>>> the
>>>>>>>>> queue
>>>>>>>>> but
>>>>>>>>> just
>>>>>>>>> setup
>>>>>>>>> the
>>>>>>>>> context.
>>>>>>>>> Then
>>>>>>>>> the
>>>>>>>>> feed
>>>>>>>>> method
>>>>>>>>> would
>>>>>>>>> block
>>>>>>>>> until
>>>>>>>>> there's a
>>>>>>>>> context set,
>>>>>>>>> and
>>>>>>>>> then
>>>>>>>>> flush
>>>>>>>>> the
>>>>>>>>> queue
>>>>>>>>> with
>>>>>>>>> blocking
>>>>>>>>> behavior.
>>>>>>>>>
>>>>>>>>> This
>>>>>>>>> is
>>>>>>>>> probably
>>>>>>>>> the
>>>>>>>>> next
>>>>>>>>> step,
>>>>>>>>> but as
>>>>>>>>> we are
>>>>>>>>> using
>>>>>>>>> AHC
>>>>>>>>> for
>>>>>>>>> async,
>>>>>>>>> it
>>>>>>>>> would
>>>>>>>>> probably
>>>>>>>>> be
>>>>>>>>> great
>>>>>>>>> if
>>>>>>>>> that
>>>>>>>>> blocking
>>>>>>>>> feed()
>>>>>>>>> method
>>>>>>>>> was
>>>>>>>>> called
>>>>>>>>> in a
>>>>>>>>> worker
>>>>>>>>> thread
>>>>>>>>> instead of
>>>>>>>>> our
>>>>>>>>> main
>>>>>>>>> thread.
>>>>>>>>> I
>>>>>>>>> won't
>>>>>>>>> use
>>>>>>>>> this
>>>>>>>>> but
>>>>>>>>> someone who
>>>>>>>>> really
>>>>>>>>> wants
>>>>>>>>> a
>>>>>>>>> non-blocking
>>>>>>>>> impl
>>>>>>>>> of
>>>>>>>>> performant
>>>>>>>>> multipart
>>>>>>>>> fileupload
>>>>>>>>> would
>>>>>>>>> probably
>>>>>>>>> need
>>>>>>>>> this,
>>>>>>>>> or
>>>>>>>>> will
>>>>>>>>> use an
>>>>>>>>> ExecutorService
>>>>>>>>> for
>>>>>>>>> the
>>>>>>>>> feeding operations
>>>>>>>>> as a
>>>>>>>>> workaround.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> again
>>>>>>>>> for
>>>>>>>>> your
>>>>>>>>> reactivity
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/9/4
>>>>>>>>> Ryan
>>>>>>>>> Lubke
>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sébastien
>>>>>>>>> Lorber
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> I've
>>>>>>>>>> integrated
>>>>>>>>>> this
>>>>>>>>>> change
>>>>>>>>>> and
>>>>>>>>>> it works
>>>>>>>>>> fine
>>>>>>>>>> except
>>>>>>>>>> a
>>>>>>>>>> little
>>>>>>>>>> detail.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I'm
>>>>>>>>>> uploading
>>>>>>>>>> files
>>>>>>>>>> to a
>>>>>>>>>> third
>>>>>>>>>> party
>>>>>>>>>> API
>>>>>>>>>> (a bit
>>>>>>>>>> like
>>>>>>>>>> S3).
>>>>>>>>>> This
>>>>>>>>>> API
>>>>>>>>>> requires
>>>>>>>>>> a
>>>>>>>>>> "sessionCode"
>>>>>>>>>> in each
>>>>>>>>>> request.
>>>>>>>>>> So there
>>>>>>>>>> is a
>>>>>>>>>> multipart
>>>>>>>>>> StringPart
>>>>>>>>>> with
>>>>>>>>>> that
>>>>>>>>>> SessionCode.
>>>>>>>>>>
>>>>>>>>>> We used
>>>>>>>>>> to have
>>>>>>>>>> a
>>>>>>>>>> cache
>>>>>>>>>> which
>>>>>>>>>> holds
>>>>>>>>>> the
>>>>>>>>>> sessionCode
>>>>>>>>>> 30min
>>>>>>>>>> per
>>>>>>>>>> user
>>>>>>>>>> so that
>>>>>>>>>> we do
>>>>>>>>>> not
>>>>>>>>>> need
>>>>>>>>>> to init
>>>>>>>>>> a
>>>>>>>>>> new
>>>>>>>>>> session
>>>>>>>>>> each
>>>>>>>>>> time.
>>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>> had
>>>>>>>>>> troubles
>>>>>>>>>> in this
>>>>>>>>>> very
>>>>>>>>>> specific
>>>>>>>>>> case:
>>>>>>>>>> when
>>>>>>>>>> I
>>>>>>>>>> upload
>>>>>>>>>> 5
>>>>>>>>>> docs
>>>>>>>>>> with
>>>>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>> session
>>>>>>>>>> code.
>>>>>>>>>> When
>>>>>>>>>> I
>>>>>>>>>> remove
>>>>>>>>>> the
>>>>>>>>>> cache
>>>>>>>>>> and
>>>>>>>>>> use
>>>>>>>>>> 5
>>>>>>>>>> different
>>>>>>>>>> session
>>>>>>>>>> codes,
>>>>>>>>>> it works
>>>>>>>>>> fine.
>>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>> guess
>>>>>>>>>> the
>>>>>>>>>> remote
>>>>>>>>>> service
>>>>>>>>>> is blocking
>>>>>>>>>> concurrent
>>>>>>>>>> uploads
>>>>>>>>>> with
>>>>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>> session
>>>>>>>>>> code.
>>>>>>>>>> I
>>>>>>>>>> don't
>>>>>>>>>> know
>>>>>>>>>> at all.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Where
>>>>>>>>>> I
>>>>>>>>>> want
>>>>>>>>>> to go
>>>>>>>>>> is that
>>>>>>>>>> I
>>>>>>>>>> wouldn't
>>>>>>>>>> have
>>>>>>>>>> expected
>>>>>>>>>> Grizzly
>>>>>>>>>> to OOM
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Avertissement:
>>>>>>>>>> Exception
>>>>>>>>>> during
>>>>>>>>>> FilterChain
>>>>>>>>>> execution
>>>>>>>>>> java.lang.OutOfMemoryError:
>>>>>>>>>> Java
>>>>>>>>>> heap
>>>>>>>>>> space
>>>>>>>>>> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>>>>>>>>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:263)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>> at com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>> at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.FilterChainContext$1.run(FilterChainContext.java:196)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.FilterChainContext.resume(FilterChainContext.java:220)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLFilter$SSLHandshakeContext.completed(SSLFilter.java:383)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:278)
>>>>>>>>>> at org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>>>>>>>>>> at com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>> at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>> at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>> at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>>>>>>>>> at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Caused
>>>>>>>>>> by:
>>>>>>>>>> java.util.concurrent.TimeoutException:
>>>>>>>>>> null
>>>>>>>>>> at org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:367)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:274)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:177)
>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>> at com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>> at com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>> at com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> multipart.body.generator.feeder.buffer=100000
>>>>>>>>>> -> size
>>>>>>>>>> of each
>>>>>>>>>> Buffer
>>>>>>>>>> sent
>>>>>>>>>> to the
>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>> transport.max.pending.bytes=1000000
>>>>>>>>>> (I tried
>>>>>>>>>> other
>>>>>>>>>> settings,
>>>>>>>>>> including
>>>>>>>>>> AUTO_SIZE)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Do you
>>>>>>>>>> have
>>>>>>>>>> any
>>>>>>>>>> idea
>>>>>>>>>> why
>>>>>>>>>> is there
>>>>>>>>>> an OOM
>>>>>>>>>> with
>>>>>>>>>> these
>>>>>>>>>> settings?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Perhaps
>>>>>>>>>> it is
>>>>>>>>>> because
>>>>>>>>>> the
>>>>>>>>>> feed()
>>>>>>>>>> method
>>>>>>>>>> of FeedableBodyGenerator
>>>>>>>>>> doesn't
>>>>>>>>>> block
>>>>>>>>>> until
>>>>>>>>>> the
>>>>>>>>>> context
>>>>>>>>>> is initialized.
>>>>>>>>>>
>>>>>>>>>> I
>>>>>>>>>> guess
>>>>>>>>>> the initializeAsynchronousTransfer
>>>>>>>>>> is called
>>>>>>>>>> only
>>>>>>>>>> once
>>>>>>>>>> the
>>>>>>>>>> connection
>>>>>>>>>> is established,
>>>>>>>>>> and
>>>>>>>>>> perhaps
>>>>>>>>>> a
>>>>>>>>>> lot
>>>>>>>>>> of Buffer
>>>>>>>>>> are
>>>>>>>>>> added
>>>>>>>>>> to the
>>>>>>>>>> queue...
>>>>>>>>> Yes,
>>>>>>>>> it's
>>>>>>>>> invoked
>>>>>>>>> once
>>>>>>>>> the request
>>>>>>>>> has been
>>>>>>>>> dispatched,
>>>>>>>>> so
>>>>>>>>> if
>>>>>>>>> the generator
>>>>>>>>> is
>>>>>>>>> fed a
>>>>>>>>> lot before
>>>>>>>>> the request,
>>>>>>>>> I
>>>>>>>>> could
>>>>>>>>> see this
>>>>>>>>> happening.
>>>>>>>>> I'll
>>>>>>>>> see what
>>>>>>>>> I
>>>>>>>>> can do
>>>>>>>>> to
>>>>>>>>> alleviate
>>>>>>>>> that
>>>>>>>>> case.
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> But
>>>>>>>>>> I'm
>>>>>>>>>> not
>>>>>>>>>> sure
>>>>>>>>>> at all
>>>>>>>>>> because
>>>>>>>>>> the
>>>>>>>>>> session
>>>>>>>>>> code
>>>>>>>>>> is transmitted
>>>>>>>>>> as a
>>>>>>>>>> BodyPart
>>>>>>>>>> and
>>>>>>>>>> I
>>>>>>>>>> get
>>>>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>> problem
>>>>>>>>>> if i
>>>>>>>>>> put
>>>>>>>>>> it as
>>>>>>>>>> the
>>>>>>>>>> first
>>>>>>>>>> or last
>>>>>>>>>> multipart.
>>>>>>>>>>
>>>>>>>>>> It's
>>>>>>>>>> not
>>>>>>>>>> a
>>>>>>>>>> big
>>>>>>>>>> deal,
>>>>>>>>>> perhaps
>>>>>>>>>> I
>>>>>>>>>> should
>>>>>>>>>> always
>>>>>>>>>> use
>>>>>>>>>> a
>>>>>>>>>> different
>>>>>>>>>> session
>>>>>>>>>> code
>>>>>>>>>> for
>>>>>>>>>> concurrent
>>>>>>>>>> operations
>>>>>>>>>> but
>>>>>>>>>> I'd
>>>>>>>>>> like
>>>>>>>>>> to be
>>>>>>>>>> sure
>>>>>>>>>> that
>>>>>>>>>> we won't
>>>>>>>>>> have
>>>>>>>>>> this
>>>>>>>>>> issue
>>>>>>>>>> in production...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2013/9/3
>>>>>>>>>> Ryan
>>>>>>>>>> Lubke
>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>
>>>>>>>>>> Good
>>>>>>>>>> catch.
>>>>>>>>>> Fixed.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sébastien
>>>>>>>>>> Lorber
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hello,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> There's
>>>>>>>>>>> a little
>>>>>>>>>>> mistake
>>>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>> grizzly
>>>>>>>>>>> ahc
>>>>>>>>>>> provider
>>>>>>>>>>> relative
>>>>>>>>>>> to
>>>>>>>>>>> the
>>>>>>>>>>> write
>>>>>>>>>>> queue
>>>>>>>>>>> size.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/b5d97efe9fe14113ea92fb1f7db192a2d090fad7/src/main/java/com/ning/http/client/providers/grizzly/GrizzlyAsyncHttpProvider.java#L419
>>>>>>>>>>>
>>>>>>>>>>> As
>>>>>>>>>>> you
>>>>>>>>>>> can
>>>>>>>>>>> see,
>>>>>>>>>>> the
>>>>>>>>>>> TransportCustomizer
>>>>>>>>>>> is
>>>>>>>>>>> called,
>>>>>>>>>>> and
>>>>>>>>>>> then
>>>>>>>>>>> the
>>>>>>>>>>> write
>>>>>>>>>>> queue
>>>>>>>>>>> size
>>>>>>>>>>> (among
>>>>>>>>>>> other
>>>>>>>>>>> things)
>>>>>>>>>>> is
>>>>>>>>>>> set
>>>>>>>>>>> to
>>>>>>>>>>> AUTO_SIZE
>>>>>>>>>>> (instead
>>>>>>>>>>> of
>>>>>>>>>>> previously
>>>>>>>>>>> UNLIMITED)
>>>>>>>>>>>
>>>>>>>>>>> clientTransport.getAsyncQueueIO().getWriter()
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> .setMaxPendingBytesPerConnection(AsyncQueueWriter.AUTO_SIZE);
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I think
>>>>>>>>>>> the
>>>>>>>>>>> default
>>>>>>>>>>> settings
>>>>>>>>>>> like
>>>>>>>>>>> this
>>>>>>>>>>> AUTO_SIZE
>>>>>>>>>>> attribute
>>>>>>>>>>> should
>>>>>>>>>>> be
>>>>>>>>>>> set
>>>>>>>>>>> before
>>>>>>>>>>> the
>>>>>>>>>>> customization
>>>>>>>>>>> of
>>>>>>>>>>> the
>>>>>>>>>>> transport,
>>>>>>>>>>> or
>>>>>>>>>>> they
>>>>>>>>>>> would
>>>>>>>>>>> override
>>>>>>>>>>> the
>>>>>>>>>>> value
>>>>>>>>>>> we
>>>>>>>>>>> customized.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This
>>>>>>>>>>> is
>>>>>>>>>>> actually
>>>>>>>>>>> my
>>>>>>>>>>> case,
>>>>>>>>>>> since
>>>>>>>>>>> I can't
>>>>>>>>>>> reproduce
>>>>>>>>>>> my
>>>>>>>>>>> "bug"
>>>>>>>>>>> which
>>>>>>>>>>> is
>>>>>>>>>>> "high
>>>>>>>>>>> memory
>>>>>>>>>>> consumption",
>>>>>>>>>>> even
>>>>>>>>>>> when
>>>>>>>>>>> using
>>>>>>>>>>> -1
>>>>>>>>>>> / UNLIMITED
>>>>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>> TransportCustomizer.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> This
>>>>>>>>>>> could
>>>>>>>>>>> work
>>>>>>>>>>> fine
>>>>>>>>>>> for
>>>>>>>>>>> me
>>>>>>>>>>> with
>>>>>>>>>>> AUTO_SIZE,
>>>>>>>>>>> but
>>>>>>>>>>> I'd
>>>>>>>>>>> rather
>>>>>>>>>>> be
>>>>>>>>>>> able
>>>>>>>>>>> to
>>>>>>>>>>> tune
>>>>>>>>>>> this
>>>>>>>>>>> parameter
>>>>>>>>>>> during
>>>>>>>>>>> load
>>>>>>>>>>> tests
>>>>>>>>>>> to
>>>>>>>>>>> see
>>>>>>>>>>> the
>>>>>>>>>>> effect.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/8/31
>>>>>>>>>>> Sebastien
>>>>>>>>>>> Lorber
>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>> i will
>>>>>>>>>>> ckeck
>>>>>>>>>>> that
>>>>>>>>>>> on
>>>>>>>>>>> monday.
>>>>>>>>>>> I can
>>>>>>>>>>> now
>>>>>>>>>>> upload
>>>>>>>>>>> a 500m
>>>>>>>>>>> file
>>>>>>>>>>> with
>>>>>>>>>>> 40m
>>>>>>>>>>> heap
>>>>>>>>>>> size
>>>>>>>>>>> ;)
>>>>>>>>>>>
>>>>>>>>>>> Envoyé
>>>>>>>>>>> de
>>>>>>>>>>> mon
>>>>>>>>>>> iPhone
>>>>>>>>>>>
>>>>>>>>>>> Le
>>>>>>>>>>> 30
>>>>>>>>>>> août
>>>>>>>>>>> 2013
>>>>>>>>>>> à 20:49,
>>>>>>>>>>> Ryan
>>>>>>>>>>> Lubke
>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>> a écrit :
>>>>>>>>>>>
>>>>>>>>>>>> I'm
>>>>>>>>>>>> going
>>>>>>>>>>>> to
>>>>>>>>>>>> be
>>>>>>>>>>>> updating
>>>>>>>>>>>> the
>>>>>>>>>>>> Grizzly
>>>>>>>>>>>> provider
>>>>>>>>>>>> such
>>>>>>>>>>>> that
>>>>>>>>>>>> AUTO_SIZE
>>>>>>>>>>>> (not
>>>>>>>>>>>> AUTO_TUNE)
>>>>>>>>>>>> is
>>>>>>>>>>>> the
>>>>>>>>>>>> default,
>>>>>>>>>>>> so
>>>>>>>>>>>> you
>>>>>>>>>>>> can
>>>>>>>>>>>> avoid
>>>>>>>>>>>> the
>>>>>>>>>>>> use
>>>>>>>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>> TransportCustomizer.
>>>>>>>>>>>>
>>>>>>>>>>>> Ryan
>>>>>>>>>>>> Lubke
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Regarding
>>>>>>>>>>>>> your
>>>>>>>>>>>>> tuning
>>>>>>>>>>>>> question,
>>>>>>>>>>>>> I would
>>>>>>>>>>>>> probably
>>>>>>>>>>>>> set
>>>>>>>>>>>>> the
>>>>>>>>>>>>> value
>>>>>>>>>>>>> to
>>>>>>>>>>>>> AsyncQueueWriter.AUTO_TUNE
>>>>>>>>>>>>> (this
>>>>>>>>>>>>> will
>>>>>>>>>>>>> be
>>>>>>>>>>>>> four
>>>>>>>>>>>>> times
>>>>>>>>>>>>> the
>>>>>>>>>>>>> socket
>>>>>>>>>>>>> write
>>>>>>>>>>>>> buffer)
>>>>>>>>>>>>> and
>>>>>>>>>>>>> see
>>>>>>>>>>>>> how
>>>>>>>>>>>>> that
>>>>>>>>>>>>> works.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> A question
>>>>>>>>>>>>>> first.
>>>>>>>>>>>>>> With
>>>>>>>>>>>>>> these
>>>>>>>>>>>>>> changes,
>>>>>>>>>>>>>> your
>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>> usage
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>> inline
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> what
>>>>>>>>>>>>>> you
>>>>>>>>>>>>>> were
>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>> for?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> way,
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>> idea
>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> 1.7.20
>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> released
>>>>>>>>>>>>>>> (with
>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> improvements?)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> wait
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> a release
>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> install
>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>> own
>>>>>>>>>>>>>>> temp
>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> Nexus
>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/8/30
>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank
>>>>>>>>>>>>>>> you,
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>> fine!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I just
>>>>>>>>>>>>>>> had
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> modify
>>>>>>>>>>>>>>> a single
>>>>>>>>>>>>>>> line
>>>>>>>>>>>>>>> after
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>> commit.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider#initializeTransport
>>>>>>>>>>>>>>> ->
>>>>>>>>>>>>>>> clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(10000);
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>> I let
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> initial
>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>> (-1)
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> won't
>>>>>>>>>>>>>>> block,
>>>>>>>>>>>>>>> canWrite
>>>>>>>>>>>>>>> always
>>>>>>>>>>>>>>> returns
>>>>>>>>>>>>>>> true
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Btw,
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> I didn't
>>>>>>>>>>>>>>> find
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> pass
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> value
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> a config
>>>>>>>>>>>>>>> attribute,
>>>>>>>>>>>>>>> neither
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> talked
>>>>>>>>>>>>>>> about.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> end,
>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> "canWrite
>>>>>>>>>>>>>>> = false"
>>>>>>>>>>>>>>> behavior?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If
>>>>>>>>>>>>>>> not,
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> set
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> v1.7.20
>>>>>>>>>>>>>>> ? thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> PS:
>>>>>>>>>>>>>>> does
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> sens
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>> number
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> bytes
>>>>>>>>>>>>>>> un
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> feed(Buffer)
>>>>>>>>>>>>>>> method
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> setMaxPendingBytesPerConnection(10000);
>>>>>>>>>>>>>>> ? do
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>> any
>>>>>>>>>>>>>>> tuning
>>>>>>>>>>>>>>> recommandation?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/8/29
>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please
>>>>>>>>>>>>>>> disregard.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sébastien,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could
>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>> also
>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>> a sample
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>> performing
>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>> feed?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -rl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sébastien,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'd
>>>>>>>>>>>>>>>>> recommend
>>>>>>>>>>>>>>>>> looking
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> Connection.canWrite()
>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> Connection.notifyCanWrite(WriteListener)
>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>>>> default,
>>>>>>>>>>>>>>>>> Grizzly
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> configure
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> async
>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>> length
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> four
>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>>>> (which
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> based
>>>>>>>>>>>>>>>>> off
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>> buffer).
>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>> exceeds
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> value,
>>>>>>>>>>>>>>>>> canWrite()
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> return
>>>>>>>>>>>>>>>>> false.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> When
>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> occurs,
>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> register
>>>>>>>>>>>>>>>>> a WriteListener
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> notified
>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>> length
>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> below
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> configured
>>>>>>>>>>>>>>>>> max
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>> simulate
>>>>>>>>>>>>>>>>> blocking
>>>>>>>>>>>>>>>>> until
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> onWritePossible()
>>>>>>>>>>>>>>>>> callback
>>>>>>>>>>>>>>>>> has
>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>> invoked.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ----------------------------------------------------------------
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> final
>>>>>>>>>>>>>>>>> FutureImpl<Boolean>
>>>>>>>>>>>>>>>>> future
>>>>>>>>>>>>>>>>> = Futures.createSafeFuture();
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> //
>>>>>>>>>>>>>>>>> Connection
>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> obtained
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> calling
>>>>>>>>>>>>>>>>> FilterChainContext.getConnection().
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> connection.notifyCanWrite(new
>>>>>>>>>>>>>>>>> WriteHandler()
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>>> void
>>>>>>>>>>>>>>>>> onWritePossible()
>>>>>>>>>>>>>>>>> throws
>>>>>>>>>>>>>>>>> Exception
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> future.result(Boolean.TRUE);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> public
>>>>>>>>>>>>>>>>> void
>>>>>>>>>>>>>>>>> onError(Throwable
>>>>>>>>>>>>>>>>> t)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> future.failure(Exceptions.makeIOException(t));
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> });
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> final
>>>>>>>>>>>>>>>>> long
>>>>>>>>>>>>>>>>> writeTimeout
>>>>>>>>>>>>>>>>> = 30;
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> future.get(writeTimeout,
>>>>>>>>>>>>>>>>> TimeUnit.MILLISECONDS);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> } catch
>>>>>>>>>>>>>>>>> (ExecutionException
>>>>>>>>>>>>>>>>> e)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> HttpTransactionContext
>>>>>>>>>>>>>>>>> httpCtx
>>>>>>>>>>>>>>>>> = HttpTransactionContext.get(connection);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> httpCtx.abort(e.getCause());
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> } catch
>>>>>>>>>>>>>>>>> (Exception
>>>>>>>>>>>>>>>>> e)
>>>>>>>>>>>>>>>>> {
>>>>>>>>>>>>>>>>> HttpTransactionContext
>>>>>>>>>>>>>>>>> httpCtx
>>>>>>>>>>>>>>>>> = HttpTransactionContext.get(connection);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> httpCtx.abort(e);
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>> http://grizzly.java.net/docs/2.3/apidocs/org/glassfish/grizzly/OutputSink.html.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Ryan,
>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>> did
>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>> tests.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>> a blocking
>>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> totally
>>>>>>>>>>>>>>>>>> useless
>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>> consuming
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> blocking
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>>> never
>>>>>>>>>>>>>>>>>> blocks
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> feeding,
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> was
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> intention
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> beginning.
>>>>>>>>>>>>>>>>>> Maybe
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> depends
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>> strategy
>>>>>>>>>>>>>>>>>> used?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I use
>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> use SameThreadIOStrategy
>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>> I don't
>>>>>>>>>>>>>>>>>> think
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> IO
>>>>>>>>>>>>>>>>>> strategy.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> end
>>>>>>>>>>>>>>>>>> I can
>>>>>>>>>>>>>>>>>> upload
>>>>>>>>>>>>>>>>>> a 70m
>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> a heap
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> 50m,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> I have
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> put
>>>>>>>>>>>>>>>>>> a Thread.sleep(30)
>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>> 100k
>>>>>>>>>>>>>>>>>> Buffer
>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>> connection
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>>>>> here,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> production
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> normally
>>>>>>>>>>>>>>>>>> a lot
>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> far
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> I know.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>> tried
>>>>>>>>>>>>>>>>>> things
>>>>>>>>>>>>>>>>>> like clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(100000);
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>> seem
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> me.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'd
>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> Grizzly
>>>>>>>>>>>>>>>>>> internals
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> block
>>>>>>>>>>>>>>>>>> when
>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>>>>> much
>>>>>>>>>>>>>>>>>> pending
>>>>>>>>>>>>>>>>>> bytes
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> send.
>>>>>>>>>>>>>>>>>> Is
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> possible?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> PS:
>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>>> been
>>>>>>>>>>>>>>>>>> able
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>> a 500mo
>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> 100mo
>>>>>>>>>>>>>>>>>> heap,
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>>>> a sleep
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> 100ms
>>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>> 100k
>>>>>>>>>>>>>>>>>> Buffer
>>>>>>>>>>>>>>>>>> sent
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> bodygenerator
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/8/29
>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> By
>>>>>>>>>>>>>>>>>> chance
>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>> I can
>>>>>>>>>>>>>>>>>> remove
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> MessageCloner
>>>>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> SSL
>>>>>>>>>>>>>>>>>> filter?
>>>>>>>>>>>>>>>>>> SSLBaseFilter$OnWriteCopyCloner
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>>>> a lot
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> memory.
>>>>>>>>>>>>>>>>>> I don't
>>>>>>>>>>>>>>>>>> really
>>>>>>>>>>>>>>>>>> understand
>>>>>>>>>>>>>>>>>> why
>>>>>>>>>>>>>>>>>> messages
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> cloned,
>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> I remove
>>>>>>>>>>>>>>>>>> this?
>>>>>>>>>>>>>>>>>> How?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/8/29
>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>> <lorber.sebastien_at_gmail.com
>>>>>>>>>>>>>>>>>> <mailto:lorber.sebastien_at_gmail.com>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>> trying
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>> a 500m
>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>> tests
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> a heap
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> 400m.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>> real
>>>>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>>>>> cases
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> files
>>>>>>>>>>>>>>>>>> under
>>>>>>>>>>>>>>>>>> 20mo
>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> reduce
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>>>>> consumption
>>>>>>>>>>>>>>>>>> because
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>> x parallel
>>>>>>>>>>>>>>>>>> uploads
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>>>>> according
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> user
>>>>>>>>>>>>>>>>>> activity.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>> try
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> check
>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> BodyGenerator
>>>>>>>>>>>>>>>>>> reduced
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>>>>> footprint
>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> almost
>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>> before.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/8/28
>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> At
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> point
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> far
>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> SSL
>>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>>>> allocation
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> concerned,
>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>> untunable.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> That
>>>>>>>>>>>>>>>>>> said,
>>>>>>>>>>>>>>>>>> feel
>>>>>>>>>>>>>>>>>> free
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> open
>>>>>>>>>>>>>>>>>> a feature
>>>>>>>>>>>>>>>>>> request.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>> second
>>>>>>>>>>>>>>>>>> question,
>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>> suggested
>>>>>>>>>>>>>>>>>> size.
>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>>>>> application
>>>>>>>>>>>>>>>>>> specific.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>> curious,
>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>> large
>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>> sending?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have
>>>>>>>>>>>>>>>>>>> seen
>>>>>>>>>>>>>>>>>>> a lot
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> buffers
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> a size
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> 33842
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> limit
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> near
>>>>>>>>>>>>>>>>>>> half
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> capacity.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Perhaps
>>>>>>>>>>>>>>>>>>> there's
>>>>>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> tune
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> consumes
>>>>>>>>>>>>>>>>>>> less
>>>>>>>>>>>>>>>>>>> memory?
>>>>>>>>>>>>>>>>>>> Is
>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> ideal
>>>>>>>>>>>>>>>>>>> Buffer
>>>>>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> feed
>>>>>>>>>>>>>>>>>>> method?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/8/28
>>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> reviewing
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> PR
>>>>>>>>>>>>>>>>>>> today,
>>>>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>>>> again!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Regarding
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> OOM:
>>>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> stands
>>>>>>>>>>>>>>>>>>> now,
>>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>> each
>>>>>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> passed
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> SSLFilter,
>>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>>>>> a buffer
>>>>>>>>>>>>>>>>>>> twice
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> order
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> accommodate
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> encrypted
>>>>>>>>>>>>>>>>>>> result.
>>>>>>>>>>>>>>>>>>> So
>>>>>>>>>>>>>>>>>>> there's
>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>> increase.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Depending
>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> socket
>>>>>>>>>>>>>>>>>>> configurations
>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>> both
>>>>>>>>>>>>>>>>>>> endpoints,
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>>> fast
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> remote
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> reading
>>>>>>>>>>>>>>>>>>> data,
>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> write
>>>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>> becoming
>>>>>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>>>>>> large.
>>>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> detect
>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>> situation,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>> I'm
>>>>>>>>>>>>>>>>>>> pretty
>>>>>>>>>>>>>>>>>>> sure
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>> Grizzly
>>>>>>>>>>>>>>>>>>> internals
>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>> currently
>>>>>>>>>>>>>>>>>>> shielded
>>>>>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>> I will
>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>> I can
>>>>>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> allow
>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> leverage
>>>>>>>>>>>>>>>>>>> this.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>> made
>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>> pull
>>>>>>>>>>>>>>>>>>>> request.
>>>>>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/pull/367
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> With
>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>> usecase
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> works,
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> uploaded
>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>> before.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> But
>>>>>>>>>>>>>>>>>>>> I didn't
>>>>>>>>>>>>>>>>>>>> notice
>>>>>>>>>>>>>>>>>>>> a big
>>>>>>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>>>>>>> improvement.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Is
>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> SSL
>>>>>>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>>>> allow
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> stream
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> body
>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>> that?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In
>>>>>>>>>>>>>>>>>>>> memory,
>>>>>>>>>>>>>>>>>>>> I have
>>>>>>>>>>>>>>>>>>>> a lot
>>>>>>>>>>>>>>>>>>>> of:
>>>>>>>>>>>>>>>>>>>> - HeapByteBuffer
>>>>>>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>> hold
>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> SSLUtils$3
>>>>>>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>> hold
>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> BufferBuffers
>>>>>>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>> hold
>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> WriteResult
>>>>>>>>>>>>>>>>>>>> Which
>>>>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>>>>> hold
>>>>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>>>> AsyncWriteQueueRecord
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Here
>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>> exemple
>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> OOM
>>>>>>>>>>>>>>>>>>>> stacktrace:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> java.lang.OutOfMemoryError:
>>>>>>>>>>>>>>>>>>>> Java
>>>>>>>>>>>>>>>>>>>> heap
>>>>>>>>>>>>>>>>>>>> space
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:227)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:404)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:319)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2503)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:132)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:101)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder$FeedBodyGeneratorOutputStream.write(MultipartBodyGeneratorFeeder.java:222)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.FilePart.sendData(FilePart.java:179)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.Part.send(Part.java:331)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.Part.sendParts(Part.java:397)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder.feed(MultipartBodyGeneratorFeeder.java:144)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Any
>>>>>>>>>>>>>>>>>>>> idea?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2013/8/27
>>>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Excellent!
>>>>>>>>>>>>>>>>>>>> Looking
>>>>>>>>>>>>>>>>>>>> forward
>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> pull
>>>>>>>>>>>>>>>>>>>> request!
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>>>>> thanks,
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> works
>>>>>>>>>>>>>>>>>>>>> fine,
>>>>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>>>>>> a pull
>>>>>>>>>>>>>>>>>>>>> request
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>>>>> tomorrow
>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> a better
>>>>>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>> Part
>>>>>>>>>>>>>>>>>>>>> classes
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>> exist.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I created
>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>> OutputStream
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> redirects
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> BodyGenerator
>>>>>>>>>>>>>>>>>>>>> feeder.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>> problem
>>>>>>>>>>>>>>>>>>>>> I currently
>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> feeder
>>>>>>>>>>>>>>>>>>>>> feeds
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>>>>>> faster
>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> async
>>>>>>>>>>>>>>>>>>>>> thread
>>>>>>>>>>>>>>>>>>>>> polling
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> :)
>>>>>>>>>>>>>>>>>>>>> I need
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> expose
>>>>>>>>>>>>>>>>>>>>> a limit
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>> queue
>>>>>>>>>>>>>>>>>>>>> size
>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>> something,
>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> that,
>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>> better
>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>> a thread
>>>>>>>>>>>>>>>>>>>>> sleep
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> slow
>>>>>>>>>>>>>>>>>>>>> down
>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> filepart
>>>>>>>>>>>>>>>>>>>>> reading
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/8/27
>>>>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yes,
>>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>> that.
>>>>>>>>>>>>>>>>>>>>> I was
>>>>>>>>>>>>>>>>>>>>> going
>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> tackle
>>>>>>>>>>>>>>>>>>>>> adding
>>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> today.
>>>>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>>>>> follow
>>>>>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> test
>>>>>>>>>>>>>>>>>>>>> out.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Ok
>>>>>>>>>>>>>>>>>>>>>> thanks!
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I think
>>>>>>>>>>>>>>>>>>>>>> I see
>>>>>>>>>>>>>>>>>>>>>> what
>>>>>>>>>>>>>>>>>>>>>> I could
>>>>>>>>>>>>>>>>>>>>>> do,
>>>>>>>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>>>>>> something
>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>> that:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator
>>>>>>>>>>>>>>>>>>>>>> bodyGenerator
>>>>>>>>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder
>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder
>>>>>>>>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder(bodyGenerator);
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Request
>>>>>>>>>>>>>>>>>>>>>> uploadRequest1
>>>>>>>>>>>>>>>>>>>>>> = new
>>>>>>>>>>>>>>>>>>>>>> RequestBuilder("POST")
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> .setUrl("url")
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> .setBody(bodyGenerator)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> .build();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> ListenableFuture<Response>
>>>>>>>>>>>>>>>>>>>>>> asyncRes
>>>>>>>>>>>>>>>>>>>>>> = asyncHttpClient
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> .prepareRequest(uploadRequest1)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> .execute(new
>>>>>>>>>>>>>>>>>>>>>> AsyncCompletionHandlerBase());
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param1","value1");
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param2","value2");
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("fileToUpload",fileInputStream);
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.end();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Response
>>>>>>>>>>>>>>>>>>>>>> uploadResponse
>>>>>>>>>>>>>>>>>>>>>> = asyncRes.get();
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Does
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> seem
>>>>>>>>>>>>>>>>>>>>>> ok
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> you?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I guess
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> could
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> interesting
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> provide
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder
>>>>>>>>>>>>>>>>>>>>>> class
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>>> Grizzly
>>>>>>>>>>>>>>>>>>>>>> since
>>>>>>>>>>>>>>>>>>>>>> some
>>>>>>>>>>>>>>>>>>>>>> other
>>>>>>>>>>>>>>>>>>>>>> people
>>>>>>>>>>>>>>>>>>>>>> may
>>>>>>>>>>>>>>>>>>>>>> want
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> achieve
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>> thing
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/8/26
>>>>>>>>>>>>>>>>>>>>>> Ryan
>>>>>>>>>>>>>>>>>>>>>> Lubke
>>>>>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com
>>>>>>>>>>>>>>>>>>>>>> <mailto:ryan.lubke_at_oracle.com>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sébastien
>>>>>>>>>>>>>>>>>>>>>> Lorber
>>>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I would
>>>>>>>>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> know
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> it's
>>>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> upload
>>>>>>>>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>>>>>> / Grizzly
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> streaming,
>>>>>>>>>>>>>>>>>>>>>> I mean
>>>>>>>>>>>>>>>>>>>>>> without
>>>>>>>>>>>>>>>>>>>>>> loading
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>> bytes
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> memory.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>>>>> behavior
>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> allocate
>>>>>>>>>>>>>>>>>>>>>> a byte[]
>>>>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>>>>> contans
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>>>>>>>>> file,
>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> means
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> OOM
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> too
>>>>>>>>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>>>> upload
>>>>>>>>>>>>>>>>>>>>>> a large
>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>>>>>> time.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I've
>>>>>>>>>>>>>>>>>>>>>> tryied
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> a Heap
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> ByteBuffer
>>>>>>>>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>>>>>>>>> managers,
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> reallocate=true/false
>>>>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>>>> no
>>>>>>>>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>>>>>>> success.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It
>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> whole
>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>> content
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> appended
>>>>>>>>>>>>>>>>>>>>>> wto
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> BufferOutputStream,
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> underlying
>>>>>>>>>>>>>>>>>>>>>> buffer
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> written.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> At
>>>>>>>>>>>>>>>>>>>>>> least
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> seems
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>>>>>> integration:
>>>>>>>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/6faf1f316e5546110b0779a5a42fd9d03ba6bc15/providers/grizzly/src/main/java/org/asynchttpclient/providers/grizzly/bodyhandler/PartsBodyHandler.java
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> So,
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>>> a way
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> patch
>>>>>>>>>>>>>>>>>>>>>> AHC
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> stream
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> I could
>>>>>>>>>>>>>>>>>>>>>> eventually
>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>> 20mo
>>>>>>>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>>>>>>> heap
>>>>>>>>>>>>>>>>>>>>>> while
>>>>>>>>>>>>>>>>>>>>>> uploading
>>>>>>>>>>>>>>>>>>>>>> a 500mo
>>>>>>>>>>>>>>>>>>>>>> file?
>>>>>>>>>>>>>>>>>>>>>> Or
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> simply
>>>>>>>>>>>>>>>>>>>>>> impossible
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> Grizzly?
>>>>>>>>>>>>>>>>>>>>>> I didn't
>>>>>>>>>>>>>>>>>>>>>> notice
>>>>>>>>>>>>>>>>>>>>>> anything
>>>>>>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> documentation.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It's
>>>>>>>>>>>>>>>>>>>>>> possible
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator.
>>>>>>>>>>>>>>>>>>>>>> But
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> you're
>>>>>>>>>>>>>>>>>>>>>> tied
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>>>>>> Multipart
>>>>>>>>>>>>>>>>>>>>>> uploads,
>>>>>>>>>>>>>>>>>>>>>> you'd
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> convert
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> multipart
>>>>>>>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> Buffers
>>>>>>>>>>>>>>>>>>>>>> manually
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator.
>>>>>>>>>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>>>>>>>>>> take
>>>>>>>>>>>>>>>>>>>>>> a closer
>>>>>>>>>>>>>>>>>>>>>> look
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> see
>>>>>>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>>> area
>>>>>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>>>>> improved.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Btw
>>>>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>> my
>>>>>>>>>>>>>>>>>>>>>> case
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>>>>>>>>> upload.
>>>>>>>>>>>>>>>>>>>>>> I receive
>>>>>>>>>>>>>>>>>>>>>> a file
>>>>>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>>>>>> CXF
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> transmit
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> a storage
>>>>>>>>>>>>>>>>>>>>>> server
>>>>>>>>>>>>>>>>>>>>>> (like
>>>>>>>>>>>>>>>>>>>>>> S3).
>>>>>>>>>>>>>>>>>>>>>> CXF
>>>>>>>>>>>>>>>>>>>>>> doesn't
>>>>>>>>>>>>>>>>>>>>>> consume
>>>>>>>>>>>>>>>>>>>>>> memory
>>>>>>>>>>>>>>>>>>>>>> bevause
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> is
>>>>>>>>>>>>>>>>>>>>>> streaming
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> large
>>>>>>>>>>>>>>>>>>>>>> fle
>>>>>>>>>>>>>>>>>>>>>> uploads
>>>>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> file
>>>>>>>>>>>>>>>>>>>>>> system,
>>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> then
>>>>>>>>>>>>>>>>>>>>>> provides
>>>>>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>>> input
>>>>>>>>>>>>>>>>>>>>>> stream
>>>>>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>>>> file.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
>