From: Sébastien Lorber <lorber.sebastien_at_gmail.com>
Date: Tue, 5 Nov 2013 17:24:49 +0100

I've encountered the same problem. Using MultipartBody I couldn't produce a
body that would be accepted by my remote server. So I used the
Parts.send(outputStream).

But it would be much more efficient if there was a way to produce the
multipart body as an unique InputStream.

Guava has a class that could help: CompositeInputStream.
It would be sufficient to concat the different InputStream parts, using in
memory inputstreams (byte array) for fixed length (small) parts like string
parts and boundaries ..
and a real stream for files.


I think this is what MultipartBody.java tries to do somehow but it is not
really readable (and as far as I remember there's an ugly catch at the end
which shadows an exception)



2013/11/5 Bongjae Chang <bongjae.chang_at_gmail.com>

> Hi Ryan,
> No, not at all. I will create a pull request.
>
> Some issues)
> 1. When I tested new logic with MultipartBody in AHC, I found some bugs of
> MultipartBody.java. I will create a separate issue about that.
> 2. About supporting transferTo(), I also tried to use the FileTransfer but
> I couldn't because the FileTransfer was not an interface and the class was
> used and referenced directly in grizzly-framework by casting. :-(
>
> Thanks!
>
> Regards,
> Bongjae Chang
>
>
> From: Ryan Lubke <ryan.lubke_at_oracle.com>
> Reply-To: <users_at_grizzly.java.net>
> Date: Tuesday, November 5, 2013 7:24 AM
> To: <users_at_grizzly.java.net>
> Subject: Re: Upload a large file without oom with Grizzly
>
> Hi Bongjae,
>
> Sounds reasonable. Would you mind creating a pull request?
>
> Thanks,
> -rl
>
> Bongjae Chang wrote:
>
> Hi Ryan and Sébastien,
>
> I'm sorry for late.
>
> When I applied the source(https://gist.github.com/slorber/7199403), it
> worked well(Thanks to Sébastien!).
>
> But I think that we can devise a better logic(I think that we can reuse
> MultipartBody class which has multipart protocol, so we don't need to
> consider the content type and boundary for multipart and can reduce buffer
> copies in FeedBodyGeneratorOutputStream).
>
> I think current multipart's logic of AHC with grizzly provider has two
> problems.
>
> 1. Before writing, it allocates all source's buffers unfortunately.( I
> already met OOM before actual writing when I tested).
>
> In GrizzlyAsyncHttpProvider's PartsBodyHandler.
> ---
> mre.writeRequest(o); // OOM before actual writing!
> …
> ctx.write(content, …);
> ---
> It can be improved with using MultipartBody(not using
> MultipartRequestEntiry#writeRequest()).
>
> Here is a simple idea.
>
> In GrizzlyAsyncHttpProvider's PartsBodyHandler.
> ---
> final Body bodyLocal = new MultipartBody(request.getParts(), contentType,
> String.valueOf(contentLength));
> …
> while(!last) {
> Buffer buffer = mm.allocate(MAX_CHUNK_SIZE);
> buffer.allowBufferDispose(true);
> final long readBytes = bodyLocal.read(buffer.toByteBuffer());
> if (readBytes > 0) {
> ...
> } else {
> buffer.dispose();
> if (readBytes < 0) {
> last = true;
> buffer = Buffers.EMPTY_BUFFER;
> } else {
> ...
> }
> }
> …
> ctx.write(content, …);
> }
> return true;
> ---
>
> 2. When I modified above 1 with MultipartBody like netty provider, I also
> met OOM in writing because the write buffer queue will be able to be full.
>
> I think this case is reasonable. For this problem, write-flow-control is
> needed like BaseFeeder, SimpleFeeder and MultipartBodyGeneratorFeeder.
>
>
> I think that it is better that the user can use the AHC without
> considering grizzly provider specific setting for multipart.
> ex) Maybe PartsBodyHandler can be improved with MultipartBody and
> BaseFeeder logic.
> In addition, multipart can have the large file so it will be better if
> grizzly provider can also support the FileTransfer in PartsBodyHandler like
> FileBodyHandler of GrizzlyAsyncHttpProvider.java.
>
> What do you think? :-)
>
> Regards,
> Bongjae Chang
>
>
> From: Ryan Lubke <ryan.lubke_at_oracle.com>
> Reply-To: <users_at_grizzly.java.net>
> Date: Tuesday, October 29, 2013 9:23 AM
> To: <users_at_grizzly.java.net>
> Subject: Re: Upload a large file without oom with Grizzly
>
> I'll take a look at it later this week and see what's possible.
>
> Sébastien Lorber wrote:
>
> Hi,
>
> Bongjae, this is code on my own project and is not part of AHC.
>
> You will find it here:
> https://gist.github.com/slorber/7199403
> It requires the last AHC release.
>
> Ryan if you want you can try to integrate it in AHC.
> Maybe it could even be made the default behavior for Grizzly based AHC
> multipart requests, and will reduce the memory footprint to a lot of users
> in the next AHC release.
>
>
> It still work fine for us until now, we had the first load tests done by
> our client.
> We don't know the test scenarios and many things were changed in addition
> to this part of the project, but at least we have good results until now
> and we'll increase the load soon.
> Won't be able to provide another feedback as i'm changing job in 3 days :)
>
>
> Bye
>
>
>
>
>
> 2013/10/28 Bongjae Chang <bongjae.chang_at_gmail.com>
>
>> Hi Ryan and Sébastien,
>>
>> I met the same issue(OOM) while I sent the large file(about 1G size) with
>> multipart in AHC.
>> When I reviewed AHC's sources, I couldn't find
>> MultipartBodyGeneratorFeeder class yet.
>>
>> Is there any updates? or where can I get the stable reference?
>>
>> Thank in advance!
>>
>> Regards,
>> Bongjae Chang
>>
>>
>> From: Sébastien Lorber <lorber.sebastien_at_gmail.com>
>> Reply-To: <users_at_grizzly.java.net>
>> Date: Wednesday, September 25, 2013 5:47 PM
>> To: "users_at_grizzly.java.net" <users_at_grizzly.java.net>
>> Subject: Re: Upload a large file without oom with Grizzly
>>
>> Thanks Ryan, will use that release :)
>>
>> No problem until now, will tell you what happens during our load tests
>>
>>
>> 2013/9/24 Ryan Lubke <ryan.lubke_at_oracle.com>
>>
>>> Sébastien,
>>>
>>> AHC 1.7.20 has been released. It's up on the oss.sonatype.org maven
>>> repository and will be sync'd with central soon-ish.
>>>
>>> -rl
>>>
>>> Ryan Lubke wrote:
>>>
>>>
>>>
>>> Sébastien Lorber wrote:
>>>
>>> Yes it works nicely :)
>>>
>>> Thanks Ryan I think everything is now working for me
>>>
>>> You're welcome. Let us know if you run into anything else, or if you
>>> have ideas on improvements, etc.
>>>
>>>
>>>
>>> 2013/9/20 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>
>>>> Just following up for closure here. Pool issue has been confirmed as
>>>> resolved.
>>>>
>>>> Sébastien Lorber wrote:
>>>>
>>>> Ok thanks :)
>>>>
>>>> Take your time
>>>> We'll keep using no connection pool (as we used to do until now anyway,
>>>> and we could probably continue with this setting for a while because it
>>>> passed our load test goals)
>>>>
>>>>
>>>> 2013/9/19 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>
>>>>> Sorry, been under the weather. I'll be investigating this issue this
>>>>> morning.
>>>>>
>>>>> Stay tuned.
>>>>>
>>>>> Sébastien Lorber wrote:
>>>>>
>>>>> So what I've found is that:
>>>>>
>>>>>
>>>>> private final Connection.CloseListener listener = new
>>>>> Connection.CloseListener() {
>>>>> @Override
>>>>> public void onClosed(Connection connection,
>>>>> Connection.CloseType closeType) throws IOException {
>>>>> if (closeType == Connection.CloseType.REMOTELY) {
>>>>> if (LOG.isInfoEnabled()) {
>>>>> LOG.info("Remote closed connection ({}).
>>>>> Removing from cache", connection.toString());
>>>>> }
>>>>> }
>>>>> GrizzlyConnectionsPool.this.removeAll(connection);
>>>>> }
>>>>> };
>>>>>
>>>>>
>>>>> public boolean removeAll(Connection connection) {
>>>>>
>>>>> if (connection == null || closed.get()) {
>>>>> return false;
>>>>> }
>>>>> connection.removeCloseListener(listener);
>>>>> boolean isRemoved = false;
>>>>> for (Map.Entry<String, DelayedExecutor.IdleConnectionQueue>
>>>>> entry : connectionsPool.entrySet()) {
>>>>> boolean removed = entry.getValue().remove(connection);
>>>>> isRemoved |= removed;
>>>>> }
>>>>> return isRemoved;
>>>>>
>>>>> }
>>>>>
>>>>>
>>>>>
>>>>> When the connection is closed remotely, the connection that we try to
>>>>> remove is never in the cache, thus isRemoved = false
>>>>>
>>>>> I guess it has something to do with the "connection discrimination"
>>>>> you previously fixed and I didn't really understand :)
>>>>>
>>>>> Do you have an idea on this problem?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> By the way, I just tested again WITHOUT the FeedableBodyGenerator.
>>>>> I run batches of 10 concurrent uploads with a max 20 connection pool.
>>>>> I run these 10 concurrent uploads multiple times.
>>>>> It works fine, but if just after the request I add a
>>>>> Thread.sleep(15000) then it seems to lead to the same problem:
>>>>>
>>>>> @Test
>>>>> public void do_test_without_ahc_streaming() throws Exception {
>>>>> List<Response> responses = runConcurrently(CONCURRENT_UPLOADS,new
>>>>> Callable<Response>() {
>>>>> @Override
>>>>> public Response call() throws Exception {
>>>>> return uploadSingleDocumentWithoutStreaming();
>>>>> }
>>>>> });
>>>>> runConcurrently(CONCURRENT_UPLOADS,new Callable<Response>() {
>>>>> @Override
>>>>> public Response call() throws Exception {
>>>>> return uploadSingleDocumentWithoutStreaming();
>>>>> }
>>>>> });
>>>>> Thread.sleep(15000);
>>>>> runConcurrently(CONCURRENT_UPLOADS,new Callable<Response>() {
>>>>> @Override
>>>>> public Response call() throws Exception {
>>>>> return uploadSingleDocumentWithoutStreaming();
>>>>> }
>>>>> });
>>>>> Thread.sleep(15000);
>>>>> runConcurrently(CONCURRENT_UPLOADS,new Callable<Response>() {
>>>>> @Override
>>>>> public Response call() throws Exception {
>>>>> return uploadSingleDocumentWithoutStreaming();
>>>>> }
>>>>> });
>>>>> Thread.sleep(15000);
>>>>> runConcurrently(CONCURRENT_UPLOADS,new Callable<Response>() {
>>>>> @Override
>>>>> public Response call() throws Exception {
>>>>> return uploadSingleDocumentWithoutStreaming();
>>>>> }
>>>>> });
>>>>> Thread.sleep(15000);
>>>>> ......................................;
>>>>> }
>>>>>
>>>>> This permits to give some time for the remote host to close the
>>>>> connection
>>>>>
>>>>>
>>>>>
>>>>> So in the end it seems that we have troubles in any case when the
>>>>> remote host closes the connection
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Btw I just saw this issue:
>>>>> https://github.com/AsyncHttpClient/async-http-client/pull/311
>>>>> I checked the code that used AHC 1.7.7 and what I can see is that the
>>>>> connection pool is bypassed even with the property=true
>>>>> I have in my logs:
>>>>> [main] DEBUG
>>>>> com.ning.http.client.providers.grizzly.GrizzlyConnectionsPool:162 - [poll]
>>>>> No existing queue for uri [https://digiposte.orsid.com:443].
>>>>>
>>>>>
>>>>> So, by updating to 1.7.20-SNAPSHOT, the connection pool has been
>>>>> magically enabled, and it doesn't seem to work well for me.
>>>>>
>>>>>
>>>>> I'll keep that pool disabled because it's the behavior of our previous
>>>>> applicative version that is near to production and works pretty fine, but
>>>>> will open a Github issue because i'm quitting my job next month and they
>>>>> want to track when they'll be able to use a connection pool :)
>>>>>
>>>>>
>>>>>
>>>>> 2013/9/19 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>
>>>>>> So it seems I can reproduce this in local.
>>>>>>
>>>>>> Disabling the ssl connection pooling solve the problem.
>>>>>>
>>>>>>
>>>>>> We didn't have this problems with v1.7.7 we previously used.
>>>>>>
>>>>>> I just tested with the last commits of 1.7.20-SNAPSHOT, the problem
>>>>>> is still here.
>>>>>> The problem doesn't appear on normal AHC requests, neither on
>>>>>> multipart file upload when I do not use the FeedableBodyGenerator so it
>>>>>> seems this is the new behavior of FeedableBodyGenerator that has a problem
>>>>>> with the ssl connection pooling. Will try to investigate this.
>>>>>>
>>>>>> By the way, your last commits about selector/worker threads seems ok
>>>>>> (didn't notice any problem with the ssl pool disabled)
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/9/18 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Unfortunatly it seems there's another problem.
>>>>>>> On our test environment I often get in the logs:
>>>>>>>
>>>>>>> Caught internal server errorjava.io.IOException: Maximum pooled
>>>>>>> connections exceeded
>>>>>>> at
>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientEventFilter.cleanup(GrizzlyAsyncHttpProvider.java:1415)
>>>>>>> ~[async-http-client-1.7.20-2846817.jar:na]
>>>>>>> at
>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientEventFilter.onHttpPacketParsed(GrizzlyAsyncHttpProvider.java:1366)
>>>>>>> ~[async-http-client-1.7.20-2846817.jar:na]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.http.HttpCodecFilter.decodeWithTransferEncoding(HttpCodecFilter.java:1176)
>>>>>>> ~[grizzly-http-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.http.HttpCodecFilter.handleRead(HttpCodecFilter.java:516)
>>>>>>> ~[grizzly-http-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.http.HttpClientFilter.handleRead(HttpClientFilter.java:161)
>>>>>>> ~[grizzly-http-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> Wrapped by: java.util.concurrent.ExecutionException:
>>>>>>> java.io.IOException: Maximum pooled connections exceeded
>>>>>>> at
>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:359)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:265)
>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>> at
>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyResponseFuture.get(GrizzlyResponseFuture.java:165)
>>>>>>> ~[async-http-client-1.7.20-2846817.jar:na]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> boolean canReturnConnection(final Connection c) {
>>>>>>>
>>>>>>> return (DO_NOT_CACHE.get(c) != null ||
>>>>>>> pool.canCacheConnection());
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>> private static HttpTransactionContext cleanup(final
>>>>>>> FilterChainContext ctx,
>>>>>>> final
>>>>>>> GrizzlyAsyncHttpProvider provider) {
>>>>>>>
>>>>>>> final Connection c = ctx.getConnection();
>>>>>>> final HttpTransactionContext context =
>>>>>>> provider.getHttpTransactionContext(c);
>>>>>>> context.provider.setHttpTransactionContext(c, null);
>>>>>>> if
>>>>>>> (!context.provider.connectionManager.canReturnConnection(c)) {
>>>>>>> context.abort(new IOException("Maximum pooled
>>>>>>> connections exceeded"));
>>>>>>> } else {
>>>>>>> if
>>>>>>> (!context.provider.connectionManager.returnConnection(context.request, c)) {
>>>>>>> ctx.getConnection().close();
>>>>>>> }
>>>>>>> }
>>>>>>>
>>>>>>> return context;
>>>>>>>
>>>>>>> }
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> We use:
>>>>>>>
>>>>>>> Key=[properties.httpclient.allowSslConnectionPool] Value=[true]
>>>>>>> Key=[properties.httpclient.maximumConnectionsPerHost] Value=[20]
>>>>>>> Key=[properties.httpclient.maximumConnectionsTotal] Value=[30]
>>>>>>> Key=[properties.httpclient.timeout.connection] Value=[10000]
>>>>>>> Key=[properties.httpclient.timeout.request] Value=[30000]
>>>>>>>
>>>>>>>
>>>>>>> In our logs, I can see there are 30 times this log:
>>>>>>>
>>>>>>> Remote closed connection (TCPNIOConnection{localSocketAddress={/
>>>>>>> 172.16.104.160:55488}, peerSocketAddress={host/172.16.4.100:443}}).
>>>>>>> Removing from cache
>>>>>>>
>>>>>>> and then it seems no connection is never available and the app is
>>>>>>> blocked.
>>>>>>>
>>>>>>> I'll try tomorrow to disable or increase the pool size, but if you
>>>>>>> have any idea of the problem please let me know.
>>>>>>> We do not run load tests, these are simple functional tests with
>>>>>>> nearly no concurrency.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 2013/9/17 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>
>>>>>>>> This is good to hear. Thanks for all the feedback and working with
>>>>>>>> us to nail this down.
>>>>>>>>
>>>>>>>>
>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> We already installed the previous snapshot in our nexus because it
>>>>>>>> works fine and I'm working on something else now but I'll give you a
>>>>>>>> feedback soon to see if this still works fine.
>>>>>>>> For us it doesn't really mater which thread is using since we do
>>>>>>>> not use Future in our applications for this case.
>>>>>>>>
>>>>>>>>
>>>>>>>> Btw I've been able deploy a main with my code running concurrent
>>>>>>>> uploads on a server which has a better connectivity with the remote API and
>>>>>>>> it seems I can upload up to 150 concurrent files of 10mb with a heap of
>>>>>>>> 250mo in 30 seconds and a throughput near 50mb/s
>>>>>>>>
>>>>>>>> I don't really know the infrastructure on which my code was
>>>>>>>> deployed but I suspect Grizzly is not the bottleneck anymore :)
>>>>>>>> When I have more than 150 concurrent uploads the remote endpoint
>>>>>>>> seems to timeout under the load.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/9/16 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I noticed something strange.
>>>>>>>>> On the FileInputStream I have, I've added a log on the close() of
>>>>>>>>> the stream which is called once the whole file has been read to be sent to
>>>>>>>>> the feeder.
>>>>>>>>>
>>>>>>>>> 1) If I perform a request (ie my session init request in the
>>>>>>>>> previous discussions) before doing my multipart upload, the thread that
>>>>>>>>> does execute the feeding is the thread that fires the request, and not a
>>>>>>>>> Grizzly worker.
>>>>>>>>> [Thread-30] INFO Will close file stream
>>>>>>>>> java.io.FileInputStream_at_27fe4315 after having read 1440304 bytes
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2) If I don't do any request before firing the multipart upload,
>>>>>>>>> the thread that does the feeding is a Grizzly threadpool worker thread:
>>>>>>>>> [Grizzly(22)] INFO Will close file stream
>>>>>>>>> java.io.FileInputStream_at_59ac4002 after having read 1440304 bytes
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is this normal? I would expect a worker thread to always be used,
>>>>>>>>> and the main applicative thread that performs the request to never be
>>>>>>>>> blocking. (but it's not such an issue for me, we don't have a reactive
>>>>>>>>> non-blocking app anyway)
>>>>>>>>>
>>>>>>>>> It's currently expected behavior. Will need to re-evaluate this
>>>>>>>>> based on the new semantics of the FeedableBodyGenerator.
>>>>>>>>>
>>>>>>>>> I've committed a change for this. If, upon testing, you find
>>>>>>>>> there is still an issue, please let us know.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On the 1st case, here's the stacktrace when the feed method is
>>>>>>>>> called:
>>>>>>>>>
>>>>>>>>> * at
>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.initializeAsynchronousTransfer(FeedableBodyGenerator.java:178)*
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$BodyGeneratorBodyHandler.doHandle(GrizzlyAsyncHttpProvider.java:2210)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.sendRequest(GrizzlyAsyncHttpProvider.java:564)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientFilter.sendAsGrizzlyRequest(GrizzlyAsyncHttpProvider.java:913)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$AsyncHttpClientFilter.handleWrite(GrizzlyAsyncHttpProvider.java:795)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.write(DefaultFilterChain.java:437)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:387)
>>>>>>>>> at
>>>>>>>>> org.glassfish.grizzly.nio.NIOConnection.write(NIOConnection.java:361)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.execute(GrizzlyAsyncHttpProvider.java:307)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$1.completed(GrizzlyAsyncHttpProvider.java:224)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$1.completed(GrizzlyAsyncHttpProvider.java:210)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$ConnectionManager.doAsyncTrackedConnection(GrizzlyAsyncHttpProvider.java:2289)
>>>>>>>>> at
>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.execute(GrizzlyAsyncHttpProvider.java:244)
>>>>>>>>> * at
>>>>>>>>> com.ning.http.client.AsyncHttpClient.executeRequest(AsyncHttpClient.java:534)*
>>>>>>>>>
>>>>>>>>> So it seems this happen when the handshake has already been done
>>>>>>>>> when *initializeAsynchronousTransfer* is called, so that we do
>>>>>>>>> not go through the HandshakeListener
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Notice that this case seems to also happen in a few threads on the
>>>>>>>>> 2nd case, but most of the threads are Grizzly workers.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2013/9/12 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thanks, it seems to work.
>>>>>>>>>>
>>>>>>>>>> I would suggest to throw IOException on the flush method since
>>>>>>>>>> the feed method is supposed to be called here
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> My implementation of flush is:
>>>>>>>>>>
>>>>>>>>>> @Override
>>>>>>>>>> public void flush() {
>>>>>>>>>> Part[] partsArray = parts.toArray(new Part[parts.size()]);
>>>>>>>>>> try ( OutputStream outputStream =
>>>>>>>>>> createFeedingOutputStream() ) {
>>>>>>>>>> Part.sendParts(outputStream,partsArray,multipartBoundary);
>>>>>>>>>> } catch (Exception e) {
>>>>>>>>>> throw new IllegalStateException("Unable to feed the
>>>>>>>>>> FeedableBodyGenerator",e);
>>>>>>>>>> }
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> Is this correct? The OutputStream redirects the bytes written to
>>>>>>>>>> the feed(Buffer) method
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> There seems to be some concurrency issue. Because the upload of 1
>>>>>>>>>> file seems fine, but when using multiple threads, I often get the following
>>>>>>>>>> stack:
>>>>>>>>>> Caused by: java.io.IOException: Stream Closed
>>>>>>>>>> at java.io.FileInputStream.readBytes(Native Method)
>>>>>>>>>> at java.io.FileInputStream.read(FileInputStream.java:242)
>>>>>>>>>> at
>>>>>>>>>> com.google.common.io.CountingInputStream.read(CountingInputStream.java:62)
>>>>>>>>>> at java.io.FilterInputStream.read(FilterInputStream.java:133)
>>>>>>>>>> at java.io.FilterInputStream.read(FilterInputStream.java:107)
>>>>>>>>>> at com.ning.http.multipart.FilePart.sendData(FilePart.java:178)
>>>>>>>>>> at com.ning.http.multipart.Part.send(Part.java:331)
>>>>>>>>>> at com.ning.http.multipart.Part.sendParts(Part.java:397)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is because the flush() method is called multiple times for
>>>>>>>>>> the same request on some cases.
>>>>>>>>>> I guess this is not supposed to happen.
>>>>>>>>>> What I understand is that the flush() method is supposed to be
>>>>>>>>>> called only once.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Using debug logging breakpoints I get the following:
>>>>>>>>>>
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.042 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 0 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.042 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 1 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 2 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 3 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.043 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 4 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 5 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 6 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.044 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 7 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 8 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 9 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.045 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 10 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.046 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 11 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.047 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 12 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.048 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 13 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.049 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 14 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.049 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 15 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.050 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 16 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.050 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 17 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.051 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 18 started
>>>>>>>>>> myapp--api-test 12/09/2013-12:20:46.051 [] [] [main] INFO
>>>>>>>>>> com.myapp.perf.DocumentUploadPerfIntegrationTest:77 - Thread 19 started
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_417de6ff
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6c0d6ef7
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_7799b411
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_1c940409
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_480f9510
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_3e888183
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_17840db8
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_2cbad94b
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_3d5d9ee8
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_51557949
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6f95de2f
>>>>>>>>>> Adding handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_346d784c
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_7799b411
>>>>>>>>>> *Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>>>>>>>>> *Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>>>>>>>>> *Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_5befaa07*
>>>>>>>>>> *Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6*
>>>>>>>>>> *Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_102873a6*
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_346d784c
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_674735a8
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_64c0a4ae
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_480f9510
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_6c0d6ef7
>>>>>>>>>> Completing and removing handshake listener for
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator_at_1daf8fd8
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> As you can see the same HandshakeListener seems to be called
>>>>>>>>>> multiple times for the same FeedableBodyGenerator
>>>>>>>>>>
>>>>>>>>>> When this happens, the stack is:
>>>>>>>>>>
>>>>>>>>>> at
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$1.onComplete(FeedableBodyGenerator.java:198)
>>>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.notifyHandshakeComplete(SSLBaseFilter.java:880)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:282)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>>>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:115)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:55)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:135)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So I tried with the following code:
>>>>>>>>>>
>>>>>>>>>> private boolean alreadyFlushed = false;
>>>>>>>>>> @Override
>>>>>>>>>> public synchronized void flush() {
>>>>>>>>>> if ( alreadyFlushed ) {
>>>>>>>>>> return;
>>>>>>>>>> }
>>>>>>>>>> startFeeding();
>>>>>>>>>> alreadyFlushed = true;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> It works fine when upload small files.
>>>>>>>>>>
>>>>>>>>>> But with larger files, I often get TimeoutException stacks for
>>>>>>>>>> some of the threads:
>>>>>>>>>>
>>>>>>>>>> Caused by: java.util.concurrent.TimeoutException
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:367)
>>>>>>>>>> at
>>>>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:274)
>>>>>>>>>> at
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.block(FeedableBodyGenerator.java:349)
>>>>>>>>>> at
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.blockUntilQueueFree(FeedableBodyGenerator.java:339)
>>>>>>>>>> at
>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator$BaseFeeder.feed(FeedableBodyGenerator.java:306)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Did I miss something?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2013/9/12 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>
>>>>>>>>>>> Committed another small change. Please make sure you're at the
>>>>>>>>>>> latest when you build.
>>>>>>>>>>>
>>>>>>>>>>> -rl
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>
>>>>>>>>>>> Okay, I've committed another set of refactorings to the
>>>>>>>>>>> FeedableBodyGenerator.
>>>>>>>>>>>
>>>>>>>>>>> For your use case, you should extend
>>>>>>>>>>> FeedableBodyGenerator.SimpleFeeder.
>>>>>>>>>>>
>>>>>>>>>>> Let me know if you run into issues.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>
>>>>>>>>>>> Yes I think it would, so that I can feed the queue at once
>>>>>>>>>>>
>>>>>>>>>>> One thread will be locked during the feeding for nothing but
>>>>>>>>>>> it's not a real problem in my usecase.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2013/9/10 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>
>>>>>>>>>>>> Would having a different listener that will be notified once
>>>>>>>>>>>> async transferring has been started work better for you?
>>>>>>>>>>>>
>>>>>>>>>>>> Something like:
>>>>>>>>>>>>
>>>>>>>>>>>> onAsyncTransferInitiated() {
>>>>>>>>>>>> // invoke your feed method
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> ?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunatly I won't be able to use the Feeder non-blocking
>>>>>>>>>>>> stuff for now, because of how the multipart request in handled in AHC
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Here's my feeding method:
>>>>>>>>>>>>
>>>>>>>>>>>> public void feed() throws IOException {
>>>>>>>>>>>> Part[] partsArray = parts.toArray(new Part[parts.size()]);
>>>>>>>>>>>> try ( OutputStream outputStream =
>>>>>>>>>>>> createFeedingOutputStream() ) {
>>>>>>>>>>>> Part.sendParts(outputStream,partsArray,multipartBoundary);
>>>>>>>>>>>> } catch (Exception e) {
>>>>>>>>>>>> throw new IllegalStateException("Unable to feed the
>>>>>>>>>>>> FeedableBodyGenerator",e);
>>>>>>>>>>>> }
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> As you can see, the multipart Parts array can only be pushed to
>>>>>>>>>>>> the OutputStream, I don't have any way to "pull" the data when the
>>>>>>>>>>>> canFeed() method is triggered.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> But I've seen that there's
>>>>>>>>>>>> a com.ning.http.multipart.MultipartBody#read that seems to provide a memory
>>>>>>>>>>>> efficient way to pull data from a Multipart body...
>>>>>>>>>>>>
>>>>>>>>>>>> Should see what I come up with this
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2013/9/10 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>
>>>>>>>>>>>>> It seems the Feeder is highly recommended but not mandatory so
>>>>>>>>>>>>> I tried without.
>>>>>>>>>>>>>
>>>>>>>>>>>>> With my existing code it seems there is a synchronization
>>>>>>>>>>>>> problem.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> The feeding threads get locked to the prematureFeed.get();
>>>>>>>>>>>>>
>>>>>>>>>>>>> So the Grizzly kernel threads are unable to acquire the lock
>>>>>>>>>>>>> required to enter the initializeAsynchronousTransfer method
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Will try with an implementation of Feeder
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2013/9/10 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hmmm it seems I have a problem with one of your maven
>>>>>>>>>>>>>> plugins. I'll try to bypass it, but for info:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ➜ ahc2 git:(ahc-1.7.x) mvn clean install
>>>>>>>>>>>>>> [WARNING]
>>>>>>>>>>>>>> [WARNING] Some problems were encountered while building the
>>>>>>>>>>>>>> effective settings
>>>>>>>>>>>>>> [WARNING] 'profiles.profile[default].
>>>>>>>>>>>>>> repositories.repository.id' must be unique but found
>>>>>>>>>>>>>> duplicate repository with id fullsix-maven-repository @
>>>>>>>>>>>>>> /home/slorber/.m2/settings.xml
>>>>>>>>>>>>>> [WARNING]
>>>>>>>>>>>>>> [INFO] Scanning for projects...
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>>> [INFO] Building Asynchronous Http Client 1.7.20-SNAPSHOT
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
>>>>>>>>>>>>>> async-http-client ---
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> [INFO] --- maven-enforcer-plugin:1.0-beta-1:enforce
>>>>>>>>>>>>>> (enforce-maven) @ async-http-client ---
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> [INFO] --- maven-enforcer-plugin:1.0-beta-1:enforce
>>>>>>>>>>>>>> (enforce-versions) @ async-http-client ---
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> [INFO] --- maven-resources-plugin:2.4.3:resources
>>>>>>>>>>>>>> (default-resources) @ async-http-client ---
>>>>>>>>>>>>>> [INFO] Using 'UTF-8' encoding to copy filtered resources.
>>>>>>>>>>>>>> [INFO] skip non existing resourceDirectory
>>>>>>>>>>>>>> /home/slorber/Bureau/ahc2/src/main/resources
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> [INFO] --- maven-compiler-plugin:2.3.2:compile
>>>>>>>>>>>>>> (default-compile) @ async-http-client ---
>>>>>>>>>>>>>> [INFO] Compiling 158 source files to
>>>>>>>>>>>>>> /home/slorber/Bureau/ahc2/target/classes
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> *[INFO] --- animal-sniffer-maven-plugin:1.6:check
>>>>>>>>>>>>>> (check-java-1.5-compat) @ async-http-client ---*
>>>>>>>>>>>>>> *[INFO] Checking unresolved references to
>>>>>>>>>>>>>> org.codehaus.mojo.signature:java15:1.0*
>>>>>>>>>>>>>> *[ERROR] Undefined reference:
>>>>>>>>>>>>>> java/io/IOException.<init>(Ljava/lang/Throwable;)V in
>>>>>>>>>>>>>> /home/slorber/Bureau/ahc2/target/classes/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.class*
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>>> [INFO] BUILD FAILURE
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>>> [INFO] Total time: 8.747s
>>>>>>>>>>>>>> [INFO] Finished at: Tue Sep 10 11:25:41 CEST 2013
>>>>>>>>>>>>>> [INFO] Final Memory: 30M/453M
>>>>>>>>>>>>>> [INFO]
>>>>>>>>>>>>>> ------------------------------------------------------------------------
>>>>>>>>>>>>>> *[ERROR] Failed to execute goal
>>>>>>>>>>>>>> org.codehaus.mojo:animal-sniffer-maven-plugin:1.6:check
>>>>>>>>>>>>>> (check-java-1.5-compat) on project async-http-client: Signature errors
>>>>>>>>>>>>>> found. Verify them and put @IgnoreJRERequirement on them. -> [Help 1]*
>>>>>>>>>>>>>> [ERROR]
>>>>>>>>>>>>>> [ERROR] To see the full stack trace of the errors, re-run
>>>>>>>>>>>>>> Maven with the -e switch.
>>>>>>>>>>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug
>>>>>>>>>>>>>> logging.
>>>>>>>>>>>>>> [ERROR]
>>>>>>>>>>>>>> [ERROR] For more information about the errors and possible
>>>>>>>>>>>>>> solutions, please read the following articles:
>>>>>>>>>>>>>> [ERROR] [Help 1]
>>>>>>>>>>>>>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2013/9/10 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ok thank you, I'll try to implement that today and will give
>>>>>>>>>>>>>>> you my feedback :)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2013/9/10 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Okay,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've committed my initial changes to the AHC repository.
>>>>>>>>>>>>>>>> Here's a summary of the changes:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Improvements to the FeedableBodyGenerator (Grizzly's).-
>>>>>>>>>>>>>>>> Don't allow queueing of data before initiateAsyncTransfer has been invoked.
>>>>>>>>>>>>>>>> In low memoryheaps, this could lead to an OOM if the source is feeding too
>>>>>>>>>>>>>>>> fast. The new behavior is toblock until initiateAsyncTransfer is called, at
>>>>>>>>>>>>>>>> which time the blocked thread may proceed withthe feed operation.-
>>>>>>>>>>>>>>>> Introduce the concept of a Feeder. Implementations are responsible, at a
>>>>>>>>>>>>>>>> high level, for:+ letting the provider know that data is available to be
>>>>>>>>>>>>>>>> fed without blocking+ allowing the registration of a callback that the
>>>>>>>>>>>>>>>> Feeder implementation may invoketo signal that more data is available, if
>>>>>>>>>>>>>>>> it wasn't available at a previous point in time.- When using a Feeder with
>>>>>>>>>>>>>>>> a secure request, the SSL handshake will be kicked off by
>>>>>>>>>>>>>>>> theinitiateAsyncTransfer call, but feeding of data will not occur until the
>>>>>>>>>>>>>>>> handshake is complete.This is necessary as the SSLFilter will queue up all
>>>>>>>>>>>>>>>> writes until the handshake is complete,and currently, the buffer isn't tied
>>>>>>>>>>>>>>>> in with the transport flow control mechanism.NOTE: This new SSL behavior is
>>>>>>>>>>>>>>>> not currently applied when invoking the feed() methodoutside the context of
>>>>>>>>>>>>>>>> a Feeder. Still need to address that.- Exposed configuration of the async
>>>>>>>>>>>>>>>> write queue limit through the FeedableBodyGenerator.This is an improvement
>>>>>>>>>>>>>>>> on using a TransportCustomizer as any configuration there istransport-wide,
>>>>>>>>>>>>>>>> and therefor applied to all Connections. By exposing it here, each
>>>>>>>>>>>>>>>> feedermay have a different byte limit.- Improved documentation for this
>>>>>>>>>>>>>>>> class*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I recommend reading through the javadoc comments in the
>>>>>>>>>>>>>>>> source [1] for FeedableBodyGenerator (comments welcome).
>>>>>>>>>>>>>>>> Additionally, I would re-work your code to leverage the
>>>>>>>>>>>>>>>> Feeder instead of calling feed() directly.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If you have issues implementing Feeder, do let us know.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If you have additional questions, again, let us know.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> -rl
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/ahc-1.7.x/src/main/java/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.java
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So in the end I've end up with an implementation that's
>>>>>>>>>>>>>>>> working for me.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I think there are 2 bugs:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) The bytes can accumulate in the FeedableBodyGenerator
>>>>>>>>>>>>>>>> queue if the initialize(ctx) method is not called fast enough.
>>>>>>>>>>>>>>>> This can be solved by using a BlockingQueue of size 1 and
>>>>>>>>>>>>>>>> the put() method.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2) Once the context is injected, the FeedableBodyGenerator
>>>>>>>>>>>>>>>> flushes the queue.
>>>>>>>>>>>>>>>> The matter is that if the connection is new, not warmed up
>>>>>>>>>>>>>>>> by a previous request, then the SSL handshake is not done yet, and it seems
>>>>>>>>>>>>>>>> that the bytes are accumulated in some part of the SSL filter which doesn't
>>>>>>>>>>>>>>>> deliver them to the connection until the handshake has completed,
>>>>>>>>>>>>>>>> so c.canWrite() continues to return true.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have replaced some part of the FeedableBodyGenerator to
>>>>>>>>>>>>>>>> test this and it works pretty fine. See what I have changed:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1)
>>>>>>>>>>>>>>>> private final BlockingQueue<BodyPart> queue = new
>>>>>>>>>>>>>>>> LinkedBlockingQueue<BodyPart>(1);
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2)
>>>>>>>>>>>>>>>> public void feed(final Buffer buffer, final boolean
>>>>>>>>>>>>>>>> last) throws IOException {
>>>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>>>> queue.put(new BodyPart(buffer, last));
>>>>>>>>>>>>>>>> } catch (InterruptedException e) {
>>>>>>>>>>>>>>>> throw new RuntimeException(e);
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> queueSize.incrementAndGet();
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> if (context != null) {
>>>>>>>>>>>>>>>> blockUntilConnectionIsReadyToWrite(context);
>>>>>>>>>>>>>>>> flushQueue(true);
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> private void
>>>>>>>>>>>>>>>> blockUntilConnectionIsReadyToWrite(FilterChainContext fcc) {
>>>>>>>>>>>>>>>> if ( !connectionIsReadyToWrite(fcc) ) {
>>>>>>>>>>>>>>>> while ( !connectionIsReadyToWrite(fcc) ) {
>>>>>>>>>>>>>>>> try { Thread.sleep(10); } catch ( Exception e ) {
>>>>>>>>>>>>>>>> throw new RuntimeException(e); }
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> private boolean
>>>>>>>>>>>>>>>> connectionIsReadyToWrite(FilterChainContext fcc) {
>>>>>>>>>>>>>>>> Connection connection = fcc.getConnection();
>>>>>>>>>>>>>>>> SSLEngine sslEngine =
>>>>>>>>>>>>>>>> SSLUtils.getSSLEngine(connection);
>>>>>>>>>>>>>>>> return sslEngine != null &&
>>>>>>>>>>>>>>>> !SSLUtils.isHandshaking(sslEngine);
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We had come to similar conclusions on this end. I'm still
>>>>>>>>>>>>>>>> working through testing the idea I mentioned previously (took longer than I
>>>>>>>>>>>>>>>> expected - sorry).
>>>>>>>>>>>>>>>> I hope to have something for you to test very soon.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Note that it will be taking the above into account as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have tried to put a while ( context == null )
>>>>>>>>>>>>>>>>> Thread.sleep but it doesn't seem to work, when the context gets injected,
>>>>>>>>>>>>>>>>> after the sleeps, there's an OOM
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So I hope you'll have more success with your alternative :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have done another test, remember my code that worked,
>>>>>>>>>>>>>>>>> which previously "warmed" the Thread with an useless request.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> private Runnable uploadSingleDocumentRunnable = new
>>>>>>>>>>>>>>>>> Runnable() {
>>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>>> public void run() {
>>>>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>>>>> getUselessSessionCode();
>>>>>>>>>>>>>>>>> Thread.sleep(X);
>>>>>>>>>>>>>>>>> uploadSingleDocument();
>>>>>>>>>>>>>>>>> } catch ( Exception e ) {
>>>>>>>>>>>>>>>>> throw new RuntimeException("file upload failed",e);
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>> };
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I have put a sleep of X between the useless warmup
>>>>>>>>>>>>>>>>> request, and the real upload request
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> What I noticed is that there is a very different behavior
>>>>>>>>>>>>>>>>> according to the value of X
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Under 10 seconds, it seems the stuff is still warm, I can
>>>>>>>>>>>>>>>>> upload the documents.
>>>>>>>>>>>>>>>>> Around 10 seconds I get a stack which seems to be
>>>>>>>>>>>>>>>>> "connection closed" or something
>>>>>>>>>>>>>>>>> Above 10 seconds, I get OOM like if the stuff wasn't warm.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The stacks I get for 10 seconds looks like
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Caused by: javax.net.ssl.SSLException: SSLEngine is CLOSED
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:295)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:133)
>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:95)
>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-SNAPSHOT.jar:na]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I think I got some other different stacks saying
>>>>>>>>>>>>>>>>> Connection Closed Remotely or something like that.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So it seems that something is bound to my thread, and it
>>>>>>>>>>>>>>>>> stays bound to it for about 10 seconds, do you have any idea what it could
>>>>>>>>>>>>>>>>> be?
>>>>>>>>>>>>>>>>> (My connection timeout setting seems to have no effect on
>>>>>>>>>>>>>>>>> this 10s threshold)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2013/9/5 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> That is one solution. I'm working out an alternative
>>>>>>>>>>>>>>>>>> right now. Stay tuned!
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Anyway it's not a problem, I think the
>>>>>>>>>>>>>>>>>> FeedableBodyGenerator.feed() method just has to block until a context has
>>>>>>>>>>>>>>>>>> been (and ThreadCache initialized) to avoid OOM errors
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What is very strange is that I tested with/without the
>>>>>>>>>>>>>>>>>>> same sessionCode with our previous code, the one not using
>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator, which has a high memory consumption.
>>>>>>>>>>>>>>>>>>> Despites the fact it had high memory consumption, it
>>>>>>>>>>>>>>>>>>> seems work fine to upload multiple documents if allocated with a large
>>>>>>>>>>>>>>>>>>> heap, and the sessionCode seems to have no effect.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On the new impl using the FeedableBodyGenerator, the
>>>>>>>>>>>>>>>>>>> sessionCode sent as a multipart bodypart seems to have an effect.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I have tried to feed the queue before sending the
>>>>>>>>>>>>>>>>>>> request to AHC, but this leads to this exception (with/without sessionCode
>>>>>>>>>>>>>>>>>>> switching)
>>>>>>>>>>>>>>>>>>> Caused by: java.util.concurrent.TimeoutException:
>>>>>>>>>>>>>>>>>>> Timeout exceeded
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider.timeout(GrizzlyAsyncHttpProvider.java:528)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$3.onTimeout(GrizzlyAsyncHttpProvider.java:361)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:383)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.utils.IdleTimeoutFilter$DefaultWorker.doWork(IdleTimeoutFilter.java:362)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.utils.DelayedExecutor$DelayedRunnable.run(DelayedExecutor.java:158)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> By the way, by using a low timeout with the same
>>>>>>>>>>>>>>>>>>>> sessioncode, I got the following NPE:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Caused by: java.lang.NullPointerException
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:184)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> GrizzlyAsyncHttpProvider.HttpTransactionContext httpCtx =
>>>>>>>>>>>>>>>>>>>> getHttpTransactionContext(c);
>>>>>>>>>>>>>>>>>>>> httpCtx.abort(e.getCause());
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I guess the httpCtx is not already available to be
>>>>>>>>>>>>>>>>>>>> aborted
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 2013/9/5 Sébastien Lorber <lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This is right, here's a log I have when I use the same
>>>>>>>>>>>>>>>>>>>>> session code, ie the remote host is blocking the data or something.
>>>>>>>>>>>>>>>>>>>>> This is obtained by running 5 parallel uploads.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 0 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 97 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 100 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 160 with allowBlocking = true*
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 0 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> java.lang.OutOfMemoryError: GC overhead limit exceeded
>>>>>>>>>>>>>>>>>>>>> Dumping heap to /ome/lorber/ureau/om ...
>>>>>>>>>>>>>>>>>>>>> Unable to create /ome/lorber/ureau/om: Le fichier
>>>>>>>>>>>>>>>>>>>>> existe
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Disconnected from the target VM, address: '
>>>>>>>>>>>>>>>>>>>>> 127.0.0.1:49268', transport: 'socket'
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Otherwise, with different session codes, I get the
>>>>>>>>>>>>>>>>>>>>> following:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 0 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 0 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 0 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> *Flushing queue of size 0 with allowBlocking = false*
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> Flushing queue of size 1 with allowBlocking = true
>>>>>>>>>>>>>>>>>>>>> ... and this continues without OOM
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> So, this seems to be the problem.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think it would be great to be able to be able to
>>>>>>>>>>>>>>>>>>>>> choose the queue impl behind that FeedableBodyGenerator, like I suggested
>>>>>>>>>>>>>>>>>>>>> in my pull request.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> See here:
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> https://github.com/slorber/async-http-client/blob/79b0c3b28a61b0aa4c4b055bca8f6be11d9ed1e6/src/main/java/com/ning/http/client/providers/grizzly/FeedableBodyGenerator.java
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Using a LinkedBlockingQueue seems to be a nice idea in
>>>>>>>>>>>>>>>>>>>>> this context, and in my case I would probably use a queue of size 1
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This would handle the blocking of the feed method,
>>>>>>>>>>>>>>>>>>>>> without having to use this:
>>>>>>>>>>>>>>>>>>>>> if (context != null) {
>>>>>>>>>>>>>>>>>>>>> flushQueue(true);
>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Or perhaps the feed() method have to wait until a
>>>>>>>>>>>>>>>>>>>>> context is set in the BodyGenerator ?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I think it would be more clear if
>>>>>>>>>>>>>>>>>>>>> the initializeAsynchronousTransfer simply didn't flush the queue but just
>>>>>>>>>>>>>>>>>>>>> setup the context.
>>>>>>>>>>>>>>>>>>>>> Then the feed method would block until there's a
>>>>>>>>>>>>>>>>>>>>> context set, and then flush the queue with blocking behavior.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This is probably the next step, but as we are using
>>>>>>>>>>>>>>>>>>>>> AHC for async, it would probably be great if that blocking feed() method
>>>>>>>>>>>>>>>>>>>>> was called in a worker thread instead of our main thread.
>>>>>>>>>>>>>>>>>>>>> I won't use this but someone who really wants a
>>>>>>>>>>>>>>>>>>>>> non-blocking impl of performant multipart fileupload would probably need
>>>>>>>>>>>>>>>>>>>>> this, or will use an ExecutorService for the feeding operations as a
>>>>>>>>>>>>>>>>>>>>> workaround.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Thanks again for your reactivity
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2013/9/4 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I've integrated this change and it works fine except
>>>>>>>>>>>>>>>>>>>>>> a little detail.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I'm uploading files to a third party API (a bit like
>>>>>>>>>>>>>>>>>>>>>> S3).
>>>>>>>>>>>>>>>>>>>>>> This API requires a "sessionCode" in each request. So
>>>>>>>>>>>>>>>>>>>>>> there is a multipart StringPart with that SessionCode.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> We used to have a cache which holds the sessionCode
>>>>>>>>>>>>>>>>>>>>>> 30min per user so that we do not need to init a new session each time.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I had troubles in this very specific case: when I
>>>>>>>>>>>>>>>>>>>>>> upload 5 docs with the same session code.
>>>>>>>>>>>>>>>>>>>>>> When I remove the cache and use 5 different session
>>>>>>>>>>>>>>>>>>>>>> codes, it works fine.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> I guess the remote service is blocking concurrent
>>>>>>>>>>>>>>>>>>>>>> uploads with the same session code. I don't know at all.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Where I want to go is that I wouldn't have expected
>>>>>>>>>>>>>>>>>>>>>> Grizzly to OOM
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Avertissement: Exception during FilterChain execution
>>>>>>>>>>>>>>>>>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>>>>>>>>>>>>>>>>>>>>> at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:238)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:405)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:320)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:263)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2500)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext$1.run(FilterChainContext.java:196)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.resume(FilterChainContext.java:220)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter$SSLHandshakeContext.completed(SSLFilter.java:383)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.notifyHandshakeComplete(SSLFilter.java:278)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleRead(SSLBaseFilter.java:275)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleRead(GrizzlyAsyncHttpProvider.java:2490)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:546)
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:113)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Caused by: java.util.concurrent.TimeoutException: null
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl$Sync.innerGet(SafeFutureImpl.java:367)
>>>>>>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.impl.SafeFutureImpl.get(SafeFutureImpl.java:274)
>>>>>>>>>>>>>>>>>>>>>> ~[grizzly-framework-2.3.5.jar:2.3.5]
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.block(FeedableBodyGenerator.java:177)
>>>>>>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.blockUntilQueueFree(FeedableBodyGenerator.java:167)
>>>>>>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:124)
>>>>>>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:94)
>>>>>>>>>>>>>>>>>>>>>> ~[async-http-client-1.7.20-204092c.jar:na]
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> multipart.body.generator.feeder.buffer=100000 -> size
>>>>>>>>>>>>>>>>>>>>>> of each Buffer sent to the FeedableBodyGenerator
>>>>>>>>>>>>>>>>>>>>>> transport.max.pending.bytes=1000000
>>>>>>>>>>>>>>>>>>>>>> (I tried other settings, including AUTO_SIZE)
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Do you have any idea why is there an OOM with these
>>>>>>>>>>>>>>>>>>>>>> settings?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Perhaps it is because the feed() method of
>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator doesn't block until the context is initialized.
>>>>>>>>>>>>>>>>>>>>>> I guess the initializeAsynchronousTransfer is called
>>>>>>>>>>>>>>>>>>>>>> only once the connection is established, and perhaps a lot of Buffer are
>>>>>>>>>>>>>>>>>>>>>> added to the queue...
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Yes, it's invoked once the request has been
>>>>>>>>>>>>>>>>>>>>>> dispatched, so if the generator is fed a lot before the request, I could
>>>>>>>>>>>>>>>>>>>>>> see this happening.
>>>>>>>>>>>>>>>>>>>>>> I'll see what I can do to alleviate that case.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> But I'm not sure at all because the session code is
>>>>>>>>>>>>>>>>>>>>>> transmitted as a BodyPart and I get the same problem if i put it as the
>>>>>>>>>>>>>>>>>>>>>> first or last multipart.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> It's not a big deal, perhaps I should always use a
>>>>>>>>>>>>>>>>>>>>>> different session code for concurrent operations but I'd like to be sure
>>>>>>>>>>>>>>>>>>>>>> that we won't have this issue in production...
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> 2013/9/3 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Good catch. Fixed.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> There's a little mistake in the grizzly ahc provider
>>>>>>>>>>>>>>>>>>>>>>> relative to the write queue size.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/b5d97efe9fe14113ea92fb1f7db192a2d090fad7/src/main/java/com/ning/http/client/providers/grizzly/GrizzlyAsyncHttpProvider.java#L419
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> As you can see, the TransportCustomizer is called,
>>>>>>>>>>>>>>>>>>>>>>> and then the write queue size (among other things) is set to AUTO_SIZE
>>>>>>>>>>>>>>>>>>>>>>> (instead of previously UNLIMITED)
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> clientTransport.getAsyncQueueIO().getWriter()
>>>>>>>>>>>>>>>>>>>>>>> .setMaxPendingBytesPerConnection(AsyncQueueWriter.AUTO_SIZE);
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I think the default settings like this AUTO_SIZE
>>>>>>>>>>>>>>>>>>>>>>> attribute should be set before the customization of the transport, or they
>>>>>>>>>>>>>>>>>>>>>>> would override the value we customized.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> This is actually my case, since I can't reproduce my
>>>>>>>>>>>>>>>>>>>>>>> "bug" which is "high memory consumption", even when using -1 / UNLIMITED in
>>>>>>>>>>>>>>>>>>>>>>> the TransportCustomizer.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> This could work fine for me with AUTO_SIZE, but I'd
>>>>>>>>>>>>>>>>>>>>>>> rather be able to tune this parameter during load tests to see the effect.
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> 2013/8/31 Sebastien Lorber <
>>>>>>>>>>>>>>>>>>>>>>> lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Thanks i will ckeck that on monday. I can now
>>>>>>>>>>>>>>>>>>>>>>>> upload a 500m file with 40m heap size ;)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Envoyé de mon iPhone
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Le 30 août 2013 à 20:49, Ryan Lubke <
>>>>>>>>>>>>>>>>>>>>>>>> ryan.lubke_at_oracle.com> a écrit :
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> I'm going to be updating the Grizzly provider such
>>>>>>>>>>>>>>>>>>>>>>>> that AUTO_SIZE (not AUTO_TUNE) is the default, so you can avoid the use of
>>>>>>>>>>>>>>>>>>>>>>>> the TransportCustomizer.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Regarding your tuning question, I would probably
>>>>>>>>>>>>>>>>>>>>>>>> set the value to AsyncQueueWriter.AUTO_TUNE (this will be four times the
>>>>>>>>>>>>>>>>>>>>>>>> socket write buffer) and see how that works.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> A question first. With these changes, your memory
>>>>>>>>>>>>>>>>>>>>>>>> usage is more inline with what you were looking for?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> By the way, do you have any idea when the 1.7.20
>>>>>>>>>>>>>>>>>>>>>>>> will be released (with these new improvements?)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We would like to know if we wait for a release or
>>>>>>>>>>>>>>>>>>>>>>>> if we install our own temp release on Nexus :)
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/30 Sébastien Lorber <
>>>>>>>>>>>>>>>>>>>>>>>> lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Thank you, it works fine!
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> I just had to modify a single line after your
>>>>>>>>>>>>>>>>>>>>>>>>> commit.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider#initializeTransport
>>>>>>>>>>>>>>>>>>>>>>>>> ->
>>>>>>>>>>>>>>>>>>>>>>>>> clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(10000);
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> If I let the initial value (-1) it won't block,
>>>>>>>>>>>>>>>>>>>>>>>>> canWrite always returns true
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> Btw, on AHC I didn't find any way to pass this
>>>>>>>>>>>>>>>>>>>>>>>>> value as a config attribute, neither the size of the write buffer you
>>>>>>>>>>>>>>>>>>>>>>>>> talked about.
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> So in the end, is there a way with current AHC
>>>>>>>>>>>>>>>>>>>>>>>>> code to use this "canWrite = false" behavior?
>>>>>>>>>>>>>>>>>>>>>>>>> If not, can you please provide a way to set this
>>>>>>>>>>>>>>>>>>>>>>>>> behavior on v1.7.20 ? thanks
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> PS: does it make sens to use the same number of
>>>>>>>>>>>>>>>>>>>>>>>>> bytes un the feed(Buffer) method and in the setMaxPendingBytesPerConnection(10000);
>>>>>>>>>>>>>>>>>>>>>>>>> ? do you have any tuning recommandation?
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/29 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Please disregard.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Could you also provide a sample of how you're
>>>>>>>>>>>>>>>>>>>>>>>>>> performing your feed?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>>>>>>>>>>>> -rl
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Ryan Lubke wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien,
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'd recommend looking at Connection.canWrite()
>>>>>>>>>>>>>>>>>>>>>>>>>> [1] and Connection.notifyCanWrite(WriteListener) [1]
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> By default, Grizzly will configure the async
>>>>>>>>>>>>>>>>>>>>>>>>>> write queue length to be four times the write buffer size (which is based
>>>>>>>>>>>>>>>>>>>>>>>>>> off the socket write buffer).
>>>>>>>>>>>>>>>>>>>>>>>>>> When this queue exceeds this value, canWrite()
>>>>>>>>>>>>>>>>>>>>>>>>>> will return false.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> When this occurs, you can register a
>>>>>>>>>>>>>>>>>>>>>>>>>> WriteListener to be notified when the queue length is below the configured
>>>>>>>>>>>>>>>>>>>>>>>>>> max and then simulate blocking
>>>>>>>>>>>>>>>>>>>>>>>>>> until the onWritePossible() callback has been
>>>>>>>>>>>>>>>>>>>>>>>>>> invoked.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> ----------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> final FutureImpl<Boolean> future =
>>>>>>>>>>>>>>>>>>>>>>>>>> Futures.createSafeFuture();
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> // Connection may be obtained by calling
>>>>>>>>>>>>>>>>>>>>>>>>>> FilterChainContext.getConnection().
>>>>>>>>>>>>>>>>>>>>>>>>>> connection.notifyCanWrite(new
>>>>>>>>>>>>>>>>>>>>>>>>>> WriteHandler() {
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>>>>>>>>>>>> public void onWritePossible() throws
>>>>>>>>>>>>>>>>>>>>>>>>>> Exception {
>>>>>>>>>>>>>>>>>>>>>>>>>> future.result(Boolean.TRUE);
>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> @Override
>>>>>>>>>>>>>>>>>>>>>>>>>> public void onError(Throwable t) {
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> future.failure(Exceptions.makeIOException(t));
>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>> });
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> try {
>>>>>>>>>>>>>>>>>>>>>>>>>> final long writeTimeout = 30;
>>>>>>>>>>>>>>>>>>>>>>>>>> future.get(writeTimeout,
>>>>>>>>>>>>>>>>>>>>>>>>>> TimeUnit.MILLISECONDS);
>>>>>>>>>>>>>>>>>>>>>>>>>> } catch (ExecutionException e) {
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpTransactionContext httpCtx =
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpTransactionContext.get(connection);
>>>>>>>>>>>>>>>>>>>>>>>>>> httpCtx.abort(e.getCause());
>>>>>>>>>>>>>>>>>>>>>>>>>> } catch (Exception e) {
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpTransactionContext httpCtx =
>>>>>>>>>>>>>>>>>>>>>>>>>> HttpTransactionContext.get(connection);
>>>>>>>>>>>>>>>>>>>>>>>>>> httpCtx.abort(e);
>>>>>>>>>>>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>>>>> http://grizzly.java.net/docs/2.3/apidocs/org/glassfish/grizzly/OutputSink.html
>>>>>>>>>>>>>>>>>>>>>>>>>> .
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> Ryan, I've did some other tests.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> It seems that using a blocking queue in the
>>>>>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator is totally useless because the thread consuming it is
>>>>>>>>>>>>>>>>>>>>>>>>>> not blocking and the queue never blocks the feeding, which was my intention
>>>>>>>>>>>>>>>>>>>>>>>>>> in the beginning. Maybe it depends on the IO strategy used?
>>>>>>>>>>>>>>>>>>>>>>>>>> I use AHC default which seems to
>>>>>>>>>>>>>>>>>>>>>>>>>> use SameThreadIOStrategy so I don't think it's related to the IO strategy.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> So in the end I can upload a 70m file with a heap
>>>>>>>>>>>>>>>>>>>>>>>>>> of 50m, but I have to put a Thread.sleep(30) between each 100k Buffer send
>>>>>>>>>>>>>>>>>>>>>>>>>> to the FeedableBodyGenerator
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> The connection with the server is not good here,
>>>>>>>>>>>>>>>>>>>>>>>>>> but in production it is normally a lot better as far as I know.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I've tried things
>>>>>>>>>>>>>>>>>>>>>>>>>> like clientTransport.getAsyncQueueIO().getWriter().setMaxPendingBytesPerConnection(100000);
>>>>>>>>>>>>>>>>>>>>>>>>>> but it doesn't seem to work for me.
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> I'd like the Grizzly internals to block when
>>>>>>>>>>>>>>>>>>>>>>>>>> there are too much pending bytes to send. Is it possible?
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> PS: I've just been able to send a 500mo file with
>>>>>>>>>>>>>>>>>>>>>>>>>> 100mo heap, but it needed a sleep of 100ms between each 100k Buffer sent to
>>>>>>>>>>>>>>>>>>>>>>>>>> the bodygenerator
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/29 Sébastien Lorber <
>>>>>>>>>>>>>>>>>>>>>>>>>> lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> By chance do you if I can remove the
>>>>>>>>>>>>>>>>>>>>>>>>>>> MessageCloner used in the SSL filter?
>>>>>>>>>>>>>>>>>>>>>>>>>>> SSLBaseFilter$OnWriteCopyCloner
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> It seems to allocate a lot of memory.
>>>>>>>>>>>>>>>>>>>>>>>>>>> I don't really understand why messages have to
>>>>>>>>>>>>>>>>>>>>>>>>>>> be cloned, can I remove this? How?
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/29 Sébastien Lorber <
>>>>>>>>>>>>>>>>>>>>>>>>>>> lorber.sebastien_at_gmail.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm trying to send a 500m file for my tests
>>>>>>>>>>>>>>>>>>>>>>>>>>>> with a heap of 400m.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> In our real use cases we would probably have
>>>>>>>>>>>>>>>>>>>>>>>>>>>> files under 20mo but we want to reduce the memory consumption because we
>>>>>>>>>>>>>>>>>>>>>>>>>>>> can have x parallel uploads on the same server according to the user
>>>>>>>>>>>>>>>>>>>>>>>>>>>> activity.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'll try to check if using this BodyGenerator
>>>>>>>>>>>>>>>>>>>>>>>>>>>> reduced the memory footprint or if it's almost like before.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/28 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> At this point in time, as far as the SSL
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> buffer allocation is concerned, it's untunable.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> That said, feel free to open a feature request.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> As to your second question, there is no
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> suggested size. This is all very application specific.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'm curious, how large of a file are you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> sending?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I have seen a lot of buffers which have a size
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> of 33842 and it seems the limit is near half the capacity.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Perhaps there's a way to tune that buffer size
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so that it consumes less memory?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is there an ideal Buffer size to send to the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feed method?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/28 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'll be reviewing the PR today, thanks again!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Regarding the OOM: as it stands now, for each
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new buffer that is passed to the SSLFilter, we allocate a buffer twice the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> size in order to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> accommodate the encrypted result. So there's
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> an increase.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Depending on the socket configurations of
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> both endpoints, and how fast the remote is reading data, it could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> be the write queue is becoming too large.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> We do have a way to detect this situation, but I'm pretty sure
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the Grizzly internals are currently shielded
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> here. I will see what I can do to allow users to leverage this.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I've made my pull request.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/pull/367
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> With my usecase it works, the file is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> uploaded like before.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But I didn't notice a big memory improvement.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Is it possible that SSL doesn't allow to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> stream the body or something like that?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> In memory, I have a lot of:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> - HeapByteBuffer
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which are hold by SSLUtils$3
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which are hold by BufferBuffers
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which are hold by WriteResult
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Which are hold by AsyncWriteQueueRecord
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Here is an exemple of the OOM stacktrace:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.lang.OutOfMemoryError: Java heap space
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLUtils.allocateOutputBuffer(SSLUtils.java:342)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter$2.grow(SSLBaseFilter.java:117)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.ensureBufferSize(SSLConnectionContext.java:392)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrap(SSLConnectionContext.java:272)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLConnectionContext.wrapAll(SSLConnectionContext.java:227)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.wrapAll(SSLBaseFilter.java:404)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLBaseFilter.handleWrite(SSLBaseFilter.java:319)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.accurateWrite(SSLFilter.java:255)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ssl.SSLFilter.handleWrite(SSLFilter.java:143)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.GrizzlyAsyncHttpProvider$SwitchingSSLFilter.handleWrite(GrizzlyAsyncHttpProvider.java:2503)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:111)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:288)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:206)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:136)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:114)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:853)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> org.glassfish.grizzly.filterchain.FilterChainContext.write(FilterChainContext.java:720)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.flushQueue(FeedableBodyGenerator.java:132)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.FeedableBodyGenerator.feed(FeedableBodyGenerator.java:101)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder$FeedBodyGeneratorOutputStream.write(MultipartBodyGeneratorFeeder.java:222)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> java.io.BufferedOutputStream.write(BufferedOutputStream.java:126)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.FilePart.sendData(FilePart.java:179)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.Part.send(Part.java:331)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.multipart.Part.sendParts(Part.java:397)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> at
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> com.ning.http.client.providers.grizzly.MultipartBodyGeneratorFeeder.feed(MultipartBodyGeneratorFeeder.java:144)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Any idea?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/27 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Excellent! Looking forward to the pull
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> request!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ryan thanks, it works fine, I'll make a pull
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> request on AHC tomorrow with a better code using the same Part classes that
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> already exist.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I created an OutputStream that redirects to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> the BodyGenerator feeder.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> The problem I currently have is that the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> feeder feeds the queue faster than the async thread polling it :)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I need to expose a limit to that queue size
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> or something, will work on that, it will be better than a thread sleep to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> slow down the filepart reading
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/27 Ryan Lubke <ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Yes, something like that. I was going to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> tackle adding something like this today. I'll follow up with something you
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> can test out.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Ok thanks!
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I think I see what I could do, probably
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> something like that:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> FeedableBodyGenerator bodyGenerator =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> new FeedableBodyGenerator();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> MultipartBodyGeneratorFeeder
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder = new MultipartBodyGeneratorFeeder(bodyGenerator);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Request uploadRequest1 = new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> RequestBuilder("POST")
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> .setUrl("url")
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> .setBody(bodyGenerator)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> .build();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> ListenableFuture<Response> asyncRes =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> asyncHttpClient
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> .prepareRequest(uploadRequest1)
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> .execute(new
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> AsyncCompletionHandlerBase());
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param1","value1");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("param2","value2");
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.append("fileToUpload",fileInputStream);
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> bodyGeneratorFeeder.end();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Response uploadResponse =
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> asyncRes.get();
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Does it seem ok to you?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I guess it could be interesting to provide
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that MultipartBodyGeneratorFeeder class to AHC or Grizzly since some other
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> people may want to achieve the same thing
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> 2013/8/26 Ryan Lubke
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> <ryan.lubke_at_oracle.com><ryan.lubke_at_oracle.com>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Sébastien Lorber wrote: Hello, I would
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> like to know if it's possible to upload a file with AHC / Grizzly in
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> streaming, I mean without loading the whole file bytes in memory. The
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> default behavior seems to allocate a byte[] which contans the whole file,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> so it means that my server can be OOM if too many users upload a large file
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> in the same time. I've tryied with a Heap and ByteBuffer memory managers,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> with reallocate=true/false but no more success. It seems the whole file
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> content is appended wto the BufferOutputStream, and then the underlying
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> buffer is written. At least this seems to be the case with AHC integration:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> https://github.com/AsyncHttpClient/async-http-client/blob/6faf1f316e5546110b0779a5a42fd9d03ba6bc15/providers/grizzly/src/main/java/org/asynchttpclient/providers/grizzly/bodyhandler/PartsBodyHandler.javaSo, is there a way to patch AHC to stream the file so that I could
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> eventually consume only 20mo of heap while uploading a 500mo file? Or is
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> this simply impossible with Grizzly? I didn't notice anything related to
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> that in the documentation. It's possible with the FeedableBodyGenerator.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> But if you're tied to using Multipart uploads, you'd have to convert the
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> multipart data to Buffers manually and send using the FeedableBodyGenerator.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> I'll take a closer look to see if this
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> area can be improved.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Btw in my case it is a file upload. I
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> receive a file with CXF and have to transmit it to a storage server (like
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> S3). CXF doesn't consume memory bevause it is streaming the large fle
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> uploads to the file system, and then provides an input stream on that file.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>