BTW, I now realize that you cannot do this with the limited Future in Java
5/6/7, you need CompletableFuture
<
http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html>
from
Java 8.
On Wed, Sep 17, 2014 at 2:10 PM, Mikael Ståldal <
mikael.staldal_at_appearnetworks.com> wrote:
> But wouldn't it be possible to do this with Java 8's CompletableFuture
> <http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletableFuture.html> only,
> why do we need Rx?
>
> On Tue, Sep 16, 2014 at 5:28 PM, Marek Potociar <marek.potociar_at_oracle.com
> > wrote:
>
>> Alas, Jersey managed async support has not been documented properly yet :(
>> Essentially, @ManagedAsync ensures that Jersey invokes the resource
>> method in a separate, custom executor service thread. By default a cached
>> thread pool
>> <http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Executors.html#newCachedThreadPool()> is
>> used to invoke the managed async methods, but you can customize it by
>> registering your own RequestExecutorProvider
>> <https://jersey.java.net/apidocs/latest/jersey/org/glassfish/jersey/spi/RequestExecutorProvider.html> in
>> your application. See also the managed async example
>> <https://github.com/jersey/jersey/tree/master/examples/server-async-managed/src/main/java/org/glassfish/jersey/examples/server/async/managed>
>> .
>>
>> Back to the Rx topic:
>> With our experimental Rx support, you can do something similar - notice
>> that we have a module supporting JDK 8 CompletionStage API too:
>> https://github.com/jersey/jersey/tree/master/incubator/rx/rx-client-java8
>>
>> So while you cannot just return a future from your resource method (at
>> this point), you can (for now) use the new Rx client API to resume the
>> injected AsyncResponse in your final completion stage implementation.
>> Something like:
>>
>> @Path("whatever")
>> public class FutureResource {
>>
>> @Uri("remote/destination")
>> private WebTarget destination;
>>
>> @GET
>> public void asyncMethod(@QueryParam("foo") String foo, @Suspended
>> AsyncResponse ar) {
>> RxCompletionStage.from(destination).path(resource).request().rx()
>> * // the get request runs in the ForkJoinPool.commonPool() by
>> default*
>> .get(String.class)
>> .whenComplete((r, t) -> {
>> if (t != null) ar.resume(t) else
>> ar.resume(r.toUpperCase());
>> });
>> }
>> }
>>
>>
>> Marek
>>
>> On 15 Sep 2014, at 13:41, Mikael Ståldal <
>> mikael.staldal_at_appearnetworks.com> wrote:
>>
>> I was not aware of the @ManagedAsync feature, it seems to be missing from
>> the user guide (https://jersey.java.net/documentation/latest/index.html).
>> How does it work?
>>
>> I was more thinking about a way to to something like this (require Java
>> 8, and that AsyncInvoker.get() return an CompletableFuture):
>>
>> @Path("whatever")
>> public class FutureResource {
>>
>> @Uri("remote/destination")
>> private WebTarget destination;
>>
>> public Future<String> asyncMethod(@QueryParam("foo") String foo) {
>> return
>> destination.path("resource").request().async().get().thenApplyAsync((r) ->
>> r.readEntity(String.class).toUpperCase()
>> );
>> }
>> }
>>
>>
>>
>>
>> On Fri, Sep 12, 2014 at 9:43 PM, Marek Potociar <
>> marek.potociar_at_oracle.com> wrote:
>>
>>> You mean something like this:
>>> https://github.com/jersey/jersey/tree/master/examples/rx-client-webapp
>>> ?
>>>
>>> The code is still steaming-fresh and I'm sure that Michal, who designed
>>> it will appreaciate any feedback... ;-)
>>>
>>> Marek
>>>
>>>
>>> On 07 Sep 2014, at 17:29, Mikael Ståldal <
>>> mikael.staldal_at_appearnetworks.com> wrote:
>>>
>>> Each and every operation async is similar to what's called Reactive
>>> programming (http://www.reactivemanifesto.org/).
>>>
>>> That would be facilitated by an easy way to connect an async method in
>>> Jersey-server with an async outbound call with Jersey-client.
>>>
>>> I guess what's needed is a way to automatically resume an async server
>>> method with the completion of a Future. Like what you can do with Play
>>> Framework.
>>>
>>> On Fri, Sep 5, 2014 at 8:16 PM, Kishore Senji <ksenji_at_gmail.com> wrote:
>>>
>>>> Thank you Marek.
>>>>
>>>> I agree to all your points and I did not say there is no advantage to
>>>> Async. I'm only referring to the example that users might think that
>>>> throughput would increase just by taking the processing to a different
>>>> thread. Throughput would only increase when that veryExpensiveOperation()
>>>> method is actually doing async IO. If it is only cpu bound, then yes the
>>>> container threads can take more requests (and they can also serve other
>>>> resource methods) but the requests to this resource method will still be
>>>> queued up and the worker threads are all busy working (spiking cpu) which
>>>> will impact the overall system performance. [Typically we have few methods
>>>> related to a domain deployed to a pool. For the client they can all be
>>>> under one end point, internally routed to the appropriate pool via ESB].
>>>> Even if the veryExpensiveOperation() is IO bound, the worker threads are
>>>> blocked waiting for the IO. This will queue up the tasks and the worker
>>>> threads cannot do any more work as they are blocked waiting for IO. This
>>>> pool of workers cannot be used for other resource methods (let us say they
>>>> also do async but have a different profile of relatively short cpu bound
>>>> tasks or quick IO) and they may have to be configured to use a different
>>>> thread pool etc.
>>>>
>>>> In short, only when each and every operation in the call stack is async
>>>> (servlet needs to be async capable, then the database driver needs to
>>>> support async or the service call this service makes needs to be done on
>>>> async io) then only we can have throughput benefits (and support same
>>>> volume of traffic with less vms) otherwise having async at one layer
>>>> (Jersey/servlet) will not help when the actual database/service call is
>>>> blocking.
>>>>
>>>> Thanks,
>>>> Kishore.
>>>>
>>>>
>>>> On Fri, Sep 5, 2014 at 9:30 AM, Marek Potociar <
>>>> marek.potociar_at_oracle.com> wrote:
>>>>
>>>>>
>>>>> On 04 Sep 2014, at 21:27, Kishore Senji <ksenji_at_gmail.com> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> The Async example is given at
>>>>> https://jersey.java.net/documentation/latest/async.html
>>>>>
>>>>> "However, in cases where a resource method execution is known to take
>>>>> a long time to compute the result, server-side asynchronous processing
>>>>> model should be used. In this model, the association between a request
>>>>> processing thread and client connection is broken. I/O container that
>>>>> handles incoming request may no longer assume that a client connection can
>>>>> be safely closed when a request processing thread returns. Instead a
>>>>> facility for explicitly suspending, resuming and closing client connections
>>>>> needs to be exposed. Note that the use of server-side asynchronous
>>>>> processing model will not improve the request processing time perceived by
>>>>> the client. *It will however increase the throughput of the server,
>>>>> by releasing the initial request processing thread back to the I/O
>>>>> container while the request may still be waiting in a queue for processing
>>>>> or the processing may still be running on another dedicated thread*.
>>>>> The released I/O container thread can be used to accept and process new
>>>>> incoming request connections."
>>>>>
>>>>> If veryExpensiveOperation() is expensive and is taking long time, then
>>>>> having it run in a different thread and releasing the request processing
>>>>> thread back to the I/O container, how would that improve the throughput?
>>>>>
>>>>>
>>>>> You are off-loading the I/O container threads, which are typically
>>>>> taken from a limited thread pool. If an I/O processing thread is blocked
>>>>> waiting, it cannot process new connections.
>>>>>
>>>>>
>>>>> If that is the case we can as well increase the number of request
>>>>> processing threads of the I/O container by the number of worker threads
>>>>> that we would use in the case of the example and not worry about Async at
>>>>> all.
>>>>>
>>>>>
>>>>> Please note that different resource methods may have different
>>>>> requirements. You typically want to configure your I/O thread pool size to
>>>>> match number of CPU cores (or sometimes CPU cores + c, where c is a
>>>>> constant < than number of cores). And then you want to make sure that only
>>>>> short computations are performed on these threads, so e.g. typically
>>>>> anything that may involve any I/O operation (disk, db, network) should
>>>>> better be coded as async, where thread context switch cost is offset by the
>>>>> overall operation cost (see also here
>>>>> <http://www.eecs.berkeley.edu/~rcs/research/interactive_latency.html>).
>>>>> Typically, also these operations tend to have specific execution
>>>>> characteristics, so a use of a dedicated thread pool with a separately
>>>>> tuned pool size is required to fine-tune the performance of the system.
>>>>>
>>>>> So advantage of using async API is that it gives you a much more
>>>>> fine-grained control over when the operation is delegated to a different
>>>>> thread pool as well as to which thread pool should the operation be
>>>>> delegated to, which is in contrast with your "one size fits all approach",
>>>>> which does nothing else then introduces the high probability of L1, L2 and
>>>>> L3 cache misses with every new request.
>>>>>
>>>>> We can take more and more connections and have them queue up (or would
>>>>> end up with creating many worker threads), but it would not necessarily
>>>>> increase throughput. It would increase throughput if the
>>>>> veryExpensiveOperation() is doing I/O over a Socket and if we use Async IO
>>>>> for that operation, then we can use minimal request threads and very small
>>>>> worker thread pool to do Async handling of the IO (or combine logic across
>>>>> multiple Service calls doing non-blocking IO, similar to Akka futures).
>>>>> This will improve the throughput as more work is done. But without
>>>>> non-blocking IO, if the veryExpensiveOperation() is either CPU bound or
>>>>> using blocking IO then the worker thread would infact be blocked for that
>>>>> time and we would end up with huge thread pool or a big queue of tasks
>>>>> waiting. Huge thread pool would not scale and big queue would also reduce
>>>>> the throughput.
>>>>>
>>>>>
>>>>> If you have an application, where the only service is the
>>>>> veryExpensiveOperation() resource method, then use of async is not likely
>>>>> to help. But frankly, how typical is that case? Often you have other
>>>>> services that would starve unnecessarily if you did not off-load the
>>>>> veryExpensiveOperation() to another thread pool.
>>>>>
>>>>>
>>>>> Nevertheless we definitely need a thread to take the processing to a
>>>>> different thread so that the container thread can be returned quickly. But
>>>>> is my understanding correct that it depends on what
>>>>> veryExpensiveOperation() does (blocking or non-blocking IO, or totally CPU
>>>>> bound computation etc) to actually improve the throughput?
>>>>>
>>>>>
>>>>> See above. I would say it does not depend on it. Obviously, in some
>>>>> cases (I/O) you would probably see better results than in others
>>>>> (CPU-intensive computation), and again it also depends on the overall
>>>>> context - other resources you need to serve, etc.
>>>>>
>>>>> Marek
>>>>>
>>>>> P.S. Interestingly, I've been just involved in a discussion, where the
>>>>> problem is that in some complex distributed systems you may start seeing
>>>>> cycles in the call graph. And if such system is implemented using
>>>>> synchronous APIs, a high system load can lead to thread pool exhaustion,
>>>>> which then leads to an inevitable system deadlock. This is another reason
>>>>> why esp. with any remote IO the use of async code is your best bet.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> Kishore.
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>> Mikael Ståldal
>>> Chief Software Architect
>>> *Appear*
>>> Phone: +46 8 545 91 572
>>> Email: mikael.staldal_at_appearnetworks.com
>>>
>>>
>>>
>>
>>
>> --
>> Mikael Ståldal
>> Chief Software Architect
>> *Appear*
>> Phone: +46 8 545 91 572
>> Email: mikael.staldal_at_appearnetworks.com
>>
>>
>>
>
>
> --
> Mikael Ståldal
> Chief Software Architect
> *Appear*
> Phone: +46 8 545 91 572
> Email: mikael.staldal_at_appearnetworks.com
>
--
Mikael Ståldal
Chief Software Architect
*Appear*
Phone: +46 8 545 91 572
Email: mikael.staldal_at_appearnetworks.com