Thank you for your opinions.
The pattern that Robert mentioned is good, but it may be a good solution
for end user web pages like flight search etc. For the Rest services, isn't
that the service is a stateless operation which gives the output for a
given input. That is there is a contract. The solution seems to break that
principle, meaning that there is an intermediate step for the client to
interact with.
Because the client has to wait anyway for the expensive operation to
complete (with async or polling option), it shouldn't matter if the
connection/socket is held. Also we typically have persistent connections
between servers (Front end app server talking to a Rest service pool will
have persistent connections)
I was referring to an use case where a REST service calls multiple service
calls (or Async IO enabled back-end systems) and aggregates/applies some
logic on top of those service calls and then gives a response back to its
clients. In this case having Async capability definitely helps as the
interface for the client does not change (clients makes a rest call and
gets a Typed response back) so the programming model does not change. For
the server because of using Async and interacting with other systems using
non-blocking IO will definitely improve the throughput of the server. (So
we can have less server instances handling the volume of requests than we
would need if we did not have Async)
It would be good if the documentation can be corrected otherwise users can
assume that if they take the processing to another thread then it will
improve overall throughput.
Thanks,
Kishore.
On Thu, Sep 4, 2014 at 12:58 PM, cowwoc <cowwoc_at_bbs.darktech.org> wrote:
> +1
>
> Sounds like a good way to go.
>
> Gili
>
>
> On 04/09/2014 3:52 PM, Robert DiFalco wrote:
>
> I tend not to use the async stuff at all for expensive operations. I want
> to spin up as many rest server processes as I need to for scalability. So
> the pattern I use is to return a SEE ALSO redirect to another URL that can
> be polled until a result shows up. The steps are basically like this:
>
> 1. REST request comes in for a long running operation.
> 2. Create a UUID for the operation's result.
> 3. Give the client a URL + '/' + UUID to poll for the result.
> 4. Server dispatches a job for the operation letting the client return
> immediately
> 5. When the job is done the result is written to REDIS with the key to
> the result being the UUID we gave to the client.
> 6. The polling routine simply polls until either a TIMEOUT is reached
> or REDIS has a value at the UUID.
>
> In this way there is no shared state in memory and you can spin up as
> many REST servers as you need to. The async stuff in Jersey is not very
> valuable to me because I don't want to tie up a connection nor require the
> result to be at the same server as the request.
>
> Just FWIW. You're requirements may be different. The above is just a
> simple pattern I follow for all long live operations (about >1s).
>
>
>
>
>
> On Thu, Sep 4, 2014 at 12:27 PM, Kishore Senji <ksenji_at_gmail.com> wrote:
>
>> Hi All,
>>
>> The Async example is given at
>> https://jersey.java.net/documentation/latest/async.html
>>
>> "However, in cases where a resource method execution is known to take a
>> long time to compute the result, server-side asynchronous processing model
>> should be used. In this model, the association between a request processing
>> thread and client connection is broken. I/O container that handles incoming
>> request may no longer assume that a client connection can be safely closed
>> when a request processing thread returns. Instead a facility for explicitly
>> suspending, resuming and closing client connections needs to be exposed.
>> Note that the use of server-side asynchronous processing model will not
>> improve the request processing time perceived by the client. *It will
>> however increase the throughput of the server, by releasing the initial
>> request processing thread back to the I/O container while the request may
>> still be waiting in a queue for processing or the processing may still be
>> running on another dedicated thread*. The released I/O container thread
>> can be used to accept and process new incoming request connections."
>>
>> If veryExpensiveOperation() is expensive and is taking long time, then
>> having it run in a different thread and releasing the request processing
>> thread back to the I/O container, how would that improve the throughput?
>>
>> If that is the case we can as well increase the number of request
>> processing threads of the I/O container by the number of worker threads
>> that we would use in the case of the example and not worry about Async at
>> all.
>>
>> We can take more and more connections and have them queue up (or would
>> end up with creating many worker threads), but it would not necessarily
>> increase throughput. It would increase throughput if the
>> veryExpensiveOperation() is doing I/O over a Socket and if we use Async IO
>> for that operation, then we can use minimal request threads and very small
>> worker thread pool to do Async handling of the IO (or combine logic across
>> multiple Service calls doing non-blocking IO, similar to Akka futures).
>> This will improve the throughput as more work is done. But without
>> non-blocking IO, if the veryExpensiveOperation() is either CPU bound or
>> using blocking IO then the worker thread would infact be blocked for that
>> time and we would end up with huge thread pool or a big queue of tasks
>> waiting. Huge thread pool would not scale and big queue would also reduce
>> the throughput.
>>
>> Nevertheless we definitely need a thread to take the processing to a
>> different thread so that the container thread can be returned quickly. But
>> is my understanding correct that it depends on what
>> veryExpensiveOperation() does (blocking or non-blocking IO, or totally CPU
>> bound computation etc) to actually improve the throughput?
>>
>> Thanks,
>> Kishore.
>>
>
>
>