users@grizzly.java.net

Re: Asynchronous request processing for ... RMI

From: Leo Romanoff <romixlev_at_yahoo.com>
Date: Mon, 5 Jul 2010 11:20:18 -0700 (PDT)

Hi Ken,

>> I know that my question is most likely beyond the scope of this mailing
>> list, but let me try ;-)

>Not really: as you noted below, many GF people are on the list.
>I am responsible for the ORB in GF, which implements RMI-IIOP.

Oh, then I'm at the right place! :-)


>> I've been using Glassfish together with Grizzly, Atmosphere, JAX-WS, etc.
>> Most of these technologies support at least some form of server-side
>> asynchronous request processing. And this is very important for
>> development
>> of scalable server-side applications.
>>
>> For my current project, I'd like to use RMI for communication between
>> clients and servers. BTW, servers are standalone Java applications (using
>> embedded GF, Grizzly, Jetty and many other things), as they need explicit
>> control over multi-threading which is prohibited by some JavaEE specs
>> like
>> EJB, etc. And I of course know that RMI is a synchronous invocation, as
>> it
>> was envisioned. But I'd like to benefit from asynchronous
>> non-thread-blocking request processing approach if possible and I'd like
>> to
>> do it in a transparent way for a client (i.e. it should look like a usual
>> local/library call). I think, it should be possible in principle, even
>> though I couldn't find anything like this yet. After all, there is a
>> socket
>> listening to requests and then triggering server-side stubs. If requests
>> could be intercepted at this point, then suspended, and the task for
>> their
>> processing could be put on the thread pool or something similar, then
>> after
>> their processing is over and response is prepared, requests could wake up
>> and response could be sent. But it is not clear for me, if such hooks are
>> exposed by the JDK and if it is possible at all.

>Have you looked at asynchronous EJBs in the EJB 3.1 specification?
>This does pretty much everything that you want, including the ease of use
>you get because any serializable Java object can be an argument or result
>of an EJB call (unlike the XML-based remoting mechanisms, which are more
>limited in their serialization capabilities). I think the current
implementation in
>GF spins a invocation thread to block waiting for the reply, while the
client thread
>continues on.

I thought about it. But my impression was that EJB 3.1 makes it look
asynchronous for the client. It is still synchronous on the server-side and
blocks the thread. Most likely, serving some 10000 long lasting requests
this way will not scale, or? Another concern - I need explicit controls
over processing threads and their management. And EJB spec still prohibits
it AFAIK.


>> Would it be possible to implement such a solution on top of Grizzly and
>> how
>> much effort it would need?

> Yes, but it's rather a lot of work. Will you ever need transactions,
> security,
> failover, or load balancing of requests? What about retry policy
> management
> for request failures?

At the moment, I don't care about most of these things. I need a good raw
performance.
But later it may become important, of course.
  
What disappoints me a bit, is the fact that all JSRs impelement these
features using a totally different way. After all, Atmosphere, Metro JAX-WS
RI with its tubes and fibers, JAX-WS RI AsyncProvider interfaces, etc all
use roughly similar pattern to implement server-side asynchrony.

As I looked a bit deeper into the implementation of the JAX-WS RI in Metro,
let me use this as an example. JAX-WS RI approach is actually rather generic
and flexible. IMHO, more flexible and well-thought through than RMI
implentations or RESTful implementations like Atmosphere and the like.
JAX-WS RI waits for incoming request and then creates a task object out of
it and puts it into a "tube assembly" which contains of multiple processing
steps. Each step is eventually handled by a dedicated thread-pool. There are
parts of the overall pipe responsible for transport handling (e.g. HTTP,
JMS, in-vm and some other transports are supported and new ones can be
defined), security, XML serialization, etc. And all that is configurable and
has APIs for building your our tubes and assemble them. Processing can be
suspended/resumed after each step using a special API. Level of
multi-threading is configurable by providing your own ThreadPools or
Executor objects. Once all required steps of the assembly has processed the
request and produced the response, it will be eventually send out.

And, BTW, the framework itself is rather protocol agnostic. It uses SOAP by
default, but AFAIK there is nothing in it that mandates that only SOAP can
be used. If one would use Java Serialization used by RMI, it should work as
well, I guess. It would be actually rather interesting, if RMI (IIOP or
JRMP) would be implemented using the same mechanisms. It would make it much
more configurable, flexible and extendable, plus increase code reuse inside
GF/JDK.

It was just an idea... In reality, there are always tons of differences in
low level details that can make such things impossible. But what do you
think about this approach?

> There is a lot of serialization machinery to deal with as well,
> plus issues like stub generation and managing the request->servant (object
> that
> implements the remote object) mapping.

>All of this is already implemented in the GF ORB for RMI-IIOP. At some
point,
>I'd like to in fact move the ORB to use Grizzly 2.x as its transport, but
that can't
>happen before GF 3.2 due to lack of resources.

Very interesting.

> It would certainly be possible to implement async RMI-IIOP directly at the
> level of
> the ORB, using syntax similar to that supported by the async EJB feature.
> A good implementation should avoid blocking threads while waiting for a
> response
> (even an internal implementation thread is probably too much in some
> cases).
> The basic need here is to modify the dynamic stub generator slightly,
> introducing a new
> API for the async case.

> The hard part is modifying the client ORB code to save and restore the
> per-request state.

Can you elaborate a bit more on this? I'm not so deep into RMI
implementations (yet ;-) to understand the problem that you describe here.

> I need to think about that part further.

Looking forward to hear more from you, once you made up your mind!

> Obviously a general
> continuation mechanism would make this easy, but failing that,
> the basic state to capture is in several ThreadLocals.
> Then it't a matter of unwinding the method return code so it can be called
> directly.

There are some attempts to support general continuations in Java (Maxine JVM
and some others), but it will take a while before it becomes mainstream.
There is also Javaflow, which could be good enough eventually. BTW, Scala
has a nice support for continuations out of the box in the recent releases.

> As far as async RMI-JRMP (the JDK version of RMI that is not IIOP based),
> I don't know anything
> about that. RMI-JRMP is not supported in GF: we use RMI-IIOP
> instead. There probably are ways to do similar things in RMI-JRMP, but I
> can't
> help with that.

OK. Thanks for your very interesting and insightful comments!

 -Leo
-- 
View this message in context: http://old.nabble.com/Asynchronous-request-processing-for-...-RMI-tp29073621p29078449.html
Sent from the Grizzly - Users mailing list archive at Nabble.com.