Hi,
Jumping :-)
D. J. Hagberg (Sun) wrote:
> Thanks for the questions, Robert -- all good I think.
>
> Robert Greig wrote:
>> On 13/06/07, D. J. Hagberg (Sun) <Dj.Hagberg_at_sun.com> wrote:
>>> It seems to me that tying up precious WorkerThread resources waiting for
>>> synchronous write's to complete (where one is not bound by the
>>> javax.servlet specification) is a bit wasteful and limiting in
>>> scalability, especially if I have a client on the other end of a slow
>>> connection, but I don't have any hard numbers to back me up on this.
>>
>> I work on the Apache Qpid project, which is an implementation of the
>> AMQP protocol (for message oriented middleware). We don't use Grizzly
>> yet but I have spent a lot of time looking at this area for Qpid.
>
> It sounds like the protocols and goals may be similar. Basically the
> Shared Shell server acts as a rendesvous point for messaging among
> clients interested in a particular "topic" (in this case a VT100-type
> shell session on a server).
>
>> What is your primary goal? Is it to maximise the number of connections
>> that can be serviced concurrently or to maximise the data throughput
>> to clients? Also, is it important for clients to be able to read data
>> as they are sending it or is it more like http where they send then
>> receive?
>
> The goals are:
> 1. To keep all the content un-snoopable and ensure no man-in-the-middle
> attacks. So we're using SSL over TCP.
> 2. To scale to a "reasonable" number of concurrent connections
> per-server, say 2000-4000.
> 3. To minimize latency in the round-trip times -- being a terminal style
> client application, seeing keystroke results in sub-second time is
> expected.
>
>> Are you in control of the client code as well as the server? One thing
>> we had to consider was memory usage - i.e. ensuring that the queues
>> did not get too big either due to badly written or just slow clients.
>
> Yes, we control both clients and servers -- the lower-level parts of the
> code are shared between client and server, and our protocol has a
> "revision" number at the start of every message. This rewrite is going
> to bump the message revision number anyway, creating an incompatible
> change. Clients will auto-update with Java Web Start so it's not a big
> deal to keep the two in sync.
>
> The only constraint is that our client code still needs to support JDK
> 1.4.2. Our Server code, which is what I am primarily concerned with
> right now, will be using 1.5 or 1.6.
>
>>> The other possibility would seem to be to *only* register an OP_WRITE
>>> interest when there are messages in the outbound queue that need to be
>>> written, triggered whenever a message is added to the outbound queue.
>>> But in this case, there is an expense involved with waking up the
>>> selector thread and registering/updating selection keys, etc.
>>
>> In our design, we have several I/O threads, half of which are
>> responsible for reading and half for writing. Connections are assigned
>> a thread on a round robin basis.
>
> It looks like the Grizzly code has a single Selector thread for the
> whole instance, then delegates to WorkerThread's to do the actual
> reading and writing. I'm pretty sure I want to split read and write
> work for the exact reasons you mention (slow clients, possibly better
> throughput).
I would think implementing a new SelectorHandler (or two) can do the
job. One for OP_READ and OP_ACCEPT, and once dedicated to OP_WRITE. The
SelectorHandler that handle the OP_WRITE would have it own Pipeline
(Thread pool) and also its own ProtocolChain, which would only contains
one ProtocolFilter (an SSLWriteFilter). I don't think having two
Selector here will significantly impact performance (this is what I did
for supporting Comet (http long polling) in Grizzly.
The current Comet implementation currently support async read/write via
a second Selector. I suspect I can generalize the implementation so it
can be used outside of Comet. The current OutputWriter main purpose is
to mimic synchronous write using temporary Selector. But as part of the
sailfin project [1], the SIP container will need to do real async write
and using the current OutputWriter is not a solution. I'm currently
working with the Ericsson team (who are on that list as well) to replace
their current NIO implementation with Grizzly. One of their request is
to have async write. Hopefully we can come with a solution that apply to
all :-)
>
>> There is certainly an expense involved with waking up the thread.
>> However, OP_WRITE interest only needs to be registered when the kernel
>> buffer is full, and this is obviously partly dependent on how quickly
>> your clients are processing data.
>
> Makes sense.
>
>> The first design we had, threads were responsible for both reads and
>> writes but we wanted to be able to read and write from a socket at the
>> same time. We found it significantly reduced the memory usage of our
>> app since the build-up in the queues was far lower.
>
> Again, I think we are on similar tracks here -- our clients can read &
> write at the same time and expect to process every message
> asynchronously. The "old" design that I am chucking out here actually
> had a TON of threads to handle this with old-school blocking I/O and
> wait/notify on message and work queues.
>
>> Sorry for providing more questions than answers but I think this is
>> such a "delicate" area.
>
> Agreed, and it makes a significant difference on the nature of the
> protocol. Plus, it sounds like both our applications have a behavior
> that is very much divergent from HTTP's typical request/response/close
> behavior. Interesting, though, that AJAX is moving (abusing?) HTTP to
> more of an asynch. messaging protocol...
>
> Thanks for the discussion. I'll take a look at the ideas in Qpid/AMQP
> when I get a chance.
Robert, I still need to look at your code, but are you using a queue
with a dedicated thread pool for OP_WRITE or a second Selector (or
something completely different?)
Thanks
-- Jeanfrancois
[1]
https://sailfin.dev.java.net/
>
> -=- D. J.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe_at_grizzly.dev.java.net
> For additional commands, e-mail: users-help_at_grizzly.dev.java.net
>