Scott Oaks wrote:
> Given that mailinator.com advertises itself as a pretty high-volume
> site, I went and studied them a bit to see what, if anything, we might
> learn about it.
>
> The first thing that strikes me is that they have really two services:
> an SMTP service and an HTTP service. For the HTTP service, they are
> using Jetty 5.1.4 -- which I believe pre-dates the use of grizzly in
> jetty. But that would be interesting to see if they ever upgrade. The
> SMTP service is custom, and that's the heart of the system. He says
> their peak load would work out to 6 milion messages/day, so thats
> 70/second. Which is not that huge a number, actually.. [And
> interestingly, he has keep-alive turned off for jetty, which is a big
> advantage of NIO as I mention later.]
>
> But the thing that fundamentally strikes me here is that the protocol
> differences really drive the architecture. SMTP is a pretty Simple
> protocol; HTTP and CORBA and other things are much more conversational.
> I'm trying to think of a better word than that -- SMTP has a short
> conversation at the begnning, and then it sends the mail. But there's no
> think/pause time, where HTTP (with keep-alive) and CORBA and things on
> top of grizzly are interactively conversant.
>
> Speed of NIO vs traditional IO aside, an interactively conversant server
> cannot have one thread per connection and scale (for the reasons I said
> before). But a simple server could scale to tens of thousands of
> connections because they could in fact use a threadpool -- the accepted
> connections get put on a queue, a thread picks them up and completes the
> entire protocol with them in a few seconds, and moves on. [Particularly
> so for mailinator, which times out the sockets pretty aggressively.]
>
> An HTTP server could use traditional I/O and just limit the keep-alives
> for 10K connections so that only a few thousand are active. But NIO will
> be faster in that case since it can keep all the connections alive. And
> a CORBA server with one thread/client couldn't scale fo 10K users at all
> (unless there's some disconnect/reconnect part of the protocol that I
> don't know about).
>
No: the GIOP specification requires that a response to a request
be sent on the same connection as the one on which the request
was received. In many ways,
this is a defect in the protocol, since a big reason for the requirement
is the fact that GIOP maintains session state per connection
(one session only per connection). But the way GIOP is defined,
a connection with pending responses must be kept open, or
the request will terminate with an error. This also means that we
have non-trivial state per connection to keep track of suspended
client threads waiting for responses on the client side. The server
side is smaller, but we still need information about pending responses
and fragmented messages not yet completely received.
The server side can get worse, because the implementation of
a remote method invocation can make more remote method invocations,
leading to lots of suspended thread stacks across multiple node,
but we do not tend to use GIOP that way in the app server.
> On the other hand, for raw I/O, it's clear that traditional I/O will be
> faster: one read system call vs. one select and one read system call
> (not to mention some inefficiencies about copying the bytebuffer data,
> at least until Alan saves us there :-). But the scalability of NIO is
> what overcomes that (particularly with HTTP keep-alive). That's the
> point we need to make sure people understand.
>
> -Scott
>
>