Given that mailinator.com advertises itself as a pretty high-volume
site, I went and studied them a bit to see what, if anything, we might
learn about it.
The first thing that strikes me is that they have really two services:
an SMTP service and an HTTP service. For the HTTP service, they are
using Jetty 5.1.4 -- which I believe pre-dates the use of grizzly in
jetty. But that would be interesting to see if they ever upgrade. The
SMTP service is custom, and that's the heart of the system. He says
their peak load would work out to 6 milion messages/day, so thats
70/second. Which is not that huge a number, actually.. [And
interestingly, he has keep-alive turned off for jetty, which is a big
advantage of NIO as I mention later.]
But the thing that fundamentally strikes me here is that the protocol
differences really drive the architecture. SMTP is a pretty Simple
protocol; HTTP and CORBA and other things are much more conversational.
I'm trying to think of a better word than that -- SMTP has a short
conversation at the begnning, and then it sends the mail. But there's no
think/pause time, where HTTP (with keep-alive) and CORBA and things on
top of grizzly are interactively conversant.
Speed of NIO vs traditional IO aside, an interactively conversant server
cannot have one thread per connection and scale (for the reasons I said
before). But a simple server could scale to tens of thousands of
connections because they could in fact use a threadpool -- the accepted
connections get put on a queue, a thread picks them up and completes the
entire protocol with them in a few seconds, and moves on. [Particularly
so for mailinator, which times out the sockets pretty aggressively.]
An HTTP server could use traditional I/O and just limit the keep-alives
for 10K connections so that only a few thousand are active. But NIO will
be faster in that case since it can keep all the connections alive. And
a CORBA server with one thread/client couldn't scale fo 10K users at all
(unless there's some disconnect/reconnect part of the protocol that I
don't know about).
On the other hand, for raw I/O, it's clear that traditional I/O will be
faster: one read system call vs. one select and one read system call
(not to mention some inefficiencies about copying the bytebuffer data,
at least until Alan saves us there :-). But the scalability of NIO is
what overcomes that (particularly with HTTP keep-alive). That's the
point we need to make sure people understand.
-Scott
On Wed, 2008-02-20 at 10:29, Scott Oaks wrote:
> One reason I tend to discount the measurements in the blog is because I
> don't know how realistic the load test is.
>
> If you have thousands of threads that are handling requests at the same
> time, the thread contention is going to overwhelm the system; if you
> have thousands of threads that are idle and only a few are handling
> requests, you'll be okay. But one of the prime reasons for having a
> threadpool-NIO-based model is to provide a throttle on the number of
> active threads at a time to make sure that the system doesn't thrash. So
> even if socket-per-thread standard I/O is faster in the simple case (and
> like Charlie, I have some questions about that too) -- it doesn't mean
> that a threadpool-based NIO model isn't better for most situations.
>
> CORBA Is an interesting case here for our testing: although we simlulate
> thousands of remote users via CORBA, they share relatively few sockets
> and hence would have relatively few threads on the server side in a
> thread-per-socket traditional I/O model. It's a case where we would want
> to make sure we weren't optimizing for the test rather than the real
> world.
>
> -Scott
>
> On Wed, 2008-02-20 at 08:16, charlie hunt wrote:
> > I will add this to the meeting agenda.
> >
> > I also think a general topic that covers this should be added to the 2.0
> > list.
> >
> > To (try to) answer your questions ... you probably know, Solaris has
> > supported POSIX thread APIs since the early 2.x version of Solaris some
> > 10 years ago.
> >
> > Since NPTL requires a 2.6 Linux kernel, it may be that the JVM does not
> > currently support NTPL due to backward compatibility? It may also
> > detect the Linux kernel version at runtime and apply a given thread
> > model? I'm gonna check with some folks on the HotSpot runtime team.
> >
> > Anything newer than, including Solaris 8 uses a 1 to 1 Solaris threads
> > to Java threads model. In Solaris 8 you have to explicitly set
> > LD_LIBRARY_PATH=/usr/lib/lwp:$LD_LIBRARY_PATH to get the 1 to 1 model.
> > But, it is the default in Solaris 9, Solaris 10 and OpenSolaris. So,
> > since Solaris 8, there has been POSIX threads API support in Solaris and
> > a 1-to-1 OS threads to Java threads mapping.
> >
> > Rather than supporting the java.net.Socket approach, I would like to
> > explore the idea of using (in effect) a temporary NIO Selector per
> > connection along with a non-blocking SocketChannel. You & I have talked
> > about this a couple times in the past. ;-) This approach would allow
> > reading as much data as there is available per read, or per write which
> > could minimize the number read / write system calls in cases where PDUs
> > are small, or where more than one PDU can be read at a time. In
> > contrast, the traditional java.net.Socket approach will block and wait
> > until it receives the number bytes you expect to read. And, typically
> > the number of bytes you expect to read is equal to the size of the PDU.
> > So, for small PDUs, you may do more system calls to read versus the
> > other approach.
> >
> > There's a couple potential things in this approach of using NPTL +
> > standard IO may be exploiting. We all know NIO is as close to bare
> > metal as it can get, i.e. there's plenty of opportunity to do things
> > inefficiently. We know that use NIO effectively you need to minimize
> > the number of read() / write() system calls, minimize underlying system
> > calls associated with the Selector and minimize thread context
> > switching. So, the use of NIO, thread pool and its interaction with
> > the thread pool can make a big difference.
> >
> > There's also the possibility that testing was done exclusively on
> > Linux. So, we don't have any information on how Solaris compares, with
> > NIO versus without.
> >
> > I wish we could see the implementation, or at least a design of the NPTL
> > & standard IO program implemented by the blogger. We know what we
> > observed when comparing a ported async web on top of Grizzly to MINA showed.
> >
> > I think there is quite a few open and unanswered questions.
> >
> > charlie ...
> >
> > Ken Cavanaugh wrote:
> > > Let's put this on the agenda too. We still use the
> > > thread-per-connection model
> > > for the SocketFactory case in CORBA: should we again consider
> > > supporting this in
> > > Grizzly?
> > > Does anyone know how Linux + NPTL compares with Solaris threads, and
> > > especially
> > > how this is used in the JVM?
> > >
> > > Thanks,
> > >
> > > Ken.
> > >
> > >
> > > -------- Original Message --------
> > > Subject: [Fwd: NIO vs NPTL+standard IO]
> > > Date: Tue, 19 Feb 2008 11:50:59 -0500
> > > From: Jeanfrancois Arcand <Jeanfrancois.Arcand_at_Sun.COM>
> > > Reply-To: dev_at_grizzly.dev.java.net
> > > To: dev_at_grizzly.dev.java.net
> > >
> > >
> > >
> > > -------- Original Message --------
> > > Subject: NIO vs NPTL+standard IO
> > > Date: Tue, 19 Feb 2008 14:07:47 +0100
> > > From: Stefano Bagnara <apache_at_bago.org>
> > > Reply-To: dev_at_mina.apache.org
> > > To: dev_at_mina.apache.org
> > >
> > > Today I found another blog post on the "usual" topic:
> > > http://mailinator.blogspot.com/2008/02/kill-myth-please-nio-is-not-faster-than.html
> > >
> > > Is this FUD or a new trend that MINA should take care of?
> > >
> > > Is there any plan to support standard IO as a transport for MINA based
> > > applications?
> > >
> > > It would be useful to understand how a MINA based application could
> > > "switch back" to multithreaded standard IO and result in better
> > > throughput on modern JVM/OS.
> > >
> > > Thank you,
> > > Stefano
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
> > > For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
> > >
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe_at_grizzly.dev.java.net
> > For additional commands, e-mail: dev-help_at_grizzly.dev.java.net
> >