Date: Wed, 26 Mar 2008 17:51:51 -0400
IBM is apparently talking alot about these appserver results from John
Campbell, which show SJSAS lagging both jBoss and Geronimo:
http://weblogs.java.net/blog/jdcampbell/archive/2008/02/top_java_5_ee_s_1.html
It's a typical simple benchmark, which means that each HTTP connection
gets a single thing and then returns and never reuses the connection. As
such, it's not particularly interesting in the real world (or is it?
maybe I'm wrong about that...), except that we keep getting dinged on
this -- recall the earlier discussion we had on the mailinator claim
that traditional I/O was faster than NIO. Faster yes, but only for
single requests and not nearly as scalable.
On the other hand, I was also doing some work with SunSolve and SJSAS
9.0, where they are using the coyote connector, and it prompted me to
revisit John Campbells benchmark (which I'd previously verified did show
jBoss beating SJSAS 9.1, at least on Linux). So I re-ran the test with
9.1 and the coyote connector (e.g. traditional I/O), and found that the
coyote connector and glassfish greatly outperformed both jBoss and
glassfish+grizzly (see attached graph).
Again, it's not the typical use case, but since it keeps cropping up,
maybe we do need to consider a single-request model in grizzly (that
uses traditional I/O after all). I had a discussion at a recent
engineering meeting with some NetBeans guys who are running JSPWiki on
9.1 and weren't happy with it's performance vs. tomcat -- and it amounts
to the same thing for them: they only do a few requests at a time, so
they don't care about the scalability. [On the other hand, do they also
really notice the time difference for a single request? I'd think
not...]
So I'm not sure what the right answer here is (and I'm not going to go
respond to John Campbell's blog by pointing out how the coyote connector
does on his test, since in the end that just confuses the issue). But it
seems like something to think about some more...
-Scott
(image/png attachment: jbench.png)