users@glassfish.java.net

Looking for input/opinions on OpenMQ/JMS vs. Grizzly for message delivery

From: Alex Sherwin <alex.sherwin_at_acadiasoft.com>
Date: Mon, 01 Jun 2009 17:10:11 -0400

I'm cross posting this to Glassfish Users and Grizzly Users as I'm
hoping to get input from both sides...

I realize this is a rather broad question, and I can only go into an
overview of the details for business reasons. With that said, heres the
kind of problem space:

- Distributed messaging (XML Message content)
- Delivery order doesn't matter
- Guaranteed delivery required
- Scalable (for load balancing, fail-over purposes)
- Security

Note that when I talk about Grizzly, I'm referring to RESTful HTTP Web
Services hosted in Glassfish served by grizzly and created with JAX-RS
using Jersey. Also, I'm not trying to bash OpenMQ/JMS, I'm trying to
apply it specifically to my problem space, and as you'll see since the
problem space isn't directly aligned with the features of the JMS spec I
feel like a lot of things it provides are actually hindering us.

Now JMS for some time now has generally been considered the right
approach here. The problem I have with JMS is the asynchronous and
in-order nature of the spec. That is of course what it is designed for,
but those features don't t specifically apply to my problem space.
Order of messaging is not important, for example.

Other problems I have with using JMS in the scenario are these:
- Requires use of EJBs/MDBs (not necessarily a bad thing, however these
are the only EJBs we use. without EJBs there is a potential to move
from an EAR to a simpler WAR).
- More complex configuration required, including but not limited to:
MDB thread pool configurations, transactionality for MDBs, clustering
for OpenMQ broker, authentication layer for OpenMQ, HADB for OpenMQ
cluster in addition to tying OpenMQ into a Glassfish cluster for load
balancing and fail-over, etc.
- Security between endpoint connections can be difficult to configure
- With multiple sites running 24/7 and sending messages to remote
Queues, there are "keep alive" issues where connections become stale etc
(stated from what we've observed during development, at least).
- Using OpenMQ/JMS with a few queues and MDBs the application(s) can
consume a large amount of heap and threads in the app server when
configured for high throughput

Usually people tout things that the JMS spec provide such a guaranteed
delivery and persistence, however from my standpoint in this situation
they are not really relevant. JMS "persistence", lets say if a message
fails to process because its next destination is not yet available,
causes a rollback which simply redelivers the message. In a case like
this, it's going to become the business codes responsibility to do
something useful with this message so that it can be attempted to be
delivered again at a later time when its destination may be available
again.

Now, using web services to deliver a message will result in the same
overall behavior; if the web service processing the message fails to
connect to the next hop, it's going to need to persist the state of the
message and try again later (same as if a message is in a
rollback/redelivery loop on a Queue). If the app server that is hosting
web service fails during processing, the requester of the web service
will not receive a successful response (non HTTP 200 OK response), and
will need to deal with it their (same as if a JMS message failed to be
delivered to the destination Queue).

Things that I think we benefit from in using Grizzly/HTTP/REST are:
- Easier to configure settings
- Uses far less resources to handle higher simultaneous throughput
(Grizzly NIO), we've tested this and seen the potential for massive
amount of throughput with far less resources then what a less capable
OpenMQ/JMS implementation was using.
- Security can be more easily tied directly into the same JAAS realms
used by our other business code
- Clustering for high availability and fail-over is greatly simplified
(no longer require any OpenMQ broker cluster, HADB, load balancing of
Queues etc). We also don't require any type of session replication for
these web services as they are completely stateless
- No persistent connections need to stay alive between endpoints (such
as remote connection factories in other scenario).

A basic summary is that I believe the overhead of using OpenMQ/JMS to
provide features we don't specifically need (or are available at
different tier when using web services) while requiring more complex
configuration and higher resource consumption make web services a much
more viable option for us.

I'm hoping to get some opinions on the matter, such as if we have any
major holes in our logic that we just haven't considered.

Any opinions are greatly appreciated

Thanks