users@glassfish.java.net

JMS recovery behavior on Glassfish Server restart

From: Theodor Richard <theodor.richard_at_googlemail.com>
Date: Thu, 17 Nov 2011 18:54:25 +0100

Hi,

first a little bit of background info:
I have a JMS Queue in my application. The message driven bean that consumes
messages from this queue sends emails. The queue is configured to retry the
message delivery two more 2 times in case of failure before it gets sent to
the Dead Message Queue. The JMS message itself contains an ID of an entity
in the database, where all the necessary information for sending the email
is stored. I also have another message driven bean that consumes messages
from the Dead Message Queue. I use this MDB to implement an exponential
back off algorithm for retries in message delivery. That is, I retrieve the
entity ID from the JMS message (that is delivered to th dead queue) and
create an EJB timer with this ID. The timeout method of the timer creates a
new JMS message with the entity ID and resubmits this message to the first
JMS queue.

Now comes the problem: Email sending in the MDB failed due to misconfigured
credentials in the email session. Email delivery is retried using the
exponential back off algorithm as described above. I entered the correct
credential in the email session and restarted Glassfish for the changes to
take effect. After the restart, the same email was delivered 10 times,
which is 9 times too much. I reproduced the issue analyzing the queue and
made the following observation:

- at any time BEFORE the server restart there were only at most one JMS
message in one of the queues (which is correct)
- after the server restart, there were 10 messages in the Email JMS Queue
and 0 messages in the Dead Message Queue
- it's 10 because 10 JMS messages were routed to the Dead Message Queue
during the retry phase in total

My question: why are these 10 messages re-delivered to the JMS upon server
restart, although they were consumed from the MDB for the dead message
queue?

Thanks,
Theo