jsr343-experts@jms-spec.java.net

[jsr343-experts] Re: (JMS_SPEC-36) Allow messages to be delivered asyncrhonously in batches

From: Nigel Deakin <nigel.deakin_at_oracle.com>
Date: Mon, 26 Sep 2011 12:17:23 +0100

On 29/07/2011 22:22, Clebert Suconic wrote:
> Maybe that's just a detail, but I thought about this being a different
> class. Maybe BatchMessageListener...
>
> With that, you would be able to set a batchListener to a Consumer.
>
>
> consumer.setBatchListener(batchListenerInstance, size, timeout);
>
> or
>
> // it could use the annotations to get the two other parameters.
> consumer.setBatchListener(batchListener);
>

I have now upated the JIRA issue to incorporate Clebert's suggestion, which I think is a good one. You can read the
updated description at http://java.net/jira/browse/JMS_SPEC-36 or below:

-------------------------------------------------------------------------------------------
It would be necessary to configure this by adding a new method to javax.jms.MessageConsumer:

void setBatchMessageListener(BatchMessageListener listener, int batchSize, long batchTimeOut)

Sets the message consumer's batch message listener. Attempting to set both a message listener and a batch message
listener on the same MessageConsumer would cause a javax.jms.IllegalStateException.

     listener The BatchMessageListener being set

     batchSize) If set to a value greater than zero, messages will be delivered in batches of up to batchSize messages.
The actual batch size used may be smaller than this, but it may never be larger.

     setBatchTimeout(long batchTimeOut The maximum number of milliseconds that the JMS provider may defer message
delivery for in order to assemble a batch of messages that is as large as possible but no larger than the batch size.

-------------------------------------------------------------------------------------------

Any comments?

(If we approve this we will need to consider adding an equivalent feature to MDBs)

Nigel


> just to keep the APi compatible with previous applications.
>
> On Fri, Jul 29, 2011 at 1:19 PM, Nigel Deakin<nigel.deakin_at_oracle.com> wrote:
>> (This is the batch processing thread)
>>
>> I have now logged this in JIRA
>> http://java.net/jira/browse/JMS_SPEC-36
>>
>> Here's what I wrote:
>>
>> "It is proposed that the JMS API be extended to allow messages to be
>> delivered to a MessageListener in batches rather than individually as in JMS
>> 1.1.
>>
>> Currently, messages are delivered asynchronously by calling the
>> {{javax.jms.MessageListener}} method {{onMessage(Message message)}}.
>>
>> However some applications could process messages more efficiently if they
>> were delivered in batches, perhaps by calling a new
>> {{javax.jms.MessageListener}} method {{onMessages(Message[] messages)}}.
>>
>> It would be necessary to configure this by adding two new methods to
>> {{javax.jms.MessageConsumer}}:
>>
>> * {{setBatchSize(int batchSize)}} If set to a value greater than zero,
>> messages will be delivered in batches of up to {{batchSize}} messages. The
>> actual batch size used may be smaller than this, but it may never be larger.
>>
>> * {{setBatchTimeout(long batchTimeOut}} The maximum number of milliseconds
>> that the JMS provider may defer message delivery for in order to assemble a
>> batch of messages that is as large as possible but no larger than the batch
>> size.
>>
>> Acknowledgement:
>>
>> * If auto-acknowledgement was being used, all messages in the batch would be
>> acknowledged together. Section 4.4.12 "Duplicate Delivery of Messages" of
>> the JMS 1.1 spec explains that when auto-acknowledgement is being used, if a
>> failure occurs, clients can't know for sure whether the message has been
>> successfully acknowledged, and so the last consumed message may may be
>> redelivered. This section would need to be extended to state that in the
>> case of batch delivery, if a failure occurs, all messages in the last batch
>> may be redelivered.
>>
>> * If dups-ok acknowledgement was being used, message acknowledgement would
>> follow existing semantics.
>>
>> * If client acknowledgement, or local or global transactions, were being
>> used then message acknowledgement would follow existing semantics, which is
>> that a call to acknowledge(), or the commit of the transaction, would
>> acknowledge all unacknowledged messages delivered by the session."
>>
>>
>> See also below...
>>
>> On 29/07/2011 16:34, Clebert Suconic wrote:
>>>
>>> Comments bellow:
>>>>
>>>> With no timeout then we will get something like:
>>>>
>>>> (no wait)
>>>> onMessages(first 20 messages)
>>>> (100ms elapses)
>>>> onMessages(1 message)
>>>> (100ms elapses)
>>>> onMessages(1 message)
>>>> (100ms elapses)
>>>> onMessages(1 message)
>>>> (100ms elapses)
>>>> onMessages(1 message)
>>>> etc etc
>>>>
>>>> You'll see that this could end up with no batching at all.
>>>>
>>>
>>>
>>> I agree the timeout is needed, but there are certain use cases where
>>> the user doesn't want a timeout.
>>>
>>> The user may want to process as soon as possible anything that's on
>>> the buffer. But on that case the user can just configure timeout = 0.
>>>
>>> The user may expect the message consumer being busy processing the
>>> messages. So, he configured the consumer to get anything that is on
>>> the buffer, and it was produced while the user was processing the last
>>> batch.
>>
>> Yes, good point.
>>
>>>
>>> It would be just a configuration option for the user.
>>>
>>> Most users will use a timeout, but some will prefer having a timeout.
>>>
>>> Like, on this real use case:
>>>
>>
>> (snip>
>>
>> That is very helpful. So the general idea is that messages should be
>> delivered in batches because the onMessage() method can process them more
>> efficiently. So it doesn't matter if the batch size decreases when the rate
>> of message delivery slows.
>>
>> OK. As I mentioned above I've now logged this in JIRA.
>>
>>>
>>> They can't use MDBs for this either. My expectation is that the EJB
>>> spec will get this as part of their spec as well.
>>
>> If we added this to JMS we would definitely want to make it available to
>> MDBs. Since the MDB spec isn't JMS you could already implement this now by
>> changing the JMS provider and JCA adapter but without making changes to the
>> app server. But we would want to standardize how this is configured in EJB
>> 3.2.
>>
>>> (Another subject is that the MDB API sucks.. it should be a single
>>> API.. or super class for JMS and EE IMO). But that's another story.
>>
>> Indeed! As you know I'm trying to keep that topic separate.
>>
>> Nigel
>>
>>
>
>
>