Nigel,

Unfortunately, my schedule at work keeps me a bit busy to do a long conference call.  I think I have an hour available from 11-12 (EST) on Monday if you want to do a call.

So I guess, my perspective is that the creation of additional connections is implicit based on runtime deployment model, rather than code build/compilation state.  If under the hood, this unified API pulled in connections as needed and allows the container to determine if its receiving an already configured Threadlocal connection or a new connection, I think its easier for application developers to understand, rather than developers need to know that they have to do it themselves. 

The reason I went the builder pattern was to allow for increased readability and DRY support.  Let's say you want to send multiple messages to the same destination, you could just do:

foo.connectionFactory(cf).destination(d).send("Hi there").send("I'm alive").send("going to sleep now").send("good night.");

Which in this scenario you only had to configure the builder once, not pass in the same arguments repeatedly to the object.  This also allows us to handle transactions better on send.  This directly ties in to what I was talking about with a producer method.  Based would allow injection of an empty MessageBuilder.  Application developers, if they chose, could provide their own producers that were configured.  I consider this a more advanced use case, since it requires stronger knowledge of both CDI and JMS to make sense of though.  I may have missed it, but there should be a new factory object to provide instances, possibly statically; similar to how JPA gives references in SE applications.

The main issue with MessagingContext is that you need to have an app server up to do injection right now, since you're assuming a valid IC for injection.  I think you're also mixing resource injection and CDI injection when you do lookup="", at least as I understood it.

John

On Mon, Nov 28, 2011 at 9:41 AM, Nigel Deakin <nigel.deakin@oracle.com> wrote:

John,

Since email is a rather inefficient way of discussing topics like this, how about I call you on the phone so we can chat about it? I'm available just about any time, tine zones permitting.

I've made further comments below, but I think the agenda of distinct issues we need to consider is now:

1. Merging the connection, session and producer

This is desirable but only possible for Java EE applications. In Java SE applications we can also merge session and producer objects, but need to allow applications to create multiple sessions from the same connection. Is it possible to devise a simplified  API which supports both without being confusing?

2. How to instantiate the "connection"

Given a connection factory (or its JNDI name), how do you create a MessagingContext or a MessageBuilder? Both without and with (C)DI?

3. Whether to use multiple method arguments (the conventional approach) or multiple method calls (the builder approach).

I think this is a separate issue from (2). The answer may become clearer (to me) when we actually define what API the conventional approach would require.

4. Sending object payloads directly, with pluggable message marshalling

I'm temporarily parking this large issue since I think we can make progress with (1-3) without solving it.

5. Avoiding JMSException

I think we're agreed that we want the new API to avoid checked exceptions if possible. I think we can focus on (1-3) first and come back to this, but it doesn't sound too difficult to me.

More below. This is a useful discussion so please don't treat any of comments as being dismissive :-)

On 25/11/2011 20:25, John D. Ament wrote:

Nigel,

On Fri, Nov 25, 2011 at 2:34 PM, Nigel Deakin <nigel.deakin@oracle.com> wrote:

John,

Thanks, there's several interesting things here.

On 25/11/2011 18:04, John D. Ament wrote:

Nigel,

Sorry, should have read my message a bit more
"plain objects" not plan objects; e.g. Strings, byte[]'s, Streams.  Rather than needing to create a JMS specific domain object.

MessageBuilder/QueueBuilder/TopicBuilder would replace something like MessageProducer/MessageConsumer and provide all APIs to the application to send and receive messages.  We'd have to define the full API set, but what I current have is here:

https://github.com/seam/jms/blob/develop/api/src/main/java/org/jboss/seam/jms/QueueBuilder.java
https://github.com/seam/jms/blob/develop/api/src/main/java/org/jboss/seam/jms/TopicBuilder.java

One thing we'll need to figure out is if we want to support this based on destination type or inspecific.

Here's one example.  Sending a Text message as a String.

https://github.com/paulbakker/ducttape/blob/master/src/main/java/ducttape/managers/SmsNotifier.java

In this example, we inject a basic topic builder (current codebase uses a default ConnectionFactory only), tell it to connect to a topic, and then send a TextMessage based on the input String.

Here's a similar example using QueueBuilder:

@Resource("jms/OrderQueue) Destination d;
@Resource("jms/ConnectionFactory") ConnectionFactory cf;
@Inject QueueBuilder queueBuilder;

Serializable order = new Order(orderId);
queueBuilder.connectionFactory(cf).destination(d).sendObject(order);

OK. Let's rewrite that with each method call on a separate line to help us understand it:

// fields
@Resource("jms/OrderQueue)
Destination orderQueue;

@Resource("jms/ConnectionFactory")
ConnectionFactory connectionFactory;

public void someMethod(

    QueueBuilder queueBuilder = // omitted - is this just a no-arg constructor?

    // set the connection factory
    queueBuilder.connectionFactory(connectionFactory)l

    // set the destination
   queueBuilder.destination(orderQueue);

   // sent the payload

   Serializable order = new Order(orderId);

   queueBuilder.sendObject(order);

I think there are three distinct innovations here:

1. You're merging the connection, session, and producer
2. You're using an API style that allows you to assemble a sequence of method calls on one line
3. The application is sending the payload directly without constructing a javax.jms.Message

Let me consider these separately:

Innovation #1: merging the connection, session, and producer

I think we're in agreement here:: this is consistent with the proposals I was discussing with Clebert a couple of weeks ago. I think we can indeed merge the connection, session, and producer. Furthermore, as you may remember me explaining, I think we can merge connection, session and async consumer, but we need to retain a separate object for sync consumption.

However I see no reason to need separate classes for queues and topics - this would be reversing the "domain unification" of JMS 1.1.

I'm currently working up my ideas on this, but the trivial sending case would look like this

    @Resource(lookup = "jms/connectionFactory")
    ConnectionFactory connectionFactory;
   
    @Resource(lookup="jms/inboundQueue")
    Queue inboundQueue;
       
     public void javaEESender() throws JMSException {
                          
        MessagingContext messagingContext = connectionFactory.createMessagingContext();
        TextMessage textMessage = messagingContext.createTextMessage("Hello world");
        messagingContext.send(inboundQueue,textMessage);
        messagingContext.close();
       
    }

The biggest issue with MessagingContext from my perspective is that you need a connection factory to create one. 

But we're both using a connection factory. I'm suggesting

MessagingContext messagingContext = connectionFactory.createMessagingContext()

and you're using

MessageBuilder messageBuilder = new MessageBuilder(); // you don't actually say how you would create one
MessageBuilder messageBuilder2 = messageBuilder.connectionFactory(connectionFactory);

 (I appreciate you wouldn't explicitly declare a variable messageBuilder2 - I'm focussing on how you would use the connection factory here)

I think the way we instantiate (or inject) the "first" object is a separate issue from whether the object implements the builder pattern or is more conventional.

This limits the ability to do DI since you need the precursor object.  Let's thrown away QueueBuilder for now and just focus on MessageBuilder (btw, the name of the objects should indicate the pattern - Builder Pattern http://en.wikipedia.org/wiki/Builder_pattern)

I see. In this particular case, this just seems to be a way of replacing:

(1) foo.method3(arg1,arg2,arg3);
with
(2) foo.method1(arg1).method2(arg2).method3(arg3);

(1) is of course just a simple method call. I think that (2) only offers a potential benefit if we would otherwise have multiple similar variations on (3).  However I don't think we do. We only need

void send(Destination destination, Message message) throws JMSException;
void send(Destination destination,  Message message,  int deliveryMode,  int priority,  long timeToLive) throws JMSException;

which correspond directly to methods that already exist on MessageProducer (which is an attraction as it doesn't introduce a completely new API style)

But I don't want to sound dogmatic here: This can be one of the issues we can continue discussing.

The fact though is that you still need to handle the JMSException somehow.  I would hope these builder interfaces just skip over the exception and maybe start throwing some new, fancy runtime exception instead...

Yes, I have the same goal.

public class Builder{
@Resource(mappedName="jms/ConnectionFactory") ConnectionFactory cf;

@Resource(mappedName="jms/OrderDestination") Destination d;

@Inject MessageBuilder messageBuilder;

//Example one, receive a fired CDI event and publish the payload of the event to a destination using the MessageBuilder paradigm.
public void handleOrders(@Observes @Completed Order o) {
messageBuilder.connectionFactory(cf).destination(d).sendObject(o);
}

//Example two, directly invoked
public void sendMessageNotification(String message) {
messageBuilder.connectionFactory(cf).destination(d).sendString(message);
}

}

In the ideal world, the application developer would proactively create a producer method (and qualifier) to support DRY within their application.

So we add a producer:

public class MessageBuilderProducer {
@Resource(mappedName="jms/ConnectionFactory")
ConnectionFactory cf;

@Resource(mappedName="jms/OrderDestination")
Destination d;

//produces a message builder that is already configured to work with the above CF and destination.
@Produces @OrderDest
public MessageBuilder createOrderDestMessageBuilder(MessageBuilder messageBuilder) {
return messageBuilder.connectionFactory(cf).destination(d);
}

Will application developers be required to provide producer methods like this? I was rather hoping we could avoid the need for them to do this.

Then the example becomes simpler.

public class Builder2{

@Inject @OrderDest MessageBuilder messageBuilder;

//Example one, receive a fired CDI event and publish the payload of the event to a destination using the MessageBuilder paradigm.
public void handleOrders(@Observes @Completed Order o) {
messageBuilder.sendObject(o);
}

//Example two, directly invoked
public void sendMessageNotification(String message) {
messageBuilder.sendString(message);
}

}

Yes, this is simple, but the complexity (such as it is) is elsewhere (in the producer method). Whether this is an overall simplification would depend on whether there are lots of calls to sendString() in the application.

 

The above is without CDI. I think it's important we have a simple API for non-CDI cases. However I envisage we could use CDI to allow MessagingContexts to be injected directly, which would also take case of the close() method.

I'm a bit confused.  We already know that we can't inject MessagingContexts to an application using CDI, since they are tightly bound to the connection factory they are related to.

This is an important topic I'd like to explore further.

I know that we've been discouraged to do something like @Inject @JMSConnectionFactory(lookup="jms/ConnectionFactory") MessagingContext mc;  Or does MessagingContext have a method that allows you to specify the factory?

That's exactly what I had been thinking of. Why did you think it was discouraged?

(The problem with my earlier DI proposals was that they required multiple related objects to be injected: connections, sessions, producers etc and CDI wasn't rich enough to allow the dependencies between these objects to be adequately expressed. Here, however, we just have a single object to inject, so I don't see the same problem(

How does CDI handle the close method automatically?  If MessagingContext was autocloseable (I'm assuming the builder would be as well), I would understand that...

I'm proposing that MessageBuilderFactory.newBuilder() be a static method on the object that uses configuration to determine what object to instantiate.
 

My suggestion is to pass in the destination as an argument to the send() method, whereas your suggestion it to set this using a setter method. My feeling is that as the MessagingContext /QueueBuilder object would hold a connection, it is relatively expensive, so we might want to reuse it with multiple destinations which means it's best to pass the destination in as an argument. We don't want to force users to create two connections so they can write to two destinations.

Here's one way to reuse a connection, but switch destinations afterwards:

public class Builder3{

@Inject @OrderDest MessageBuilder messageBuilder;

@Resource(mappedName="jms/ConfirmationQueue") Destination confirms;

//Example three, reusing connection factory, but switching destination after sending the first object.
public void notifyOrderAndSendConfirmation(Order o) {
//First line sends to the default OrderDestination destination
messageBuilder.sendObject(o).
//this line sends it to the ConfirmationQueue
destination(confirms).sendObject(o);
}

}

So we still are in one connection, but use a different destination to send the message to.

OK. Again, as I mentioned above when I commented on the builder pattern, the issue we need to consider is whether this is simpler than simply passing in the destination as an argument  in each call to send().

 

My example above doesn't address the idea of sending the payload directly. That's something I'd like to change. See my comment later.

Note that we can only merge connection and session for Java EE applications. In Java SE applications we need to retain the ability to use multiple threads with a connection. I'm looking for an API which supports both  modes without looking too inconsistent. My idea is to provide  ConnectionFactory.createMessagingContext() for Java EE users, and Connection.createMessagingContext() for Java SE users who want multiple threads per  connection.

That is going to be really confusing.  Why are we making the API dependent on the runtime deployment model?

I'm not sure why you describe this as a "runtime deployment model". It is a fundamental concept of JMS that a Session can only be used by a single thread of control at a time (some reasons are given in Section 2.9 of the JMS 1.1 spec). Whatever the new API looks like, we'll need to be able to point to an object and say that, like a Session, it is a single-threaded context for producing and consuming messages, and applications will need to respect the threading restrictions on its use. This will be the object that has the commit() method on it, and the object used to define message order.

Given that applications that need to use JMS from multiple threads at the same time may not want to simply create a separate connection (i.e. socket) for each thread we need to give them control of the relationship between the socket (a limited physical resource) and the application thread.

Obviously that's what the existing JMS 1.1 API does now - and it's why we currently have separate connection and session objects.

However Java EE already restricts connections to a single session (so that connections can be pooled), gives us scope to merge these objects - as both you and I are proposing. However since we can't do that in Java SE, which is why I'm suggesting we need to keep a separate connection object for that case.

I'd welcome other suggestions for how to achieve this.

2. You're using an API style that allows you to assemble a cascade of method calls on one line

As I said, I think this is just a matter of API style. Does this pattern have a name in Java? (It reminds me of method cascading in Smalltalk).

Personally I would prefer a "setter" method to follow the standard Javadoc convention.

Sometimes you need to be bold... :-)
In this case, you're not necessarily setting an internal.  Let's suppose the builder interface actually was building upon Connection, MessageProducer and MessageConsumer (more like a wrapper API, the current ones are this right now).  when you do .destination(Destination d) or .destination(String mappedName) you're not necessarily changing an internal String or Destination, you're creating a new MessageProducer based on the input.
 

3. The application is sending the payload directly without constructing a javax.jms.Message

This is a big area which I haven't really got my head round yet, especially after the suggestions that were made to me last week. I agree we should try to define an API which makes this possible. Whether we use my MessagingContext idea or your QueueBuilder idea, the issues around sending and receiving object paylloads are the same.

I think we need to be clear what our goals are for this: I think it is more than just avoiding another intermediate object. Users need to be able to plug their own converters (e.g. to convert an XML object type to a TextMessage, ObjectMessage or ByteArray, whichever they prefer). Users also need to be able to construct a TextMessage by writing to a Stream. Vendors need to continue to be able to handle very large messages efficiently. There are probably other requirements as well. I'm not sure exactly what we need here and need to think about this a bit more before I can make any useful comments.

So, do we need to provide an internal, eg JMSProtocolWire that devs and other APIs can implement to convert objects into serialized formats.  E.g. a JAXBProtocolWire that took in Objects and turns them to XML as the of a TextMessage, automatically?  Then support this interface in the API?

I don't know. I'll need to spend more time thinking about it.

Nigel

 

Nigel

John

On Fri, Nov 25, 2011 at 12:41 PM, Nigel Deakin <nigel.deakin@oracle.com> wrote:

John,

Thank you for the ideas.

What is the purpose of MessageBuilder, TopicBuilder or QueueBuilder objects? Are these standard interfaces implemented by the JMS provider? What methods would they have?

What's a "plan object"?

Can you give a few examples of what the application code would look like?

Nigel

On 24/11/2011 00:06, John D. Ament wrote:

All,

I wanted to propose a possible solution to this issue.  I had previously not wanted to propose it since I wanted to avoid new APIs to solve the issue.  Its clear though that we need to have a new API to manage CDI capabilities.  I want to respond specifically to Nigel's comments from Devoxx separately, so that will come shortly.

The idea is to provide a simple builder API for working with messages, both sending and receiving.  I've thrown around in my head a few ideas, such as MessageBuilder, TopicBuilder or QueueBuilder.  I would break the interface into sets of methods:

1. Set of methods responsible for configuring the builder.
2. Set of methods responsible for creating messages from the builder.
3. Set of methods responsible for sending messages and plan objects (not necessarily POJOs?) to destinations.

For the first part, we would introduce methods for setting the connection factory, transactionality and acknowledge mode.  I would not include destinations here, since I would like to be able to connect to multiple destinations.

For the second group, I would create overloaded methods javax.jms.Message newMessage(X param) where X could be one of Serializable, String, Map<T,R>, byte[], char[], Stream, etc.  The returned type would be the correctly typed method.

For the third group, I would add similar methods for sendXMessage(X param), mostly for readability.  These would match up to each of the message types.  In addition, we would support a send that takes a Message object.

So here's how all of this correlates with CDI..

You should be able to inject this builder object (or perhaps a builder factory) to your application.  The injection should always be a blank slate.  In addition, you should be able to leverage reusability of code to make your own customer builder.  Let's suppose the following injection point...

@Inject MessageBuilder builder;

Then let's say you want to map the qualifier @EnterpriseSystem to have the connection factory found at jms/Enterprise and set transacted to true, you should be able to handle that with CDI:

@Inject MessageBuilder builder;

@Resource(mappedName="jms/Enterprise")
private ConnectionFactory cf;

@Produces
@EnterpriseSystem
public MessageBuilder createEnterpriseBuilder() {
        return builder.connectionFactory(cf).transacted();
}

So that when you did

@Inject @EnterpriseSystem MessageBuilder builder;


So basically, JMS would expose an interface (or maybe more than one?) that handled injectable components.  They would not be bound to a connection factory at the creation level, but instead be configured by the application developer.  We allow for DRY support but put the ownership on the application developer.

I hope this is clear enough for everyone, though I'd love for feedback.

John

On Wed, Nov 23, 2011 at 7:37 AM, Pete Muir <pmuir@bleepbleep.org.uk> wrote:

I spent some time looking at the DI problem that Nigel, Reza and Siva kindly explained to me at JavaOne. I can't find a good way to support DI cleanly with the current object structure in JMS.

Having spent a long time looking at DI, I would also highlight that using DI to fix an API is normally likely to fail. It's a much better idea to fix the underlying API to accurately reflect the use cases that people are commonly trying to solve, whilst still not making those other use cases impossible!

So I would just like to fully support Clebert's approach here!

On 22 Oct 2011, at 01:54, Clebert Suconic wrote:

> Maybe that's a crazy idea. What if we took a totally new approach?
> Why do we still need connections sessions consumers and producers?
>
> What if we only had producers and consumers?
>
> You could maybe have producer extending AbstractSession and have commit, rollback... Etc
>
> The issue with annotations would be gone if we take an approach like that.
>
>
> Consider this a brain storm please. Don't flame me if that's a crazy idea. Especially that I'm writing this in a Friday through the iPhone while drinking some wine :)
>
> Have a nice weekend.
> Sent from my iPhone
>
> On Oct 21, 2011, at 7:39 PM, Reza Rahman <reza_rahman@lycos.com> wrote:
>
>> It's a point I made before. If we leave things be on this for JMS 2, it gives projects/products like Seam 3 JMS and Resin room to evolve further to bring about proven/mature solutions that real-world customers use successfully in significant numbers. Currently, Seam 3 JMS is still in beta and Resin has a design and no implementation yet. I think that's the real underlying problem we have rather than any fundamental usability issue with what we need to do.
>>
>>
>> On 10/21/2011 7:48 AM, Nigel Deakin wrote:
>>> On 21/10/2011 03:12, Reza Rahman wrote:
>>>> Not to put too negative of a spin on this, but I don't think it would be terrible for non-standard solutions in this problem space to evolve a bit more.
>>>
>>> There are rather too many negatives in that sentence for me to understand what you mean. Can you say a bit more?
>>>
>>> Nigel
>>>
>>>>
>>>> That being said, we should still give this an honest try I think...
>>>>
>>>> On 10/19/2011 7:24 AM, Nigel Deakin wrote:
>>>>> Dear All,
>>>>>
>>>>> It's time for an update on progress on proposals to allow the injection of JMS objects into Java EE and Java SE applications.
>>>>>
>>>>> The last update you had from me on his subject was on 7th September, when I circulated minutes from a call I had with EG members Reza (Rahman) and John (Ament) to discuss John's AtInject proposals which were circulated earlier.
>>>>>
>>>>> Since then Reza, John and I have had one or two further calls and extensive email correspondence. I wrote a new document, based on the ideas in John's, which attempted to define a set of annotations which could be used to inject JMS objects into applications.  An updated version of this document is attached to this message. It lists a fairly complete set of possible annotations to inject almost all JMS objects, but it leaves a number of important issues unresolved, and until we can resolve these issues this document is simply a statement of desire rather than a realistic practical proposal.
>>>>>
>>>>> The unresolved issues are listed in the document, but in summary, the main ones are
>>>>> * The relationship between injected objects
>>>>> * Avoiding repetition on annotations
>>>>> * Injected objects cannot be local variables
>>>>> * Scope of injected variables
>>>>> * Java SE support
>>>>>
>>>>> It is important to appreciate that if we can't resolve these issues then we will probably need to abandon the document and start again.
>>>>>
>>>>> When I was at JavaOne earlier this month Reza and I had a meeting with Pete Muir, spec lead for CDI (Contexts and Dependency Injection). He offered to work with us to see whether it would be possible to achieve what we wanted in a reasonably standard manner using CDI - either the existing version or a future version.
>>>>>
>>>>> Since it's been a few weeks since I gave the full expert group (and user list) an update on this, please do feel free to ask questions about the attached document, make comments, or raise issues. Also, if you think you have ideas on how to resolve the unresolved issues please say so!
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Nigel
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> -----
>>>>> No virus found in this message.
>>>>> Checked by AVG -
>>>>> www.avg.com
>>>>>
>>>>> Version: 2012.0.1831 / Virus Database: 2092/4562 - Release Date: 10/19/11
>>>>>
>>>>
>>> No virus found in this message.
>>> Checked by AVG - www.avg.com
>>> Version: 2012.0.1831 / Virus Database: 2092/4565 - Release Date: 10/21/11
>>>
>>