users@glassfish.java.net

Has anyone managed to set cache coordination in cluster? Hard to believe!!!

From: <glassfish_at_javadesktop.org>
Date: Tue, 03 Nov 2009 19:42:30 PST

Hi guys

When it comes to choose an application server for your production environment, there are a few choices. We went for Glassfish (Reference Implementation).

When it came to choosing a persistence provider, we went for EclipseLink (RI).

When it came to set up the cache co-ordination in our cluster, we went for the JMS option (as per EclipseLink recommendation)


GlassFish 2.1 (in cluster with two node agents in two separate machines, one instance each nodeagent) (JEE5 reference implementation)
EclipseLink 1.2 (JPA reference implementation)
JMS resources for the cache coordination

As per the recommendation, I went for the JMS option: On the book, this task can be accomplised by either setting up a bunch of properties in persistence.xml or implementing a SesisonCustomizer

After realising that the Persistence Unit Propeties where not always being processed, I decided to go for the session customizer option
, this meant goind to the pretty bad docs at eclipse.org wiki, a bit of fortune telling and days of waiting for help from the forums

after that I managed to get it more or less like this:

--------------------------------------------------------------------------------

public void customize(Session session) throws Exception {
RemoteCommandManager rcm = new RemoteCommandManager((CommandProcessor) session);
java.util.Properties props = new java.util.Properties();
props.put(Context.PROVIDER_URL, );
props.put(Context.INITIAL_CONTEXT_FACTORY, getInitialContextFactory());

JMSTopicTransportManager tm = new JMSTopicTransportManager(rcm);
tm.setTopicHostUrl(getTopicHostURL());
tm.setInitialContextFactoryName("com.sun.appserv.naming.S1ASCtxFactory");
tm.setTopicName(getJmsTopicName());
tm.setTopicConnectionFactoryName(getJmsConnectionFactoryName());
tm.setUserName(getUserName());
tm.setPassword(getPassword());




--------------------------------------------------------------------------------


The first couple of days, me and a couple of mates here, broke our heads trying to work out the right url for the getTopicHostURL(), As we are deploying to a cluster, we didn't know which of the instances in the cluster should we use as the "TopicHost" and what where we going to do if that instance failed. In the beggining we thought that the url for looking up the jms resources would ve like mq://machine:7676 or like.


For any application running in glassfish (weather clustered or not), to find these two JMS resources it is as easy as
new InitialContext().lookup("jms/myTopic")
new InitialContext().lookup("jms/myConnectionFactory")

Jet, for EclipseLink seems to see a hell of a lot more complex:

If I don't provide the
tm.setInitialContextFactoryName("com.sun.appserv.naming.S1ASCtxFactory");
it will use a WebLogic InitialContextFactory for the InitialContext lookup

If I dont' provide the getTopicHostURL() It will use some other missleading value, starting with ormi:// and the name of the machine in which it is running
If I don't provide the user name, it will use "admin" while glassfish prefers "guest"

And a series of other make it more difficult and complex than what it really is.

After a few days, not being eabke ti get the thing going, I downloaded the EclipseLink sources, which I thought would be more informative than the wiki or the docs.

In the end, as I couldn't understand at the time which topicHostURL to use and for me it was so easy to find these resources I overrided the method where the Eclipselink JMSTransportManager looks up for the resources and avoiding the lookup I would pass it the resources I had already looked up: It came to something like this:

final TopicConnectionFactory connectionFactory = (TopicConnectionFactory) ic.lookup(cfName);
final Topic topic = (Topic) ic.lookup(topicName);

JMSTopicTransportManager jmsTtm = new JMSTopicTransportManager(cm2) {

@Override
protected JMSTopicRemoteConnection createConnection(boolean isLocalConnectionBeingCreated) throws RemoteCommandManagerException {
Context remoteHostContext = null;
try {


TopicConnection topicConnection = connectionFactory.createTopicConnection();
// external connection is a puiblisher; local connection is a subscriber
return new JMSTopicRemoteConnection(rcm, topicConnection, topic, isLocalConnectionBeingCreated);
} catch (Exception ex) {
ex.printStackTrace();
RemoteCommandManagerException rcmException;
if (isLocalConnectionBeingCreated) {
rcmException = RemoteCommandManagerException.errorCreatingLocalJMSConnection(topicName, connectionFactoryName, getRemoteContextProperties(), ex);
} else {
rcmException = RemoteCommandManagerException.errorCreatingJMSConnection(topicName, connectionFactoryName, getRemoteContextProperties(), ex);
}
throw rcmException;
} finally {
if (remoteHostContext != null) {
try {
remoteHostContext.close();
} catch (NamingException namingException) {
// ignore
}
}
}
}

@Override
public Context getRemoteHostContext(String url) {
throw new RuntimeException("This method should never be called");


try {
//return super.getRemoteHostContext(url);
return new InitialContext();
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException();
}


}

@Override
protected Topic getTopic(Context remoteHostContext) {
log.info("JMSTopicTransportManager patch returning topic: " + topic);
return topic;
}

@Override
protected TopicConnectionFactory getTopicConnectionFactory(Context remoteHostContext) {
log.info("JMSTopicTransportManager patch returning cf: " + tcf);
return connectionFactory;
}
};


Anyways, Once I managed to let this JMSTopicTransportManager find its Topic and TopicConnectionFactory, the connection tries to start, but this is what happens:


Exception: com.sun.messaging.jms.JMSException: ADD_CONSUMER_REPLY(15) C4036:
A broker error occurred. :412 B4135: Cannot add durable consumer null. No
ClientID was set on connection. user=guest, broker=localhost:37676(59279)

As usual, I copy the exception, paste it on google search box, click search, first result: open bug in glassfish, one year ago, still not resolved. And as it has "apparently already been addressed in V3" why bother with V2.

Apparently, eventhough eclipselink tries to create a basic JMS consumer MQ attempts to create a "durable-Consumer" (this only happens when running in clustered environment), for which it needs a connection Id, (that you basically can't set unless you are in an application client container, there you go)


https://glassfish.dev.java.net/issues/show_bug.cgi?id=6580

Spent another bunch of hours trying to get my head around the "work around's" but I tell you it wasn't easy. At this point it was like... So now what... now that we have spent three good months building our application on glassfish+eclipselink, what do we do? try to switch to Jboss?, try to switch to Hibernate?, get rid of the cluster and deploy to a standalone server?

Now guys, tell me the truth, has anyone been eable to set this JMS cache coordination in a glassfish cluster? It seems really hard to believe.


Pablo.

p.s. apparently swictching the MQ to EMBEDDED solves the problem
[Message sent by forum member 'pablopina' (roastedbanana_at_yahoo.com)]

http://forums.java.net/jive/thread.jspa?messageID=370411