I've been trying to resolve this issue for a while now with a great deal of confusion and frustration. We are deploying to GlassFish v2ur2 in our production environment. This is a clustered environment using a REMOTE JMS cluster. (2 glassfish instances and 2 openMQ JMS brokers) Our application used EclipseLink 1.1.1 as its JPA implementaiton. The issue that we have is that we want to utilize the EclipseLink Cache Coordination functionality. This will allow both instances of our application to stay in sync with each other. Ensuring that when one instance updates an object in the cache the other application instance receives those updates or invalidates the object. EclipseLink does this very easily using a JMS Topic.
The problem that we have run into stems from an open GlassFish bug #6580.
https://glassfish.dev.java.net/issues/show_bug.cgi?id=6580
The issue in short is that when EclipseLink attempts to subscribe to the Topic an Exception is raised.
com.sun.messaging.jms.JMSException: [ADD_CONSUMER_REPLY(15)] [C4036]: A
broker error occurred. :[412] [B4135]: Cannot add durable consumer null. No
ClientID was set on connection. user=guest, broker=localhost:37676(59279)
I'm told that this is an erroneous error message and in fact the exception is caused by the fact that the glassfish is attempting to share the connection. Per lindaschneider, I should be able to work around this issue by simply providing a clientID, but that would not resolve my issue. From her explanation,
What does share mean ???
For durables what it means is:
if you create two durables with the same name and same clientID on
different servers ... they will be considered a "single" durable.
e.g.:
You send 10 messages to topic foo [and create two subscribers with a
clientID of cid1 and a durable name of dname]
[gf instance 1: cid1, dname on foo] gets 5 messages
[gf instance 2: cid1, dname on foo] - gets 5 messages
For non-durable subscribers:
If you create two subscribers on the same topic with the same clientID,
they will share messages (same as the above durables case)
From the above explanation, our application would not want the subscription shared as described above. We want all messages that are produced to be consumed by both instances of the application equally. This is the only way that the entity cache can be kept in sync between the 2 application instances. So the correct behavior would in fact be to not share the subscription. GlassFish admin console does not provide us a way to turn off sharing, nor is there any other way to accomplish this via configuration. So I manage to handle this in Spring by applying advice to the connection factory when a connection is requested:
@Aspect
public class TopicConnectionFactoryAdvice {
private static Field ConnectionAdapter_xac;
private static Field ConnectionImpl_imqEnableSharedSubscriptions;
private static Field ConnectionImpl_clientID;
private static int counter = 0;
static {
// work-around for:
//
https://glassfish.dev.java.net/issues/show_bug.cgi?id=6580
try {
Class<?> ConnectionAdapter = Class.forName("com.sun.messaging.jms.ra.ConnectionAdapter");
ConnectionAdapter_xac = ConnectionAdapter.getDeclaredField("xac");
ConnectionAdapter_xac.setAccessible(true);
Class<?> ConnectionImpl = Class.forName("com.sun.messaging.jmq.jmsclient.ConnectionImpl");
ConnectionImpl_imqEnableSharedSubscriptions = ConnectionImpl.getDeclaredField("imqEnableSharedSubscriptions");
ConnectionImpl_imqEnableSharedSubscriptions.setAccessible(true);
ConnectionImpl_clientID = ConnectionImpl.getDeclaredField("clientID");
ConnectionImpl_clientID.setAccessible(true);
} catch (Exception e) {
e.printStackTrace(System.out);
}
}
private Logger logger = Logger.getLogger(getClass());
@Around(value="execution(public * javax.jms.TopicConnectionFactory.createTopicConnection(..))")
protected TopicConnection createTopicConnection(ProceedingJoinPoint pjp) throws Throwable {
// work-around for GlassFish bug
TopicConnection connection = (TopicConnection) pjp.proceed();
try {
if (connection != null) {
Object xac = ConnectionAdapter_xac.get(connection);
// Disable Sharing in the container. (Glassfish bug: 6580)
ConnectionImpl_imqEnableSharedSubscriptions.set(xac, false);
ConnectionImpl_clientID.set(xac, getClientID());
} else {
logger.debug("Connection is null");
}
} catch (Exception e) {
logger.error("Exception while applying Around Advice!", e);
}
return connection;
}
private String getClientID() {
String clientID = "unknown";
try {
java.net.InetAddress localInetAddress = java.net.InetAddress.getLocalHost();
clientID = localInetAddress.getHostName();
} catch ( Exception e ) {
logger.debug("fail_to_acquire_hostname", e);
}
clientID += "-" + ++counter;
return clientID;
}
}
The issue now is that when we deploy our application into our test environment, that is clustered the same as Production, we see instability on the server. One server becomes unstable and inevitably runs out of memory. (This does not happen on my local desktop clustered environment) The solution in test was to disable one JMS instance. This seems to resolve the instability but leaves me wondering if the environment is configured correctly? as my local environment seems to work well.
Still with me? Thanks...
Our infrastructure team contacted a Sun consultant that helped us configure the environments in the past and the consultants recommendation was to not use the managed JMS objects for EclipseLink but rather we should be using JGroups for our messaging communications. Now this seems rather odd to me. Why should I simply turn my back on the Managed JMS Objects in the container in preference for some other library, that will inevitably go directly to the JMS servers for the communication? Does this really make sense? Is this really the recommendation of Sun?
Any insight into this would be greatly appreciated, and thanks again for taking the time to read through all of this.
[Message sent by forum member 'cmathrusse' (cmathrusse)]
http://forums.java.net/jive/thread.jspa?messageID=353465