Amy -
Thank you for the assistance. I have not tried GF3.1b34 yet, primarily
because the issue only affects both of our production servers and I do not
want to change the configuration too much. However, I have discovered the
underlying cause of the problem and I would like some advice regarding how
to fix it.
The WidgetHelperBean class, the one containing the intializeWidget() method
seen in the stacktrace, is structured as such (symbols abbreviated to
protect the mailing-list ). This class has no JMS resources and doesn't do
anything special with JMS or JTA:
@Stateless
@PersistenceContext(unitName = "vhm", name = "VhmEM")
public class WidgetHelperBean implements WidgetHelper {
@EJB QuestionManager questionMgr;
@Override
public WidgetInitInfo initializeWidget (long e, Class c, Long ci, boolean
m, int s, String ip) {
// No programmatic JMS or JTA stuff at all
// ...
questionMgr.listUserQuestionsForEvent(e, QuestionListSort.POPULAR,
0, Math.max(initSize, INITIAL_PAGE_SIZE));
// ...
}
}
The @EJB is injected from a local bean that looks like:
@Stateless
@PersistenceContext(unitName = "vhm", name = "VhmEM")
public class QuestionManagerBean implements QuestionManager {
@Resource(name="ConnectionFactory",
mappedName="ConnectionFactory")
private ConnectionFactory connectionFactory;
private Connection connection;
@Resource(name=NewContent.TOPIC,
mappedName= NewContent.TOPIC)
private Topic newContentTopic;
@PostConstruct
public void makeConnection() {
connection = JmsUtils.createConnection(connectionFactory);
}
@PreDestroy
public void endConnection() {
JmsUtils.closeConnection(connection);
}
// Called by WidgetHelperBean.initializeWidget above
@TransactionAttribute(TransactionAttributeType.SUPPORTS)
public UserQuestionList listUserQuestionsForEvent (
long eventId, QuestionListSort sort) {
// Loads data from JPA, NO calls to JMS or JTA
}
// Not called by WidgetHelperBean or by ANY methods within that TX
private void announceNewQuestion (Question question) {
Session session = null;
try {
session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
MessageProducer producer = session.createProducer(newContentTopic);
ObjectMessage msg = session.createObjectMessage();
// Send msg
if (msg != null && producer != null) {
msg.setObject(question);
msg.setStringProperty(NewContent.TYPE, Question.TYPE);
msg.setIntProperty(NewContent.CAUSE, NewContent.CAUSE_SUBMITTED);
producer.send(msg);
}
}
catch (JMSException je) {
throw new IllegalStateException(
"MessageProducer could not be created or send message", je);
}
catch (Exception ex) {
log.error("Something terrible happened", ex);
}
finally {
VHMFileUtils.safeClose(session);
}
}
// ...
}
For completion sake, here are the 3 utility functions used in the previous
code snippet:
public class JmsUtils {
private static final Logger log = LoggerFactory.getLogger(JmsUtils.class);
public static Connection createConnection (ConnectionFactory factory) {
final String msg = "Could not connect to JMS.";
try {
return factory.createConnection();
}
catch (JMSException e) {
log.error(msg, e);
throw new IllegalStateException(msg, e);
}
}
public static void closeConnection (Connection connection) {
final String msg = "Could not close JMS connection";
if (connection != null) {
try {
connection.close();
}
catch (JMSException e) {
log.error(msg);
throw new IllegalStateException(msg, e);
}
}
}
}
public class VHMFileUtils {
static final Logger log = LoggerFactory.getLogger(VHMFileUtils.class);
public static void safeClose (Session s) {
try {
if (s != null) {
s.close();
}
}
catch (Throwable t) {
log.error("Failed to close session!", t);
}
}
}
Anyways, there you have it. The error only happens on our production
systems and it doesn't happen consistently. I removed the @PostConstruct
and @PreConstruct methods from QuestionManagerBean as well as any usage of
the QuestionManagerBean.announceQuestion() function. The error does not
occur in this configuration!
Hooray! So my guess is that our production system creates more instances of
the WidgetHelperBean (and by extension, the QuestionManagerBean as well) due
to higher load. This in turn causes more JMS connections to be created.
Can there be too many JMS connections? I'm following the recipe given by
Sun for EJB2+JMS and presented in "EJB3 In Action" by Panda, et al.
What is the solution? Do we need to ensure that the ConnectionFactory's
max-pool-size is at least as large as number-of-distinct-ejbs *
ejb-max-pool-size? Or, should we only create connections on-demand (within
the QuestionManagerBean.announceQuestion method)? Or, is something just
terribly wrong? Certainly, the problem exposes itself in a very
non-intuitive way.
Anyways, I hope this helps, and I hope you will have some advice. If you
need anything else (Config files, system information, etc..) please let me
know.
Thank you,
Paul G