Folks,
Looking for some insight on this one. I'm using the
sun-jms-adapter as our connector to OpenMQ.
We are SunAS 9.1 ur2 in a clustered environment. At some point,
we started getting NPEs when
accessing our JMS objects (ConnectionFactories in particular).
This gets reported as as the following:
[#|2009-01-14T08:11:40.942-0600|WARNING|sun-appserver9.1|javax.en
terprise.system.stream.err|_ThreadID=2066;_ThreadName=httpSSLWork
erThread-50011-1;_RequestID=49b9cf44-0580-494d-93b2-f175a7edd24a;
|.stc.jmsjca.core.JConnection.createSessionByApplication(JConnect
ion.java:153)
at
com.stc.jmsjca.core.JConnection.createSession(JConnection.java:31
4)
at
com.stc.jmsjca.core.WConnection.createSession(WConnection.java:94
)
at sun.reflect.GeneratedMethodAccessor295.invoke(Unknown
Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodA
ccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.springframework.jms.connection.TransactionAwareConnectionFact
oryProxy$TransactionAwareConnectionInvocationHandler.invoke(Trans
actionAwareConnectionFactoryProxy.java:258)
at $Proxy554.createSession(Unknown Source)
at
org.springframework.jms.support.JmsAccessor.createSession(JmsAcce
ssor.java:196)
at
org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java
:450)
... 165 more
Caused by: java.lang.NullPointerException
After some investigation into the actual cause of the NPE (yes,
printing that stack trace would have been helpful),
I find the following:
Caused by: java.lang.NullPointerException
at
com.sun.enterprise.resource.PoolManagerImpl.getResourceFromPool(P
oolManagerImpl.java:248)
at
com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManag
erImpl.java:176)
at
com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetCo
nnection(ConnectionManagerImpl.java:337)
at
com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConne
ction(ConnectionManagerImpl.java:189)
at
com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConne
ction(ConnectionManagerImpl.java:165)
at
com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConne
ction(ConnectionManagerImpl.java:158)
at
com.stc.jmsjca.core.JConnection.createSessionByApplication(JConne
ction.java:131)
... 36 more
So its not the adapter - its the server. This code looks like
the following:
public ResourceHandle getResourceFromPool(ResourceSpec spec,
ResourceAllocator alloc, ClientSecurityInfo info, Transaction
tran) throws PoolingException {
ResourcePool pool = getPool( spec.getConnectionPoolName()
);
// pool.getResource() has been modified to:
// - be able to create new resource if needed
// - block the caller until a resource is acquired
or
// the max-wait-time expires
return pool.getResource(spec, alloc, tran); <<<<< this
is line 248
}
So clearly, getPool returns null and this code never checks for
that.
I've tried rebooting the cluster (which for us is stop/start
instance1 and then stop/start instance2) and I've tried rebooting
the DAS. I also believe that a full stop on the cluster (both
instances) has been done as well. I now get the NPEs even
after all the reboots.
I've turned on logging (FINEST) on the RAR and for Connectors and
I have also turned on Monitoring for Connectors
and have the logs for those startups. Monitoring doesn't seem to
work for this pool so that was pointless. And the logs
show all kinds of pools getting added but then removed with a
line about them being empty. We do have the pools
set to a steady state of 0 since the RAR actually pools the
connections/sessions. Its like they get created to deal with
JNDI
and then removed for some reason after that.
Things seem to be bound to the JNDI but the internal pool is not
there. I'm not sure how to get around this.
I've thought about adding a JSP to a web app that gets the pool
table and printing it out just to see.
I've also noticed a CVS commit from 3 months ago to
PoolManagerImpl (appsrv-core) related to cleaning up resources
after transaction commit. I'm nervous that key functionality is
not in our version but I don't understand the tags to know
which release is which.
I put together a sample program and when I went to deploy it, I
kept getting JNDI errors. I didn't understand since
everything was correct (targets/enabled state). It turned out
that my resource-ref was listed as false in the domain.xml
but true on the screen. This was easily reproducable by
asadmin-creating them into the server and then retargetting
them to the cluster and resetting enabled to true. If I add them
via the gui directly (no asadmin), then they go in correctly.
The common thread is that when enabled=false, I also see the
pools getting added and then removed. But the JNDI
is not there in one case and seems to still be there in the other
case.
Looking for clues as to what to do - let me know whatever you
might need. I can get it for you. Thanks in advance.
At this point I can't get my server to reboot and applications to
deploy properly.
Brian