Jagadish,
I'll get the domain.xml(s) and logs together. I've noticed issues when the DAS domain.xml and the instance/nodeagent domain.xmls get out of sync. I've also noticed issues where the admin says something different from the xml (enabled is true vs false). I don't have a test case - I have a sample app but it runs on my local cluster. I've tried to setup local as close to the failing cluster as possible but it still works for me. That's why I think its part of the domain.xml or our domains are setup in such a way that there is cross-talk (bad port assignments).
We are using Spring's JmsTemplate as you can see from the stack trace so we are holding on to the ConnectionFactory and the Destination.
I'll send files when I get them.
Brian
----- Original message -----
From: "Jagadish Prasath Ramu" <Jagadish.Ramu_at_Sun.COM>
To: users_at_glassfish.dev.java.net
Date: Thu, 15 Jan 2009 17:19:54 +0530
Subject: Re: NPEs during PoolManagerImpl.getResourceFromPool with Connector Connection Pools
Hi Brian,
Can you post your domain.xml snapshot & server.log from DAS & cluster
instances when you face the issue [FINEST for "connectors" &
"resource-adapter" module log levels] ?
If you have exact steps to reproduce the issue (or a test-case), that
would be helpful.
* Somehow the pool is removed (via delete-connector-resource,
delete-connector-connection-pool or disable connector resource). So, the
NPE is an after effect.
* Does your application cache the connection factory ? (eg: as an
instance variable and is used during a web request after the
resource/pool is removed)
Thanks,
-Jagadish
On Wed, 2009-01-14 at 15:30 -0600, Brian Repko wrote:
>
> Folks,
>
> Looking for some insight on this one. I'm using the sun-jms-adapter
> as our connector to OpenMQ.
> We are SunAS 9.1 ur2 in a clustered environment. At some point, we
> started getting NPEs when
> accessing our JMS objects (ConnectionFactories in particular). This
> gets reported as as the following:
>
> [#|2009-01-14T08:11:40.942-0600|WARNING|sun-appserver9.1|
> javax.enterprise.system.stream.err|
> _ThreadID=2066;_ThreadName=httpSSLWorkerThread-50011-1;_RequestID=49b9cf44-0580-494d-93b2-f175a7edd24a;|.stc.jmsjca.core.JConnection.createSessionByApplication(JConnection.java:153)
> at
> com.stc.jmsjca.core.JConnection.createSession(JConnection.java:314)
> at
> com.stc.jmsjca.core.WConnection.createSession(WConnection.java:94)
> at sun.reflect.GeneratedMethodAccessor295.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at
> org.springframework.jms.connection.TransactionAwareConnectionFactoryProxy$TransactionAwareConnectionInvocationHandler.invoke(TransactionAwareConnectionFactoryProxy.java:258)
> at $Proxy554.createSession(Unknown Source)
> at
> org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
> at
> org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:450)
> ... 165 more
> Caused by: java.lang.NullPointerException
>
> After some investigation into the actual cause of the NPE (yes,
> printing that stack trace would have been helpful),
> I find the following:
>
> Caused by: java.lang.NullPointerException
> at
> com.sun.enterprise.resource.PoolManagerImpl.getResourceFromPool(PoolManagerImpl.java:248)
> at
> com.sun.enterprise.resource.PoolManagerImpl.getResource(PoolManagerImpl.java:176)
> at
> com.sun.enterprise.connectors.ConnectionManagerImpl.internalGetConnection(ConnectionManagerImpl.java:337)
> at
> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:189)
> at
> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:165)
> at
> com.sun.enterprise.connectors.ConnectionManagerImpl.allocateConnection(ConnectionManagerImpl.java:158)
> at
> com.stc.jmsjca.core.JConnection.createSessionByApplication(JConnection.java:131)
> ... 36 more
>
> So its not the adapter - its the server. This code looks like the
> following:
>
> public ResourceHandle getResourceFromPool(ResourceSpec spec,
> ResourceAllocator alloc, ClientSecurityInfo info, Transaction tran)
> throws PoolingException {
> ResourcePool pool = getPool( spec.getConnectionPoolName() );
> // pool.getResource() has been modified to:
> // - be able to create new resource if needed
> // - block the caller until a resource is acquired or
> // the max-wait-time expires
> return pool.getResource(spec, alloc, tran); <<<<< this is
> line 248
> }
>
> So clearly, getPool returns null and this code never checks for that.
>
> I've tried rebooting the cluster (which for us is stop/start instance1
> and then stop/start instance2) and I've tried rebooting
> the DAS. I also believe that a full stop on the cluster (both
> instances) has been done as well. I now get the NPEs even
> after all the reboots.
>
> I've turned on logging (FINEST) on the RAR and for Connectors and I
> have also turned on Monitoring for Connectors
> and have the logs for those startups. Monitoring doesn't seem to work
> for this pool so that was pointless. And the logs
> show all kinds of pools getting added but then removed with a line
> about them being empty. We do have the pools
> set to a steady state of 0 since the RAR actually pools the
> connections/sessions. Its like they get created to deal with JNDI
> and then removed for some reason after that.
>
> Things seem to be bound to the JNDI but the internal pool is not
> there. I'm not sure how to get around this.
> I've thought about adding a JSP to a web app that gets the pool table
> and printing it out just to see.
>
> I've also noticed a CVS commit from 3 months ago to PoolManagerImpl
> (appsrv-core) related to cleaning up resources
> after transaction commit. I'm nervous that key functionality is not
> in our version but I don't understand the tags to know
> which release is which.
>
> I put together a sample program and when I went to deploy it, I kept
> getting JNDI errors. I didn't understand since
> everything was correct (targets/enabled state). It turned out that my
> resource-ref was listed as false in the domain.xml
> but true on the screen. This was easily reproducable by
> asadmin-creating them into the server and then retargetting
> them to the cluster and resetting enabled to true. If I add them via
> the gui directly (no asadmin), then they go in correctly.
>
> The common thread is that when enabled=false, I also see the pools
> getting added and then removed. But the JNDI
> is not there in one case and seems to still be there in the other
> case.
>
> Looking for clues as to what to do - let me know whatever you might
> need. I can get it for you. Thanks in advance.
> At this point I can't get my server to reboot and applications to
> deploy properly.
>
> Brian
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe_at_glassfish.dev.java.net
For additional commands, e-mail: users-help_at_glassfish.dev.java.net