users@glassfish.java.net

Glassfish v2.1 stop working

From: Paulo Reis <casmeiron_at_gmail.com>
Date: Mon, 25 Jan 2010 17:24:46 -0200

Hello guys,

I have an APP (EAR with few EJBıs3.1) running on GF v2.1 on linux.
Everythinı runs perfectly almost all the time but in some cases the GF just
get frozen and I canıt even stop the domain, I have to kill it.

After a lot of research, Iıve end up executing the follow command when the
problem happens:

asadmin generate-jvm-report --type=thread > thread_dump.txt

So, for my surprise, Iıve got (just the resume, too much text):

**** Thread Dump for Java Virtual Machine [App Server Instance Name: server]
****
Full Java Thread Dump Java HotSpot(TM) Server VM 16.0-b13 Sun Microsystems
Inc.
Number of threads: 554
Number of daemon threads: 541


90% of the threads are on WAITING state as:


Thread "p: thread-pool-1; w: 50" thread-id 1,638 thread-stateWAITINGWaiting
on lock: oracle.toplink.essentials.internal.helper.ConcurrencyManager_at_eaec66
     at: java.lang.Object.wait(Native Method)
     at: java.lang.Object.wait(Object.java:485)
     at:
oracle.toplink.essentials.internal.helper.ConcurrencyManager.acquire(Concurr
encyManager.java:111)
     at:
oracle.toplink.essentials.internal.identitymaps.CacheKey.acquire(CacheKey.ja
va:117)
     at:
oracle.toplink.essentials.internal.identitymaps.IdentityMap.acquireLock(Iden
tityMap.java:120)
     at:
oracle.toplink.essentials.internal.identitymaps.IdentityMapManager.acquireLo
ck(IdentityMapManager.java:142)
     at:
oracle.toplink.essentials.internal.sessions.IdentityMapAccessor.acquireLock(
IdentityMapAccessor.java:104)
     at:
oracle.toplink.essentials.internal.sessions.IdentityMapAccessor.acquireLock(
IdentityMapAccessor.java:94)
     at:
oracle.toplink.essentials.internal.descriptors.ObjectBuilder.buildObject(Obj
ectBuilder.java:506)
     at:
oracle.toplink.essentials.internal.descriptors.ObjectBuilder.buildObject(Obj
ectBuilder.java:389)
     at:
oracle.toplink.essentials.queryframework.ObjectLevelReadQuery.buildObject(Ob
jectLevelReadQuery.java:386)
     at:
oracle.toplink.essentials.queryframework.ReadObjectQuery.executeObjectLevelR
eadQuery(ReadObjectQuery.java:374)


So my APP just stop and I have to kill the entire domain and restart to make
it work again.

In my researches Iıve found somethinı like this related to a bug and a patch
number 5442078 for toplink but not sure about it.

Did you guys have some clue about what can I do to fix it?

Also, my APP does some simply selects, no custom Lock system is being used,
just the normal approach by EntityManager.

Thanks in adavance.