Hi, as already described in the forum entry
http://www.java.net/forum/topic/glassfish/glassfish/deadlock-transaction-recovery-glassfish-3122,
we faced some strange behavior during Glassfish startup resulting in deadlock
somewhere between transaction recovery and startup singletons. Unfortunately
our application is quite big and we could not create a smaller application
reproducing the bug. But we did some investigations of the thread dumps and
debugging of Glassfish core using Glassfish source code. Here is what we
found and what we found looks like a Glassfish bug ;): RecoveryHelperThread
started by Glassfish is blocked on
com.sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:62) by
trying to obtain ApplicationLoaderService from the habitat
(ConnectorsRecoveryResourceHandler.java:217). Seems that it tries to get some
connector stuff and for that it needs ApplicationLoaderService to wait for
connector to initialize I think. We use ActiveMQ RAR, but it is deployed
independently of our Web application as a separate connector application (so
no big master EAR). So RecoveryHelperThread blocks on trying to obtain
ApplicationLoaderService. And it can't get ApplicationLoaderService since it
is already blocked by the main thread in AppServerStartup line 253.
AppServerStartup tries at that moment to load our Web application and to
execute our startup singletons using ApplicationLoaderService. Our startup
singletons are doing some transactional stuff. So they try to open a
transaction. And they block on
com.sun.jts.CosTransactions.EventSemaphore.waitEvent since
RecoveryHelperThread is not ready yet with its recovery. And it can't since
it is blocked on ApplicationLoaderService which is blocked by main which is
blocked on CosTransactions.EventSemaphore. Deadlock. Or am I wrong? I
attached the stack traces of RecoveryHelperThread and the main thread. Best
regards, Sergiy
--
[Message sent by forum member 'sbarlabanov']
View Post: http://forums.java.net/node/894269