dev@glassfish.java.net

Re: intermittent deadlock in GF due to logging

From: Jerome Dochez <Jerome.Dochez_at_Sun.COM>
Date: Fri, 10 Apr 2009 12:01:03 -0700

looks like I need to lock more than just the logger creation.

thanks, jerome

On Apr 10, 2009, at 11:45 AM, Amy Roh wrote:

> Just want to share that I still observe a deadlock using the latest
> workspace from this morning on mac (no custom logging handler has
> been added)
>
> =========
>
> startDomainUnix:
> [exec]
> [exec] Launching Watchdog in a separate JVM. The Domain is now
> remote-restart enabled.
> [exec]
> [exec] Domain (domain1) did not respond in 90 seconds. It means
> it is still coming up
> [exec] or it has failed to come up. Check server.log for details.
>
> BUILD FAILED
> /Users/Amy/glassfish-v3/v3/appserv-tests/config/run.xml:32: The
> following error occurred while executing this line:
> /Users/Amy/glassfish-v3/v3/appserv-tests/config/run.xml:38: exec
> returned: 1
>
> Total time: 1 minute 54 seconds
> in flushAll , creating new testSuiteHash
> bash-3.2$ jps
> 20756 Jps
> 20752 ASWatchdog
> 20753 ASMain
> 132
> bash-3.2$ jstack 20753
> 2009-04-10 11:39:29
> Full thread dump Java HotSpot(TM) 64-Bit Server VM (1.6.0_07-b06-57
> mixed mode):
>
> "Attach Listener" daemon prio=9 tid=0x0000000101a93000
> nid=0x1347d9000 waiting on condition
> [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "RMI TCP Accept-8686" daemon prio=5 tid=0x0000000101a5b800
> nid=0x1346d6000 runnable [0x00000001346d5000..0x00000001346d5ad0]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> - locked <0x0000000110852348> (a java.net.SocksSocketImpl)
> at java.net.ServerSocket.implAccept(ServerSocket.java:453)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.executeAcceptLoop(TCPTransport.java:369)
> at sun.rmi.transport.tcp.TCPTransport
> $AcceptLoop.run(TCPTransport.java:341)
> at java.lang.Thread.run(Thread.java:637)
>
> "AutoDeployer" daemon prio=5 tid=0x0000000101a52000 nid=0x1345d3000
> waiting for monitor entry [0x00000001345d2000..0x00000001345d2ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000010facca78> (a
> java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:246)
> at
> org
> .glassfish
> .deployment
> .autodeploy
> .AutoDeployDirectoryScanner.<clinit>(AutoDeployDirectoryScanner.java:
> 66)
> at
> org
> .glassfish
> .deployment.autodeploy.AutoDeployer.deployAll(AutoDeployer.java:385)
> at
> org
> .glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:
> 343)
> at
> org
> .glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:
> 331)
> at org.glassfish.deployment.autodeploy.AutoDeployService
> $1.run(AutoDeployService.java:200)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
>
> "Thread-6" prio=5 tid=0x0000000101a51800 nid=0x1344d0000 in
> Object.wait() [0x00000001344cf000..0x00000001344cfad0]
> java.lang.Thread.State: RUNNABLE
> at com.sun.grizzly.tcp.Request.<init>(Request.java:185)
> at com.sun.grizzly.http.ProcessorTask.initialize(ProcessorTask.java:
> 475)
> at
> com
> .sun
> .grizzly.http.SelectorThread.rampUpProcessorTask(SelectorThread.java:
> 916)
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1008)
> at
> com
> .sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
> 1088)
> at
> com
> .sun
> .enterprise
> .v3
> .services
> .impl.GrizzlyServiceListener.start(GrizzlyServiceListener.java:84)
> at com.sun.enterprise.v3.services.impl.GrizzlyProxy
> $1.run(GrizzlyProxy.java:209)
>
> "Thread-5" prio=5 tid=0x0000000101a4d800 nid=0x1343cd000 waiting for
> monitor entry [0x00000001343cc000..0x00000001343ccad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.Logger.getLogger(Logger.java:274)
> - waiting to lock <0x00000001252f1550> (a java.lang.Class for
> java.util.logging.Logger)
> at com.sun.grizzly.util.http.Cookies.<clinit>(Cookies.java:76)
> at com.sun.grizzly.tcp.Request.<init>(Request.java:185)
> at com.sun.grizzly.http.ProcessorTask.initialize(ProcessorTask.java:
> 475)
> at
> com
> .sun
> .grizzly.http.SelectorThread.rampUpProcessorTask(SelectorThread.java:
> 916)
> at com.sun.grizzly.http.SelectorThread.run(SelectorThread.java:1008)
> at
> com
> .sun.grizzly.http.SelectorThread.startEndpoint(SelectorThread.java:
> 1088)
> at
> com
> .sun
> .enterprise
> .v3
> .services
> .impl.GrizzlyServiceListener.start(GrizzlyServiceListener.java:84)
> at com.sun.enterprise.v3.services.impl.GrizzlyProxy
> $1.run(GrizzlyProxy.java:209)
>
> "DynamicReloader" daemon prio=5 tid=0x0000000101a09000
> nid=0x1342ca000 in Object.wait()
> [0x00000001342c9000..0x00000001342c9ad0]
> java.lang.Thread.State: TIMED_WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001107b9530> (a java.util.TaskQueue)
> at java.util.TimerThread.mainLoop(Timer.java:509)
> - locked <0x00000001107b9530> (a java.util.TaskQueue)
> at java.util.TimerThread.run(Timer.java:462)
>
> "pool-1-thread-4" prio=5 tid=0x00000001019f9800 nid=0x13419e000
> waiting on condition [0x000000013419d000..0x000000013419dad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001107b9640> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "Thread-3" daemon prio=5 tid=0x00000001019f2000 nid=0x134001000
> waiting for monitor entry [0x0000000133fff000..0x0000000134000ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000010facca78> (a
> java.util.logging.LogManager)
> at java.util.logging.Logger.getLogger(Logger.java:275)
> - locked <0x00000001252f1550> (a java.lang.Class for
> java.util.logging.Logger)
> at com.sun.jmx.remote.util.ClassLogger.<init>(ClassLogger.java:37)
> at
> javax
> .management
> .remote
> .JMXConnectorServerFactory.<clinit>(JMXConnectorServerFactory.java:
> 182)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread
> $RMIConnectorStarter.startRMIConnector(ConnectorStartupService.java:
> 282)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread.startConnector(ConnectorStartupService.java:
> 309)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread.run(ConnectorStartupService.java:339)
>
> "pool-3-thread-1" daemon prio=5 tid=0x0000000101992000
> nid=0x133e99000 waiting for monitor entry
> [0x0000000133e97000..0x0000000133e98ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at java.util.logging.Logger.getLogger(Logger.java:274)
> - waiting to lock <0x00000001252f1550> (a java.lang.Class for
> java.util.logging.Logger)
> at java.util.logging.LogManager.addLogger(LogManager.java:399)
> - locked <0x000000010facca78> (a java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
> at
> com
> .sun
> .enterprise
> .util.i18n.StringManagerBase.<clinit>(StringManagerBase.java:101)
> at
> com
> .sun
> .enterprise
> .transaction
> .jts
> .ResourceRecoveryManagerImpl
> .<clinit>(ResourceRecoveryManagerImpl.java:89)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun
> .reflect
> .NativeConstructorAccessorImpl
> .newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingConstructorAccessorImpl
> .newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at java.lang.Class.newInstance0(Class.java:355)
> at java.lang.Class.newInstance(Class.java:308)
> at
> com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:72)
> at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:
> 76)
> at
> com
> .sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
> 58)
> - locked <0x0000000110853138> (a
> com.sun.hk2.component.SingletonInhabitant)
> at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
> at
> com
> .sun
> .hk2
> .component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
> at com.sun.enterprise.v3.server.AppServerStartup
> $2.run(AppServerStartup.java:197)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "Thread-1" prio=5 tid=0x0000000101991000 nid=0x133c3b000 waiting on
> condition [0x0000000133c3a000..0x0000000133c3aad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001106c4240> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at
> com
> .sun.enterprise.server.logging.GFFileHandler.log(GFFileHandler.java:
> 479)
> at com.sun.enterprise.server.logging.GFFileHandler
> $3.run(GFFileHandler.java:238)
>
> "pool-2-thread-1" daemon prio=5 tid=0x000000010198c000
> nid=0x133b08000 waiting on condition
> [0x0000000133b07000..0x0000000133b07ad0]
> java.lang.Thread.State: TIMED_WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001106c4428> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at
> java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1963)
> at java.util.concurrent.DelayQueue.take(DelayQueue.java:164)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:582)
> at java.util.concurrent.ScheduledThreadPoolExecutor
> $DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:575)
> at
> java
> .util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:
> 946)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:906)
> at java.lang.Thread.run(Thread.java:637)
>
> "GlassFish Kernel Main Thread" prio=5 tid=0x0000000101982800
> nid=0x133d96000 waiting for monitor entry
> [0x0000000133d93000..0x0000000133d95ad0]
> java.lang.Thread.State: BLOCKED (on object monitor)
> at
> java
> .util.logging.LogManager.initializeGlobalHandlers(LogManager.java:871)
> - waiting to lock <0x000000010facca78> (a
> java.util.logging.LogManager)
> at java.util.logging.LogManager.access$900(LogManager.java:129)
> at java.util.logging.LogManager
> $RootLogger.getHandlers(LogManager.java:994)
> at java.util.logging.Logger.log(Logger.java:468)
> at java.util.logging.Logger.doLog(Logger.java:494)
> at java.util.logging.Logger.log(Logger.java:517)
> at
> com
> .sun
> .enterprise
> .v3
> .admin
> .adapter.AdminConsoleAdapter.setStateMsg(AdminConsoleAdapter.java:368)
> at
> com
> .sun
> .enterprise
> .v3
> .admin
> .adapter.AdminConsoleAdapter.initState(AdminConsoleAdapter.java:445)
> at
> com
> .sun
> .enterprise
> .v3.admin.adapter.AdminConsoleAdapter.init(AdminConsoleAdapter.java:
> 431)
> at
> com
> .sun
> .enterprise
> .v3
> .admin
> .adapter.AdminConsoleAdapter.postConstruct(AdminConsoleAdapter.java:
> 385)
> at
> com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:
> 160)
> at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:
> 90)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> com
> .sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:87)
> at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:
> 77)
> at
> com
> .sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
> 58)
> - locked <0x00000001056f0918> (a
> com.sun.hk2.component.SingletonInhabitant)
> at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
> at
> com
> .sun
> .hk2
> .component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
> at org.jvnet.hk2.component.Habitat$1.get(Habitat.java:252)
> at java.util.AbstractList$Itr.next(AbstractList.java:345)
> at
> com
> .sun
> .enterprise
> .v3
> .services
> .impl.GrizzlyService.registerNetworkProxy(GrizzlyService.java:400)
> at
> com
> .sun
> .enterprise
> .v3
> .services
> .impl.GrizzlyService.registerNetworkProxy(GrizzlyService.java:392)
> at
> com
> .sun
> .enterprise
> .v3.services.impl.GrizzlyService.postConstruct(GrizzlyService.java:
> 293)
> at
> com.sun.hk2.component.AbstractWombImpl.inject(AbstractWombImpl.java:
> 160)
> at com.sun.hk2.component.ConstructorWomb$1.run(ConstructorWomb.java:
> 90)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> com
> .sun.hk2.component.ConstructorWomb.initialize(ConstructorWomb.java:87)
> at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:
> 77)
> at
> com
> .sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
> 58)
> - locked <0x00000001106c4748> (a
> com.sun.hk2.component.SingletonInhabitant)
> at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
> at
> com
> .sun
> .hk2
> .component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
> at
> com
> .sun.enterprise.v3.server.AppServerStartup.run(AppServerStartup.java:
> 212)
> at com.sun.enterprise.v3.server.AppServerStartup
> $1.run(AppServerStartup.java:124)
>
> "pool-1-thread-3" prio=5 tid=0x0000000101969000 nid=0x133a05000
> waiting on condition [0x0000000133a04000..0x0000000133a04ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001106082a0> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "pool-1-thread-2" prio=5 tid=0x0000000101965800 nid=0x1338e3000
> waiting on condition [0x00000001338e2000..0x00000001338e2ad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x00000001106084f8> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "pool-1-thread-1" prio=5 tid=0x0000000101964800 nid=0x1337e0000
> waiting on condition [0x00000001337df000..0x00000001337dfad0]
> java.lang.Thread.State: WAITING (parking)
> at sun.misc.Unsafe.park(Native Method)
> - parking to wait for <0x0000000110608750> (a
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> at java.util.concurrent.locks.AbstractQueuedSynchronizer
> $ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
> at
> java.util.concurrent.ArrayBlockingQueue.take(ArrayBlockingQueue.java:
> 317)
> at org.jvnet.hk2.config.Transactions$Notifier
> $1.run(Transactions.java:142)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
>
> "{felix.fileinstall.poll=5000,
> felix.fileinstall.bundles.new.start=false, felix.fileinstall.dir=/
> Users/Amy/v3-4-10/v3/distributions/glassfish/target/glassfish/
> modules/, felix.fileinstall.debug=1}" daemon prio=5
> tid=0x00000001018fe800 nid=0x13333f000 waiting on condition
> [0x000000013333e000..0x000000013333ead0]
> java.lang.Thread.State: TIMED_WAITING (sleeping)
> at java.lang.Thread.sleep(Native Method)
> at
> org
> .apache.felix.fileinstall.DirectoryWatcher.run(DirectoryWatcher.java:
> 122)
>
> "Configuration Updater" daemon prio=5 tid=0x00000001018fe000
> nid=0x13323c000 in Object.wait()
> [0x000000013323b000..0x000000013323bad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001101b59e8> (a java.util.LinkedList)
> at java.lang.Object.wait(Object.java:485)
> at org.apache.felix.cm.impl.UpdateThread.run(UpdateThread.java:67)
> - locked <0x00000001101b59e8> (a java.util.LinkedList)
>
> "telnetconsole.Listener" daemon prio=5 tid=0x00000001018f3000
> nid=0x133139000 runnable [0x0000000133138000..0x0000000133138ad0]
> java.lang.Thread.State: RUNNABLE
> at java.net.PlainSocketImpl.socketAccept(Native Method)
> at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:384)
> - locked <0x00000001101b5ad0> (a java.net.SocksSocketImpl)
> at java.net.ServerSocket.implAccept(ServerSocket.java:453)
> at java.net.ServerSocket.accept(ServerSocket.java:421)
> at org.apache.felix.shell.remote.Listener
> $Acceptor.run(Listener.java:137)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixPackageAdmin" daemon prio=5 tid=0x00000001018e8000
> nid=0x133036000 in Object.wait()
> [0x0000000133035000..0x0000000133035ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001101b5d50> (a
> org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Object.wait(Object.java:485)
> at
> org
> .apache.felix.framework.PackageAdminImpl.run(PackageAdminImpl.java:
> 314)
> - locked <0x00000001101b5d50> (a
> org.apache.felix.framework.PackageAdminImpl)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixStartLevel" daemon prio=5 tid=0x00000001018e7000
> nid=0x132f33000 in Object.wait()
> [0x0000000132f32000..0x0000000132f32ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001101b6538> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at
> org.apache.felix.framework.StartLevelImpl.run(StartLevelImpl.java:241)
> - locked <0x00000001101b6538> (a java.util.ArrayList)
> at java.lang.Thread.run(Thread.java:637)
>
> "FelixDispatchQueue" daemon prio=5 tid=0x000000010186b800
> nid=0x132756000 in Object.wait()
> [0x0000000132755000..0x0000000132755ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010fb3f7c8> (a java.util.ArrayList)
> at java.lang.Object.wait(Object.java:485)
> at
> org
> .apache
> .felix.framework.util.EventDispatcher.run(EventDispatcher.java:875)
> - locked <0x000000010fb3f7c8> (a java.util.ArrayList)
> at org.apache.felix.framework.util.EventDispatcher.access
> $000(EventDispatcher.java:50)
> at org.apache.felix.framework.util.EventDispatcher
> $1.run(EventDispatcher.java:102)
> at java.lang.Thread.run(Thread.java:637)
>
> "OSGi Framework Launcher" daemon prio=5 tid=0x0000000101887800
> nid=0x131fe6000 in Object.wait()
> [0x0000000131fe4000..0x0000000131fe5ad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x00000001101b9e40> (a
> org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:
> 48)
> - locked <0x00000001101b9e40> (a
> org.apache.felix.framework.util.ThreadGate)
> at org.apache.felix.framework.Felix.waitForStop(Felix.java:785)
> at org.apache.felix.main.Main.main(Main.java:215)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun
> .reflect
> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingMethodAccessorImpl
> .invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at com.sun.enterprise.glassfish.bootstrap.ASMainFelix
> $1.run(ASMainFelix.java:91)
> at java.lang.Thread.run(Thread.java:637)
>
> "Low Memory Detector" daemon prio=5 tid=0x0000000101844800
> nid=0x131983000 runnable [0x0000000000000000..0x0000000000000000]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread1" daemon prio=9 tid=0x0000000101843800
> nid=0x131880000 waiting on condition
> [0x0000000000000000..0x000000013187e3e0]
> java.lang.Thread.State: RUNNABLE
>
> "CompilerThread0" daemon prio=9 tid=0x0000000101841800
> nid=0x13177d000 waiting on condition
> [0x0000000000000000..0x000000013177b3e0]
> java.lang.Thread.State: RUNNABLE
>
> "Signal Dispatcher" daemon prio=9 tid=0x0000000101841000
> nid=0x13167a000 runnable [0x0000000000000000..0x00000001316797d0]
> java.lang.Thread.State: RUNNABLE
>
> "Finalizer" daemon prio=8 tid=0x0000000101829800 nid=0x131560000 in
> Object.wait() [0x000000013155f000..0x000000013155fad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010fac9f10> (a java.lang.ref.ReferenceQueue
> $Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
> - locked <0x000000010fac9f10> (a java.lang.ref.ReferenceQueue$Lock)
> at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
> at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
>
> "Reference Handler" daemon prio=10 tid=0x0000000101828800
> nid=0x13145d000 in Object.wait()
> [0x000000013145c000..0x000000013145cad0]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010fac9b88> (a java.lang.ref.Reference$Lock)
> at java.lang.Object.wait(Object.java:485)
> at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
> - locked <0x000000010fac9b88> (a java.lang.ref.Reference$Lock)
>
> "main" prio=5 tid=0x0000000101801000 nid=0x100401000 in
> Object.wait() [0x0000000100400000..0x0000000100400c40]
> java.lang.Thread.State: WAITING (on object monitor)
> at java.lang.Object.wait(Native Method)
> - waiting on <0x000000010fb17bf0> (a java.lang.Thread)
> at java.lang.Thread.join(Thread.java:1161)
> - locked <0x000000010fb17bf0> (a java.lang.Thread)
> at java.lang.Thread.join(Thread.java:1214)
> at
> com
> .sun
> .enterprise
> .glassfish.bootstrap.ASMainFelix.launchOSGiFW(ASMainFelix.java:110)
> at
> com
> .sun.enterprise.glassfish.bootstrap.ASMainOSGi.run(ASMainOSGi.java:
> 137)
> at com.sun.enterprise.glassfish.bootstrap.ASMain.main(ASMain.java:65)
>
> "VM Thread" prio=9 tid=0x0000000101820800 nid=0x13135a000 runnable
>
> "VM Periodic Task Thread" prio=10 tid=0x0000000101846000
> nid=0x131a86000 waiting on condition
>
> "Exception Catcher Thread" prio=10 tid=0x0000000101802000
> nid=0x100504000 runnable
> JNI global references: 689
>
>
> Found one Java-level deadlock:
> =============================
> "AutoDeployer":
> waiting to lock monitor 0x00000001338eee38 (object
> 0x000000010facca78, a java.util.logging.LogManager),
> which is held by "pool-3-thread-1"
> "pool-3-thread-1":
> waiting to lock monitor 0x000000013214c448 (object
> 0x00000001252f1550, a java.lang.Class),
> which is held by "Thread-3"
> "Thread-3":
> waiting to lock monitor 0x00000001338eee38 (object
> 0x000000010facca78, a java.util.logging.LogManager),
> which is held by "pool-3-thread-1"
>
> Java stack information for the threads listed above:
> ===================================================
> "AutoDeployer":
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000010facca78> (a
> java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:246)
> at
> org
> .glassfish
> .deployment
> .autodeploy
> .AutoDeployDirectoryScanner.<clinit>(AutoDeployDirectoryScanner.java:
> 66)
> at
> org
> .glassfish
> .deployment.autodeploy.AutoDeployer.deployAll(AutoDeployer.java:385)
> at
> org
> .glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:
> 343)
> at
> org
> .glassfish.deployment.autodeploy.AutoDeployer.run(AutoDeployer.java:
> 331)
> at org.glassfish.deployment.autodeploy.AutoDeployService
> $1.run(AutoDeployService.java:200)
> at java.util.TimerThread.mainLoop(Timer.java:512)
> at java.util.TimerThread.run(Timer.java:462)
> "pool-3-thread-1":
> at java.util.logging.Logger.getLogger(Logger.java:274)
> - waiting to lock <0x00000001252f1550> (a java.lang.Class for
> java.util.logging.Logger)
> at java.util.logging.LogManager.addLogger(LogManager.java:399)
> - locked <0x000000010facca78> (a java.util.logging.LogManager)
> at com.sun.logging.LogDomains.getLogger(LogDomains.java:331)
> at
> com
> .sun
> .enterprise
> .util.i18n.StringManagerBase.<clinit>(StringManagerBase.java:101)
> at
> com
> .sun
> .enterprise
> .transaction
> .jts
> .ResourceRecoveryManagerImpl
> .<clinit>(ResourceRecoveryManagerImpl.java:89)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun
> .reflect
> .NativeConstructorAccessorImpl
> .newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun
> .reflect
> .DelegatingConstructorAccessorImpl
> .newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at java.lang.Class.newInstance0(Class.java:355)
> at java.lang.Class.newInstance(Class.java:308)
> at
> com.sun.hk2.component.ConstructorWomb.create(ConstructorWomb.java:72)
> at com.sun.hk2.component.AbstractWombImpl.get(AbstractWombImpl.java:
> 76)
> at
> com
> .sun.hk2.component.SingletonInhabitant.get(SingletonInhabitant.java:
> 58)
> - locked <0x0000000110853138> (a
> com.sun.hk2.component.SingletonInhabitant)
> at com.sun.hk2.component.LazyInhabitant.get(LazyInhabitant.java:107)
> at
> com
> .sun
> .hk2
> .component.AbstractInhabitantImpl.get(AbstractInhabitantImpl.java:60)
> at com.sun.enterprise.v3.server.AppServerStartup
> $2.run(AppServerStartup.java:197)
> at java.util.concurrent.Executors
> $RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.runTask(ThreadPoolExecutor.java:885)
> at java.util.concurrent.ThreadPoolExecutor
> $Worker.run(ThreadPoolExecutor.java:907)
> at java.lang.Thread.run(Thread.java:637)
> "Thread-3":
> at java.util.logging.LogManager.getLogger(LogManager.java:550)
> - waiting to lock <0x000000010facca78> (a
> java.util.logging.LogManager)
> at java.util.logging.Logger.getLogger(Logger.java:275)
> - locked <0x00000001252f1550> (a java.lang.Class for
> java.util.logging.Logger)
> at com.sun.jmx.remote.util.ClassLogger.<init>(ClassLogger.java:37)
> at
> javax
> .management
> .remote
> .JMXConnectorServerFactory.<clinit>(JMXConnectorServerFactory.java:
> 182)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread
> $RMIConnectorStarter.startRMIConnector(ConnectorStartupService.java:
> 282)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread.startConnector(ConnectorStartupService.java:
> 309)
> at org.glassfish.admin.mbeanserver.ConnectorStartupService
> $ConnectorsStarterThread.run(ConnectorStartupService.java:339)
>
> Found 1 deadlock.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe_at_glassfish.dev.java.net
> For additional commands, e-mail: dev-help_at_glassfish.dev.java.net
>